Bayesian Compendium

Preface
Contents
1 Introduction to Bayesian Thinking
	1.1 Bayesian Thinking
	1.2 A Murder Mystery
	1.3 Bayes' Theorem
		1.3.1 Implications of Bayes' Theorem
		1.3.2 The Odds-Form of Bayes' Theorem and a Simple Application
2 Introduction to Bayesian Science
	2.1 Measuring, Modelling and Science: The Three Basic Equations
	2.2 Terminological Confusion
	2.3 Process-Based Models Versus Empirical Models
	2.4 Errors and Uncertainties in Modelling
		2.4.1 Errors and Uncertainties in Model Drivers
		2.4.2 Errors and Uncertainties in Model Parameters
		2.4.3 Errors and Uncertainties in Model Structure
		2.4.4 Forward Propagation of Uncertainty to Model Outputs
	2.5 Bayes and Science
	2.6 Bayesian Parameter Estimation
3 Assigning a Prior Distribution
	3.1 Quantifying Uncertainty and MaxEnt
	3.2 Final Remarks for Priors
4 Assigning a Likelihood Function
	4.1 Expressing Knowledge About Data Error in the Likelihood Function
	4.2 What to Measure
5 Deriving the Posterior Distribution
	5.1 Analytically Solving Bayes' Theorem: Conjugacy
	5.2 Numerically `Solving' Bayes' Theorem: Sampling-Based Methods
6 Sampling from Any Distribution by MCMC
	6.1 MCMC
	6.2 MCMC in Two Lines of R-Code
	6.3 The Metropolis Algorithm
7 Sampling from the Posterior Distribution by MCMC
	7.1 MCMC and Bayes
		7.1.1 MCMC and Models
		7.1.2 The Need for Log-Transformations in MCMC
	7.2 Bayesian Calibration of a 2-Parameter Model Using the Metropolis Algorithm
		7.2.1 The Metropolis Algorithm
		7.2.2 Failed Application of MCMC Using the Default Settings
	7.3 Bayesian Calibration of a 3-Parameter Model Using the Metropolis Algorithm
	7.4 More MCMC Diagnostics
8 Twelve Ways to Fit a Straight Line
	8.1 Hidden Equivalences
	8.2 Our Data
	8.3 The Normal Equations for Ordinary Least Squares Regression (OLS)
		8.3.1 Uncertainty Quantification
	8.4 Regression Using Generalised Least Squares (GLS)
		8.4.1 From GLS to WLS and OLS
	8.5 The Lindley and Smith (LS72) Equations
	8.6 Regression Using the Kalman Filter
	8.7 Regression Using the Conditional Multivariate Gaussian
	8.8 Regression Using Graphical Modelling (GM)
	8.9 Regression Using a Gaussian Process (GP)
	8.10 Regression Using Accept-Reject Sampling
	8.11 Regression Using MCMC with the Metropolis Algorithm
	8.12 Regression Using MCMC with Gibbs Sampling
	8.13 Regression Using JAGS
	8.14 Comparison of Methods
9 MCMC and Complex Models
	9.1 Process-Based Models (PBMs)
		9.1.1 A Simple PBM for Vegetation Growth: The Expolinear Model
	9.2 Bayesian Calibration of the Expolinear Model
	9.3 More Complex Models
10 Bayesian Calibration and MCMC: Frequently Asked Questions
	10.1 The MCMC Algorithm
	10.2 Data and Likelihood Function
	10.3 Parameters and Prior
	10.4 Code Efficiency and Computational Issues
	10.5 Results from the Bayesian Calibration
11 After the Calibration: Interpretation, Reporting, Visualization
	11.1 Interpreting the Posterior Distribution and Model Diagnostics
	11.2 Reporting
	11.3 Visualising Uncertainty
12 Model Ensembles: BMC and BMA
	12.1 Model Ensembles, Integrated Likelihoods and Bayes Factors
	12.2 Bayesian Model Comparison (BMC)
	12.3 Bayesian Model Averaging (BMA)
	12.4 BMC and BMA of Two Process-Based Models
		12.4.1 EXPOL5 and EXPOL6
		12.4.2 Bayesian Calibration of EXPOL6's Parameters
		12.4.3 BMC and BMA of EXPOL5 and EXPOL6
13 Discrepancy
	13.1 Treatment of Discrepancy in Single-Model Calibration
	13.2 Treatment of Discrepancy in Model Ensembles
14 Gaussian Processes and Model Emulation
	14.1 Model Emulation
	14.2 Gaussian Processes (GP)
	14.3 An Example of Emulating a One-Input, One-Output Model
		14.3.1 Analytical Formulas for GP-calibration and Prediction
		14.3.2 Using R-package \texttt{geoR} for GP-calibration and prediction
	14.4 An example of emulating a process-based model (\texttt{EXPOL6})
		14.4.1 Training Set
		14.4.2 Calibration of the Emulator
		14.4.3 Testing the Emulator
	14.5 Comments on Emulation
15 Graphical Modelling (GM)
	15.1 Gaussian Bayesian Networks (GBN)
		15.1.1 Conditional Independence
	15.2 Three Mathematically Equivalent Specifications of a Multivariate Gaussian
		15.2.1 Switching Between the Three Different Specifications of the Multivariate Gaussian
	15.3 The Simplest DAG Is the Causal One!
	15.4 Sampling from a GBN and Bayesian Updating
		15.4.1 Updating a GBN When Information About Nodes Becomes Available
	15.5 Example I: A 4-Node GBN Demonstrating DAG Design, Sampling and Updating
	15.6 Example II: A 5-Node GBN in the form of a Linear Chain
	15.7 Examples III & IV: All Relationships in a GBN are Linear
		15.7.1 Example III: A GBN Representing Univariate Linear Dependency
		15.7.2 Example IV: A GBN Representing Multivariate Stochastic Linear Relations
	15.8 Example V: GBNs can do Geostatistical Interpolation
	15.9 Comments on Graphical Modelling
16 Bayesian Hierarchical Modelling (BHM)
	16.1 Why Hierarchical Modelling?
	16.2 Comparing Non-hierarchical and Hierarchical Models
		16.2.1 Model A: Global Intercept and Slope, Not Hierarchical
		16.2.2 Model B: Cv-Specific Intercepts and Slopes, Not Hierarchical
		16.2.3 Model C: Cv-Specific Intercepts and Slopes, Hierarchical
		16.2.4 Comparing Models A, B and C
	16.3 Applicability of BHM
17 Probabilistic Risk Analysis and Bayesian Decision Theory
	17.1 Risk, Hazard and Vulnerability
		17.1.1 Theory for Probabilistic Risk Analysis (PRA)
	17.2 Bayesian Decision Theory (BDT)
		17.2.1 Value of Information
	17.3 Graphical Modelling as a Tool to Support BDT
18 Approximations to Bayes
	18.1 Approximations to Bayesian Calibration
	18.2 Approximations to Bayesian Model Comparison
19 Linear Modelling: LM, GLM, GAM and Mixed Models
	19.1 Linear Models
	19.2 LM
	19.3 GLM
	19.4 GAM
	19.5 Mixed Models
	19.6 Parameter Estimation
		19.6.1 Software
20 Machine Learning
	20.1 The Family Tree of Machine Learning Approaches
	20.2 Neural Networks
		20.2.1 Bayesian Calibration of a Neural Network
		20.2.2 Preventing Overfitting
	20.3 Outlook for Machine Learning
21 Time Series and Data Assimilation
	21.1 Sampling from a Gaussian Process (GP)
	21.2 Data Assimilation Using the Kalman Filter (KF)
		21.2.1 A More General Formulation of KF
	21.3 Time Series, KF and Complex Dynamic Models
22 Spatial Modelling and Scaling Error
	22.1 Spatial Models
	22.2 Geostatistics Using a GP
	22.3 Geostatistics Using geoR
	22.4 Adding a Nugget
	22.5 Estimating All GP-hyperparameters
	22.6 Spatial Upscaling Error
23 Spatio-Temporal Modelling and Adaptive Sampling
	23.1 Spatio-Temporal Modelling
	23.2 Adaptive Sampling
	23.3 Comments on Spatio-Temporal Modelling and Adaptive Sampling
24 What Next?
	24.1 Some Crystal Ball Gazing
	24.2 Further Reading
	24.3 Closing Words
Appendix A Notation and Abbreviations
	Notation and Abbreviations
	Abbreviations
Appendix B Mathematics for Modellers
	How to read an equation
	Dimension Checking of Linear Algebra
Appendix C Probability Theory for Modellers
	Notation
	Probability Distributions
	Product Rule of Probability
	Law of Total Probability
	Bayes' Theorem
	Sequential Bayesian Updating
	Gaussian Probability Distributions
Appendix D R
	Basic R Commands
Appendix E Bayesian Software
Appendix F Solutions to Exercises
	Chapter 1摥映數爠eflinkChIntroBayesThink11
	Chapter 5摥映數爠eflinkChPosterior55
	Chapter 6摥映數爠eflinkChMCMCany66
	Chapter 7摥映數爠eflinkChMCMCpost77
	Chapter 8摥映數爠eflinkChTwelveWays88
	Chapter 14摥映數爠eflinkChGPemulation1414
	Chapter 15摥映數爠eflinkChGM1515
	Chapter 16摥映數爠eflinkChBHM1616
	Chapter 20摥映數爠eflinkChMachineLearning2020
	Chapter 21摥映數爠eflinkChTimeSeries2121
References
Index