Practical Bayesian Inference. A Primer for Physical Scientists

Preface......Page 2
The three doors problem......Page 5
Probability......Page 7
Expectation, variance, and moments......Page 11
Univariate probability distributions......Page 14
Cumulative distribution function and quantile function......Page 24
Multiple variables......Page 25
Combinations and permutations......Page 29
Random number generation......Page 32
Change of variables......Page 34
The three doors problem revisited......Page 36
Estimators......Page 37
Noise, measurement errors, and measurement models......Page 38
reduction......Page 40
Population properties vs sample estimates......Page 45
The mean is not necessarily the most efficient estimator......Page 46
Outliers and robust statistics......Page 48
Errors, accuracy, and precision......Page 50
Propagation and combination of uncertainties......Page 51
Introduction to data modelling......Page 56
Bayesian model comparison......Page 57
Data modelling with parametric models......Page 62
Making decisions......Page 67
Estimating one parameter from one data point......Page 68
An inference story......Page 75
One-dimensional linear regression......Page 77
Linear models......Page 84
Maximum likelihood and maximum posterior probability......Page 85
Multi-dimensional linear regression......Page 90
One-dimensional regression with nonlinear functions......Page 92
Least squares with errors on both axes......Page 95
Bias-variance decomposition......Page 96
Bayesian analysis of coin tossing......Page 99
Likelihoods can be arbitrarily small and their absolute values are irrelevant......Page 110
Assigning priors......Page 113
Some other conjugate priors......Page 121
Summarizing distributions......Page 122
Conditional and marginal distributions......Page 126
Inferring the parameters of a Gaussian......Page 127
A two-parameter problem: estimating amplitude and background......Page 134
The quadratic approximation......Page 143
Density estimation......Page 146
Why we need efficient sampling......Page 155
Uses of integration in Bayesian inference......Page 157
Monte Carlo integration......Page 158
Monte Carlo sampling......Page 160
Markov Chain Monte Carlo......Page 162
R code......Page 168
Fitting a straight line with unknown noise......Page 174
Fitting a quadratic curve with unknown noise......Page 191
A mixture model: fitting a straight line with an outlier model......Page 194
Fitting curves with arbitrary error bars on both axes......Page 200
Fitting a straight line with unknown noise......Page 205
Fitting a quadratic curve with unknown noise......Page 222
A mixture model: fitting a straight line with an outlier model......Page 225
Fitting curves with arbitrary error bars on both axes......Page 231
The principles of frequentist statistics and hypothesis testing......Page 236
One-sample hypothesis tests......Page 238
Two-sample hypothesis tests......Page 247
Hypothesis testing in linear modelling......Page 249
distribution......Page 250
Issues with frequentist hypothesis testing......Page 254
Bayesian model comparison......Page 257
Example of an analytic evidence calculation: is a coin fair?......Page 259
Example of a numerical evidence calculation: is there evidence for a non-zero gradient?......Page 263
Comparing Gaussians (or other distributions)......Page 270
How the evidence accounts for model complexity......Page 271
Other ways of computing the evidence......Page 274
Other measures for model comparison: AIC and BIC......Page 277
The stopping problem......Page 280
Issues with Bayesian model comparison......Page 284
Cross-validation......Page 287
Regularization in regression......Page 292
Regression with basis functions......Page 298
Regression kernels......Page 305
A non-parametric smoothing problem......Page 309
Numerical optimization (mode finding)......Page 312
Bootstrap resampling......Page 314
Refs......Page 320