Model selection and model averaging

Cover......Page 1
Half-title......Page 3
Series-title......Page 4
Title......Page 5
Copyright......Page 6
Dedication......Page 7
Contents......Page 9
Preface......Page 13
Acknowledgements......Page 14
A guide to notation......Page 16
1.1 Introduction......Page 21
1.2 Egyptian skull development......Page 23
1.3 Who wrote ‘The Quiet Don’?......Page 27
1.4 Survival data on primary biliary cirrhosis......Page 30
1.5 Low birthweight data......Page 33
1.6 Football match prediction......Page 35
1.7 Speedskating......Page 37
1.8 Preview of the following chapters......Page 39
1.9 Notes on the literature......Page 40
2.1 Information criteria for balancing fit with complexity......Page 42
2.2 Maximum likelihood and the Kullback–Leibler distance......Page 43
2.3 AIC and the Kullback–Leibler distance......Page 48
2.4 Examples and illustrations......Page 52
2.5 Takeuchi’s model-robust information criterion......Page 63
2.6 Corrected AIC for linear regression and autoregressive time series......Page 64
2.7 AIC, corrected AIC and bootstrap-AIC for generalised linear models......Page 66
2.8 Behaviour of AIC for moderately misspecified models......Page 69
2.9 Cross-validation......Page 71
2.10 Outlier-robust methods......Page 75
2.11 Notes on the literature......Page 84
Exercises......Page 86
3.1 Examples and illustrations of the BIC......Page 90
3.2 Derivation of the BIC......Page 98
3.2.1 Posterior probability of a model......Page 99
3.2.2 BIC, BIC., BICexact......Page 100
3.2.3 A robust version of the BIC using M-estimators......Page 101
3.3 Who wrote ‘The Quiet Don’?......Page 102
3.4 The BIC and AIC for hazard regression models......Page 105
3.5 The deviance information criterion......Page 110
3.6 Minimum description length......Page 114
3.7 Notes on the literature......Page 116
Exercises......Page 117
4.1 Comparing selectors: consistency, efficiency and parsimony......Page 119
4.2 Prototype example: choosing between two normal models......Page 122
4.3 Strong consistency and the Hannan–Quinn criterion......Page 126
4.4 Mallows’s Cp and its outlier-robust versions......Page 127
4.5 Efficiency of a criterion......Page 128
4.6 Efficient order selection in an autoregressive process and the FPE......Page 130
4.7 Efficient selection of regression variables......Page 131
4.8 Rates of convergence......Page 132
4.9 Taking the best of both worlds?......Page 133
4.10 Notes on the literature......Page 134
Exercises......Page 135
5.1 Some concrete examples......Page 137
5.2 Large-sample framework for the problem......Page 139
5.2.1 A fixed true model......Page 140
5.2.2 Asymptotic distributions under local misspecification......Page 141
5.3 A precise tolerance limit......Page 144
5.4 Tolerance regions around parametric models......Page 146
5.5 Computing tolerance thresholds and radii......Page 148
5.6 How the 5000-m time influences the 10,000-m time......Page 150
5.7 Large-sample calculus for AIC......Page 157
Exercises......Page 160
6.1 Estimators and notation in submodels......Page 165
6.2 The focussed information criterion, FIC......Page 166
6.3 Limit distributions and mean squared errors in submodels......Page 168
6.4 A bias-modified FIC......Page 170
6.5 Calculation of the FIC......Page 173
6.6.1 FIC in logistic regression models......Page 174
6.6.2 FIC in the normal linear regression model......Page 180
6.6.3 FIC in a skewed regression model......Page 183
6.6.4 FIC for football prediction......Page 187
6.6.5 FIC for speedskating prediction......Page 189
6.6.6 FIC in generalised linear models......Page 191
6.7 Exact mean squared error calculations for linear regression......Page 192
6.8 The FIC for Cox proportional hazard regression models......Page 194
6.9 Average-FIC......Page 199
6.10 A Bayesian focussed information criterion......Page 203
6.11 Notes on the literature......Page 208
Exercises......Page 209
7.1 Estimators-post-selection......Page 212
7.2 Smooth AIC, smooth BIC and smooth FIC weights......Page 213
7.3 Distribution of model average estimators......Page 215
7.4 What goes wrong when we ignore model selection?......Page 219
7.4.1 The degree of over-optimism......Page 220
7.4.2 The inflated type I error......Page 224
7.5.1 Correcting the standard error......Page 226
7.5.3 Simulation from the A distribution......Page 228
7.5.4 A two-stage confidence procedure......Page 230
7.6.1 Shrinkage and ridge regression......Page 231
7.6.2 Thresholding in wavelet smoothing......Page 233
7.7 Bayesian model averaging......Page 236
7.8 A frequentist view of Bayesian model averaging......Page 240
7.9 Bayesian model selection with canonical normal priors......Page 243
7.10 Notes on the literature......Page 244
Exercises......Page 245
8.1 The principle of order selection......Page 247
8.2 Asymptotic distribution of the order selection test......Page 249
8.3 The probability of overfitting......Page 252
8.4 Score-based tests......Page 256
8.5 Two or more covariates......Page 258
8.6.1 The original Neyman smooth test......Page 260
8.6.2 Data-driven Neyman smooth tests......Page 261
8.7 A comparison between AIC and the BIC for model testing......Page 262
8.8 Goodness-of-fit monitoring processes for regression models......Page 263
8.9 Notes on the literature......Page 265
Exercises......Page 266
9.1 AIC and BIC selection for Egyptian skull development data......Page 268
9.2 Low birthweight data: FIC plots and FIC selection per stratum......Page 272
9.3 Survival data on PBC: FIC plots and FIC selection......Page 276
9.4 Speedskating data: averaging over covariance structure models......Page 279
Exercises......Page 286
10.1 Model selection in mixed models......Page 289
10.1.1 AIC for linear mixed models......Page 290
10.1.2 REML versus ML......Page 291
10.1.3 Consistent model selection criteria......Page 292
10.2 Boundary parameters......Page 293
10.2.1 Maximum likelihood theory with a boundary parameter......Page 294
10.2.2 Maximum likelihood theory with several boundary parameters......Page 297
10.3 Finite-sample corrections......Page 301
10.4 Model selection with missing data......Page 302
10.5 When p and q grow with n......Page 304
10.6 Notes on the literature......Page 305
The (not so) Quiet Don......Page 307
Football matches......Page 308
Blood groups A, B, AB, O......Page 309
Survival for oropharynx carcinoma......Page 310
CH4 concentrations......Page 311
Birds on islands......Page 312
References......Page 313
Author index......Page 326
Subject index......Page 330