Linear Models: An Integrated Approach

1.1 The linear model ......Page 23
1.2 Why a linear model ......Page 25
1.3 Description of the linear model and notations ......Page 26
1.4 Scope of the linear model ......Page 28
1.5 Related models ......Page 31
1.6 Uses of the linear model ......Page 33
1.7 A tour through the rest of the book ......Page 35
1.8 Exercises ......Page 38
2.1 Matrices and vectors ......Page 45
2.2 Inverses and generalized inverses ......Page 50
2.3 Vector space and projection ......Page 53
2.4 Column space ......Page 58
2.5 Matrix decompositions ......Page 62
2.6 Lowner order ......Page 67
2.7 Solution of linear equations ......Page 69
2.8 Optimization of quadratic forms and functions ......Page 70
2.9 Exercises ......Page 74
3.1 Covariance adjustment ......Page 77
3.2 Basic distributions ......Page 80
3.3 Distribution of quadratic forms ......Page 82
3.4 Regression ......Page 83
3.5 Basic concepts of inference ......Page 88
3.6 Point estimation ......Page 92
3.7 Bayesian estimation ......Page 100
3.8 Tests of hypotheses ......Page 104
3.9 Confidence region ......Page 107
3.10 Exercises ......Page 109
4 Estimation in the Linear Model ......Page 115
4.1.1 Linear unbiased estimator and linear zero function ......Page 116
4.1.2 Estimability and identifiability ......Page 119
4.2 Least squares estimation ......Page 122
4.3 Best linear unbiased estimation ......Page 124
4.4 Maximum likelihood estimation ......Page 129
4.5 Fitted value, residual and leverage ......Page 130
4.6 Dispersions ......Page 133
4.7.1 A basis set of linear zero functions ......Page 135
4.7.2 A natural estimator of error variance ......Page 138
4.7.3 A decomposition of the sum of squares ......Page 139
4.8 Reparametrization ......Page 140
4.9 Linear restrictions ......Page 142
4.10 Nuisance parameters ......Page 148
4.11.1 The case of normal distribution ......Page 150
4.11.2 The symmetric non-normal case ......Page 153
4.12 Collinearity in the linear model ......Page 156
4.13 Exercises ......Page 159
5.1 Distribution of the estimators ......Page 169
5.2.1 Confidence interval for a single LPF ......Page 170
5.2.2 Confidence region for a vector LPF ......Page 172
5.2.3 Simultaneous confidence intervals ......Page 173
5.2.4 Confidence band for regression surface ......Page 178
5.3.1 Testability of linear hypotheses ......Page 180
5.3.2 Hypotheses with a single degree of freedom ......Page 184
5.3.3 Decomposing the sum of squares ......Page 186
5.3.4 Generalized likelihood ratio test and ANOVA table ......Page 188
5.3.5 Special cases ......Page 190
5.3.6 Power of the generalized likelihood ratio test ......Page 193
5.3.7 Multiple comparisons ......Page 194
5.3.8 Nested hypotheses ......Page 195
5.4 Prediction in the linear model ......Page 196
5.4.1 Best linear unbiased predictor ......Page 197
5.4.2 Prediction interval ......Page 198
5.4.4 Tolerance interval ......Page 201
5.5 Consequences of collinearity ......Page 204
5.6 Exercises ......Page 207
6 Analysis of Variance in Basic Designs ......Page 213
6.1 Optimal design ......Page 215
6.2.1 The model ......Page 216
6.2.2 Estimation of model parameters ......Page 218
6.2.3 Analysis of variance ......Page 221
6.2.4 Multiple comparisons of group means ......Page 222
6.3 Two-way classified data ......Page 224
6.3.1 Single observation per cell ......Page 225
6.3.2 Interaction in two-way classified data ......Page 229
6.3.3 Multiple observations per cell: balanced data ......Page 234
6.3.4 Unbalanced data ......Page 238
6.4 Multiple treatment/block factors ......Page 241
6.5 Nested models ......Page 242
6.6.2 Uses of the model ......Page 246
6.6.3 Estimation of parameters ......Page 248
6.6.4 Tests of hypotheses ......Page 250
6.6.5 ANCOVA table and adjustment for covariate ......Page 251
6.7 Exercises ......Page 254
7 General Linear Model ......Page 265
7.1 Why study the singular model ......Page 266
7.2.1 Checking for model consistency ......Page 268
7.2.2 LUE, LZF, estimability and identifiability ......Page 270
7.3.1 BLUE, fitted values and residuals ......Page 273
7.3.2 Dispersions ......Page 277
7.3.3 The nonsingular case ......Page 279
7.4 Estimation of error variance ......Page 280
7.5 Maximum likelihood estimation ......Page 283
7.6 Weighted least squares estimation ......Page 285
7.7 Some recipes for obtaining the BLUE ......Page 287
7.7.1 \'Unified theory\'of least squares estimation ......Page 288
7.7.2 The inverse partitioned matrix approach ......Page 290
7.7.3 A constrained least squares approach ......Page 293
7.8 Information matrix and Cramer-Rao bound ......Page 294
7.9.1 Linear restrictions in the general linear model ......Page 297
7.9.2 Improved estimation through restrictions ......Page 300
7.9.3 Stochastic restrictions ......Page 301
7.9.4 Inequality constraints ......Page 304
7.10 Model with nuisance parameters ......Page 306
7.11 Tests of hypotheses ......Page 309
7.12 Confidence regions ......Page 312
7.13.1 Best linear unbiased predictor ......Page 313
7.13.2 Prediction and tolerance intervals ......Page 315
7.13.3 Inference through finite population sampling ......Page 316
7.14 Exercises ......Page 319
8 Misspecified or Unknown Dispersion ......Page 327
8.1 Misspecified dispersion matrix ......Page 328
8.1.1 When dispersion misspecification can be tolerated ......Page 330
8.1.2 Efficiency of least squares estimators ......Page 337
8.1.3 Effect on the estimated variance of LSEs ......Page 344
8.2.1 An estimator based on prior information ......Page 346
8.2.2 Maximum likelihood estimator ......Page 348
8.2.3 Translation invariance and REML ......Page 350
8.2.4 A two-stage estimator ......Page 352
8.3 Mixed effects and variance components ......Page 354
8.3.1 Identifiability and estimability ......Page 355
8.3.2 ML and REML methods ......Page 358
8.3.3 ANOVA methods ......Page 364
8.3.4 Minimum norm quadratic unbiased estimator ......Page 367
8.3.5 Best quadratic unbiased estimator ......Page 373
8.3.6 Further inference in the mixed model ......Page 374
8.4.1 Serially correlated observations ......Page 375
8.4.2 Models for spatial data ......Page 378
8.5.1 Combining experiments: meta-analysis ......Page 380
8.5.2 Systematic heteroscedasticity ......Page 383
8.6 Some problems of signal processing ......Page 384
8.7 Exercises ......Page 386
9 Updates in the General Linear Model ......Page 393
9.1.1 A simple case ......Page 394
9.1.2 General case: linear zero functions gained ......Page 397
9.1.3 General case: update equations ......Page 400
9.1.4 Application to model diagnostics ......Page 405
9.1.5 Design augmentation ......Page 407
9.1.6 Recursive prediction and Kalman filter ......Page 411
9.2 Exclusion of observations ......Page 419
9.2.1 A simple case ......Page 420
9.2.2 General case: linear zero functions lost ......Page 422
9.2.3 General case: update equations ......Page 424
9.2.4 Deletion diagnostics ......Page 426
9.2.5 Missing plot substitution ......Page 429
9.3 Exclusion of explanatory variables ......Page 432
9.3.1 A simple case ......Page 434
9.3.2 General case: linear zero functions gained ......Page 435
9.3.3 General case: update equations ......Page 437
9.4 Inclusion of explanatory variables ......Page 439
9.4.1 A simple case ......Page 440
9.4.2 General case: linear zero functions lost ......Page 441
9.4.3 General case: update equations ......Page 443
9.4.4 Application to regression model building ......Page 444
9.5 Data exclusion and variable inclusion ......Page 445
9.6 Exercises ......Page 446
10 Multivariate Linear Model ......Page 451
10.1 Description of the multivariate linear model ......Page 452
10.2 Best linear unbiased estimation ......Page 453
10.3 Unbiased estimation of error dispersion ......Page 457
10.4.1 Estimator of mean ......Page 461
10.4.2 Estimator of error dispersion ......Page 462
10.4.3 REML estimator of error dispersion ......Page 463
10.5.1 Effect on estimable LPFs, LZFs and BLUEs ......Page 464
10.5.2 Change in error sum of squares and products ......Page 465
10.5.3 Change in \'BLUE\' and mean squared error matrix ......Page 466
10.6.1 Generalized likelihood ratio test ......Page 467
10.6.2 Roy\'s union-intersection test ......Page 470
10.6.3 Other tests ......Page 471
10.6.4 A more general hypothesis ......Page 475
10.6.5 Multiple comparisons ......Page 476
10.6.6 Test for additional information ......Page 477
10.7 Linear prediction and confidence regions ......Page 479
10.8.2 Two-sample problem ......Page 482
10.8.3 Multivariate ANOVA ......Page 483
10.8.4 Growth models ......Page 484
10.9 Exercises ......Page 485
11 Linear Inference — Other Perspectives ......Page 491
11.1.1 General theory ......Page 492
11.1.2 Basis set of BLUEs ......Page 498
11.1.3 A decomposition of the response ......Page 501
11.1.4 Estimation and error spaces ......Page 505
11.2.1 Admissible linear estimator ......Page 508
11.2.2 Bayes linear estimator ......Page 511
11.2.3 Minimax linear estimator ......Page 514
11.3.1 Subset estimator ......Page 522
11.3.2 Principal components estimator ......Page 525
11.3.3 Ridge estimator ......Page 528
11.3.4 Shrinkage estimator ......Page 530
11.4.1 Best linear minimum bias estimator ......Page 532
11.5 A geometric view of BLUE in the Linear Model ......Page 534
11.5.1 The homoscedastic case ......Page 535
11.5.2 The effect of linear restrictions ......Page 537
11.5.3 The general linear model ......Page 538
11.6 Large sample properties of estimators ......Page 543
11.7 Exercises ......Page 547
Solutions to Odd-Numbered Exercises ......Page 555
Bibliography and Author Index ......Page 609
Index ......Page 629