Semiparametric regression

Half-title......Page 2
Series-title......Page 3
Title......Page 4
Copyright......Page 5
Dedication......Page 6
Contents......Page 7
Preface......Page 13
Acknowledgments......Page 14
Guide to Notation......Page 15
1 Introduction......Page 17
1.1 Assessing the Carcinogenicity of Phenolphthalein......Page 19
1.2 Salinity and Fishing in North Carolina......Page 20
1.3 Management of a Retirement Fund......Page 21
1.5 Term Structure of Interest Rates......Page 23
1.6 Air Pollution and Mortality in Milan: The Harvesting Effect......Page 27
2.2 Linear Regression Models......Page 31
2.2.1 General Linear Model......Page 35
2.3 Regression Diagnostics......Page 36
2.3.1 Influence Diagnostics......Page 40
2.3.2 Autocorrelation......Page 42
2.4.1 Confidence and Prediction Intervals......Page 44
2.4.2 Inference about the Regression Coefficients......Page 46
2.4.3 t-Statistics and p-Values......Page 47
2.4.5 Inference about New Observations......Page 48
2.4.6 Estimation of Sigma......Page 49
2.4.7 Extra Sums of Squares and Hypothesis Testing......Page 50
2.5 Parametric Additive Models......Page 52
2.5.1 Displaying Additive Fits......Page 53
2. 5.1.1 Vertical Alignment......Page 54
2.5.1.2 Variability Bands......Page 55
2.5.1.3 Partial Residuals......Page 56
2.5.1.4 Derivative Plots......Page 57
2.5.2 Degrees of Freedom......Page 58
2.6 Model Selection......Page 60
2.7 Polynomial Regression Models......Page 62
2.8 Nonlinear Regression......Page 64
2.9 Transformations in Regression......Page 67
2.9.1 LIDAR Monitoring of Air Pollutants......Page 70
2.11 Summary of Formulas......Page 71
3.1 Introduction......Page 73
3.2 Preliminary Ideas......Page 74
3.3 Practical Implementation......Page 78
3.4 Automatic Knot Selection......Page 80
3.5 Penalized Spline Regression......Page 81
3.6 Quadratic Spline Bases......Page 83
3.7 Other Spline Models and Bases......Page 85
3.7.1 B-Splines......Page 86
3.7.2 Natural Cubic Splines......Page 87
3.7.3 Radial Basis Functions......Page 88
3.8 Other Penalties......Page 90
3.9 General Definition of a Penalized Spline......Page 91
3.11 Error of a Smoother......Page 92
3.12 Rank of a Smoother......Page 94
3.13 Degrees of Freedom of a Smoother......Page 96
3.14 Residual Degrees of Freedom......Page 98
3.15.1 Local Polynomial Fitting......Page 100
3.15.2 Series-Based Smoothers......Page 102
3.16 Choosing a Scatterplot Smoother......Page 103
3.17 Bibliographic Notes......Page 104
3.18 Summary of Formulas......Page 105
4.2 Mixed Models......Page 107
4.2.1 Degrees-of-Freedom Interpretation......Page 109
4.3 Prediction......Page 111
4.3.1 Best Linear Prediction (BLP)......Page 112
4.3.2 Application to Pig Weight Example......Page 113
4.5.1 Estimation of Fixed Effects......Page 114
4.5.3 Best Linear Unbiased Prediction (BLUP)......Page 115
4.5.4 Estimation of Covariance Matrices......Page 116
4.6 Estimated BLUP (EBLUP)......Page 117
4.7 Standard Error Estimation......Page 118
4.7.1 Summary of Fit to Pig Weights......Page 119
4.8.1 Normal Theory Tests......Page 120
4.8.2 Likelihood Ratio Tests......Page 121
4.8.3 Restricted Likelihood Ratio Tests......Page 123
4.9 Penalized Splines as BLUPs......Page 124
4.11 Summary of Formulas......Page 126
5.1 Introduction......Page 128
5.2 The Likelihood Approach......Page 129
5.3.1 Cross-Validation (CV)......Page 130
5.3.2 Generalized Cross-Validation (GCV)......Page 132
5.3.2.1 Age and Income Data......Page 133
5.3.3 Mallows’s C Criterion......Page 134
5.3.3.2 Relationship between GCV and C......Page 135
5.4 Caveats of Automatic Parameter Selection......Page 136
5.5.1 Varying the Number of Knots......Page 139
5.5.2 Varying the Degree of the Regression Spline......Page 140
5.5.3 Default Choices for Knot Locations......Page 141
5.6.2 Full-Search Algorithm......Page 143
5.6.3 A Simulation Study......Page 144
5.6.4 Fossil Data......Page 145
5.8 Summary of Formulas......Page 147
6.2 Variability Bands......Page 149
6.3 Confidence and Prediction Intervals......Page 151
6.4 Inference for Penalized Splines......Page 153
6.5 Simultaneous Confidence Bands......Page 158
6.6 Testing the Adequacy of Parametric Models......Page 161
6.6.1 Restricted Likelihood Ratio Tests......Page 162
6.6.2 F-Test Approach......Page 163
6.7 Testing for No Effect......Page 165
6.7.1 F-Test for No Effect......Page 166
6.8 Inference Using First Derivatives......Page 167
6.8.1 Derivative Estimation via Penalized Splines......Page 169
6.8.3 LIDAR Data......Page 170
6.9 Testing for Existence of a Feature......Page 172
6.10 Bibliographical Notes......Page 174
6.11 Summary of Formulas......Page 175
7.2 Beyond Scatterplot Smoothing......Page 177
7.3 Semiparametric Binary Offset Model......Page 178
7.5 General Parametric Component......Page 180
7.6 Inference......Page 183
7.7 Bibliographical Notes......Page 184
8.1 Introduction......Page 186
8.2 Fitting an Additive Model......Page 187
8.3 Degrees of Freedom......Page 190
8.4 Smoothing Parameter Selection......Page 192
8.4.1 Upper Cape Cod Birthweight Data......Page 195
8.5 Hypothesis Testing......Page 197
8.5.2 F-tests......Page 198
8.6 Model Selection......Page 199
8.6.3 MCMC Model Selection Algorithms......Page 200
8.7 Bibliographical Notes......Page 201
9.2 Additive Mixed Models......Page 202
9.2.1 Additive Model Extension......Page 204
9.2.2 Serially Correlated Errors......Page 206
9.3 Subject-Specific Curves......Page 207
9.4 Bibliographical Notes......Page 208
10.2 Binary Response Data......Page 210
10.3 Logistic Regression......Page 211
10.4 Other Generalized Linear Models......Page 213
10.4.1 Poisson Regression and Overdispersion......Page 214
10.4.3 The Gamma GLM: A Model with a Constant Coefficient of Variation......Page 215
10.5 Iteratively Reweighted Least Squares......Page 216
10.7 Overdispersion and Variance Functions: Pseudolikelihood......Page 217
10.8 Generalized Linear Mixed Models......Page 219
10.8.1 Estimation of Model Parameters......Page 220
10.8.2 Penalized Quasilikelihood (PQL)......Page 221
10.8.5 Fitting via Expectation Maximization......Page 222
10.8.6 Bayesian Fitting via Markov Chain Monte Carlo......Page 223
10.8.8 Standard Error Estimation......Page 224
10.9 Deviance......Page 225
10.10.1 Fitting a Logistic Regression......Page 226
10.10.2 Standard Error Estimation in Logistic Regression......Page 227
10.10.4 Derivation of PQL......Page 228
10.11 Bibliographical Notes......Page 229
11.1 Introduction......Page 230
11.2 Generalized Scatterplot Smoothing......Page 231
11.2.1 Application to Wage–Union Data......Page 232
11.3 Generalized Additive Mixed Models......Page 233
11.4 Degrees-of-Freedom Approximations......Page 235
11.6 Hypothesis Testing......Page 236
11.8 Density Estimation......Page 237
11.9 Bibliographical Notes......Page 238
12.1 Introduction......Page 239
12.2 Binary-by-Continuous Interaction Models......Page 240
12.3.1 Modularity of Spline Models......Page 242
12.3.2 Example: Ragweed Pollen Revisited......Page 243
12.3.3 Discrete-by-Continuous Interactions......Page 245
12.3.4 Interactions in Additive Models......Page 246
12.3.5 Generalized Additive Models with Interactions......Page 247
12.3.6 Pollen Data......Page 248
12.4 Varying Coefficient Models......Page 250
12.5 Continuous-by-Continuous Interactions......Page 251
12.6 Bibliographical Notes......Page 253
13.1 Introduction......Page 254
13.2 Choice of Bivariate Basis Functions......Page 256
13.3 Kriging......Page 258
13.3.1 The Kriging Algorithm......Page 263
13.4 General Radial Smoothing......Page 264
13.4.2 Positive Definitization......Page 267
13.4.4 Low-Rank Radial Smoothers......Page 268
13.4.5 Higher-Dimensional Radial Smoothers......Page 269
13.4.6 Choice of Knots......Page 271
13.5 Default Automatic Bivariate Smoother......Page 272
13.6 Geoadditive Models......Page 274
13.9 Appendix: Equivalence of BLUP using Z and Z......Page 275
13.10 Bibliographical Notes......Page 276
14.1 Introduction......Page 277
14.2 Formulation......Page 279
14.3 Application to the LIDAR Data......Page 280
14.4 Quasilikelihood and Variance Functions......Page 282
14.5 Bibliographical Notes......Page 283
15.1 Introduction......Page 284
15.2 Formulation......Page 285
15.3 The Expectation Maximization (EM) Algorithm......Page 286
15.5 Sensitivity Analysis Example......Page 289
15.6 Bibliographical Notes......Page 291
16.1 Introduction......Page 292
16.2.1 Markov Chain Monte Carlo......Page 293
16.2.2 Credible Sets......Page 294
16.3 Scatterplot Smoothing......Page 295
16.3.1 Application to LIDAR Data......Page 298
16.4 Linear Mixed Models......Page 301
16.5 Generalized Linear Mixed Models......Page 304
16.5.1 Probit Mixed Models......Page 305
16.5.1.1 Union and Wages Data Revisited......Page 306
16.6 Rao–Blackwellization......Page 307
16.7 Bibliographical Notes......Page 308
17.1 Introduction......Page 309
17.2 A Local Penalty Method......Page 310
17.3 Completely Automatic Algorithm......Page 311
17.4 Bayesian Inference......Page 312
17.5.1 Effects of the Tuning Parameters......Page 314
17.5.2 The Automatic Algorithms......Page 315
17.5.3 Bayesian Inference......Page 318
17.6 LIDAR Example......Page 320
17.7.2 Simulations of an Additive Model......Page 321
17.8 Bibliographical Notes......Page 323
18.3 Salinity and Fishing in North Carolina......Page 324
18.4 Management of a Retirement Fund......Page 329
18.5 Biomonitoring of Airborne Mercury......Page 330
18.6 Term Structure of Interest Rates......Page 331
18.7 Air Pollution and Mortality in Milan: The Harvesting Effect......Page 335
19.2 Minimalist Statistics......Page 336
19.3.3 Nonquadratic Penalties......Page 337
19.3.5 Missing Data......Page 338
19.3.9 Diagnostics......Page 339
19.3.11 Constrained Smoothing......Page 340
19.4 Future Research......Page 341
A.2.2 Eigenvalues and Eigenvectors......Page 342
A.2.5 Elementwise Function Notation......Page 343
A.2.7 Triangular Matrices......Page 344
A.2.11 Matrix Square Root......Page 345
A.2.12 Derivative Vector and Hessian Matrix......Page 346
A.3.1 Vectors and Vector Spaces......Page 347
A.3.4 Bases......Page 348
A.4.2 Covariance Matrix of a Random Vector......Page 349
A.4.6 Multivariate Normal Distribution......Page 350
A.6 Bibliographical Notes......Page 351
B.1.1 Demmler–Reinsch Orthogonalization......Page 352
B.1.1.1 Justification of Algorithm A.1......Page 354
B.1.1.2 S-PLUS Implementation of Algorithm A.1......Page 355
B.1.1.3 MATLAB Implementation of Algorithm A.1......Page 356
B.1.2 QR Decomposition......Page 366
B.2 Computation of Covariance Matrix Estimators......Page 367
B.3.1.1 S-PLUS Functions......Page 369
B.3.2 S-PLUS Mixed Model Functions......Page 370
B.3.3 SAS Mixed Model Procedures......Page 372
B.3.4.1 Logistic Semiparametric Models......Page 373
B.3.4.5 Other Models......Page 376
Bibliography......Page 377
Author Index......Page 391
Notation Index......Page 396
Example Index......Page 397
Subject Index......Page 398