Regression for Categorical Data

Contents......Page 3
Preface......Page 7
Categorical Data: Examples and Basic Concepts......Page 9
Organization of This Book......Page 13
Basic Components of Structured Regression......Page 14
Classical Linear Regression......Page 23
Exercises......Page 35
Distribution Models for Binary Responses and Basic Concepts......Page 37
Linking Response and Explanatory Variables......Page 41
The Logit Model......Page 45
The Origins of the Logistic Function and the Logit Model......Page 56
Exercises......Page 57
Basic Structure......Page 59
Generalized Linear Models for Continuous Responses......Page 61
GLMs for Discrete Responses......Page 64
Further Concepts......Page 68
Modeling of Grouped Data......Page 70
Maximum Likelihood Estimation......Page 71
Inference......Page 75
Goodness-of-Fit for Grouped Observations......Page 80
Computation of Maximum Likelihood Estimates......Page 83
Hat Matrix for Generalized Linear Models......Page 84
Quasi-Likelihood Modeling......Page 86
Exercises......Page 87
4 Modeling of Binary Data......Page 89
Maximum Likelihood Estimation......Page 90
Discrepancy between Data and Fit......Page 95
Diagnostic Checks......Page 101
Structuring the Linear Predictor......Page 109
Comparing Non-Nested Models......Page 121
Explanatory Value of Covariates......Page 122
Further Reading......Page 127
Exercises......Page 128
Alternative Links in Binary Regression......Page 130
The Missing Link......Page 137
Overdispersion......Page 139
Conditional Likelihood......Page 145
Exercises......Page 147
6 Regularization & Variable Selection for Parametric Models......Page 149
Classical Subset Selection......Page 150
Regularization by Penalization......Page 151
Boosting Methods......Page 169
Simultaneous Selection of Link Function and Predictors......Page 176
Categorical Predictors......Page 179
Bayesian Approach......Page 184
Exercises......Page 185
7 Regression Analysis of Count Data......Page 187
The Poisson Distribution......Page 188
Poisson Regression Model......Page 191
Inference for the Poisson Regression Model......Page 192
Poisson Regression with an Offset......Page 196
Poisson Regression with Overdispersion......Page 198
Negative Binomial Model and Alternatives......Page 200
Zero-Inflated Counts......Page 204
Hurdle Models......Page 206
Further Reading......Page 209
Exercises......Page 210
8 Multinomial Response Models......Page 212
The Multinomial Distribution......Page 214
The Multinomial Logit Model......Page 215
Structuring the Predictor......Page 220
Logit Model as Multivariate Generalized Linear Model......Page 222
Inference for Multicategorical Response Models......Page 223
Multinomial Models with Hierarchically Structured Response......Page 228
Discrete Choice Models......Page 231
Nested Logit Model......Page 236
Regularization for the Multinomial Model......Page 238
Further Reading......Page 243
Exercises......Page 244
9 Ordinal Response Models......Page 246
Cumulative Models......Page 248
Sequential Models......Page 257
Further Properties and Comparison of Models......Page 260
Alternative Models......Page 262
Inference for Ordinal Models......Page 266
Exercises......Page 270
Univariate Generalized Non-Parametric Regression......Page 273
Non-Parametric Regression with Multiple Covariates......Page 289
Structured Additive Regression......Page 293
Functional Data and Signal Regression......Page 311
Further Reading......Page 317
Exercises......Page 318
Regression and Classification Trees......Page 320
Multivariate Adaptive Regression Splines......Page 331
Exercises......Page 332
12 Analysis of Contingency Tables - Log-Linear & Graphical Models......Page 334
Types of Contingency Tables......Page 335
Log-Linear Models for Two-Way Tables......Page 338
Log-Linear Models for Three-Way Tables......Page 341
Specific Log-Linear Models......Page 344
Log-Linear and Graphical Models for Higher Dimensions......Page 348
Collapsibility......Page 351
Log-Linear Models and the Logit Model......Page 352
Inference for Log-Linear Models......Page 353
Model Selection and Regularization......Page 357
Mosaic Plots......Page 360
Further Reading......Page 361
Exercises......Page 362
13 Multivariate Response Models......Page 365
Conditional Modeling......Page 367
Marginal Parametrization and Generalized Log-Linear Models......Page 372
General Marginal Models: Association as Nuisance and GEEs......Page 373
Marginal Homogeneity......Page 387
Further Reading......Page 394
Exercises......Page 395
14 Random Effects Models & Finite Mixtures......Page 397
Linear Random Effects Models for Gaussian Data......Page 398
Generalized Linear Mixed Models......Page 404
Estimation Methods for Generalized Mixed Models......Page 409
Multicategorical Response Models......Page 418
The Marginalized Random Effects Model......Page 421
Semiparametric Mixed Models......Page 422
Finite Mixture Models......Page 424
Further Reading......Page 428
Exercises......Page 429
15 Prediction & Classification......Page 430
Basic Concepts of Prediction......Page 431
Methods for Optimal Classification......Page 439
Basics of Estimated Classification Rules......Page 446
Parametric Classification Methods......Page 452
Non-Parametric Methods......Page 458
Neural Networks......Page 469
Examples......Page 472
Variable Selection in Classification......Page 474
Prediction of Ordinal Outcomes......Page 475
Model-Based Prediction......Page 481
Further Reading......Page 482
Exercises......Page 483
A.1 Discrete Distributions......Page 486
A.2 Continuous Distributions......Page 488
B.1 Linear Algebra......Page 491
B.2 Taylor Approximation......Page 492
B.3 Conditional Expectation, Distribution......Page 494
B.4 EM Algorithm......Page 495
C.1 Simplification of Penalties......Page 497
C.2 Linear Constraints......Page 499
C.3 Fisher Scoring with Penalty Term......Page 500
D.1 Kullback-Leibler Distance......Page 501
E.1 Laplace Approximation......Page 505
E.2 Gauss-Hermite Integration......Page 506
E.3 Inversion of Pseudo-Fisher Matrix......Page 508
Examples......Page 509
Biblo......Page 512
Index......Page 544