A Companion for Mathematical Statistics

Preface......Page 3
Probability and Distribution Theory......Page 19
Some Important Probability Definitions and Facts......Page 20
Probability and Probability Distributions......Page 21
Definitions and Properties of Expected Values......Page 33
Relations among Random Variables......Page 40
Entropy......Page 45
Generating Functions......Page 47
Functionals of the CDF; Distribution ``Measures\'\'......Page 52
Transformations of Random Variables......Page 56
Decomposition of Random Variables......Page 60
Order Statistics......Page 62
Sequences of Spaces, Events, and Random Variables......Page 63
The Borel-Cantelli Lemmas......Page 64
Exchangeability and Independence of Sequences......Page 66
Types of Convergence......Page 68
Expectations of Sequences; Sequences of Expectations......Page 77
Convergence of Functions......Page 78
Asymptotic Distributions......Page 79
Limit Theorems......Page 83
Laws of Large Numbers......Page 84
Central Limit Theorems......Page 86
Other Limiting Distributions......Page 89
Power Series Expansions......Page 90
Hermite Polynomials......Page 91
Expansion of the Characteristic Functions......Page 92
Cumulants and Expected Values......Page 93
The Edgeworth Expansion......Page 94
Conditional Expectation: Definition and Properties......Page 95
Projections......Page 99
Conditional Probability Distributions......Page 100
Stochastic Processes......Page 101
Probability Models for Stochastic Processes......Page 105
Markov Chains......Page 107
Martingales......Page 109
Families of Probability Distributions......Page 112
Characterizing a Family of Distributions......Page 115
Shapes of the Probability Density......Page 118
``Regular\'\' Families......Page 120
The Exponential Class of Families......Page 121
Parametric-Support Families......Page 125
Transformation Group Families......Page 126
Complete Families......Page 127
Elliptical Families......Page 128
The Family of Normal Distributions......Page 129
Functions of Normal Random Variables......Page 131
Notes and Further Reading......Page 135
Exercises......Page 143
Basic Statistical Concepts......Page 151
Inferential Information in Statistics......Page 154
Statistical Inference......Page 158
Sufficiency, Ancillarity, Minimality, and Completeness......Page 162
Information and the Information Inequality......Page 168
Prediction......Page 172
Other Issues in Statistical Inference......Page 173
Statistical Inference: Approaches and Methods......Page 174
Likelihood......Page 176
The Empirical Cumulative Distribution Function......Page 180
Fitting Expected Values......Page 186
Fitting Probability Distributions......Page 187
Summary and Preview......Page 188
Efficiency of Estimators......Page 190
Estimating Equations......Page 191
Decisions, Losses, Risks, and Optimal Actions......Page 193
Approaches to Minimizing the Risk......Page 198
Admissibility......Page 201
Minimaxity......Page 203
Probability Statements in Statistical Inference......Page 204
Tests of Hypotheses......Page 205
Confidence Sets......Page 210
Asymptotic Inference......Page 214
Consistency......Page 215
Asymptotic Expectation......Page 217
Asymptotic Properties and Limiting Properties......Page 218
Jackknife......Page 223
Bootstrap......Page 225
Methods of Estimating Variance-Covariance Matrices......Page 226
Notes and Further Reading......Page 227
Exercises......Page 232
The Bayesian Paradigm......Page 233
Bayesian Estimation......Page 242
Properties of Bayes Estimators......Page 243
Examples......Page 244
Markov Chain Monte Carlo......Page 247
Bayesian Testing......Page 259
The Bayes Factor......Page 263
Bayesian Tests of a Simple Hypothesis......Page 266
Probability Statements in Statistical Inference......Page 267
Lindley\'s Paradox......Page 268
Credible Regions......Page 269
Decision-Theoretic Approach......Page 270
Other Optimality Considerations......Page 271
Notes and Further Reading......Page 273
Exercises......Page 275
Uniformly Minimum Variance Unbiased Point Estimation......Page 277
Unbiased Point Estimators......Page 278
Unbiasedness and Squared-Error Loss; UMVUE......Page 279
Other Properties of UMVUEs......Page 283
Lower Bounds on the Variance of Unbiased Estimators......Page 285
Expectation Functionals and U-Statistics......Page 286
Properties of U-Statistics......Page 291
Method of Moments Estimators......Page 292
V-Statistics......Page 293
Asymptotic Efficiency......Page 294
Asymptotically Efficient Estimators......Page 295
Estimation in Linear Models......Page 298
Estimation in Survey Samples of Finite Populations......Page 310
Notes and Further Reading......Page 313
Exercises......Page 314
The Likelihood Function and Its Use in Statistical Inference......Page 315
Definition and Examples......Page 319
Properties of MLEs......Page 328
The Score Function and the Likelihood Equations......Page 333
Finding an MLE......Page 335
Asymptotic Distributions of MLEs and RLEs......Page 351
Asymptotic Efficiency of MLEs and RLEs......Page 352
Inconsistent MLEs......Page 354
Properties of GEE Estimators......Page 356
Application: MLEs in Generalized Linear Models......Page 357
Generalized Linear Models......Page 358
Fitting Generalized Linear Models......Page 360
Generalized Additive Models......Page 361
Quasi-likelihood Methods......Page 364
Nonparametric and Semiparametric Models......Page 365
Notes and Further Reading......Page 367
Exercises......Page 370
Testing Statistical Hypotheses......Page 371
Statistical Hypotheses......Page 372
Optimal Tests......Page 378
The Neyman-Pearson Fundamental Lemma......Page 381
Uniformly Most Powerful Tests......Page 385
Unbiasedness of Tests......Page 386
UMP Unbiased (UMPU) Tests......Page 387
Likelihood Ratio Tests......Page 388
Wald Tests and Score Tests......Page 390
Multiple Tests......Page 395
Notes and Further Reading......Page 397
Exercises......Page 398
Confidence Sets......Page 401
Introduction: Construction and Properties......Page 402
Optimal Confidence Sets......Page 406
Asymptotic Confidence Sets......Page 410
Bootstrap Confidence Sets......Page 411
Notes and Further Reading......Page 417
Transformations......Page 419
Transformation Groups......Page 420
Invariant and Equivariant Statistical Procedures......Page 421
Equivariant Point Estimation......Page 425
Invariant Tests......Page 429
Equivariant Confidence Sets......Page 430
Notes and Further Reading......Page 431
Nonparametric Inference......Page 433
Perturbations of Probability Distributions......Page 435
Robust Inference......Page 440
Sensitivity of Statistical Functions......Page 442
Sensitivity of Estimators......Page 445
Notes and Further Reading......Page 447
Nonparametric Estimation of Functions......Page 449
General Methods for Estimating Functions......Page 451
Pointwise Properties of Function Estimators......Page 454
Global Properties of Estimators of Functions......Page 456
Nonparametric Probability Density Estimation......Page 461
Histogram Estimators......Page 464
Kernel Estimators......Page 472
Choice of Window Widths......Page 477
Orthogonal Series Estimators......Page 478
Exercises......Page 480
Statistical Mathematics......Page 481
Sets......Page 484
Sets and Spaces......Page 487
Binary Operations and Algebraic Structures......Page 495
Linear Spaces......Page 497
The Real Number System......Page 502
Some Useful Mathematical Tools and Operations......Page 515
Basic Concepts of Measure Theory......Page 526
Functions and Images......Page 535
Measure......Page 538
Sets in IR and IRd......Page 546
Real-Valued Functions over Real Domains......Page 553
Integration......Page 558
Derivatives......Page 568
Real Function Spaces......Page 569
Continuous Time Stochastic Processes......Page 582
Integration with Respect to Stochastic Differentials......Page 589
Inner Products, Norms, and Metrics......Page 597
Matrices and Vectors......Page 598
Vector/Matrix Derivatives and Integrals......Page 609
Optimization of Functions......Page 620
Vector Random Variables......Page 625
Transition Matrices......Page 626
Overview of Optimization......Page 630
Alternating Conditional Optimization......Page 635
Notes and Further Reading......Page 637
Exercises......Page 641
Important Probability Distributions......Page 647
Preliminaries......Page 657
Pr(X Ai) and Pr(X Ai) or Pr(X Ai)......Page 658
Pr(X A) and E(f(X))......Page 659
E(f(X)) and f(E(X))......Page 661
E(f(X,Y)) and E(g(X)) and E(h(Y))......Page 664
Multivariate Extensions......Page 667
Notes and Further Reading......Page 668
General Notation......Page 669
General Mathematical Functions and Operators......Page 671
Sets, Measure, and Probability......Page 675
Linear Spaces and Matrices......Page 676
Bibliography......Page 681
Index......Page 683