Numerical Methods of Statistics

Cover......Page 1
Half-title......Page 3
Series-title......Page 5
Title......Page 7
Copyright......Page 8
Contents......Page 9
Preface to the Second Edition......Page 15
Preface to the First Edition......Page 17
1.1 Introduction......Page 19
1.2 Computers......Page 21
1.3 Software and Computer Languages......Page 23
1.4 Data Structures......Page 26
1.5 Programming Practice......Page 27
1.6 Some Comments on R......Page 28
References......Page 30
2.1 Introduction......Page 31
2.2 Positional Number Systems......Page 32
2.3 Fixed Point Arithmetic......Page 35
2.4 Floating Point Representations......Page 38
2.5 Living with Floating Point Inaccuracies......Page 41
2.6 The Pale and Beyond......Page 46
2.7 Conditioned Problems and Stable Algorithms......Page 50
Programs and Demonstrations......Page 52
Exercises......Page 53
References......Page 56
3.1 Introduction......Page 58
3.2 Matrix Operations......Page 59
3.3 Solving Triangular Systems......Page 61
3.4 Gaussian Elimination......Page 62
3.5 Cholesky Decomposition......Page 68
3.6 Matrix Norms......Page 72
3.7 Accuracy and Conditioning......Page 73
3.8 Matrix Computations in R......Page 78
Programs and Demonstrations......Page 79
Exercises......Page 81
References......Page 83
4.2 Full Elimination with Complete Pivoting......Page 85
4.3 Banded Matrices......Page 89
4.4 Applications to ARMA Time-Series Models......Page 91
4.5 Toeplitz Systems......Page 94
4.6 Sparse Matric......Page 98
4.7 Iterative Methods......Page 100
4.8 Linear Programming......Page 102
Programs and Demonstrations......Page 105
Exercises......Page 106
References......Page 108
5.1 Introduction......Page 109
5.2 Condition of the Regression Problem......Page 111
5.3 Solving the Normal Equations......Page 114
5.4 Gram–Schmidt Orthogonalization......Page 115
5.5 Householder Transformations......Page 118
5.6 Householder Transformations for Least Squares......Page 119
5.7 Givens Transformations......Page 122
5.8 Givens Transformations for Least Squares......Page 123
5.9 Regression Diagnostics......Page 125
5.10 Hypothesis Tests......Page 128
5.11 Conjugate Gradient Methods......Page 130
5.12 Doolittle, the Sweep, and All Possible Regressions......Page 133
5.13 Alternatives to Least Squares......Page 136
5.14 Comments......Page 138
Exercises......Page 140
References......Page 143
6.2 Theory......Page 146
6.3 Power Methods......Page 148
6.4 The Symmetric Eigenproblem and Tridiagonalization......Page 151
6.5 The QR Algorithm......Page 153
6.6 Singular Value Decomposition......Page 155
(B) Principal Components......Page 158
(C) Moore–Penrose Pseudoinverse......Page 159
(E) Canonical Correlation......Page 160
(F) Procrustes Rotation......Page 161
6.8 Complex Singular Value Decomposition......Page 162
Programs and Demonstrations......Page 164
Exercises......Page 165
References......Page 168
7.1 Introduction......Page 169
7.2 Interpolation......Page 171
7.3 Interpolating Splines......Page 174
7.4 Curve Fitting with Splines: Smoothing and Regression......Page 177
7.5 Mathematical Approximation......Page 181
7.6 Practical Approximation Techniques......Page 186
7.7 Computing Probability Functions......Page 188
(A) Normal Distribution......Page 189
(C) Student’s t Distribution......Page 190
(D) Chi-Square, Poisson, and Incomplete Gamma......Page 192
(E) F and Beta Distributions......Page 193
(F) Inverse Normal......Page 194
Programs and Demonstrations......Page 195
Exercises......Page 197
References......Page 201
8.1 Introduction......Page 204
8.2 Safe Univariate Methods: Lattice Search, Golden Section, and Bisection......Page 206
8.3 Root Finding......Page 209
8.4 First Digression: Stopping and Condition......Page 215
8.5 Multivariate Newton’s Methods......Page 217
8.6 Second Digression: Numerical Differentiation......Page 218
8.7 Minimization and Nonlinear Equations......Page 221
8.8 Condition and Scaling......Page 226
8.9 Implementation......Page 228
8.10 A Non-Newton Method: Nelder-Mead......Page 229
Programs and Demonstrations......Page 231
Exercises......Page 232
References......Page 235
9.1 Introduction......Page 237
9.2 Notation and Asymptotic Theory of Maximum Likelihood......Page 238
9.3 Information, Scoring, and Variance Estimates......Page 244
9.4 An Extended Example......Page 246
9.5 Concentration, Iteration, and the EM Algorithm......Page 248
9.6 Multiple Regression in the Context of Maximum Likelihood......Page 254
9.7 Generalized Linear Models......Page 255
9.8 Nonlinear Regression......Page 260
9.9 Parameterizations and Constraints......Page 264
Programs and Demonstrations......Page 269
Exercises......Page 270
References......Page 273
10.1 Introduction......Page 275
(A) Simulation Experiments in Statistics......Page 276
(B) Hypothesis Tests......Page 278
(C) Bayesian Analysis......Page 279
10.3 One-Dimensional Quadrature......Page 282
10.4 Numerical Integration in Two or More Variables......Page 289
(A) Integration over a Triangle......Page 290
(B) Integration on Surface of a Sphere......Page 294
(C) The Curse and Monte Carlo Integration......Page 295
(A) Testing Random Number Generators......Page 296
(B) Linear Congruential Generators......Page 298
(C) Shift Register Methods......Page 300
(D) Recommendations......Page 302
10.6 Quasi–Monte Carlo Integration......Page 304
10.7 Strategy and Tactics......Page 309
Programs and Demonstrations......Page 313
Exercises......Page 315
References......Page 317
11.1 Introduction......Page 321
(A) Transformations......Page 322
(B) Acceptance/Rejection......Page 323
(C) Ratio of Uniforms......Page 325
11.3 Algorithms for Continuous Distributions......Page 326
(A) Normal Distribution......Page 327
(B) Exponential Distribution......Page 330
(C) Student’s t and Cauchy......Page 331
(D) Gamma, Chi-Square, and Chi......Page 334
(E) Logistic and Laplace......Page 335
(F) Beta, F, and Dirichlet......Page 336
(G) Noncentral Chi-Square, F, and t......Page 337
(I) Multivariate Normal and t; Wishart......Page 338
11.4 General Methods for Discrete Distributions......Page 339
(A) Discrete Inversion......Page 340
(C) Ratio of Uniforms......Page 341
(D) Walker’s Alias Method......Page 342
11.5 Algorithms for Discrete Distributions......Page 343
(B) Binomial......Page 344
(C) Poisson......Page 345
(D) Hypergeometric......Page 347
(A) Random Permutations......Page 348
(B) Random Sampling......Page 349
(C) Random Contingency Tables......Page 351
11.7 Accuracy in Random Number Generation......Page 352
Programs and Demonstrations......Page 355
Exercises......Page 356
References......Page 359
12.2 Distribution and Density Estimation......Page 361
(A) Pearson’s Chi-Square......Page 368
(B) Kolmogorov–Smirnov......Page 369
(C) Anderson–Darling......Page 370
12.4 Importance Sampling and Weighted Observations......Page 371
12.5 Testing Importance Sampling Weights......Page 377
12.6 Laplace Approximations......Page 379
12.7 Randomized Quadrature......Page 381
12.8 Spherical–Radial Methods......Page 383
Programs and Demonstrations......Page 388
Exercises......Page 390
References......Page 391
13.1 Introduction......Page 393
13.2 Markov Chains......Page 395
13.3 Gibbs Sampling......Page 396
13.4 Metropolis–Hastings Algorithm......Page 401
13.5 Time-Series Analysis......Page 404
13.6 Adaptive Acceptance/Rejection......Page 408
(A) Plot the Data......Page 412
(B) Gelman and Rubin......Page 413
(C) Geweke......Page 414
(F) Dickey–Fuller......Page 415
Exercises......Page 416
References......Page 418
14.2 Divide and Conquer......Page 421
14.3 Sorting Algorithms......Page 423
14.4 Fast Order Statistics and Related Problems......Page 426
14.5 Fast Fourier Transform......Page 427
14.6 Convolutions and the Chirp-z Transform......Page 431
14.7 Statistical Applications of the FFT......Page 433
(A) Time Series......Page 434
(B) Characteristic Functions of Discrete Random Variables......Page 435
(C) Convolutions of Continuous Random Variables by Discretization......Page 438
(D) Inverting the Characteristic Function of Continuous Random Variables......Page 440
(E) Weighted Sums of Chi-Square Random Variables......Page 442
14.8 Combinatorial Problems......Page 443
(B) Subsets of Size K from N......Page 445
(C) All Permutations......Page 446
Programs and Demonstrations......Page 447
Exercises......Page 451
References......Page 454
Author Index......Page 457
Subject Index......Page 462