Computational Statistics

Contents......Page 18
Preface......Page 8
Part I: Preliminaries......Page 23
 Introduction to Part I......Page 25
 Mathematical and Statistical Preliminaries......Page 27
  1.1 Discovering Structure in Data......Page 28
  1.2 Mathematical Tools for Identifying Structure in Data......Page 32
  1.3 Data-Generating Processes; Probability Distributions......Page 51
  1.4 Statistical Inference......Page 59
  1.5 Probability Statements in Statistical Inference......Page 74
  1.6 Modeling and Computational Inference......Page 78
  1.7 The Role of the Empirical Cumulative Distribution Function......Page 81
  1.8 The Role of Optimization in Inference......Page 87
  Notes and Further Reading......Page 96
  Exercises......Page 97
Part II: Statistical Computing......Page 103
 Introduction to Part II......Page 105
 Computer Storage and Arithmetic......Page 107
  2.1 The Fixed-Point Number System......Page 108
  2.2 The Floating-Point Number System......Page 110
  2.3 Errors......Page 119
  Notes and Further Reading......Page 123
  Exercises......Page 124
 Algorithms and Programming......Page 129
  3.1 Error in Numerical Computations......Page 131
  3.2 Algorithms and Data......Page 135
  3.3 Efficiency......Page 138
  3.4 Iterations and Convergence......Page 150
  3.5 Programming......Page 156
  3.6 Computational Feasibility......Page 159
  Notes and Further Reading......Page 160
  Exercises......Page 164
 Approximation of Functions and Numerical Quadrature......Page 169
  4.1 Function Approximation and Smoothing......Page 175
  4.2 Basis Sets in Function Spaces......Page 182
  4.3 Orthogonal Polynomials......Page 189
  4.4 Splines......Page 200
  4.5 Kernel Methods......Page 204
  4.6 Numerical Quadrature......Page 206
  4.7 Monte Carlo Methods for Quadrature......Page 214
  Notes and Further Reading......Page 219
  Exercises......Page 221
 Numerical Linear Algebra......Page 225
  5.1 General Computational Considerations for Vectors and Matrices......Page 227
  5.2 Gaussian Elimination and Elementary Operator Matrices......Page 231
  5.3 Matrix Decompositions......Page 237
  5.4 Iterative Methods......Page 243
  5.5 Updating a Solution to a Consistent System......Page 249
  5.6 Overdetermined Systems; Least Squares......Page 250
  5.7 Other Computations with Matrices......Page 257
  Notes and Further Reading......Page 258
  Exercises......Page 259
 Solution of Nonlinear Equations and Optimization......Page 263
  6.1 Finding Roots of Equations......Page 266
  6.2 Unconstrained Descent Methods in Dense Domains......Page 283
  6.3 Unconstrained Combinatorial and Stochastic Optimization......Page 297
  6.4 Optimization under Constraints......Page 306
  6.5 Computations for Least Squares......Page 313
  6.6 Computations for Maximum Likelihood......Page 316
  Notes and Further Reading......Page 320
  Exercises......Page 323
  7.1 Randomness of Pseudorandom Numbers......Page 327
  7.2 Generation of Nonuniform Random Numbers......Page 329
  7.3 Acceptance/Rejection Method Using a Markov Chain......Page 335
  7.4 Generation of Multivariate Random Variates......Page 337
  7.5 Data-Based Random Number Generation......Page 340
  7.6 Software for Random Number Generation......Page 342
  Exercises......Page 351
Part III: Methods of Computational Statistics......Page 355
 Introduction to Part III......Page 357
 Graphical Methods in Computational Statistics......Page 359
  8.1 Smoothing and Drawing Lines......Page 363
  8.2 Viewing One, Two, or Three Variables......Page 366
  8.3 Viewing Multivariate Data......Page 377
  Notes and Further Reading......Page 387
  Exercises......Page 390
 Tools for Identification of Structure in Data......Page 393
  9.1 Transformations......Page 395
  9.2 Measures of Similarity and Dissimilarity......Page 405
  Exercises......Page 419
 Estimation of Functions......Page 423
  10.1 General Approaches to Function Estimation......Page 425
  10.2 Pointwise Properties of Function Estimators......Page 429
  10.3 Global Properties of Estimators of Functions......Page 432
  Exercises......Page 436
 Monte Carlo Methods for Statistical Inference......Page 439
  11.1 Monte Carlo Estimation......Page 440
  11.2 Simulation of Data from a Hypothesized Model: Monte Carlo Tests......Page 444
  11.4 Random Sampling from Data......Page 446
  11.5 Reducing Variance in Monte Carlo Methods......Page 447
  11.6 Software for Monte Carlo......Page 451
  Notes and Further Reading......Page 452
  Exercises......Page 453
 Data Randomization, Partitioning, and Augmentation......Page 457
  12.1 Randomization Methods......Page 458
  12.2 Cross Validation for Smoothing and Fitting......Page 462
  12.3 Jackknife Methods......Page 464
  Notes and Further Reading......Page 470
  Exercises......Page 471
 Bootstrap Methods......Page 475
  13.1 Bootstrap Bias Corrections......Page 476
  13.2 Bootstrap Estimation of Variance......Page 478
  13.3 Bootstrap Confidence Intervals......Page 479
  13.4 Bootstrapping Data with Dependencies......Page 483
  13.5 Variance Reduction in Monte Carlo Bootstrap......Page 484
  Notes and Further Reading......Page 486
  Exercises......Page 487
Part IV: Exploring Data Density and Relationships......Page 491
 Introduction to Part IV......Page 493
 Estimation of Probability Density Functions Using Parametric Models......Page 497
  14.1 Fitting a Parametric Probability Distribution......Page 498
  14.2 General Families of Probability Distributions......Page 499
  14.3 Mixtures of Parametric Families......Page 502
  14.4 Statistical Properties of Density Estimators Based on Parametric Families......Page 504
  Notes and Further Reading......Page 505
  Exercises......Page 506
  15.1 The Likelihood Function......Page 509
  15.2 Histogram Estimators......Page 512
  15.3 Kernel Estimators......Page 521
  15.4 Choice of Window Widths......Page 526
  15.5 Orthogonal Series Estimators......Page 527
  15.6 Other Methods of Density Estimation......Page 528
  Notes and Further Reading......Page 531
  Exercises......Page 532
 Statistical Learning and Data Mining......Page 537
  16.1 Clustering and Classification......Page 541
  16.2 Ordering and Ranking Multivariate Data......Page 560
  16.3 Linear Principal Components......Page 570
  16.4 Variants of Principal Components......Page 582
  16.5 Projection Pursuit......Page 586
  16.6 Other Methods for Identifying Structure......Page 594
  16.7 Higher Dimensions......Page 595
  Notes and Further Reading......Page 600
  Exercises......Page 602
 Statistical Models of Dependencies......Page 607
  17.1 Regression and Classification Models......Page 610
  17.2 Probability Distributions in Models......Page 619
  17.3 Fitting Models to Data......Page 622
  17.4 Classification......Page 642
  17.5 Transformations......Page 650
  Notes and Further Reading......Page 656
  Exercises......Page 658
Appendices......Page 663
 Monte Carlo Studies in Statistics......Page 665
  A.1 Simulation as an Experiment......Page 666
  A.2 Reporting Simulation Experiments......Page 667
  A.3 An Example......Page 668
  A.4 Computer Experiments......Page 675
  Exercises......Page 677
 Some Important Probability Distributions......Page 679
  C.1 General Notation......Page 685
  C.2 Computer Number Systems......Page 687
  C.3 Notation Relating to Random Variables......Page 688
  C.4 General Mathematical Functions and Operators......Page 690
  C.5 Models and Data......Page 697
 Solutions and Hints for Selected Exercises......Page 699
 Bibliography......Page 711
  E.1 Literature in Computational Statistics......Page 712
  E.3 References to the Literature......Page 715
 B......Page 737
 C......Page 738
 D......Page 739
 F......Page 740
 H......Page 741
 J......Page 742
 L......Page 743
 N......Page 744
 P......Page 745
 R......Page 746
 S......Page 747
 T......Page 748
 Z......Page 749