Probability and Statistics

PROBABILITY AND STATISTICS, 2ND ED.......Page 1
Title Page......Page 2
Copyright Page......Page 3
Preface......Page 4
Contents......Page 8
1.1 The History of Probability......Page 14
The Frequency Interpretation of Probability......Page 15
The Classical Interpretation of Probability......Page 16
The Subjective Interpretation of Probability......Page 17
Types of Experiments......Page 18
The Mathematical Theory of Probability......Page 19
Relations of Set Theory......Page 20
Operations of Set Theory......Page 21
Exercises......Page 25
Axioms and Basic Theorems......Page 26
Further Properties of Probability......Page 28
Exercises......Page 29
Simple Sample Spaces......Page 31
1.7 Counting Methods......Page 33
Multiplication Rule......Page 34
Permutations......Page 35
The Birthday Problem......Page 37
Exercises......Page 38
Combinations......Page 39
Binomial Coefficients......Page 40
The Tennis Tournament......Page 42
Exercises......Page 43
1.9 Multinomial Coefficients......Page 45
The Union of Three Events......Page 49
The Union of a Finite Number of Events......Page 52
The Matching Problem......Page 53
Exercises......Page 55
1.11 Independent Events......Page 56
Independence of Two Events......Page 57
Independence of Several Events......Page 59
The Collector\'s Problem......Page 62
Exercises......Page 63
Perfect Forecasts......Page 65
Guaranteed Winners......Page 66
1.13 Supplementary Exercises......Page 67
2.1 The Definition of Conditional Probability......Page 70
Conditional Probability for Independent Events......Page 72
The Multiplication Rule for Conditional Probabllilles......Page 73
The Game of Craps......Page 74
Exercises......Page 76
Probability and Partitions......Page 77
Statement and Proof of Bayes\' Theorem......Page 79
Computation of Posterior Probabilities In More Than One Stage......Page 81
Exercises......Page 83
Stochastic Processes......Page 85
Markov Chains......Page 86
The Transition Matrix......Page 88
The Initial Probability Vector......Page 91
Exercises......Page 92
Statement of the Problem......Page 95
Solution of the Problem......Page 96
Exercises......Page 99
Optimal Selection......Page 100
The Form of the Best Procedure......Page 101
The Best Procedure......Page 102
The Simple but Interesting Limiting Value......Page 105
Parlor Games......Page 106
2.6 Supplementary Exercises......Page 107
Definition of a Random Variable......Page 110
The Distribution of a Random Variable......Page 111
Discrete Distributions......Page 112
The Binomial Distribution......Page 113
Exercises......Page 114
Nonuniqueness of the p.d.f.......Page 115
The Uniform Distribution on an Interval......Page 116
Mixed Distributions......Page 119
Exercises......Page 120
Definition and Basic Properties......Page 121
Determining Probabilities from the Distribution Function......Page 124
The d.f. of a Continuous Distribution......Page 125
Exercises......Page 126
Discrete Joint Distributions......Page 128
Continuous Joint Distributions......Page 130
Bivariate Distribution Functions......Page 133
Exercises......Page 136
Deriving a Marginal p.f. or a Marginal p.d.f.......Page 138
Independent Random Variables......Page 140
Exercises......Page 145
Discrete Conditional Distributions......Page 147
Continuous Conditional Distributions......Page 148
Construction of the Joint Distribution......Page 150
Exercises......Page 152
Joint Distributions......Page 155
Marginal Distributions......Page 156
Conditional Distributions......Page 159
Exercises......Page 162
Variable with a Continuous Distribution......Page 163
Direct Derivation of the Probability Density Function......Page 165
The Probability Integral Transformation......Page 167
Tables of Random Digits......Page 168
Exercises......Page 170
Variables with a Discrete Joint Distribution......Page 171
Variables with a Continuous Joint Distribution......Page 172
Transformation of a Multivariate Probability Density Function......Page 174
Linear Transformations......Page 178
The Sum of Two Random Variables......Page 179
The Range......Page 181
Exercises......Page 182
Conditioning on a Particular Value......Page 184
Conditioning on the Equality of Two Random Variables......Page 186
3.11 Supplementary Exercises......Page 187
Expectation for a Discrete Distribution......Page 192
Interpretation of the Expectation......Page 193
The Expectation of a Function......Page 196
Exercises......Page 199
Basic Theorems......Page 200
The Mean of a Binomial Distribution......Page 202
Expected Number of Matches......Page 203
Expectation of a Product......Page 204
Expectation for Nonnegative Discrete Distributions......Page 205
Exercises......Page 206
Definitions of the Variance and the Standard Deviation......Page 207
Properties of the Variance......Page 208
The Variance of the Binomial Distribution......Page 211
Existence of Moments......Page 212
Moment Generating Functions......Page 214
Properties of Moment Generating Functions......Page 215
Exercises......Page 218
The Median......Page 219
Comparison of the Mean and the Median......Page 221
Minimizing the Mean Absolute Error......Page 222
Exercises......Page 224
Correlation......Page 226
Properties of Covariance and Correlation......Page 227
Exercises......Page 230
Definition and Basic Properties......Page 232
Prediction......Page 234
Exercises......Page 237
The Markov and Chebyshev Inequalities......Page 239
Properties of the Sample Mean......Page 240
The Law of Large Numbers......Page 242
Exercises......Page 244
Utility Functions......Page 246
Examples of Utility Functions......Page 248
Selling a Lottery Ticket......Page 249
Exercises......Page 250
4.10 Supplementary Exercises......Page 252
The Bernoulli Distribution......Page 256
Bernoulli Trials......Page 257
The Binomial Distribution......Page 258
Exercises......Page 259
Definition of the Hypergeometrlc Distribution......Page 260
Extending the Definition of Binomial Coefficients......Page 261
The Mean and Variance for a Hypergeometric Distribution......Page 262
Comparison of Sampling Methods......Page 263
Exercises......Page 264
Definition and Properties of the Poisson Distribution......Page 265
The Poisson Process......Page 267
The Poisson Approximation to the Binomial Distribution......Page 269
Exercises......Page 270
Definition of the Negative Binomial Distribution......Page 271
Other Properties of Negative Binomial and Geometric Distributions......Page 273
Exercises......Page 275
Importance of the Normal Distribution......Page 276
Properties of the Normal Distribution......Page 277
The Standard Normal Distribution......Page 280
Comparisons of Normal Distributions......Page 281
Linear Combinations of Normally Distributed Variables......Page 283
Exercises......Page 285
Statement ot the Theorem......Page 287
Convergence in Distribution......Page 292
Exercises......Page 294
Approximating a Histogram......Page 296
Exercises......Page 298
The Gamma Function......Page 299
The Gamma Distribution......Page 301
The Exponential Distribution......Page 302
Life Tests......Page 304
Exercises......Page 305
Definition of the Beta Distribution......Page 307
Moments of the Beta Distribution......Page 308
Exercises......Page 309
Definition of the Multinomial Distribution......Page 310
Relation Between the Multinomial and Binomial Distributions......Page 311
Exercises......Page 312
Definition of the Bivariate Normal Distribution......Page 313
Marginal and Conditional Distributions......Page 315
Linear Combinations......Page 317
Exercises......Page 319
5.13 Supplementary Exercises......Page 320
Parameters......Page 324
Statistical Decision Problems......Page 325
The Prior Distribution......Page 326
The Posterior Distribution......Page 329
The Likelihood Function......Page 330
Sequential Observations......Page 332
Exercises......Page 333
Sampling from a Bernoulli Distribution......Page 334
Sampling from a Poisson Distribution......Page 336
Sampling from a Normal Distribution......Page 337
Exercises......Page 340
Nature of an Estimation Problem......Page 343
Definition of a Bayes Estimator......Page 344
Different Loss Functions......Page 345
The Bayes Estimate for Large Samples......Page 348
Exercises......Page 349
Limitations of Bayes Estimators......Page 351
Definition of a Maximum Likelihood Estimator......Page 352
Examples of Maximum Likelihood Estimators......Page 353
Exercises......Page 359
Invariance......Page 361
Numerical Computation......Page 362
Consistency......Page 363
Sampling Plans......Page 364
The Likelihood Principle......Page 366
Exercises......Page 367
Definition of a Statistic......Page 369
Definition of a Sufficient Statistic......Page 370
The Factorization Criterion......Page 371
Exercises......Page 375
Definition of Jointly Sufficient Statistics......Page 377
Minimal Sufficient Statistics......Page 379
Maximum Likelihood Estimators and Bayes Estimators as Sufficient Statistics......Page 381
Exercises......Page 382
The Mean Squared Error of an Estimator......Page 384
Conditional Expectation When a Sufficient Statistic Is Known......Page 385
Limitation of the Use of Sufficient Statistics......Page 388
Exercises......Page 389
6.10 Supplementary Exercises......Page 390
Statistics and Estimators......Page 394
Exercises......Page 395
Definition of the Distribution......Page 396
Properties of the Distribution......Page 397
Exercises......Page 398
Independence of the Sample Mean and Sample Variance......Page 399
Orthogonal Matrices......Page 400
Proof of the Independence of the Sample Mean and Sample Variance......Page 402
Estimation of the Mean and Variance......Page 404
Exercises......Page 405
Definition of the Distribution......Page 406
Moments of the t Distribution......Page 408
Relation to Random Samples from a Normal Distribution......Page 409
Exercises......Page 410
Confidence Intervals for the Mean of a Normal Distribution......Page 411
Confidence Intervals for an Arbitrary Parameter......Page 412
Shortcoming of Confidence Intervals......Page 413
Exercises......Page 414
The Precision of a Normal Distribution......Page 415
A Conjugate Family of Prior Distributions......Page 416
The Marginal Distribution of the Mean......Page 418
A Numerical Example......Page 420
Exercises......Page 423
Definition of an Unbiased Estimator......Page 424
Unbiased Estimation of the Variance......Page 425
Estimation of the Variance of a Normal Dlstrlbullon......Page 427
Discussion of the Concept of Unbiased Estimation......Page 428
Exercises......Page 430
Definition and Properties of Fisher Information......Page 433
The Information Inequality......Page 437
Efficient Estimators......Page 439
Properties of Maximum Likelihood Estimators for Large Samples......Page 440
The Delta Method......Page 442
Exercises......Page 443
7.9 Supplementary Exercises......Page 446
The Null and Alternative Hypotheses......Page 450
The Power Function......Page 451
Simple and Composite Hypotheses......Page 453
Exercises......Page 454
Two Types of Errors......Page 455
Optimal Tests......Page 456
Choosing a Level of Significance......Page 462
Bayes Test Procedures......Page 464
Exercises......Page 465
Finite Number of Parameter Values and Finite Number of Decisions......Page 469
Bayes Decision Procedures......Page 470
Exercises......Page 475
Definition of a Uniformly Most Powerful Test......Page 479
Monotone Likelihood Ratio......Page 480
One-Sided Alternatives......Page 481
Exercises......Page 486
General Form of the Procedure......Page 490
Selection of the Test Procedure......Page 491
Composite Null Hypothesis......Page 492
Equivalence of Confidence Sets and Tests......Page 494
Exercises......Page 496
Testing Hypotheses About the Mean of a Normal Distribution When the Variance Is Unknown......Page 498
Derivation of the t Test......Page 499
Properties of the t Test......Page 502
Testing with a Two-Sided Alternative......Page 503
Exercises......Page 504
Tail Areas......Page 507
Tail Areas for a Two-Sided Alternative Hypothesis......Page 508
Statistically Significant Results......Page 509
The Bayesian Approach......Page 510
Exercises......Page 511
Definition of the F Distribution......Page 512
Properties of the F Distribution......Page 514
Derivation of the F Test......Page 515
Properties of the F Test......Page 516
Exercises......Page 518
Derivation of the Two-Sample t Test......Page 519
Properties of the Two-Sample t Test......Page 521
Two-Sided Alternatives and Unequal Variances......Page 523
Exercises......Page 524
8.10 Supplementary Exercises......Page 525
Description of Nonparametric Problems......Page 532
The χ² Test......Page 533
Testing Hypotheses About a Proportion......Page 534
Testing Hypotheses About a Continuous Distribution......Page 535
Discussion of the Test Procedure......Page 536
Exercises......Page 537
Composite Null Hypotheses......Page 539
The χ² Test for Composite Null Hypotheses......Page 540
Determining the Maximum Likelihood Estimates......Page 541
Testing Whether a Distribution Is Normal......Page 542
Testing Composite Hypotheses About an Arbitrary Distribution......Page 543
Exercises......Page 544
Independence in Contingency Tables......Page 547
The χ² Test of Independence......Page 549
Exercises......Page 550
Samples from Several Populations......Page 553
The χ² Test of HomogeneIty......Page 555
Comparing Two or More Proportions......Page 556
Correlated 2 × 2 Tables......Page 557
Exercises......Page 558
Aggregation and Disaggregation......Page 561
The Paradox Explained......Page 562
Exercises......Page 563
The Sample Distribution Function......Page 565
The Kolmogorov–Smirnov Test of a Simple Hypothesis......Page 567
The Kolmogorov–Smirnov Test for Two Samples......Page 571
Exercises......Page 572
Confidence Intervals and Tests for the Median......Page 574
Confidence Intervals and Tests for Quantiles......Page 576
Estimating the Median......Page 577
Trimmed Means......Page 578
Comparison of the Estimators......Page 579
Exercises......Page 582
Comparative Experiments and Matched Pairs......Page 584
The Sign Test......Page 585
The Wilcoxon Signed-Ranks Test......Page 586
Ties......Page 589
Exercises......Page 590
Comparing Two Distributions......Page 593
The Wilcoxon–Mann–Whitney Ranks Test......Page 594
Exercises......Page 597
9.11 Supplementary Exercises......Page 599
Fitting a Straight Line......Page 606
The Least-Squares Line......Page 608
Fitting a Polynomial by the Method of Least Squares......Page 610
Fitting a Linear Function of Several Variables......Page 612
Exercises......Page 615
Regression Functions......Page 617
Simple Linear Regression......Page 618
The Distribution of the Least-Squares Estimators......Page 619
The Gauss–Markov Theorem for Simple Linear Regression......Page 621
Design of the Experiment......Page 622
Prediction......Page 623
Exercises......Page 624
Joint Distribution of the Estimators......Page 625
Tests of Hypotheses about the Regression Coefficients......Page 630
Confidence Intervals and Confidence Sets......Page 636
The Analysis of Residuals......Page 638
Exercises......Page 639
Use of the Term \"Regression\"......Page 641
The Normal Distribution......Page 643
The General Linear Model......Page 644
Maximum Likelihood Estimators......Page 645
Explicit Form of the Estimators......Page 646
Mean Vector and Covariance Matrix......Page 647
The Joint Distribution of the Estimators......Page 650
Testing Hypotheses......Page 651
Multiple Linear Regression......Page 652
Screening Regression Equations......Page 653
Exercises......Page 654
The One-Way Layout......Page 657
Partitioning a Sum of Squares......Page 659
Testing Hypotheses......Page 662
Exercises......Page 663
The Two-Way Layout with One Observation In Each Cell......Page 665
Estimating the Parameters......Page 667
Partitioning the Sum of Squares......Page 668
Testing Hypotheses......Page 670
Exercises......Page 673
The Two-Way Layout with K Observations in Each Cell......Page 675
Partitioning the Sum of Squares......Page 678
Testing Hypotheses......Page 680
The Two-Way Layout with Unequal Numbers of Observations in the Cells......Page 684
Exercises......Page 685
10.9 Supplementary Exercises......Page 687
References......Page 692
Tables......Page 694
Answers to Even-Numbered Exercises......Page 710
Index......Page 730