Probability: A Graduate Course

Contents......Page 8
Preface......Page 5
Outline of Contents......Page 16
Notation and Symbols......Page 19
1 Probability Theory: An Introduction......Page 22
2 Basics from Measure Theory......Page 23
2.1 Sets......Page 24
2.2 Collections of Sets......Page 26
2.3 Generators......Page 28
2.4 A Metatheorem and Some Consequences......Page 30
3 The Probability Space......Page 31
3.1 Limits and Completeness......Page 32
3.2 An Approximation Lemma......Page 34
3.3 The Borel Sets on R......Page 35
4 Independence; Conditional Probabilities......Page 37
4.1 The Law of Total Probability; Bayes\' Formula......Page 38
4.2 Independence of Collections of Events......Page 39
4.3 Pair-wise Independence......Page 40
5 The Kolmogorov Zero-one Law......Page 41
6 Problems......Page 43
1 Definition and Basic Properties......Page 46
1.1 Functions of Random Variables......Page 49
2.1 Distribution Functions......Page 51
2.2 Integration: A Preview......Page 53
2.3 Decomposition of Distributions......Page 57
2.4 Some Standard Discrete Distributions......Page 60
2.6 The Cantor Distribution......Page 61
2.7 Two Perverse Examples......Page 63
3.1 Random Vectors......Page 64
3.2 Random Elements......Page 66
4.1 Definitions......Page 67
4.2 Basic Properties......Page 69
5 Expectation; Convergence......Page 75
6 Indefinite Expectations......Page 79
7 A Change of Variables Formula......Page 81
8 Moments, Mean, Variance......Page 83
9.1 Finite-dimensional Product Measures......Page 85
9.2 Fubini\'s Theorem......Page 86
9.3 Partial Integration......Page 87
9.4 The Convolution Formula......Page 88
10 Independence......Page 89
10.3 Pair-wise Independence......Page 92
10.4 The Kolmogorov Zero-one Law Revisited......Page 93
11 The Cantor Distribution......Page 94
12 Tail Probabilities and Moments......Page 95
13 Conditional Distributions......Page 100
14 Distributions with Random Parameters......Page 102
15 Sums of a Random Number of Random Variables......Page 104
15.1 Applications......Page 106
16.1 Random Walks......Page 109
16.2 Renewal Theory......Page 110
16.3 Renewal Theory for Random Walks......Page 111
16.5 Sequential Analysis......Page 112
16.6 Replacement Based on Age......Page 113
17.2 Records......Page 114
18.1 The Borel-Cantelli Lemmas 1 and 2......Page 117
18.2 Some (Very) Elementary Examples......Page 119
18.3 Records......Page 122
18.4 Recurrence and Transience of Simple Random Walks......Page 123
18.6 Pair-wise Independence......Page 125
18.7 Generalizations Without Independence......Page 126
18.8 Extremes......Page 128
18.9 Further Generalizations......Page 130
19 A Convolution Table......Page 134
20 Problems......Page 135
1 Tail Probabilities Estimated via Moments......Page 139
2 Moment Inequalities......Page 147
3 Covariance; Correlation......Page 150
4 Interlude on L[sup(p)]-spaces......Page 151
5 Convexity......Page 152
6 Symmetrization......Page 153
7 Probability Inequalities for Maxima......Page 158
8 The Marcinkiewics-Zygmund Inequalities......Page 166
9 Rosenthal\'s Inequality......Page 171
10 Problems......Page 173
1 Definition and Basics......Page 176
1.1 Uniqueness; Inversion......Page 178
1.2 Multiplication......Page 183
1.3 Some Further Results......Page 184
2.1 The Cantor Distribution......Page 185
2.2 The Convolution Table Revisited......Page 187
2.3 The Cauchy Distribution......Page 189
2.4 Symmetric Stable Distributions......Page 190
2.5 Parseval\'s Relation......Page 191
3 Two Surprises......Page 192
4 Refinements......Page 194
5.1 The Multivariate Normal Distribution......Page 199
5.2 The Mean and the Sample Variance Are Independent......Page 202
6 The Cumulant Generating Function......Page 203
7 The Probability Generating Function......Page 205
7.1 Random Vectors......Page 207
8 The Moment Generating Function......Page 208
8.2 Two Boundary Cases......Page 210
9 Sums of a Random Number of Random Variables......Page 211
10 The Moment Problem......Page 213
10.1 The Moment Problem for Random Sums......Page 215
11 Problems......Page 216
5 Convergence......Page 220
1 Definitions......Page 221
1.1 Continuity Points and Continuity Sets......Page 222
1.2 Measurability......Page 224
1.3 Some Examples......Page 225
2 Uniqueness......Page 226
3 Relations Between Convergence Concepts......Page 228
3.1 Converses......Page 231
4 Uniform Integrability......Page 233
5.1 Almost Sure Convergence......Page 237
5.2 Convergence in Probability......Page 239
5.3 Convergence in Distribution......Page 241
6 Distributional Convergence Revisited......Page 244
6.1 Scheffé\'s Lemma......Page 245
7 A Subsequence Principle......Page 248
8 Vague Convergence; Helly\'s Theorem......Page 249
8.1 Vague Convergence......Page 250
8.2 Helly\'s Selection Principle......Page 251
8.3 Vague Convergence and Tightness......Page 253
8.4 The Method of Moments......Page 256
9.1 The Characteristic Function......Page 257
9.2 The Cumulant Generating Function......Page 259
9.3 The (Probability) Generating Function......Page 260
9.4 The Moment Generating Function......Page 261
10 Convergence of Functions of Random Variables......Page 262
10.1 The Continuous Mapping Theorem......Page 264
11 Convergence of Sums of Sequences......Page 266
11.1 Applications......Page 268
11.2 Converses......Page 271
11.3 Symmetrization and Desymmetrization......Page 274
12 Cauchy Convergence......Page 275
13 Skorohod\'s Representation Theorem......Page 277
14 Problems......Page 279
6 The Law of Large Numbers......Page 284
1.1 Convergence Equivalence......Page 285
1.2 Distributional Equivalence......Page 286
1.4 Moments and Tails......Page 287
2 A Weak Law for Partial Maxima......Page 288
3 The Weak Law of Large Numbers......Page 289
3.1 Two Applications......Page 295
4 A Weak Law Without Finite Mean......Page 297
4.1 The St. Petersburg Game......Page 302
5 Convergence of Series......Page 303
5.1 The Kolmogorov Convergence Criterion......Page 305
5.2 A Preliminary Strong Law......Page 307
5.3 The Kolmogorov Three-series Theorem......Page 308
5.4 Lévy\'s Theorem on the Convergence of Series......Page 311
6 The Strong Law of Large Numbers......Page 313
7 The Marcinkiewicz-Zygmund Strong Law......Page 317
8 Randomly Indexed Sequences......Page 320
9.1 Normal Numbers......Page 324
9.3 Renewal Theory for Random Walks......Page 325
9.4 Records......Page 326
10 Uniform Integrability; Moment Convergence......Page 328
11 Complete Convergence......Page 330
11.1 The Hsu-Robbins-Erdős Strong Law......Page 331
11.2 Complete Convergence and the Strong Law......Page 333
12.1 Convergence Rates......Page 334
12.2 Counting Variables......Page 339
12.3 The Case r = p Revisited......Page 340
12.4 Random Indices......Page 341
13 Problems......Page 342
7 The Central Limit Theorem......Page 347
2 The Lindeberg-Lévy-Feller Theorem......Page 348
2.1 Lyapounov\'s Condition......Page 357
2.2 Remarks and Complements......Page 358
2.3 Pair-wise Independence......Page 361
2.4 The Central Limit Theorem for Arrays......Page 362
3 Anscombe\'s Theorem......Page 363
4 Applications......Page 366
4.1 The Delta Method......Page 367
4.3 Renewal Theory for Random Walks......Page 368
4.4 Records......Page 369
5 Uniform Integrability; Moment Convergence......Page 370
6 Remainder Term Estimates......Page 372
6.1 The Berry-Esseen Theorem......Page 373
6.2 Proof of the Berry-Esseen Theorem 6.2......Page 375
7.1 Rates of Rates......Page 380
7.2 Non-uniform Estimates......Page 381
7.4 Records......Page 382
7.6 Large Deviations......Page 383
7.7 Convergence Rates......Page 384
7.8 Precise Asymptotics......Page 389
7.9 A Short Outlook on Extensions......Page 392
8 Problems......Page 394
8 The Law of the Iterated Logarithm......Page 400
1 The Kolmogorov and Hartman-Wintner LILs......Page 401
2 Exponential Bounds......Page 402
3 Proof of the Hartman-Wintner Theorem......Page 404
4 Proof of the Converse......Page 413
5 The LIL for Subsequences......Page 415
5.1 A Borel-Cantelli Sum for Subsequences......Page 418
5.2 Proof of Theorem 5.2......Page 419
6 Cluster Sets......Page 421
6.1 Proofs......Page 423
7.1 Hartman-Wintner via Berry-Esseen......Page 429
7.2 Examples Not Covered by Theorems 5.2 and 5.1......Page 430
7.3 Further Remarks on Sparse Subsequences......Page 431
7.4 An Anscombe LIL......Page 433
7.6 Record Times......Page 434
7.7 Convergence Rates......Page 435
7.9 The Other LIL......Page 436
8 Problems......Page 437
9 Limit Theorems; Extensions and Generalizations......Page 439
1 Stable Distributions......Page 440
2 The Convergence to Types Theorem......Page 443
3 Domains of Attraction......Page 446
3.1 Sketch of Preliminary Steps......Page 449
3.2 Proof of Theorems 3.2 and 3.3......Page 451
3.3 Two Examples......Page 454
3.4 Two Variations......Page 455
3.5 Additional Results......Page 456
4 Infinitely Divisible Distributions......Page 458
5 Sums of Dependent Random Variables......Page 464
6.1 Max-stable and Extremal Distributions......Page 467
6.2 Domains of Attraction......Page 472
6.3 Record Values......Page 473
7 The Stein-Chen Method......Page 475
8 Problems......Page 480
10 Martingales......Page 483
1 Conditional Expectation......Page 484
1.1 Properties of Conditional Expectation......Page 487
1.2 Smoothing......Page 490
1.3 The Rao-Blackwell Theorem......Page 491
1.4 Conditional Moment Inequalities......Page 492
2 Martingale Definitions......Page 493
2.2 Two Equivalent Definitions......Page 495
3 Examples......Page 497
4 Orthogonality......Page 503
5 Decompositions......Page 505
6 Stopping Times......Page 507
7 Doob\'s Optional Sampling Theorem......Page 511
8 Joining and Stopping Martingales......Page 513
9 Martingale Inequalities......Page 517
10.1 Garsia\'s Proof......Page 524
10.2 The Upcrossings Proof......Page 527
10.3 Some Remarks on Additional Proofs......Page 529
10.4 Some Questions......Page 530
11 The Martingale {E(Z | F[sub(n)])}......Page 531
12 Regular Martingales and Submartingales......Page 532
12.1 A Main Martingale Theorem......Page 533
12.2 A Main Submartingale Theorem......Page 534
12.4 Regular Martingales Revisited......Page 535
13 The Kolmogorov Zero-one Law......Page 536
14.1 Finiteness of Moments......Page 537
14.2 The Wald Equations......Page 538
14.3 Tossing a Coin Until Success......Page 541
14.4 The Gambler\'s Ruin Problem......Page 542
14.5 A Converse......Page 545
15 Regularity......Page 547
15.1 First Passage Times for Random Walks......Page 551
15.2 Complements......Page 553
15.3 The Wald Fundamental Identity......Page 554
16 Reversed Martingales and Submartingales......Page 557
16.1 The Law of Large Numbers......Page 560
16.2 U-statistics......Page 563
17 Problems......Page 564
1 Taylor Expansion......Page 570
2 Mill\'s Ratio......Page 573
3 Sums and Integrals......Page 574
4 Sums and Products......Page 575
5 Convexity; Clarkson\'s Inequality......Page 576
6 Convergence of (Weighted) Averages......Page 579
7 Regularly and Slowly Varying Functions......Page 581
8 Cauchy\'s Functional Equation......Page 583
9 Functions and Dense Sets......Page 585
References......Page 591
C......Page 603
D......Page 606
E......Page 607
G......Page 608
I......Page 609
L......Page 610
M......Page 611
P......Page 612
R......Page 613
S......Page 614
U......Page 616
Z......Page 617