Probability: A Graduate Course

Cover......Page 1
Probability: A Graduate Course......Page 3
Preface to the First Edition......Page 5
Preface to the Second Edition......Page 8
Contents......Page 9
Outline of Contents......Page 17
Notation and Symbols......Page 20
10 Martingales......Page 23
2 Basics from Measure Theory......Page 24
2.1 Sets......Page 25
2.2 Collections of Sets......Page 27
2.3 Generators......Page 29
2.4 A Metatheorem and Some Consequences......Page 31
3 The Probability Space......Page 32
3.1 Limits and Completeness......Page 33
3.2 An Approximation Lemma......Page 35
3.3 The Borel Sets on R......Page 36
4 Independence; Conditional Probabilities......Page 38
4.1 The Law of Total Probability; Bayes\' Formula......Page 39
4.2 Independence of Collections of Events......Page 40
4.3 Pair-wise Independence......Page 41
5 The Kolmogorov Zero--one Law......Page 42
6 Problems......Page 44
1 Definition and Basic Properties......Page 47
1.1 Functions of Random Variables......Page 49
2.1 Distribution Functions......Page 52
2.2 Integration: A Preview......Page 54
2.3 Decomposition of Distributions......Page 58
2.5 Some Standard Absolutely Continuous Distributions......Page 61
2.6 The Cantor Distribution......Page 62
3.1 Random Vectors......Page 64
4 Expectation; Definitions and Basics......Page 67
4.1 Definitions......Page 68
4.2 Basic Properties......Page 70
5 Expectation; Convergence......Page 76
6 Indefinite Expectations......Page 80
7 A Change of Variables Formula......Page 82
8 Moments, Mean, Variance......Page 84
9.1 Finite-Dimensional Product Measures......Page 86
9.3 Partial Integration......Page 87
9.4 The Convolution Formula......Page 89
10 Independence......Page 90
10.3 Pair-Wise Independence......Page 93
10.4 The Kolmogorov Zero--One Law Revisited......Page 94
11 The Cantor Distribution......Page 95
12 Tail Probabilities and Moments......Page 96
13 Conditional Distributions......Page 101
14 Distributions with Random Parameters......Page 103
15 Sums of a Random Number of Random Variables......Page 105
15.1 Applications......Page 107
16.1 Random Walks......Page 110
16.2 Renewal Theory......Page 111
16.3 Renewal Theory for Random Walks......Page 112
16.5 Sequential Analysis......Page 113
16.6 Replacement Based on Age......Page 114
17.2 Records......Page 115
18.1 The Borel--Cantelli Lemmas 1 and 2......Page 118
18.2 Some (Very) Elementary Examples......Page 120
18.3 Records......Page 123
18.4 Recurrence and Transience of Simple Random Walks......Page 124
18.6 Pair-Wise Independence......Page 126
18.7 Generalizations Without Independence......Page 127
18.8 Extremes......Page 128
18.9 Further Generalizations......Page 131
20 Problems......Page 135
1 Tail Probabilities Estimated Via Moments......Page 140
2 Moment Inequalities......Page 148
3 Covariance: Correlation......Page 151
4 Interlude on Lp-Spaces......Page 152
5 Convexity......Page 153
6 Symmetrization......Page 154
7 Probability Inequalities for Maxima......Page 159
8 The Marcinkiewics--Zygmund Inequalities......Page 167
9 Rosenthal\'s Inequality......Page 172
10 Problems......Page 174
1 Definition and Basics......Page 177
1.1 Uniqueness; Inversion......Page 179
1.2 Multiplication......Page 184
1.3 Some Further Results......Page 185
2.1 The Cantor Distribution......Page 186
2.2 The Convolution Table Revisited......Page 188
2.3 The Cauchy Distribution......Page 190
2.4 Symmetric Stable Distributions......Page 191
2.5 Parseval\'s Relation......Page 192
3 Two Surprises......Page 193
4 Refinements......Page 195
5.1 The Multivariate Normal Distribution......Page 200
5.2 The Mean and the Sample Variance Are Independent......Page 203
6 The Cumulant Generating Function......Page 204
7 The Probability Generating Function......Page 206
7.1 Random Vectors......Page 208
8 The Moment Generating Function......Page 209
8.2 Two Boundary Cases......Page 211
9 Sums of a Random Number of Random Variables......Page 212
10 The Moment Problem......Page 214
10.1 The Moment Problem for Random Sums......Page 216
11 Problems......Page 217
1 Definitions......Page 222
1.1 Continuity Points and Continuity Sets......Page 223
1.2 Measurability......Page 225
1.3 Some Examples......Page 226
2 Uniqueness......Page 227
3 Relations Between Convergence Concepts......Page 228
3.1 Converses......Page 231
4 Uniform Integrability......Page 233
5 Convergence of Moments......Page 237
5.1 Almost Sure Convergence......Page 238
5.2 Convergence in Probability......Page 239
5.3 Convergence in Distribution......Page 242
6 Distributional Convergence Revisited......Page 244
6.1 Scheffé\'s Lemma......Page 246
7 A Subsequence Principle......Page 248
8.1 Vague Convergence......Page 250
8.2 Helly\'s Selection Principle......Page 251
8.3 Vague Convergence and Tightness......Page 254
8.4 The Method of Moments......Page 256
9.1 The Characteristic Function......Page 257
9.3 The (Probability) Generating Function......Page 260
9.4 The Moment Generating Function......Page 261
10 Convergence of Functions of Random Variables......Page 263
10.1 The Continuous Mapping Theorem......Page 264
11 Convergence of Sums of Sequences......Page 266
11.1 Applications......Page 268
11.2 Converses......Page 271
11.3 Symmetrization and Desymmetrization......Page 274
12 Cauchy Convergence......Page 275
13 Skorohod\'s Representation Theorem......Page 277
14 Problems......Page 279
1.1 Convergence Equivalence......Page 285
1.2 Distributional Equivalence......Page 286
1.4 Moments and Tails......Page 287
2 A Weak Law for Partial Maxima......Page 288
3 The Weak Law of Large Numbers......Page 289
3.1 Two Applications......Page 295
4 A Weak Law Without Finite Mean......Page 297
4.1 The St. Petersburg Game......Page 302
5 Convergence of Series......Page 303
5.1 The Kolmogorov Convergence Criterion......Page 305
5.2 A Preliminary Strong Law......Page 307
5.3 The Kolmogorov Three-Series Theorem......Page 308
5.4 Lévy\'s Theorem on the Convergence of Series......Page 311
6 The Strong Law of Large Numbers......Page 313
7 The Marcinkiewicz--Zygmund Strong Law......Page 317
8 Randomly Indexed Sequences......Page 320
9.1 Normal Numbers......Page 324
9.3 Renewal Theory for Random Walks......Page 325
9.4 Records......Page 326
10 Uniform Integrability; Moment Convergence......Page 328
11 Complete Convergence......Page 330
11.1 The Hsu--Robbins--Erdős Strong Law......Page 331
11.2 Complete Convergence and the Strong Law......Page 333
12.1 Convergence Rates......Page 334
12.2 Counting Variables......Page 339
12.3 The Case r=p Revisited......Page 340
12.4 Random Indices......Page 341
13 Problems......Page 342
2 The Lindeberg--Lévy--Feller Theorem......Page 349
2.1 Lyapounov\'s Condition......Page 358
2.2 Remarks and Complements......Page 359
2.3 Pair-Wise Independence......Page 362
2.4 The Central Limit Theorem for Arrays......Page 363
3 Anscombe\'s Theorem......Page 364
4.1 The Delta Method......Page 368
4.2 Stirling\'s Formula......Page 369
4.3 Renewal Theory for Random Walks......Page 370
4.4 Records......Page 371
5 Uniform Integrability; Moment Convergence......Page 372
6.1 The Berry--Esseen Theorem......Page 374
6.2 Proof of the Berry--Esseen Theorem 6.2......Page 376
7.1 Rates of Rates......Page 382
7.3 Renewal Theory......Page 383
7.5 Local Limit Theorems......Page 384
7.6 Large Deviations......Page 385
7.7 Convergence Rates......Page 386
7.8 Precise Asymptotics......Page 391
7.9 A Short Outlook on Extensions......Page 394
8 Problems......Page 396
1 The Kolmogorov and Hartman--Wintner LILs......Page 403
2 Exponential Bounds......Page 404
3 Proof of the Hartman--Wintner Theorem......Page 406
4 Proof of the Converse......Page 415
5 The LIL for Subsequences......Page 417
5.1 A Borel--Cantelli Sum for Subsequences......Page 420
5.2 Proof of Theorem 5.2......Page 421
6 Cluster Sets......Page 423
6.1 Proofs......Page 425
7.1 Hartman--Wintner via Berry--Esseen......Page 431
7.2 Examples Not Covered by Theorems 5.2 and 5.1......Page 432
7.3 Further Remarks on Sparse Subsequences......Page 433
7.4 An Anscombe LIL......Page 435
7.6 Record Times......Page 436
7.7 Convergence Rates......Page 437
7.9 The Other LIL......Page 438
7.10 Delayed Sums......Page 439
8 Problems......Page 440
1 Stable Distributions......Page 443
2 The Convergence to Types Theorem......Page 446
3 Domains of Attraction......Page 449
3.1 Sketch of Preliminary Steps......Page 452
3.2 Proof of Theorems 3.2 and 3.3......Page 454
3.3 Two Examples......Page 457
3.4 Two Variations......Page 458
3.5 Additional Results......Page 459
4 Infinitely Divisible Distributions......Page 461
5 Sums of Dependent Random Variables......Page 467
6.1 Max-Stable and Extremal Distributions......Page 470
6.2 Domains of Attraction......Page 475
6.3 Record Values......Page 476
7 The Stein--Chen Method......Page 478
8 Problems......Page 483
1 Conditional Expectation......Page 487
1.1 Properties of Conditional Expectation......Page 490
1.2 Smoothing......Page 493
1.3 The Rao--Blackwell Theorem......Page 494
1.4 Conditional Moment Inequalities......Page 495
2 Martingale Definitions......Page 496
2.2 Two Equivalent Definitions......Page 498
3 Examples......Page 500
4 Orthogonality......Page 506
5 Decompositions......Page 507
6 Stopping Times......Page 510
7 Doob\'s Optional Sampling Theorem......Page 514
8 Joining and Stopping Martingales......Page 516
9 Martingale Inequalities......Page 520
10.1 Garsia\'s Proof......Page 527
10.2 The Upcrossings Proof......Page 530
10.3 Some Remarks on Additional Proofs......Page 533
10.6 A Central Limit Theorem?......Page 534
11 The Martingale E(Z|F_n)......Page 535
12.1 A Main Martingale Theorem......Page 536
12.2 A Main Submartingale Theorem......Page 537
12.3 Two Non-regular Martingales......Page 538
13 The Kolmogorov Zero--One Law......Page 539
14.1 Finiteness of Moments......Page 540
14.2 The Wald Equations......Page 542
14.3 Tossing a Coin Until Success......Page 544
14.4 The Gambler\'s Ruin Problem......Page 545
14.5 A Converse......Page 549
15 Regularity......Page 551
15.1 First Passage Times for Random Walks......Page 554
15.2 Complements......Page 556
15.3 The Wald Fundamental Identity......Page 557
16 Reversed Martingales and Submartingales......Page 560
16.1 The Law of Large Numbers......Page 564
16.2 U-Statistics......Page 566
17 Problems......Page 568
AppendixSome Useful Mathematics......Page 574
References......Page 596
Index......Page 606