A First Course in Stochastic Processes, Second Edition

Front Cover......Page 1
A First Course In Stochastic Processes......Page 4
Copyright Page......Page 5
Table of Contents\n......Page 6
Preface......Page 12
Preface to First Edition......Page 16
1. Review of Basic Terminology and Properties of Random Variables and Distribution Functions......Page 20
2. Two Simple Examples of Stochastic Processes......Page 39
3. Classification of General Stochastic Processes......Page 45
4. Defining a Stochastic Process......Page 51
Elementary Problems......Page 52
Problems......Page 55
References......Page 63
1. Definitions......Page 64
2. Examples of Markov Chains......Page 66
3. Transition Probability Matrices of a Markov Chain......Page 77
4. Classification of States of a Markov Chain......Page 78
5. Recurrence......Page 81
6. Examples of Recurrent Markov Chains......Page 86
7. More on Recurrence......Page 91
Elementary Problems......Page 92
Problems......Page 96
Notes......Page 98
References......Page 99
1. Discrete Renewal Equation......Page 100
2. Proof of Theorem 1.1......Page 106
3. Absorption Probabilities......Page 108
4. Criteria for Recurrence......Page 113
5. A Queueing Example......Page 115
6. Another Queueing Model......Page 121
7. Random Walk......Page 125
Elementary Problems......Page 127
Problems......Page 131
Reference......Page 135
1. General Pure Birth Processes and Poisson Processes......Page 136
2. More about Poisson Processes......Page 142
3. A Counter Model......Page 147
4. Birth and Death Processes......Page 150
5. Differential Equations of Birth and Death Processes......Page 154
6. Examples of Birth and Death Processes......Page 156
7. Birth and Death Processes with Absorbing States......Page 164
8. Finite State Continuous Time Markov Chains......Page 169
Elementary Problems......Page 171
Problems......Page 177
Notes......Page 184
References......Page 185
1. Definition of a Renewal Process and Related Concepts......Page 186
2. Some Examples of Renewal Processes......Page 189
3. More on Some Special Renewal Processes......Page 192
4. Renewal Equations and the Elementary Renewal Theorem......Page 200
5. The Renewal Theorem......Page 208
6. Applications of the Renewal Theorem......Page 211
7. Generalizations and Variations on Renewal Processes......Page 216
8. More Elaborate Applications of Renewal Theory......Page 231
9. Superposition of Renewal Processes......Page 240
Elementary Problems......Page 247
Problems......Page 249
Reference......Page 256
1. Preliminary Definitions and Examples......Page 257
2. Supermartingales and Submartingales......Page 267
3. The Optional Sampling Theorem......Page 272
4. Some Applications of the Optional Sampling Theorem......Page 282
5. Martingale Convergence Theorems......Page 297
6. Applications and Extensions of the Martingale Convergence Theorems......Page 306
7. Martingales with Respect to σ-Fields......Page 316
8. Other Martingales......Page 332
Elementary Problems......Page 344
Problems......Page 349
References......Page 358
1. Background Material......Page 359
2. Joint Probabilities for Brownian Motion......Page 362
3. Continuity of Paths and the Maximum Variables......Page 364
4. Variations and Extensions......Page 370
5. Computing Some Functionals of Brownian Motion by Martingale Methods......Page 376
6. Multidimensional Brownian Motion......Page 384
7. Brownian Paths......Page 390
Elementary Problems......Page 402
Problems......Page 405
References......Page 410
1. Discrete Time Branching Processes......Page 411
2. Generating Function Relations for Branching Processes......Page 413
3. Extinction Probabilities......Page 415
4. Examples......Page 419
5. Two-Type Branching Processes......Page 423
6. Multi-Type Branching Processes......Page 430
7. Continuous Time Branching Processes......Page 431
8. Extinction Probabilities for Continuous Time Branching Processes......Page 435
9. Limit Theorems for Continuous Time Branching Processes......Page 438
10. Two-Type Continuous Time Branching Process......Page 443
11. Branching Processes with General Variable Lifetime......Page 450
Elementary Problems......Page 455
Problems......Page 457
Reference......Page 461
1. Definitions and Examples......Page 462
2. Mean Square Distance......Page 470
3. Mean Square Error Prediction......Page 480
4. Prediction of Covariance Stationary Processes......Page 489
5. Ergodic Theory and Stationary Processes......Page 493
6. Applications of Ergodic Theory......Page 508
7. Spectral Analysis of Covariance Stationary Processes......Page 521
8. Gaussian Systems......Page 529
9. Stationary Point Processes......Page 535
10. The Level-Crossing Problem......Page 538
Elementary Problems......Page 543
Problems......Page 546
Notes......Page 553
References......Page 554
1. The Spectral Theorem......Page 555
2. The Frobenius Theory of Positive Matrices......Page 561
Index......Page 572