Finite Markov Chains

FINITE MARKOV CHAINS......Page 1
Undergraduate Texts in Mathematics......Page 2
Title Page......Page 3
Copyright Page......Page 4
Preface......Page 5
Preface to the Second Printing......Page 7
Table of Contents......Page 9
§ 1.1 Sets.......Page 13
§ 1.2 Statements.......Page 14
§ 1.3 Order relations.......Page 15
§ 1.4 Communication relations.......Page 17
§ 1.5 Probability measures.......Page 19
§ 1.6 Conditional probability.......Page 21
§ 1.7 Functions on a possibility space.......Page 22
§ 1.8 Mean and variance of a function.......Page 24
§ 1.9 Stochastic processes.......Page 26
§ 1.10 Summability of sequences and series.......Page 30
§ 1.11 Matrices.......Page 31
§ 2.1 Definition of a Markov process and a Markov chain.......Page 36
§ 2.2 Examples.......Page 38
§ 2.3 Connection with matrix theory.......Page 44
§ 2.4 Classification of states and chains.......Page 47
§ 2.5 Problems to be studied.......Page 50
Exercises for Chapter 2......Page 51
§ 3.1 Introduction.......Page 55
§ 3.2 The fundamental matrix.......Page 57
§ 3.3 Applications of the fundamental matrix.......Page 61
§ 3.4 Examples......Page 67
§ 3.5 Extension of results.......Page 70
Exercises for Chapter 3......Page 78
§ 4.1 Basic theorems.......Page 81
§ 4.2 Law of large numbers for regular Markov chains.......Page 85
§ 4.3 The fundamental matrix for regular chains.......Page 87
§ 4.4 First passage times.......Page 90
§ 4.5 Variance of the first passage time.......Page 94
§ 4.6 Limiting covariance.......Page 96
§ 4.7 Comparison of two examples.......Page 102
§ 4.8 The general two-state case.......Page 106
Exercises for Chapter 4......Page 107
§ 5.1 Fundamental matrix.......Page 111
§ 5.2 Examples of cyclic chains.......Page 114
§ 5.3 Reverse Markov chains.......Page 117
Exercises for Chapter 5......Page 122
§ 6.1 Application of absorbing chain theory to ergodic chains.......Page 124
§ 6.2 Application of ergodic chain theory to absorbing Markov chains.......Page 129
§ 6.3 Combining states.......Page 135
§ 6.4 Weak lumpability.......Page 144
§ 6.5 Expanding a Markov chain.......Page 152
Exercises for Chapter 6......Page 157
§ 7.1 Random walks.......Page 161
§ 7.2 Applications to sports.......Page 173
§ 7.3 Ehrenfest model for diffusion.......Page 179
§ 7.4 Applications to genetics.......Page 188
§ 7.5 Learning theory.......Page 194
§ 7.6 Applications to mobility theory.......Page 203
§ 7.7 The open Leontief model.......Page 212
2—Basic Definitions......Page 219
3—Basic Quantities for Absorbing Chains......Page 220
5—Basic Quantities for Ergodic Chains......Page 221
7—Some Basic Examples......Page 222
A Class of Linear Equations......Page 223
A Special Case......Page 227
Ergodic Chains......Page 229
Applications......Page 232
Historical Notes......Page 235
References......Page 236
Undergraduate Texts in Mathematics (continued from ii)......Page 237
Back Cover......Page 238