Discrete-Time Semi-Markov Random Evolutions and Their Applications

Preface
Contents
Acronyms
Notation
1 Discrete-Time Stochastic Calculus in Banach Space
	1.1 Introduction
	1.2 Random Elements and Discrete-Time Martingales in a Banach Space
	1.3 Martingale Characterization of Markov and Semi-Markov Chains
		1.3.1 Martingale Characterization of Markov Chains
		1.3.2 Martingale Characterization of Markov Processes
		1.3.3 Martingale Characterization of Semi-Markov Processes
	1.4 Operator Semigroups and Their Generators
	1.5 Martingale Problem in a Banach Space
	1.6 Weak Convergence in a Banach Space
	1.7 Reducible-Invertible Operators and Their Perturbations
		1.7.1 Reducible-Invertible Operators
		1.7.2 Perturbation of Reducible-Invertible Operators
2 Discrete-Time Semi-Markov Chains
	2.1 Introduction
	2.2 Semi-Markov Chains
		2.2.1 Definitions
		2.2.2 Classification of States
		2.2.3 Markov Renewal Equation and Theorem
	2.3 Discrete- and Continuous-Time Connection
	2.4 Compensating Operator and Martingales
	2.5 Stationary Phase Merging
	2.6 Semi-Markov Chains in Merging State Space
		2.6.1 The Ergodic Case
		2.6.2 The Non-ergodic Case
	2.7 Concluding Remarks
3 Discrete-Time Semi-Markov Random Evolutions
	3.1 Introduction
	3.2 Discrete-time Random Evolution with Underlying Markov Chain
	3.3 Definition and Properties of DTSMRE
	3.4 Discrete-Time Stochastic Systems
		3.4.1 Additive Functionals
		3.4.2 Geometric Markov Renewal Chains
		3.4.3 Dynamical Systems
	3.5 Discrete-Time Stochastic Systems in Series Scheme
	3.6 Concluding Remarks
4 Weak Convergence of DTSMRE in Series Scheme
	4.1 Introduction
	4.2 Weak Convergence Results
		4.2.1 Averaging
		4.2.2 Diffusion Approximation
		4.2.3 Normal Deviations
		4.2.4 Rates of Convergence in the Limit Theorems
	4.3 Proof of Theorems
		4.3.1 Proof of Theorem 4.1
		4.3.2 Proof of Theorem 4.2
		4.3.3 Proof of Theorem 4.3
		4.3.4 Proof of Proposition 4.1
	4.4 Applications of the Limit Theorems to Stochastic Systems
		4.4.1 Additive Functionals
		4.4.2 Geometric Markov Renewal Processes
		4.4.3 Dynamical Systems
		4.4.4 Estimation of the Stationary Distribution
		4.4.5 U-Statistics
		4.4.6 Rates of Convergence for Stochastic Systems
	4.5 Concluding Remarks
5 DTSMRE in Reduced Random Media
	5.1 Introduction
	5.2 Definition and Properties
	5.3 Average and Diffusion Approximation
		5.3.1 Averaging
		5.3.2 Diffusion Approximation
		5.3.3 Normal Deviations
	5.4 Proof of Theorems
		5.4.1 Proof of Theorem 5.1
		5.4.2 Proof of Theorem 5.2
	5.5 Application to Stochastic Systems
		5.5.1 Additive Functionals
		5.5.2 Dynamical Systems
		5.5.3 Geometric Markov Renewal Chains
		5.5.4 U-Statistics
	5.6 Concluding Remarks
6 Controlled Discrete-Time Semi-Markov Random Evolutions
	6.1 Introduction
	6.2 Controlled Discrete-Time Semi-Markov Random Evolutions
		6.2.1 Definition of CDTSMREs
		6.2.2 Examples
		6.2.3 Dynamic Programming for Controlled Models
	6.3 Limit Theorems for Controlled Semi-Markov Random Evolutions
		6.3.1 Averaging of CDTSMREs
		6.3.2 Diffusion Approximation of DTSMREs
		6.3.3 Normal Approximation
	6.4 Applications to Stochastic Systems
		6.4.1 Controlled Additive Functionals
		6.4.2 Controlled Geometric Markov Renewal Processes
		6.4.3 Controlled Dynamical Systems
		6.4.4 The Dynamic Programming Equations for Limiting Models in Diffusion Approximation
			6.4.4.1 DPE/HJB Equation for the Limiting CAF in DA (see Sect.6.4.1)
			6.4.4.2 DPE/HJB Equation for the Limiting CGMRP in DA (see Sect.6.4.2)
			6.4.4.3 DPE/HJB Equation for the Limiting CDS in DA (see Sect.6.4.3)
	6.5 Solution of Merton Problem for the Limiting CGMRP in DA
		6.5.1 Introduction
		6.5.2 Utility Function
		6.5.3 Value Function or Performance Criterion
		6.5.4 Solution of Merton Problem: Examples
		6.5.5 Solution of Merton Problem
	6.6 Rates of Convergence in Averaging and Diffusion Approximations
	6.7 Proofs
		6.7.1 Proof of Theorem 6.1
		6.7.2 Proof of Theorem 6.2
		6.7.3 Proof of Theorem 6.3
		6.7.4 Proof of Proposition 6.1
	6.8 Concluding Remarks
7 Epidemic Models in Random Media
	7.1 Introduction
	7.2 From the Deterministic to Stochastic SARS Model
	7.3 Averaging of Stochastic SARS Models
	7.4 SARS Model in Merging Semi-Markov Random Media
	7.5 Diffusion Approximation of Stochastic SARS Models in Semi-Markov Random Media
	7.6 Concluding remarks
8 Optimal Stopping of Geometric Markov Renewal Chains and Pricing
	8.1 Introduction
	8.2 GMRC and Embedded Markov-Modulated (B,S)-Security Markets
		8.2.1 Definition of the GMRC
		8.2.2 Statement of the Problem: Optimal Stopping Rule
	8.3 GMRP as Jump Discrete-Time Semi-Markov Random Evolution
	8.4 Martingale Properties of GMRC
	8.5 Optimal Stopping Rules for GMRC
	8.6 Martingale Properties of Discount Price and Discount Capital
	8.7 American Option Pricing Formulae for embedded Markov-modulated (B,S)-Security markets
	8.8 European Option Pricing Formula for Embedded Markov-Modulated (B,S)-Security Markets
	8.9 Proof of Theorems
	8.10 Concluding Remarks
A Markov Chains
	A.1 Transition Function
	A.2 Irreducible Markov Chains
	A.3 Recurrent Markov Chains
	A.4 Invariant Measures
	A.5 Uniformly Ergodic Markov Chains
Bibliography
Index