Relative Optimization of Continuous-Time and Continuous-State Stochastic Systems

Preface
Acknowledgements
Contents
Notation and Terminology
1 Introduction
	1.1 A Brief Introduction to Relative Optimization
	1.2 Relative Optimization Versus Dynamic Programming
		1.2.1 The Optimization Problem
		1.2.2 Dynamic Programming
		1.2.3 Relative Optimization
		1.2.4 Comparison of Two Approaches
		1.2.5 Intuitive Explanations
	1.3 Main Results of the Book
	1.4 Philosophical and Historical Remarks
		1.4.1 A Philosophical View on Relative Optimization
		1.4.2 History of Development
	References
2 Optimal Control of Markov Processes: Infinite-Horizon
	2.1 Introduction
	2.2 The Markov Process
	2.3 Optimization of Long-Run Average
		2.3.1 State Comparability and Performance Potentials
		2.3.2 Conditions for State Comparability
		2.3.3 Performance-Difference Formula
		2.3.4 Performance Optimization
	2.4 Bias Optimality
		2.4.1 Bias Potential
		2.4.2 The Bias-Difference Formula
		2.4.3 The Space of Average Optimal Policies
		2.4.4 Bias Optimality Conditions
	2.5 Optimization of Multi-class Markov Processes
		2.5.1 State Classification
		2.5.2 Performance-Difference Formula
		2.5.3 Performance Optimization
	2.6 Optimization of Discounted Rewards
	2.7 Special Cases, Extensions, and Discussions
	References
3 Optimal Control of Diffusion Processes
	3.1 Fundamental Mathematics
		3.1.1 Stochastic Differential Equations
		3.1.2 Stochastic Calculus
		3.1.3 Solutions to Stochastic Differential Equations
		3.1.4 Application in Finance: The Black–Scholes Equation
	3.2 Stochastic Calculus with Non-smooth Features
		3.2.1 Local Time, Ito–Tanaka Formula, and the Skorokhod Problem
		3.2.2 Stochastic Calculus for Semi-smooth Functions
		3.2.3 The One-Dimensional System
		3.2.4 Stochastic Calculus in Relative Time
	3.3 Long-Run Average Optimization (Single Class)
		3.3.1 Performance-Difference Formula
		3.3.2 Performance Optimization
	3.4 Finite-Horizon Control Problems
		3.4.1 Main Results
		3.4.2 When the Value Function Cannot Be Reached
		3.4.3 Time-Dependent Problems
	3.5 Optimal Stopping
		3.5.1 A General Formulation
		3.5.2 Pure Optimal Stopping
		3.5.3 Illustrative Examples
	3.6 Singular Control
		3.6.1 Formulated with Reflecting Points
		3.6.2 Optimality Conditions
	References
4 Degenerate Diffusion Processes
	4.1 Multi-class Structure of Degenerate Diffusion Processes
		4.1.1 Transient and Recurrent States
		4.1.2 W-Ergodic and Branching States
	4.2 Semi-smoothness of Potential Functions
		4.2.1 Motivating Examples
		4.2.2 The Proof of Semi-smoothness with Finite Horizon
		4.2.3 Potential Functions for Long-Run Average
		4.2.4 Extensions
	4.3 Optimization of Degenerate Processes
		4.3.1 Long-Run Average
		4.3.2 Finite Horizon
		4.3.3 Optimal Stopping
		4.3.4 Singular Control
	References
5 Multi-dimensional Diffusion Processes
	5.1 Optimal Control with Smooth Performance Functions
		5.1.1 Multi-dimensional Diffusion Processes and Ito Formula
		5.1.2 Control Problems
	5.2 Calculus of Semi-smooth Functions
		5.2.1 Definitions
		5.2.2 Smooth Quadrants and Taylor Expansion
		5.2.3 Properties of Semi-smooth Functions
	5.3 Stochastic Calculus of Semi-smooth Functions
		5.3.1 Tanaka Formula
		5.3.2 Calculus in Relative Time
	5.4 Control Problems with Semi-smooth Performance Functions
		5.4.1 System Dynamics on Degenerate Curves
		5.4.2 Semi-smoothness of Performance Functions
		5.4.3 Finite-Horizon Stochastic Control
	5.5 Discussions
	References
6 Performance-Derivative-Based Optimization
	6.1 First-Order Optimality Condition
	6.2 Optimization with Distorted Probability
	References
Appendix A Stochastic Diffusions
A.1 Brownian Motions
A.1.1 The Preliminaries
A.1.1.1 Mathematical Definition of Brownian Motion
A.1.2 Probability Distributions
A.1.2.1 Probabilities of First Passage Times
A.1.2.2 Probability of the Maximum
A.1.3 Total and Quadratic Variations
A.1.3.1 Total Variation
A.1.3.2 Quadratic Variation
A.2 Stochastic Calculus
A.2.1 Stochastic Integrations
A.2.1.1 Definition
A.2.1.2 Stochastic Differentials
A.2.1.3 Stochastic Differential Equations
A.2.2 Ito Formula
A.3 Diffusion Processes as Markov Processes
A.3.1 Infinitesimal Generator
A.3.1.1 Dynkin's Formula
A.4 Fokker–Planck Equation
Appendix B Stochastic Calculus with Non-smooth Features
B.1 Reflected Brownian Motion and Skorokhod Problem
B.1.1 Definition
B.1.2 Local Time
B.2 Reflected Diffusion Processes
B.2.1 Solution to (B.13)
B.2.2 The Order of E[d ξ(t)]
B.3 More than One Reflecting Points
B.4 Stochastic Calculus for Semi-smooth Functions
B.4.1 Ito–Tanaka Formula for Semi-smooth Functions
B.4.2 Intuitive Explanation
B.4.2.1 First-Order Non-smoothness
B.4.2.2 Second-Order Semi-smooth Functions
B.4.2.3 At the Boundary Point
Appendix C Solutions
C.1 Problems of Chap. 3摥映數爠eflinkch:scdp33
C.2 Problems of Chap. 5摥映數爠eflinkch:multi55
Index