Controlled Markov Processes and Viscosity Solutions

Contents......Page 7
Preface to Second Edition......Page 11
Preface......Page 13
Notation......Page 15
I.1 Introduction......Page 18
I.2 Examples......Page 19
I.3 Finite time horizon problems......Page 22
I.4 Dynamic programming principle......Page 26
I.5 Dynamic programming equation......Page 28
I.6 Dynamic programming and Pontryagin’s principle......Page 35
I.7 Discounted cost with infinte horizon......Page 42
I.8 Calculus of variations I......Page 50
I.9 Calculus of variations II......Page 54
I.10 Generalized solutions to Hamilton-Jacobi equations......Page 59
I.11 Existence theorems......Page 66
I.12 Historical remarks......Page 72
II.1 Introduction......Page 74
II.2 Examples......Page 77
II.3 An abstract dynamic programming principle......Page 79
II.4 Defintion......Page 84
II.5 Dynamic programming and viscosity property......Page 89
II.6 Properties of viscosity solutions......Page 90
II.7 Deterministic optimal control and viscosity solutions......Page 95
II.8 Viscosity solutions: fist order case......Page 100
II.9 Uniqueness: fist order case......Page 106
II.10 Continuity of the value function......Page 116
II.11 Discounted cost with infinte horizon......Page 122
II.12 State constraint......Page 123
II.13 Discussion of boundary conditions......Page 128
II.14 Uniqueness: fist-order case......Page 131
II.15 Pontryagin’s maximum principle (continued)......Page 132
II.16 Historical remarks......Page 134
III.1 Introduction......Page 136
III.2 Markov processes and their evolution operators......Page 137
III.3 Autonomous (time-homogeneous) Markov processes......Page 140
III.4 Classes of Markov processes......Page 141
; stochastic di erential equations......Page 144
III.6 Controlled Markov processes......Page 147
III.7 Dynamic programming: formal description......Page 148
III.8 A Verifiction Theorem; finte time horizon......Page 151
III.9 Infinte Time Horizon......Page 156
III.10 Viscosity solutions......Page 162
III.11 Historical remarks......Page 165
IV.1 Introduction......Page 168
IV.2 Finite time horizon problem......Page 169
IV.3 Hamilton-Jacobi-Bellman PDE......Page 172
IV.4 Uniformly parabolic case......Page 178
IV.5 Infinte time horizon......Page 181
IV.6 Fixed finte time horizon problem: Preliminary estimates......Page 188
IV.7 Dynamic programming principle......Page 193
IV.8 Estimates for fist order di erence quotients......Page 199
IV.9 Estimates for second-order di erence quotients......Page 203
IV.10 Generalized subsolutions and solutions......Page 207
IV.11 Historical remarks......Page 214
V.1 Introduction......Page 216
V.2 Dynamic programming principle......Page 217
V.3 Viscosity property......Page 222
V.4 An equivalent formulation......Page 227
V.5 Semiconvex, concave approximations......Page 231
V.6 Crandall-Ishii Lemma......Page 233
V.7 Properties of......Page 235
V.8 Comparison......Page 236
V.9 Viscosity solutions in......Page 239
V.10 Historical remarks......Page 242
VI.1 Introduction......Page 244
VI.2 Risk sensitivity......Page 245
VI.3 Logarithmic transformations for Markov di usions......Page 247
VI.4 Auxiliary stochastic control problem......Page 252
VI.5 Bounded region......Page 255
VI.6 Small noise limits......Page 256
VI.7 H-infinty norm of a nonlinear system......Page 262
VI.8 Risk sensitive control......Page 267
VI.9 Logarithmic transformations for Markov processes......Page 272
VI.10 Historical remarks......Page 276
VII.1 Introduction......Page 278
VII.2 Examples......Page 280
VII.3 Barles and Perthame procedure......Page 282
VII.4 Discontinuous viscosity solutions......Page 283
VII.5 Terminal condition......Page 286
VII.6 Boundary condition......Page 288
VII.7 Convergence......Page 289
VII.8 Comparison......Page 290
VII.9 Vanishing viscosity......Page 297
VII.10 Large deviations for exit probabilities......Page 299
VII.11 Weak comparison principle in......Page 307
VII.12 Historical remarks......Page 309
VIII.1 Introduction......Page 310
VIII.2 Formal discussion......Page 311
VIII.3 Singular stochastic control......Page 313
VIII.4 Verifiction theorem......Page 316
VIII.5 Viscosity solutions......Page 328
VIII.6 Finite fuel problem......Page 334
VIII.7 Historical remarks......Page 336
IX.1 Introduction......Page 338
IX.2 Controlled discrete time Markov chains......Page 339
IX.3 Finite di erence approximations to HJB equations......Page 341
IX.4 Convergence of finte di erence approximations I......Page 348
IX.5 Convergence of finte di erence approximations. II......Page 353
IX.6 Historical remarks......Page 363
X.2 Financial market model......Page 364
X.3 Merton portfolio problem......Page 365
X.4 General utility and duality......Page 366
X.5 Portfolio selection with transaction costs......Page 371
X.6 Derivatives and the Black-Scholes price......Page 377
X.7 Utility pricing......Page 379
X.8 Super-replication with portfolio constraints......Page 381
X.9 Buyer’s price and the no-arbitrage interval......Page 382
X.10 Portfolio constraints and duality......Page 383
X.11 Merton problem with random parameters......Page 385
X.12 Historical remarks......Page 389
XI.1 Introduction......Page 392
XI.2 Static games......Page 393
XI.3 Di erential game formulation......Page 394
XI.4 Upper and lower value functions......Page 398
XI.5 Dynamic programming principle......Page 399
XI.6 Value functions as viscosity solutions......Page 401
XI.7 Risk sensitive control limit game......Page 404
XI.8 Time discretizations......Page 407
XI.9 Strictly progressive strategies......Page 409
XI.10 Historical remarks......Page 412
A Duality Relationships......Page 414
B Dynkin’s Formula for Random Evolutions with Markov Chain Parameters......Page 416
C Extension of Lipschitz Continuous Functions; Smoothing......Page 418
D Stochastic Di erential Equations: Random Coeÿcients......Page 420
References......Page 426
Index......Page 442