Dynamic Programming

Preface......Page bellman0005_0001.djvu
Contents......Page bellman0017_0001.djvu
1. Introduction......Page bellman0026_0001.djvu
2. A multi-stage allocation process......Page bellman0027_0001.djvu
3. Discussion......Page bellman0028_0001.djvu
4. Functional equation approach......Page bellman0030_0001.djvu
5. Discussion......Page bellman0032_0001.djvu
7. A "smoothing" problem......Page bellman0033_0001.djvu
8. Infinite stage approximation......Page bellman0034_0001.djvu
9. Existence and uniqueness theorems......Page bellman0035_0001.djvu
11. Approximation in policy space......Page bellman0039_0001.djvu
12. Properties of the solution - I: Convexity......Page bellman0042_0001.djvu
13. Properties of the solution - II: Concavity......Page bellman0043_0001.djvu
14. Properties of the solution - III: Concavity......Page bellman0045_0001.djvu
15. An "ornery" example......Page bellman0048_0001.djvu
16. A particular example - I......Page bellman0049_0001.djvu
17. A particular example - II......Page bellman0051_0001.djvu
18. Approximation and stability......Page bellman0052_0001.djvu
19. Time-dependent processes......Page bellman0053_0001.djvu
20. Multi-activity processes......Page bellman0054_0001.djvu
21. Multi-dimensional structure theorems......Page bellman0056_0001.djvu
22. Locating the unique maximum of a concave function......Page bellman0057_0001.djvu
23. Continuity and memory......Page bellman0060_0001.djvu
24. Stohastic allocation processes......Page bellman0061_0001.djvu
25. Functional equations......Page bellman0062_0001.djvu
Exercises and Research Problems for Chapter I......Page bellman0063_0001.djvu
Bibliography and Comments for Chapter I......Page bellman0082_0001.djvu
2. Stohastic gold-mining......Page bellman0084_0001.djvu
3. Enumerative treatment......Page bellman0085_0001.djvu
5. Infinite stage approximation......Page bellman0086_0001.djvu
6. Existence and uniqueness......Page bellman0087_0001.djvu
7. Approximation in policy space and monotone convergence......Page bellman0088_0001.djvu
8. The solution......Page bellman0089_0001.djvu
10. Some generalizations......Page bellman0092_0001.djvu
11. The form of f(x,y)......Page bellman0094_0001.djvu
12. The problem for a finite number of stages......Page bellman0095_0001.djvu
13. A three-choice problem......Page bellman0097_0001.djvu
14. A stability theorem......Page bellman0099_0001.djvu
Exercises and Research Problems for Chapter II......Page bellman0100_0001.djvu
Bibliography and Comments for Chapter II......Page bellman0102_0001.djvu
2. Discussion of the two preceding processes......Page bellman0104_0001.djvu
4. Mathematical formulation - I. A discrete deterministic process......Page bellman0106_0001.djvu
5. Mathematical formulation - II. A discrete stohastic process ......Page bellman0108_0001.djvu
6. Mathematical formulation - III. A continuous deterministic process......Page bellman0109_0001.djvu
9. Casuality and optimality......Page bellman0110_0001.djvu
10. Approximation in policy space......Page bellman0111_0001.djvu
Exercises and Research Problems for Chapter III......Page bellman0113_0001.djvu
Bibliography and Comments for Chapter III......Page bellman0138_0001.djvu
1. Introduction......Page bellman0139_0001.djvu
2. A Fundamental inequality......Page bellman0140_0001.djvu
3. Equations of Type One......Page bellman0142_0001.djvu
4. Equations of Type Two......Page bellman0144_0001.djvu
5. Monotone convergence......Page bellman0145_0001.djvu
6. Stability theorems......Page bellman0146_0001.djvu
7. Some directions of generalization......Page bellman0147_0001.djvu
8. An equation of the third type......Page bellman0148_0001.djvu
9. An "optimal inventory" equation......Page bellman0152_0001.djvu
Exercises and Research Problems for Chapter IV......Page bellman0155_0001.djvu
Bibliography and Comments for Chapter IV......Page bellman0174_0001.djvu
1. Introduction......Page bellman0175_0001.djvu
2. Formulation of the general problem......Page bellman0176_0001.djvu
A. Finite total time period......Page bellman0177_0001.djvu
D. Unbounded time period - one period lag in supply......Page bellman0179_0001.djvu
3. A simple observation......Page bellman0180_0001.djvu
4. Constant stock level - preliminary discussion......Page bellman0181_0001.djvu
5. Proportional cost - one-dimensional case......Page bellman0182_0001.djvu
6. Proportional cost - multi-dimensional case......Page bellman0187_0001.djvu
7. Finite time period......Page bellman0189_0001.djvu
9. Non-proportional penalty cost - red tape......Page bellman0192_0001.djvu
11. The form of the general solution......Page bellman0194_0001.djvu
12. Fixed costs......Page bellman0195_0001.djvu
14. Unbounded process - one period time lag......Page bellman0196_0001.djvu
15. Convex cost function - unbounded process......Page bellman0199_0001.djvu
Appendix Chapter V - The Renewal Equation......Page bellman0200_0001.djvu
Exercises and Research Problems for Chapter V......Page bellman0201_0001.djvu
Bibliography and Comments for Chapter V......Page bellman0205_0001.djvu
1. Introduction......Page bellman0206_0001.djvu
2. A General class of multi-stage production problems......Page bellman0207_0001.djvu
3. Discussion of the preceding model......Page bellman0210_0001.djvu
4. Functional equations......Page bellman0211_0001.djvu
5. A Continuous version......Page bellman0212_0001.djvu
6. Notation......Page bellman0214_0001.djvu
8. The basic functional equation......Page bellman0215_0001.djvu
10. Application of the partial differential equation......Page bellman0216_0001.djvu
11. A Particular example......Page bellman0217_0001.djvu
12. A Dual problem......Page bellman0220_0001.djvu
13. Verification of the solution given in 11......Page bellman0223_0001.djvu
14. Computational solution......Page bellman0225_0001.djvu
15. Nonlinear problems ......Page bellman0226_0001.djvu
Exercises and Research Problems for Chapter VI......Page bellman0227_0001.djvu
Bibliography and Comments for Chapter VI......Page bellman0228_0001.djvu
1. Introduction......Page bellman0230_0001.djvu
2. Preliminaries......Page bellman0232_0001.djvu
4. The solution......Page bellman0234_0001.djvu
5. The modified w solution......Page bellman0237_0001.djvu
6. The equilibrium solution......Page bellman0238_0001.djvu
7. A short-time w solution......Page bellman0240_0001.djvu
8. Description of solution and proof......Page bellman0241_0001.djvu
Bibliography and Comments for Chapter VII......Page bellman0244_0001.djvu
1. Introduction......Page bellman0245_0001.djvu
2. Continuous versions - I: A differential approach......Page bellman0246_0001.djvu
4. Preliminary discussion......Page bellman0247_0001.djvu
5. Mixing at a point......Page bellman0249_0001.djvu
7. Derivation of the differential equations......Page bellman0250_0001.djvu
8. The variational procedure......Page bellman0251_0001.djvu
9. The behavior of Ki.......Page bellman0252_0001.djvu
10. The solution for T=infinity......Page bellman0253_0001.djvu
11. Solution for finite total time......Page bellman0254_0001.djvu
12. The three-choice problem......Page bellman0256_0001.djvu
13. Some lemmas and preliminary results......Page bellman0257_0001.djvu
14. Mixed policies......Page bellman0258_0001.djvu
15. The solution for infinite time, D>0......Page bellman0259_0001.djvu
16. D<0......Page bellman0263_0001.djvu
17. The case r3=r4......Page bellman0264_0001.djvu
18. Nonlinear utility - two choice problem......Page bellman0265_0001.djvu
Bibliography and Comments for Chapter VIII......Page bellman0267_0001.djvu
1. Introduction......Page bellman0268_0001.djvu
2. A new approach......Page bellman0269_0001.djvu
3. Max_y \int_0^{\infty} F(x,y)dt......Page bellman0271_0001.djvu
5. The two dimensional case......Page bellman0274_0001.djvu
6. Max_y \int_0^T F(x,y)dt......Page bellman0275_0001.djvu
7. Max_y \int_0^T F(x,y)dt  under the Constraint 0<=y<=x......Page bellman0276_0001.djvu
8. Computational solution......Page bellman0277_0001.djvu
9. Discussion......Page bellman0278_0001.djvu
10. An example......Page bellman0279_0001.djvu
11. A discrete version......Page bellman0281_0001.djvu
12. A convergence proof......Page bellman0283_0001.djvu
13. Max_y \int_0^T F(x,y,t)dt......Page bellman0286_0001.djvu
14. Generalization and discussion......Page bellman0287_0001.djvu
15. Integral constraints......Page bellman0288_0001.djvu
16. Further remarks concerning numerical solution......Page bellman0289_0001.djvu
17. Eigenvalue problem......Page bellman0290_0001.djvu
18. The first formulation......Page bellman0291_0001.djvu
19. An approximate solution......Page bellman0292_0001.djvu
21. Discrete approximations......Page bellman0293_0001.djvu
22. Successive approximation......Page bellman0294_0001.djvu
23. Monotone approximation......Page bellman0295_0001.djvu
25. Minimum maximum deviation......Page bellman0296_0001.djvu
Exercises and Research Problems for Chapter IX......Page bellman0297_0001.djvu
Bibliography and Comments for Chapter IX......Page bellman0305_0001.djvu
1. Introduction......Page bellman0306_0001.djvu
2. A Single-stage discrete game......Page bellman0307_0001.djvu
3. The min-max theorem......Page bellman0308_0001.djvu
4. Continous games......Page bellman0309_0001.djvu
6. Games of survival......Page bellman0310_0001.djvu
7. Pursuit games......Page bellman0311_0001.djvu
8. General formulation......Page bellman0312_0001.djvu
9. The principle of optimality and functional equations......Page bellman0314_0001.djvu
10. More general process......Page bellman0315_0001.djvu
11. A basic lemma......Page bellman0317_0001.djvu
12. Existence and uniqueness......Page bellman0318_0001.djvu
13. Proof of results......Page bellman0320_0001.djvu
14. Alternate proof of existence......Page bellman0322_0001.djvu
16. Effectiveness of solution......Page bellman0323_0001.djvu
18. One-side min-max......Page bellman0325_0001.djvu
19. Existence and uniqueness for games of survival......Page bellman0326_0001.djvu
20. An approximation......Page bellman0329_0001.djvu
21. Non-zero-sum games - games of survival......Page bellman0330_0001.djvu
23. Proof of the extended min-max theorem......Page bellman0331_0001.djvu
24. A rationale for non-zero sum games......Page bellman0332_0001.djvu
Exercises and Research Problems for Chapter X......Page bellman0333_0001.djvu
Bibliography and Comments for Chapter X......Page bellman0338_0001.djvu
2. Markovian decision processes......Page bellman0340_0001.djvu
3. Notation......Page bellman0342_0001.djvu
4. A lemma......Page bellman0343_0001.djvu
5. Existence and uniqueness - I......Page bellman0344_0001.djvu
6. Existence and uniqueness - II......Page bellman0347_0001.djvu
8. The Riccati equation......Page bellman0348_0001.djvu
9. Approximation in policy space......Page bellman0349_0001.djvu
10. Discrete versions......Page bellman0351_0001.djvu
11. The recurrence relation......Page bellman0353_0001.djvu
13. Generalization of a von Neumann result......Page bellman0355_0001.djvu
Exercises and Research Problems for Chapter XI......Page bellman0357_0001.djvu
Bibliography and Comments for Chapter XI......Page bellman0359_0001.djvu
Index of applications......Page bellman0361_0001.djvu
Name and subject index......Page bellman0362_0001.djvu