Stochastic Control for Econometric Models

Preface......Page 11
Introduction......Page 14
I Deterministic Control......Page 16
Quadratic Linear Problems......Page 17
Problem Statement......Page 19
Solution Method......Page 24
General Nonlinear Models......Page 32
Problem Statement......Page 34
Quadratic Linear Approximation Method......Page 35
Gradient Methods......Page 39
Special Problems......Page 41
Large Model Size......Page 42
Example of Deterministic Control......Page 43
System Equations......Page 44
The Criterion Function......Page 50
II Passive-Learning Stochastic Control......Page 51
Additive Uncertainty......Page 52
Uncertainty in economic problems......Page 53
Methods of Modeling Uncertainty......Page 54
Learning: Passive and Active......Page 55
Additive Error Terms......Page 58
Multiplicative Uncertainty......Page 59
Statement of the Problem......Page 60
Period N......Page 63
Period N-1......Page 65
Period k......Page 69
Expected Values of Matrix Products......Page 70
Example of Passive-Learning Control......Page 71
The Problem......Page 72
The Optimal Control for Period 0......Page 73
Projections of Means and Covariances to Period 1......Page 78
III Active-Learning Stochastic Control......Page 82
Overview......Page 84
Problem Statement......Page 87
The Monte Carlo Procedure......Page 90
Search for the Optimal Control in Period k......Page 91
The Update......Page 95
Other Algorithms......Page 96
Nonlinear Active-Learning Control......Page 97
Problem Statement......Page 98
Computing the Approximate Cost-to-Go......Page 100
Obtaining a Deterministic Approximation for the Cost-to-Go......Page 110
Projection of Covariance Matrices......Page 112
Summary of the Search for the Optimal Control in Period k......Page 116
Quadratic Linear Active-Learning Control......Page 117
Original System......Page 118
Augmented System......Page 120
The Approximate Optimal Cost-to-Go......Page 121
Dual-Control Algorithm......Page 127
Search for the Optimal Control......Page 130
Updating State and Parameter Estimates......Page 133
Example: The MacRae Problem......Page 134
Problem Statement: MacRae Problem......Page 135
Initialization......Page 137
Search for Optimal Control......Page 138
The Search......Page 144
Example: Model with Measurement Error......Page 147
Introduction......Page 148
The Model and Data......Page 149
Adaptive Versus Certainty-Equivalence Policies......Page 153
Results from a Single Monte Carlo Run......Page 154
Time Paths of Variables and of Parameter Estimates......Page 156
Decomposition of the Cost-to-Go......Page 169
Summary......Page 179
Second-Order Expansion of System Eqs......Page 180
Expected Value of Matrix Products......Page 183
The Expected Value of a Quadratic Form......Page 184
The Expected Value of a Matrix Triple Product......Page 185
Equivalence of Matrix Riccati Recursions......Page 186
Second-Order Kalman Filter......Page 190
Alternate Forms of Cost-to-Go Expression......Page 198
Expectation of Prod of Quadratic Forms......Page 201
Fourth Moment of Normal Distribution: Scalar Case......Page 203
Fourth Moment of Normal Distribution: Vector Case......Page 204
Proof......Page 214
Certainty-Equivalence Cost-To-Go Problem......Page 217
Matrix Recursions for Augmented System......Page 220
Vector Recursions for Augmented System......Page 230
Proof That Term in Cost-To-Go is Zero......Page 235
Updating the Augmented State Covariance......Page 238
Deriv of Sys Equations wrt Parameters......Page 242
Projection of the Augmented State Vector......Page 246
Updating the Augmented State Vector......Page 250
Sequential Certainty-Equiv Method......Page 252
The Reestimation Method......Page 254
Components of the Cost-to-Go......Page 256
The Measurement-Error Covariance......Page 259
Data for Deterministic Problem......Page 263
Solution to Measurement Error Model......Page 267
Random Elements......Page 268
Results......Page 271
Changes in the Second Edition......Page 277
Bibliography......Page 279