Adaptive dynamic programming: single and multiple controllers

Preface......Page 6
Contents......Page 8
About the Authors......Page 13
Symbols......Page 15
  1.1.1 Continuous-Time LQR......Page 16
  1.1.2 Discrete-Time LQR......Page 17
 1.2 Adaptive Dynamic Programming......Page 18
 1.3 Review of Matrix Algebra......Page 20
 References......Page 21
 2.1 Introduction......Page 22
 2.3 The Data-Based Identifier......Page 24
 2.4 Derivation of the Iterative ADP Algorithm  with Convergence Analysis......Page 26
 2.5 Neural Network Implementation of the Iterative Control Algorithm......Page 32
 2.6 Simulation Study......Page 33
 2.7 Conclusions......Page 35
 References......Page 37
 3.1 Introduction......Page 39
 3.2 Problem Statement......Page 40
  3.3.1 The Novel ADP Iteration Algorithm......Page 44
  3.3.2 Convergence Analysis of the Improved Iteration Algorithm......Page 47
  3.3.3 Neural Network Implementation of the Iteration ADP Algorithm......Page 52
 3.4 Simulation Study......Page 54
 3.5 Conclusion......Page 59
 References......Page 62
 4.1 Introduction......Page 63
 4.2 Problem Formulation......Page 64
 4.3 Derivation of the ADP Algorithm for Time-Delay Systems......Page 65
 4.4 Neural Network Implementation for the Multi-objective Optimal Control Problem of Time-Delay Systems......Page 68
 4.5 Simulation Results......Page 69
 4.6 Conclusions......Page 75
 References......Page 76
 5.1 Introduction......Page 77
 5.2 Problem Statement......Page 79
 5.3 SIANN Architecture-Based Classification......Page 80
 5.4 Optimal Control Based on ADP......Page 83
  5.4.1 Model Neural Network......Page 84
  5.4.2 Critic Network and Action Network......Page 88
 5.5 Simulation Study......Page 96
 References......Page 105
 6.1 Introduction......Page 108
 6.2 Motivations and Preliminaries......Page 109
  6.3.1 Critic Network......Page 112
  6.3.2 Action Network......Page 114
  6.3.3 Design of the Compensation Controller......Page 115
  6.3.4 Stability Analysis......Page 116
 6.4 Simulation Study......Page 120
 References......Page 123
 7.1 Introduction......Page 125
 7.2 Problem Statement......Page 126
 7.3 Off-Policy Optimal Control Method......Page 127
  7.3.1 Convergence Analysis of Off-Policy PI Algorithm......Page 129
  7.3.2 Implementation Method of Off-Policy Iteration Algorithm......Page 131
 7.4 Simulation Study......Page 134
 References......Page 137
 8.1 Introduction......Page 139
 8.2 Problem Formulation and Preliminaries......Page 140
  8.3.1 Description of Approximation-Error ADP Algorithm......Page 142
  8.3.2 Convergence Analysis of The Iterative ADP Algorithm......Page 144
 8.4 Simulation Study......Page 148
 References......Page 156
 9.1 Introduction......Page 158
 9.2 Problem Statement......Page 159
  9.3.1 On-Policy IRL for Nonzero Disturbance......Page 162
  9.3.2 Off-Policy IRL for Nonzero Disturbance......Page 163
  9.3.3 NN Approximation for Actor-Critic Structure......Page 165
  9.4.1 Disturbance Compensation Off-Policy Controller Design......Page 168
  9.4.2 Stability Analysis......Page 169
 9.5 Simulation Study......Page 172
 References......Page 174
 10.1 Introduction......Page 176
 10.2 Preliminaries and Assumptions......Page 177
  10.3.1 Derivation of The Iterative ADP Method......Page 180
  10.3.2 The Procedure of the Method......Page 185
  10.3.3 The Properties of the Iterative ADP Method......Page 187
 10.4 Neural Network Implementation......Page 201
  10.4.1 The Model Network......Page 202
  10.4.2 The Critic Network......Page 203
  10.4.3 The Action Network......Page 204
 10.5 Simulation Study......Page 206
 References......Page 215
 11.1 Introduction......Page 217
 11.2 Motivations and Preliminaries......Page 218
  11.3.1 Derivation of Off-Policy Algorithm......Page 223
  11.3.2 Implementation Method for Off-Policy Algorithm......Page 224
  11.3.3 Stability Analysis......Page 228
 11.4 Simulation Study......Page 229
 References......Page 234
 12.1 Introduction......Page 236
 12.2 Problem Statement......Page 237
 12.3 Multi-player Learning PI Solution for NZS Games......Page 238
  12.4.1 Derivation of Off-Policy Algorithm......Page 243
  12.4.2 Implementation Method for Off-Policy Algorithm......Page 245
  12.4.3 Stability Analysis......Page 247
 12.5 Simulation Study......Page 251
 References......Page 257
 13.1 Introduction......Page 259
  13.2.1 Graph Theory......Page 260
  13.2.2 Synchronization and Tracking Error Dynamic Systems......Page 261
 13.3 Optimal Distributed Cooperative Control for Multi-agent …......Page 263
  13.3.1 Cooperative Performance Index Function......Page 264
  13.3.2 Nash Equilibrium......Page 265
  13.4.1 Derivation of the Heterogeneous Multi-agent Differential Graphical Games......Page 267
  13.4.2 Properties of the Developed Policy Iteration Algorithm......Page 268
  13.4.3 Heterogeneous Multi-agent Policy Iteration Algorithm......Page 272
 13.5 Simulation Study......Page 273
 References......Page 277
Index......Page 278