Reinforcement Learning: Optimal Feedback Control with Industrial Applications (Advances in Industrial Control)

Series Editor’s Foreword
Preface
Acknowledgements
Contents
Abbreviations
1 Background on Reinforcement Learning and Optimal Control
	1.1 Fundamentals of Reinforcement Learning and Recall
	1.2 Fundamentals of Optimal Control with Dynamic Programming
	1.3 Architecture and Performance of Networked Control System
	1.4 The State of the Art and Contributions
	References
2 Hinfty Control Using Reinforcement Learning
	2.1 Hinfty State Feedback Control of Multi-player Systems
		2.1.1 Problem Statement
		2.1.2 Solving the Multi-player Zero-Sum Game
		2.1.3 Off-Policy Game Q-Learning Technique
		2.1.4 Simulation Results
	2.2 Hinfty Output Feedback Control of Multi-player Systems
		2.2.1 Problem Statement
		2.2.2 Solving the Multi-player Zero-Sum Game
		2.2.3 Off-Policy Game Q-Learning Technique
		2.2.4 Simulation Results
	2.3 Conclusion
	References
3 Robust Tracking Control and Output Regulation
	3.1 Optimal Robust Control Problem Statement
	3.2 Theoretical Solutions
		3.2.1 Solving the Regulator Equations with Known Dynamics
		3.2.2 Solving Problem 3.2 with Known Dynamics
	3.3 Data-Driven Solutions
		3.3.1 Data-Driven OPCG Q-Learning
		3.3.2 No Bias Analysis of the Solution for the Proposed Algorithm
	3.4 Illustrative Examples
	3.5 Conclusion
	References
4 Interleaved Robust Reinforcement Learning
	4.1 Robust Controller Design and Simplified HJB Equation
	4.2 Interleaved RL for Approximating Robust Control
		4.2.1 Theoretical Analysis
	4.3 Illustrative Examples
	4.4 Conclusion
	References
5 Optimal Networked Controller and Observer Design
	5.1 Off-Policy Q-Learning for Single-Player Networked Control Systems
		5.1.1 Problem Formulation
		5.1.2 Optimal Observer Design
		5.1.3 Optimal Controller Design
		5.1.4 Simulation Results
	5.2 Off-Policy Q-Learning for Multi-player Networked Control Systems
		5.2.1 Problem Formulation
		5.2.2 Main Results
		5.2.3 Illustrative Example
	5.3 Conclusion
	References
6 Interleaved Q-Learning
	6.1 Optimal Control for Affine Nonlinear Systems
		6.1.1 Problem Statement
		6.1.2 On-Policy Q-Learning Formulation
	6.2 Off-Policy Q-Learning Technique
		6.2.1 Off-Policy and Q-Learning
		6.2.2 Derivation of Off-Policy Q-Learning Algorithm
		6.2.3 No Bias of Off-Policy Q-Learning Algorithm
	6.3 Neural Network-Based Off-Policy Interleaved Q-Learning
		6.3.1 Model Neural Network
		6.3.2 Actor Neural Network
		6.3.3 Critic Neural Network
		6.3.4 Interleaved Q-Learning
		6.3.5 Optimal Control for Linear Systems
	6.4 Illustrative Examples
	6.5 Conclusion
	References
7 Off-Policy Game Reinforcement Learning
	7.1 Graphical Game for Optimal Synchronization
		7.1.1 Preliminaries
		7.1.2 Multi-agent Graphical Games
		7.1.3 Off-Policy Reinforcement Learning Algorithm
		7.1.4 Simulation Examples
	7.2 Nonzero-Sum Game for Cooperative Optimization
		7.2.1 Problem Statement
		7.2.2 Solving the Nonzero-Sum Game Problems
		7.2.3 Finding Ki* (i=1,2,…,n) by the On-Policy Approach
		7.2.4 Finding Ki* (i=1,2,…,n) by the Off-Policy Approach
		7.2.5 Simulation Results
	7.3 Conclusion
	References
8 Industrial Applications of Game Reinforcement Learning
	8.1 Rougher Flotation Processes and Plant-Wide Optimization Control
	8.2 Optimal Operational Control for Industrial Process
		8.2.1 Problem Statement
		8.2.2 Hinfty Tracking Control for Operational Processes
		8.2.3 Solving the Hinfty Tracking Control Problem
		8.2.4 Off-Policy Reinforcement Learning Algorithm
		8.2.5 Simulation Results
	8.3 Optimal Set-Point Design for Rougher Flotation Processes with Multiple Cells
		8.3.1 Problem Statement
		8.3.2 Hinfty Tracking Control for Rougher Flotation Processes
		8.3.3 On-Policy Q-Learning Based on Zero-Sum Game
		8.3.4 Off-Policy Q-Learning Algorithm
		8.3.5 Optimum Set-Point Selector
		8.3.6 Simulation Results
	8.4 Plant-Wide Optimization Using Game Reinforcement Learning
		8.4.1 Problem Statement
		8.4.2 Nonzero-Sum Graphical Game for Solving Multi-objective Optimization Problem
		8.4.3 Model Free Solution to Nonzero-Sum Graphical Game
		8.4.4 Industrial Application in Iron Ore Processing
	8.5 Conclusion
	References
Appendix  Appendix A
A.1 Matrix Calculus
A.2 Some Notation
Index