Reinforcement Learning with Hybrid Quantum Approximation in the NISQ Context

Abstract
Acknowledgements
Contents
List of Figures
List of Tables
1 Motivation: Complex Attacker-Defender Scenarios—The Eternal Conflict
	1.1 Reinforcement Learning and Attacker-Defender Scenarios
		1.1.1 Today's Challenges in Reinforcement Learning and More Complex Attacker-Defender Scenarios
	1.2 Quantum Computing—An Opportunity for Reinforcement Learning in Complex Attacker-Defender Scenarios
	1.3 Specifying the Research Questions
2 The Information Game—A special Attacker-Defender Scenario
	2.1 The 2D Game
	2.2 High Complexity 3D Game
3 Reinforcement Learning and Bellman's Principle of Optimality
	3.1 A Short Mathematical Introduction to Reinforcement Learning
		3.1.1 Value Functions and Bellman's Principle of Optimality
	3.2 From Bellman to Q-learning—The Tabular Approach
	3.3 Approximation Techniques in Reinforcement Learning
		3.3.1 DQN—Advanced Value Approximation
		3.3.2 Policy Gradient—Policy Approximation
	3.4 Policy vs. Value-based Methods in Attacker-Defender Scenarios
4 Quantum Reinforcement Learning—Connecting Reinforcement Learning and Quantum Computing
	4.1 Quantum Reinforcement Learning Methods
	4.2 Projective Simulation Methods
	4.3 Quantum Hybrid Approximation Methods
	4.4 Defining the Research Questions
5 Approximation in Quantum Computing
	5.1 Quantum Variational Circuits—A Quantum Approximator
6 Advanced Quantum Policy Approximation in Policy Gradient Reinforcement Learning
	6.1 Central idea: Quantum Variational Circuits' Components
		6.1.1 Parameter Encoding
		6.1.2 Variational Form
		6.1.3 Post Processing
	6.2 Experimental Framework & Hyper-Parameter Optimization
7 Applying Quantum REINFORCE to the Information Game
	7.1 Experimental Set-Up: Optimal Parameter Configuration
	7.2 Results
		7.2.1 The Simple Problem—A First Approach
		7.2.2 Enlarging the State Space—The 2D8 Game
		7.2.3 High Complexity with the 3D Game
	7.3 Discussion
8 Evaluating Quantum REINFORCE on IBM's Quantum Hardware
	8.1 Evaluating the Trained Algorithm
	8.2 The Full Training Loop on the Quantum Hardware
	8.3 Increasing the Level of Detail
	8.4 Summary and Answering the Research Question
9 Future Steps in Quantum Reinforcement Learning for Complex Scenarios
	9.1 Characteristics of NISQ devices
	9.2 Improved Data Encoding
	9.3 Analysis of Quantum Variational Circuits in Quantum Policy Gradient methods
10 Conclusion
A Bibliography