Applied Reinforcement Learning with Python. With OpenAI Gym, Tensorflow and Keras

Contents......Page 3
Introduction......Page 7
1 Intro to Reinforcement Learning......Page 8
History of Reinforcement Learning......Page 9
MDPs & their Relation to RL......Page 10
RL Algorithms & RL Frameworks......Page 14
Q Learning......Page 17
Applications of Reinforcement Learning......Page 19
Sonic the Hedgehog......Page 23
Conclusion......Page 24
OpenAI Gym......Page 25
Policy-based Learning......Page 26
Policy Gradients Explained Mathematically......Page 28
Gradient Ascent applied to Policy Optimization......Page 30
Using Vanilla Policy Gradients on the Cart Pole Problem......Page 31
What Are Discounted Rewards and Why Do We Use Them?......Page 35
Drawbacks to Policy Gradients......Page 42
Proximal Policy Optimization (PPO) and Actor-Critic Models......Page 43
Implementing PPO & Solving Super Mario Bros.......Page 44
Working with a More Difficult Reinforcement Learning Challenge......Page 53
Dockerizing Reinforcement Learning Experiments......Page 56
Results of the Experiment......Page 58
Conclusion......Page 59
Q Learning......Page 60
Temporal Difference (TD) Learning......Page 62
Epsilon-Greedy Algorithm......Page 64
Frozen Lake solved with Q Learning......Page 65
Deep Q Learning......Page 70
Playing Doom with Deep Q Learning......Page 71
Training & Performance......Page 78
Double Q Learning & Double Deep Q Networks......Page 79
Conclusion......Page 80
What is Market Making?......Page 82
Trading Gym......Page 86
Why RL for this Problem......Page 87
Synthesizing Order Book Data with Trading Gym......Page 89
Generating Order Book Data with Trading Gym......Page 90
Experimental Design......Page 92
RL Approach 1 Policy Gradients......Page 95
RL Approach 2 Deep Q Network......Page 96
Results & Discussion......Page 98
Conclusion......Page 99
Overview of Sonic the Hedgehog......Page 100
Downloading the Game......Page 101
Writing the Code for the Environment......Page 103
A3C Actor-Critic......Page 108
Conclusion......Page 116
Market Making Model Utilities......Page 118
Policy Gradient Utilities......Page 120
Models......Page 121
Cart Pole Example......Page 130
Super Mario Example......Page 135
Frozen Lake Example......Page 139
Doom Example......Page 144
Market Making Example......Page 151
Sonic Example......Page 163
Index......Page 169