Practical Simulations for Machine Learning

Cover
Copyright
Table of Contents
Preface
	Resources Used in This Book
	Audience and Approach
	Organization of This Book
	Using This Book
		Our Tasks
	Conventions Used in This Book
	Using Code Examples
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
Part I. The Basics of Simulation and Synthesis
	Chapter 1. Introducing Synthesis and Simulation
		A Whole New World of ML
		The Domains
			Simulation
			Synthesis
		The Tools
			Unity
			PyTorch via Unity ML-Agents
			Unity ML-Agents Toolkit
			Unity Perception
		The Techniques
			Reinforcement Learning
			Imitation Learning
			Hybrid Learning
			Summary of Techniques
		Projects
			Simulation Projects
			Synthesis Projects
		Summary and Next Steps
	Chapter 2. Creating Your First Simulation
		Everybody Remembers Their First Simulation
		Our Simulation
		Setting Up
		Creating the Unity Project
		Packages All the Way Down
		The Environment
			The Floor
			The Target
		The Agent
			Starting and Stopping the Agent
			Letting the Agent Observe the Environment
			Letting the Agent Take Actions in the Environment
			Giving the Agent Rewards for Its Behavior
			Finishing Touches for the Agent
			Providing a Manual Control System for the Agent
		Training with the Simulation
			Monitoring the Training with TensorBoard
			When the Training Is Complete
		What’s It All Mean?
		Coming Up Next
	Chapter 3. Creating Your First Synthesized Data
		Unity Perception
		The Process
			Using Unity Perception
		Creating the Unity Project
		Creating a Scene
			Getting the Dice Models
			A Very Simple Scene
		Preparing for Synthesis
			Testing the Scenario
			Setting Up Our Labels
			Checking the Labels
		What’s Next?
Part II. Simulating Worlds for Fun and Profit
	Chapter 4. Creating a More Advanced Simulation
		Setting Up the Block Pusher
		Creating the Unity Project
		The Environment
			The Floor
			The Walls
			The Block
			The Goal
			The Agent
			The Environment
		Training and Testing
	Chapter 5. Creating a Self-Driving Car
		Creating the Environment
			The Track
			The Car
			Setting Up for ML
		Training the Simulation
			Training
			When the Training Is Complete
	Chapter 6. Introducing Imitation Learning
		Simulation Environment
			Creating the Ground
			Creating the Goal
			The Name’s Ball, Agent Ball
			The Camera
		Building the Simulation
			Agent Components
			Adding Heuristic Controls
			Observations and Goals
		Generating Data and Training
			Creating Training Data
			Configuring for Training
			Begin Training
			Running with Our Trained Model
		Understanding and Using Imitation Learning
	Chapter 7. Advanced Imitation Learning
		Meet GAIL
		Do What I Say and Do
			A GAIL Scenario
			Modifying the Agent’s Actions
			Modifying the Observations
			Resetting the Agent
			Updating the Agent Properties
			Demonstration Time
		Training with GAIL
		Running It and Beyond
	Chapter 8. Introducing Curriculum Learning
		Curriculum Learning in ML
		A Curriculum Learning Scenario
		Building in Unity
			Creating the Ground
			Creating the Target
			The Agent
		Building the Simulation
			Making the Agent an Agent
			Actions
			Observations
			Heuristic Controls for Humans
		Creating the Curriculum
			Resetting the Environment
			Curriculum Config
		Training
		Running It
		Curriculum Versus Other Approaches
		What’s Next?
	Chapter 9. Cooperative Learning
		A Simulation for Cooperation
			Building the Environment in Unity
			Coding the Agents
			Coding the Environment Manager
			Coding the Blocks
			Finalizing the Environment and Agents
		Training for Cooperation
		Cooperative Agents or One Big Agent
	Chapter 10. Using Cameras in Simulations
		Observations and Camera Sensors
		Building a Camera-Only Agent
			Coding the Camera-Only Agent
			Adding a New Camera for the Agent
			Seeing What the Agent’s Camera Sees
		Training the Camera-Based Agent
		Cameras and You
	Chapter 11. Working with Python
		Python All the Way Down
		Experimenting with an Environment
		What Can Be Done with Python?
			Using Your Own Environment
			Completely Custom Training
		What’s the Point of Python?
	Chapter 12. Under the Hood and Beyond
		Hyperparameters (and Just Parameters)
			Parameters
			Reward Parameters
			Hyperparameters
		Algorithms
		Unity Inference Engine and Integrations
			Using the ML-Agents Gym Wrapper
			Side Channels
Part III. Synthetic Data, Real Results
	Chapter 13. Creating More Advanced Synthesized Data
		Adding Random Elements to the Scene
			Randomizing the Floor Color
			Randomizing the Camera Position
		What’s Next?
	Chapter 14. Synthetic Shopping
		Creating the Unity Environment
		A Perception Camera
		Faking It Until You Make It
		Using Synthesized Data
Index
About the Authors
Colophon