Fundamentals of Deep Learning

Cover
Copyright
Table of Contents
Preface
	Prerequisites and Objectives
	How Is This Book Organized?
	Conventions Used in This Book
	Using Code Examples
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgements
		Nithin and Nikhil
		Joe
Chapter 1. Fundamentals of Linear Algebra for Deep Learning
	Data Structures and Operations
		Matrix Operations
		Vector Operations
		Matrix-Vector Multiplication
	The Fundamental Spaces
		The Column Space
		The Null Space
	Eigenvectors and Eigenvalues
	Summary
Chapter 2. Fundamentals of Probability
	Events and Probability
	Conditional Probability
	Random Variables
	Expectation
	Variance
	Bayes’ Theorem
	Entropy, Cross Entropy, and KL Divergence
	Continuous Probability Distributions
	Summary
Chapter 3. The Neural Network
	Building Intelligent Machines
	The Limits of Traditional Computer Programs
	The Mechanics of Machine Learning
	The Neuron
	Expressing Linear Perceptrons as Neurons
	Feed-Forward Neural Networks
	Linear Neurons and Their Limitations
	Sigmoid, Tanh, and ReLU Neurons
	Softmax Output Layers
	Summary
Chapter 4. Training Feed-Forward Neural Networks
	The Fast-Food Problem
	Gradient Descent
	The Delta Rule and Learning Rates
	Gradient Descent with Sigmoidal Neurons
	The Backpropagation Algorithm
	Stochastic and Minibatch Gradient Descent
	Test Sets, Validation Sets, and Overfitting
	Preventing Overfitting in Deep Neural Networks
	Summary
Chapter 5. Implementing Neural Networks in PyTorch
	Introduction to PyTorch
	Installing PyTorch
	PyTorch Tensors
		Tensor Init
		Tensor Attributes
		Tensor Operations
	Gradients in PyTorch
	The PyTorch nn Module
	PyTorch Datasets and Dataloaders
	Building the MNIST Classifier in PyTorch
	Summary
Chapter 6. Beyond Gradient Descent
	The Challenges with Gradient Descent
	Local Minima in the Error Surfaces of Deep Networks
	Model Identifiability
	How Pesky Are Spurious Local Minima in Deep Networks?
	Flat Regions in the Error Surface
	When the Gradient Points in the Wrong Direction
	Momentum-Based Optimization
	A Brief View of Second-Order Methods
	Learning Rate Adaptation
		AdaGrad—Accumulating Historical Gradients
		RMSProp—Exponentially Weighted Moving Average of Gradients
		Adam—Combining Momentum and RMSProp
	The Philosophy Behind Optimizer Selection
	Summary
Chapter 7. Convolutional Neural Networks
	Neurons in Human Vision
	The Shortcomings of Feature Selection
	Vanilla Deep Neural Networks Don’t Scale
	Filters and Feature Maps
	Full Description of the Convolutional Layer
	Max Pooling
	Full Architectural Description of Convolution Networks
	Closing the Loop on MNIST with Convolutional Networks
	Image Preprocessing Pipelines Enable More Robust Models
	Accelerating Training with Batch Normalization
	Group Normalization for Memory Constrained Learning Tasks
	Building a Convolutional Network for CIFAR-10
	Visualizing Learning in Convolutional Networks
	Residual Learning and Skip Connections for Very Deep Networks
	Building a Residual Network with Superhuman Vision
	Leveraging Convolutional Filters to Replicate Artistic Styles
	Learning Convolutional Filters for Other Problem Domains
	Summary
Chapter 8. Embedding and Representation Learning
	Learning Lower-Dimensional Representations
	Principal Component Analysis
	Motivating the Autoencoder Architecture
	Implementing an Autoencoder in PyTorch
	Denoising to Force Robust Representations
	Sparsity in Autoencoders
	When Context Is More Informative than the Input Vector
	The Word2Vec Framework
	Implementing the Skip-Gram Architecture
	Summary
Chapter 9. Models for Sequence Analysis
	Analyzing Variable-Length Inputs
	Tackling seq2seq with Neural N-Grams
	Implementing a Part-of-Speech Tagger
	Dependency Parsing and SyntaxNet
	Beam Search and Global Normalization
	A Case for Stateful Deep Learning Models
	Recurrent Neural Networks
	The Challenges with Vanishing Gradients
	Long Short-Term Memory Units
	PyTorch Primitives for RNN Models
	Implementing a Sentiment Analysis Model
	Solving seq2seq Tasks with Recurrent Neural Networks
	Augmenting Recurrent Networks with Attention
	Dissecting a Neural Translation Network
	Self-Attention and Transformers
	Summary
Chapter 10. Generative Models
	Generative Adversarial Networks
	Variational Autoencoders
	Implementing a VAE
	Score-Based Generative Models
	Denoising Autoencoders and Score Matching
	Summary
Chapter 11. Methods in Interpretability
	Overview
	Decision Trees and Tree-Based Algorithms
	Linear Regression
	Methods for Evaluating Feature Importance
		Permutation Feature Importance
		Partial Dependence Plots
	Extractive Rationalization
	LIME
	SHAP
	Summary
Chapter 12. Memory Augmented Neural Networks
	Neural Turing Machines
	Attention-Based Memory Access
	NTM Memory Addressing Mechanisms
	Differentiable Neural Computers
	Interference-Free Writing in DNCs
	DNC Memory Reuse
	Temporal Linking of DNC Writes
	Understanding the DNC Read Head
	The DNC Controller Network
	Visualizing the DNC in Action
	Implementing the DNC in PyTorch
	Teaching a DNC to Read and Comprehend
	Summary
Chapter 13. Deep Reinforcement Learning
	Deep Reinforcement Learning Masters Atari Games
	What Is Reinforcement Learning?
	Markov Decision Processes
		Policy
		Future Return
		Discounted Future Return
	Explore Versus Exploit
		ϵ-Greedy
		Annealed   ϵ-Greedy
	Policy Versus Value Learning
	Pole-Cart with Policy Gradients
		OpenAI Gym
		Creating an Agent
		Building the Model and Optimizer
		Sampling Actions
		Keeping Track of History
		Policy Gradient Main Function
		PGAgent Performance on Pole-Cart
	Trust-Region Policy Optimization
	Proximal Policy Optimization
	Q-Learning and Deep Q-Networks
		The Bellman Equation
		Issues with Value Iteration
		Approximating the Q-Function
		Deep Q-Network
		Training DQN
		Learning Stability
		Target Q-Network
		Experience Replay
		From Q-Function to Policy
		DQN and the Markov Assumption
		DQN’s Solution to the Markov Assumption
		Playing Breakout with DQN
		Building Our Architecture
		Stacking Frames
		Setting Up Training Operations
		Updating Our Target Q-Network
		Implementing Experience Replay
		DQN Main Loop
		DQNAgent Results on Breakout
	Improving and Moving Beyond DQN
		Deep Recurrent Q-Networks
		Asynchronous Advantage Actor-Critic Agent
		UNsupervised REinforcement and Auxiliary Learning
	Summary
Index
About the Authors
Colophon