Tensorflow for Deep Learning

Cover
Copyright
Table of Contents
Preface
	Conventions Used in This Book
	Using Code Examples
	O’Reilly Safari
	How to Contact Us
	Acknowledgments
Chapter 1. Introduction to Deep Learning
	Machine Learning Eats Computer Science
	Deep Learning Primitives
		Fully Connected Layer
		Convolutional Layer
		Recurrent Neural Network Layers
		Long Short-Term Memory Cells
	Deep Learning Architectures
		LeNet
		AlexNet
		ResNet
		Neural Captioning Model
		Google Neural Machine Translation
		One-Shot Models
		AlphaGo
		Generative Adversarial Networks
		Neural Turing Machines
	Deep Learning Frameworks
		Limitations of TensorFlow
	Review
Chapter 2. Introduction to TensorFlow Primitives
	Introducing Tensors
		Scalars, Vectors, and Matrices
		Matrix Mathematics
		Tensors
		Tensors in Physics
		Mathematical Asides
	Basic Computations in TensorFlow
		Installing TensorFlow and Getting Started
		Initializing Constant Tensors
		Sampling Random Tensors
		Tensor Addition and Scaling
		Matrix Operations
		Tensor Types
		Tensor Shape Manipulations
		Introduction to Broadcasting
	Imperative and Declarative Programming
		TensorFlow Graphs
		TensorFlow Sessions
		TensorFlow Variables
	Review
Chapter 3. Linear and Logistic Regression with TensorFlow
	Mathematical Review
		Functions and Differentiability
		Loss Functions
		Gradient Descent
		Automatic Differentiation Systems
	Learning with TensorFlow
		Creating Toy Datasets
		New TensorFlow Concepts
	Training Linear and Logistic Models in TensorFlow
		Linear Regression in TensorFlow
		Logistic Regression in TensorFlow
	Review
Chapter 4. Fully Connected Deep Networks
	What Is a Fully Connected Deep Network?
	“Neurons” in Fully Connected Networks
		Learning Fully Connected Networks with Backpropagation
		Universal Convergence Theorem
		Why Deep Networks?
	Training Fully Connected Neural Networks
		Learnable Representations
		Activations
		Fully Connected Networks Memorize
		Regularization
		Training Fully Connected Networks
	Implementation in TensorFlow
		Installing DeepChem
		Tox21 Dataset
		Accepting Minibatches of Placeholders
		Implementing a Hidden Layer
		Adding Dropout to a Hidden Layer
		Implementing Minibatching
		Evaluating Model Accuracy
		Using TensorBoard to Track Model Convergence
	Review
Chapter 5. Hyperparameter Optimization
	Model Evaluation and Hyperparameter Optimization
	Metrics, Metrics, Metrics
		Binary Classification Metrics
		Multiclass Classification Metrics
		Regression Metrics
	Hyperparameter Optimization Algorithms
		Setting Up a Baseline
		Graduate Student Descent
		Grid Search
		Random Hyperparameter Search
		Challenge for the Reader
	Review
Chapter 6. Convolutional Neural Networks
	Introduction to Convolutional Architectures
		Local Receptive Fields
		Convolutional Kernels
		Pooling Layers
		Constructing Convolutional Networks
		Dilated Convolutions
	Applications of Convolutional Networks
		Object Detection and Localization
		Image Segmentation
		Graph Convolutions
		Generating Images with Variational Autoencoders
	Training a Convolutional Network in TensorFlow
		The MNIST Dataset
		Loading MNIST
		TensorFlow Convolutional Primitives
		The Convolutional Architecture
		Evaluating Trained Models
		Challenge for the Reader
	Review
Chapter 7. Recurrent Neural Networks
	Overview of Recurrent Architectures
	Recurrent Cells
		Long Short-Term Memory (LSTM)
		Gated Recurrent Units (GRU)
	Applications of Recurrent Models
		Sampling from Recurrent Networks
		Seq2seq Models
	Neural Turing Machines
	Working with Recurrent Neural Networks in Practice
	Processing the Penn Treebank Corpus
		Code for Preprocessing
		Loading Data into TensorFlow
		The Basic Recurrent Architecture
		Challenge for the Reader
	Review
Chapter 8. Reinforcement Learning
	Markov Decision Processes
	Reinforcement Learning Algorithms
		Q-Learning
		Policy Learning
		Asynchronous Training
	Limits of Reinforcement Learning
	Playing Tic-Tac-Toe
		Object Orientation
		Abstract Environment
		Tic-Tac-Toe Environment
		The Layer Abstraction
		Defining a Graph of Layers
	The A3C Algorithm
		The A3C Loss Function
		Defining Workers
		Training the Policy
		Challenge for the Reader
	Review
Chapter 9. Training Large Deep Networks
	Custom Hardware for Deep Networks
	CPU Training
		GPU Training
		Tensor Processing Units
		Field Programmable Gate Arrays
		Neuromorphic Chips
	Distributed Deep Network Training
		Data Parallelism
		Model Parallelism
	Data Parallel Training with Multiple GPUs on Cifar10
		Downloading and Loading the DATA
		Deep Dive on the Architecture
		Training on Multiple GPUs
		Challenge for the Reader
	Review
Chapter 10. The Future of Deep Learning
	Deep Learning Outside the Tech Industry
		Deep Learning in the Pharmaceutical Industry
		Deep Learning in Law
		Deep Learning for Robotics
		Deep Learning in Agriculture
	Using Deep Learning Ethically
	Is Artificial General Intelligence Imminent?
	Where to Go from Here?
Index
About the Authors
Colophon