Deep Learning for Computer Vision with Python: ImageNet Bundle

1 Introduction
	1.1 I Studied Deep Learning the Wrong Way…This Is the Right Way
	1.2 Who This Book Is For
		1.2.1 Just Getting Started in Deep Learning?
		1.2.2 Already a Seasoned Deep Learning Practitioner?
	1.3 Book Organization
		1.3.1 Volume #1: Starter Bundle
		1.3.2 Volume #2: Practitioner Bundle
		1.3.3 Volume #3: ImageNet Bundle
		1.3.4 Need to Upgrade Your Bundle?
	1.4 Tools of the Trade: Python, Keras, and Mxnet
		1.4.1 What About TensorFlow?
		1.4.2 Do I Need to Know OpenCV?
	1.5 Developing Our Own Deep Learning Toolset
	1.6 Summary
2 What Is Deep Learning?
	2.1 A Concise History of Neural Networks and Deep Learning
	2.2 Hierarchical Feature Learning
	2.3 How "Deep" Is Deep?
	2.4 Summary
3 Image Fundamentals
	3.1 Pixels: The Building Blocks of Images
		3.1.1 Forming an Image From Channels
	3.2 The Image Coordinate System
		3.2.1 Images as NumPy Arrays
		3.2.2 RGB and BGR Ordering
	3.3 Scaling and Aspect Ratios
	3.4 Summary
4 Image Classification Basics
	4.1 What Is Image Classification?
		4.1.1 A Note on Terminology
		4.1.2 The Semantic Gap
		4.1.3 Challenges
	4.2 Types of Learning
		4.2.1 Supervised Learning
		4.2.2 Unsupervised Learning
		4.2.3 Semi-supervised Learning
	4.3 The Deep Learning Classification Pipeline
		4.3.1 A Shift in Mindset
		4.3.2 Step #1: Gather Your Dataset
		4.3.3 Step #2: Split Your Dataset
		4.3.4 Step #3: Train Your Network
		4.3.5 Step #4: Evaluate
		4.3.6 Feature-based Learning versus Deep Learning for Image Classification
		4.3.7 What Happens When my Predictions Are Incorrect?
	4.4 Summary
5 Datasets for Image Classification
	5.1 MNIST
	5.2 Animals: Dogs, Cats, and Pandas
	5.3 CIFAR-10
	5.4 SMILES
	5.5 Kaggle: Dogs vs. Cats
	5.6 Flowers-17
	5.7 CALTECH-101
	5.8 Tiny ImageNet 200
	5.9 Adience
	5.10 ImageNet
		5.10.1 What Is ImageNet?
		5.10.2 ImageNet Large Scale Visual Recognition Challenge (ILSVRC)
	5.11 Kaggle: Facial Expression Recognition Challenge
	5.12 Indoor CVPR
	5.13 Stanford Cars
	5.14 Summary
6 Configuring Your Development Environment
	6.1 Libraries and Packages
		6.1.1 Python
		6.1.2 Keras
		6.1.3 Mxnet
		6.1.4 OpenCV, scikit-image, scikit-learn, and more
	6.2 Configuring Your Development Environment?
	6.3 Preconfigured Virtual Machine
	6.4 Cloud-based Instances
	6.5 How to Structure Your Projects
	6.6 Summary
7 Your First Image Classifier
	7.1 Working with Image Datasets
		7.1.1 Introducing the “Animals” Dataset
		7.1.2 The Start to Our Deep Learning Toolkit
		7.1.3 A Basic Image Preprocessor
		7.1.4 Building an Image Loader
	7.2 k-NN: A Simple Classifier
		7.2.1 A Worked k-NN Example
		7.2.2 k-NN Hyperparameters
		7.2.3 Implementing k-NN
		7.2.4 k-NN Results
		7.2.5 Pros and Cons of k-NN
	7.3 Summary
8 Parameterized Learning
	8.1 An Introduction to Linear Classification
		8.1.1 Four Components of Parameterized Learning
		8.1.2 Linear Classification: From Images to Labels
		8.1.3 Advantages of Parameterized Learning and Linear Classification
		8.1.4 A Simple Linear Classifier With Python
	8.2 The Role of Loss Functions
		8.2.1 What Are Loss Functions?
		8.2.2 Multi-class SVM Loss
		8.2.3 Cross-entropy Loss and Softmax Classifiers
	8.3 Summary
9 Optimization Methods and Regularization
	9.1 Gradient Descent
		9.1.1 The Loss Landscape and Optimization Surface
		9.1.2 The “Gradient” in Gradient Descent
		9.1.3 Treat It Like a Convex Problem (Even if It’s Not)
		9.1.4 The Bias Trick
		9.1.5 Pseudocode for Gradient Descent
		9.1.6 Implementing Basic Gradient Descent in Python
		9.1.7 Simple Gradient Descent Results
	9.2 Stochastic Gradient Descent (SGD)
		9.2.1 Mini-batch SGD
		9.2.2 Implementing Mini-batch SGD
		9.2.3 SGD Results
	9.3 Extensions to SGD
		9.3.1 Momentum
		9.3.2 Nesterov's Acceleration
		9.3.3 Anecdotal Recommendations
	9.4 Regularization
		9.4.1 What Is Regularization and Why Do We Need It?
		9.4.2 Updating Our Loss and Weight Update To Include Regularization
		9.4.3 Types of Regularization Techniques
		9.4.4 Regularization Applied to Image Classification
	9.5 Summary
10 Neural Network Fundamentals
	10.1 Neural Network Basics
		10.1.1 Introduction to Neural Networks
		10.1.2 The Perceptron Algorithm
		10.1.3 Backpropagation and Multi-layer Networks
		10.1.4 Multi-layer Networks with Keras
		10.1.5 The Four Ingredients in a Neural Network Recipe
		10.1.6 Weight Initialization
		10.1.7 Constant Initialization
		10.1.8 Uniform and Normal Distributions
		10.1.9 LeCun Uniform and Normal
		10.1.10 Glorot/Xavier Uniform and Normal
		10.1.11 He et al./Kaiming/MSRA Uniform and Normal
		10.1.12 Differences in Initialization Implementation
	10.2 Summary
11 Convolutional Neural Networks
	11.1 Understanding Convolutions
		11.1.1 Convolutions versus Cross-correlation
		11.1.2 The “Big Matrix” and “Tiny Matrix" Analogy
		11.1.3 Kernels
		11.1.4 A Hand Computation Example of Convolution
		11.1.5 Implementing Convolutions with Python
		11.1.6 The Role of Convolutions in Deep Learning
	11.2 CNN Building Blocks
		11.2.1 Layer Types
		11.2.2 Convolutional Layers
		11.2.3 Activation Layers
		11.2.4 Pooling Layers
		11.2.5 Fully-connected Layers
		11.2.6 Batch Normalization
		11.2.7 Dropout
	11.3 Common Architectures and Training Patterns
		11.3.1 Layer Patterns
		11.3.2 Rules of Thumb
	11.4 Are CNNs Invariant to Translation, Rotation, and Scaling?
	11.5 Summary
12 Training Your First CNN
	12.1 Keras Configurations and Converting Images to Arrays
		12.1.1 Understanding the keras.json Configuration File
		12.1.2 The Image to Array Preprocessor
	12.2 ShallowNet
		12.2.1 Implementing ShallowNet
		12.2.2 ShallowNet on Animals
		12.2.3 ShallowNet on CIFAR-10
	12.3 Summary
13 Saving and Loading Your Models
	13.1 Serializing a Model to Disk
	13.2 Loading a Pre-trained Model from Disk
	13.3 Summary
14 LeNet: Recognizing Handwritten Digits
	14.1 The LeNet Architecture
	14.2 Implementing LeNet
	14.3 LeNet on MNIST
	14.4 Summary
15 MiniVGGNet: Going Deeper with CNNs
	15.1 The VGG Family of Networks
		15.1.1 The (Mini) VGGNet Architecture
	15.2 Implementing MiniVGGNet
	15.3 MiniVGGNet on CIFAR-10
		15.3.1 With Batch Normalization
		15.3.2 Without Batch Normalization
	15.4 Summary
16 Learning Rate Schedulers
	16.1 Dropping Our Learning Rate
		16.1.1 The Standard Decay Schedule in Keras
		16.1.2 Step-based Decay
		16.1.3 Implementing Custom Learning Rate Schedules in Keras
	16.2 Summary
17 Spotting Underfitting and Overfitting
	17.1 What Are Underfitting and Overfitting?
		17.1.1 Effects of Learning Rates
		17.1.2 Pay Attention to Your Training Curves
		17.1.3 What if Validation Loss Is Lower than Training Loss?
	17.2 Monitoring the Training Process
		17.2.1 Creating a Training Monitor
		17.2.2 Babysitting Training
	17.3 Summary
18 Checkpointing Models
	18.1 Checkpointing Neural Network Model Improvements
	18.2 Checkpointing Best Neural Network Only
	18.3 Summary
19 Visualizing Network Architectures
	19.1 The Importance of Architecture Visualization
		19.1.1 Installing graphviz and pydot
		19.1.2 Visualizing Keras Networks
	19.2 Summary
20 Out-of-the-box CNNs for Classification
	20.1 State-of-the-art CNNs in Keras
		20.1.1 VGG16 and VGG19
		20.1.2 ResNet
		20.1.3 Inception V3
		20.1.4 Xception
		20.1.5 Can We Go Smaller?
	20.2 Classifying Images with Pre-trained ImageNet CNNs
		20.2.1 Classification Results
	20.3 Summary
21 Case Study: Breaking Captchas with a CNN
	21.1 Breaking Captchas with a CNN
		21.1.1 A Note on Responsible Disclosure
		21.1.2 The Captcha Breaker Directory Structure
		21.1.3 Automatically Downloading Example Images
		21.1.4 Annotating and Creating Our Dataset
		21.1.5 Preprocessing the Digits
		21.1.6 Training the Captcha Breaker
		21.1.7 Testing the Captcha Breaker
	21.2 Summary
22 Case Study: Smile Detection
	22.1 The SMILES Dataset
	22.2 Training the Smile CNN
	22.3 Running the Smile CNN in Real-time
	22.4 Summary
23 Your Next Steps
	23.1 So, What's Next?