Deep Learning Pipeline: Building A Deep Learning Model With TensorFlow

Table of Contents
About the Authors
About the Technical Reviewer
Introduction
Part I: Introduction
	Chapter 1: A Gentle Introduction
		Information Theory, Probability Theory, and Decision Theory
			Information Theory
			Probability Theory
			Decision Theory
		Introduction to Machine Learning
			Predictive Analytics and Its Connection with Machine learning
			Machine Learning Approaches
				Supervised Learning
				Unsupervised Learning
				Semisupervised Learning
				Checkpoint
				Reinforcement Learning
		From Machine Learning to Deep Learning
		Lets’ See What Some Heroes of Machine Learning Say About the Field
			Connections Between Machine Learning and Deep Learning
			Difference Between ML and DL
				In Machine Learning
				In Deep Learning
				What Have We Learned Here?
			Why Should We Learn About Deep Learning (Advantages of Deep learning)?
			Disadvantages of Deep Learning (Cost of Greatness)
		Introduction to Deep Learning
		Machine Learning Mathematical Notations
		Summary
	Chapter 2: Setting Up Your Environment
		Background
			Python 2 vs. Python 3
		Installing Python
			Python Packages
			IPython
				Installing IPython
			Jupyter
				Installing Jupyter
					What Is an ipynb File?
		Packages Used in the Book
			NumPy
			SciPy
			Pandas
			Matplotlib
			NLTK
			Scikit-learn
			Gensim
			TensorFlow
				Installing on Mac or Linux distributions
				Installing on Windows
			Keras
		Summary
	Chapter 3: A Tour Through the Deep Learning Pipeline
		Deep Learning Approaches
			What Is Deep Learning
			Biological Deep Learning
			What Are Neural Networks Architectures?
		Deep Learning Pipeline
			Define and Prepare Problem
			Summarize and Understand Data
			Process and Prepare Data
			Evaluate Algorithms
			Improve Results
		Fast Preview of the TensorFlow Pipeline
			Tensors—the Main Data Structure
			First Session
			Data Flow Graphs
			Tensor Properties
				Tensor Rank
				Tensor Shape
		Summary
	Chapter 4: Build Your First Toy TensorFlow app
		Basic Development of TensorFlow
			Hello World with TensorFlow
			Simple Iterations
			Prepare the Input Data
			Doing the Gradients
		Linear Regression
			Why Linear Regression?
			What Is Linear Regression?
			Dataset Description
			Full Source Code
		XOR Implementation Using TensorFlow
			Full Source Code
		Summary
Part II: Data
	Chapter 5: Defining Data
		Defining Data
		Why Should You Read This Chapter?
		Structured, Semistructured, and Unstructured Data
		Tidy Data
		Divide and Conquer
		Tabular Data
			Quantitative​ vs. ​Qualitative​ Data
			Example—the Titanic
			Divide and Conquer
			Making a Checkpoint
			The Four Levels of Data
				Measure of Center
			The Nominal Level
				Mathematical Operations Allowed for Nominal
				Measures of Center for Nominal
				What Does It Mean to be a Nominal Level Type?
			The Ordinal Level
				Examples of Being Ordinal
				What Data Is Like at the Ordinal Level
				Mathematical Operations Allowed for Ordinal
				Measures of Center for Ordinal
			Quick Recap and Check
			The Interval Level
			Examples of Interval Level Data
			What Data Is Like at the Interval Level
			Mathematical Operations Allowed for Interval
				Measures of Center for Interval
				Measures of Variation for Interval
					Standard Deviation
			The Ratio Level
				Examples
				Measures of Center for Ratio
				Problems with the Ratio Level
			Summarizing All Levels Table 5-1
		Text Data
			What Is Text Processing and What Is the Level of Importance of Text Processing?
			IMDB—Example
			Images Data
				Type of Images (2-D, 3-D, 4-D)
					2-D Data
					3-D Data
					4-D Data
				Example—MNIST
				Example—CIFAR-10
		Summary
	Chapter 6: Data Wrangling and Preprocessing
		The Data Fields Pipelines Revisited
			Giving You a Reason
			Where Is Data Cleaning in the Process?
		Data Loading and Preprocessing
			Fast and Easy Data Loading
		Missing Data
			Empties
			Is It Ever Useful to Fill Missing Data Using a Zero Instead of an Empty or Null?
			Managing Missing Features
		Dealing with Big Datasets
		Accessing Other Data Formats
		Data Preprocessing
		Data Augmentation
			Image Crop
			Crop and Resize
			Crop to Bounding Box
			Flipping
			Rotate Image
			Translation
			Transform
			Adding Salt and Pepper Noise
			Convert RGB to Grayscale
			Change Brightness
			Adjust Contrast
			Adjust Hue
			Adjust Saturation
		Categorical and Text data
			Data Encoding
			Performing One-Hot Encoding on Nominal Features
			Can You Spot the Problem?
			A Special Type of Data: Text
			So Far, Everything Has Been Pretty Good, Hasn’t It?
			Tokenization, Stemming, and Stop Words
				What Are Tokenizing and Tokenization?
				The Bag-of-Words (BoW) Model
					What is the BoW?
		Summary
	Chapter 7: Data Resampling
		Creating Training and Test Sets
		Cross-Validation
			Validation Set Technique
			Leave-One-Out Cross-Validation (LOOCV)
			K-Fold Cross-Validation
		Bootstrap
			Bootstrap in Statistics
			Tips to Use Bootstrap (Resampling with Replacement)
		Generators
			What Are Keras Generators?
			Data Generator
			Callback
		Summary
	Chapter 8: Feature Selection and Feature Engineering
		Dataset Used in This Chapter
		Dimensionality Reduction—Questions to Answer
			What Is Dimensionality Reduction?
			When Should I Use Dimensionality Reduction?
		Unsupervised Dimensionality Reduction via Principal Component Analysis (PCA)
			Total and Explained Variance
		Feature Selection and Filtering
		Principal Component Analysis
			Nonnegative Matrix Factorization
		Sparse PCA
		Kernel PCA
		Atom Extraction and Dictionary Learning
		Latent Dirichlet Allocation (LDA)
			Latent Dirichlet Allocation (LDA in NLP)
			Code Example Using gensim
		LDA vs. PCA
		ZCA Whitening
		Summary
Part III: TensorFlow
	Chapter 9: Deep Learning Fundamentals
		Perceptron
			Single Perceptron
			Multilayer Perceptron
			Recap
		Different Neural Network Layers
			Input Layer
			Hidden Layer(s)
			Output Layer
		Shallow vs. Deep Neural Networks
		Activation Functions
			Types of Activation Functions
			Recap
		Gradient Descent
			Recap
		Batch vs. Stochastic vs. Mini-Batch Gradient Descent
			Batch Gradient Descent
			Stochastic Gradient Descent
			Mini-batch Gradient Descent
			Recap
		Loss function and Backpropagation
			Loss Function
			Backpropagation
				The Four Fundamental Equations Behind Backpropagation
		Exploding Gradients
			Re-Design the Network Model
			Use Long Short-Term Memory Networks
			Use Gradient Clipping
			Use Weight Regularization
		Vanishing Gradients
			Vanishing Gradients Problem
		TensorFlow Basics
			Placeholder vs. Variable vs. Constant
			Gradient-Descent Optimization Methods from a Deep-Learning Perspective
			Learning Rate in the Mini-batch Approach to Stochastic Gradient Descent
		Summary
	Chapter 10: Improving Deep Neural Networks
		Optimizers in TensorFlow
			The Notation to Use
		Momentum
			Nesterov Accelerated Gradient
			Adagrad
			Adadelta
			RMSprop
			Adam
			Nadam (Adam + NAG)
		Choosing the Learning Rate
		Dropout Layers and Regularization
		Normalization Techniques
			Batch Normalization
			Weight Normalization
			Layer Normalization
			Instance Normalization
			Group Normalization
		Summary
	Chapter 11: Convolutional Neural Network
		What is a Convolutional Neural Network
		Convolution Operation
			One-Dimensional Convolution
			Two-Dimensional Convolution
			Padding and Stride
		Common Image-Processing Filters
			Mean and Median Filters
			Gaussian Filter
			Sobel Edge-Detection Filter
			Identity Transform
		Convolutional Neural Networks
		Layers of Convolutional Neural Networks
			Input Layer
			Convolutional layer
			Pooling Layer
		Backpropagation Through the Convolutional and Pooling Layers
		Weight Sharing Through Convolution and Its Advantages
		Translation Equivariance and Invariance
		Case Study—Digit Recognition on the CIFAR-10 Dataset
		Summary
	Chapter 12: Sequential Models
		Recurrent Neural Networks
		Language Modeling
		Backpropagation Through Time
		Vanishing and Exploding Gradient Problems in RNN
		The Solution to Vanishing and Exploding Gradients Problems in RNNs
		Long Short-Term Memory
		Case Study—Digit Identification on the MNIST Dataset
		Gated Recurrent Unit
		Bidirectional RNN (Bi-RNN)
		Summary
Part IV: Applying What You’ve Learned
	Chapter 13: Selected Topics in Computer Vision
		Different Architectures in Convolutional Neural Networks
			LeNet
			AlexNet
			VGG
			ResNet
		Transfer Learning
			What Is a Pretrained Model, and Why Use It?
			How to Use a Pretrained Model?
			Ways to Fine-Tune the Model
			Pretrained VGG19
		Summary
	Chapter 14: Selected Topics in Natural Language Processing
		Vector Space Model
		Vector Representation of Words
		Word2Vec
			Continuous Bag of Words
				Implementing Continuous Bag of Words
			Skip-Gram Model for Word Embeddings
				Implementing Skip-Gram
			GloVe
		Summary
	Chapter 15: Applications
		Case Study—Tabular Dataset
			Understanding the Dataset
				Scratching the Surface
				Digging Deeper
			Preprocessing Dataset
			Building the Model
		Case Study—IMDB Movie Review Data with Word2Vec
		Case Study—Image Segmentation
		Summary
Index