Mastering PyTorch: Create and deploy deep learning models from CNNs to multimodal models, LLMs and beyond, 2nd Edition

Cover
Copyright
Contributors
Table of Contents
Preface
Chapter 1: Overview of Deep Learning Using PyTorch
	A refresher on deep learning
		Activation functions
	Optimization schedule
	Exploring the PyTorch library in contrast to TensorFlow
		Tensor modules
		PyTorch modules
			torch.nn
			torch.optim
			torch.utils.data
		Training a neural network using PyTorch
	Summary
	Reference list
Chapter 2: Deep CNN Architectures
	Why are CNNs so powerful?
	Evolution of CNN architectures
	Developing LeNet from scratch
		Using PyTorch to build LeNet
		Training LeNet
		Testing LeNet
	Fine-tuning the AlexNet model
		Using PyTorch to fine-tune AlexNet
	Running a pretrained VGG model
	Exploring GoogLeNet and Inception v3
		Inception modules
		1x1 convolutions
		Global average pooling
		Auxiliary classifiers
		Inception v3
	Discussing ResNet and DenseNet architectures
		ResNet
		DenseNet
	Understanding EfficientNets and the future of CNN architectures
	Summary
	References
Chapter 3: Combining CNNs and LSTMs
	Building a neural network with CNNs and LSTMs
		Text encoding demo
	Building an image caption generator using PyTorch
		Downloading the image captioning datasets
		Preprocessing caption (text) data
		Preprocessing image data
		Defining the image captioning data loader
		Defining the CNN-LSTM model
		Training the CNN-LSTM model
		Generating image captions using the trained model
	Summary
	References
Chapter 4: Deep Recurrent Model Architectures
	Exploring the evolution of recurrent networks
		Types of recurrent neural networks
		RNNs
		Bidirectional RNNs
		LSTMs
		Extended and bidirectional LSTMs
		Multi-dimensional RNNs
		Stacked LSTMs
		GRUs
		Grid LSTMs
		Gated orthogonal recurrent units
	Training RNNs for sentiment analysis
		Loading and preprocessing the text dataset
		Instantiating and training the model
	Building a bidirectional LSTM
		Loading and preprocessing the text dataset
		Instantiating and training the LSTM model
	Discussing GRUs and attention-based models
		GRUs and PyTorch
		Attention-based models
	Summary
	References
Chapter 5: Advanced Hybrid Models
	Building a transformer model for language modeling
		Reviewing language modeling
		Understanding the transformer model architecture
		Defining a transformer model in PyTorch
			Loading and processing the dataset
			Training the transformer model
	Developing a RandWireNN model from scratch
		Understanding RandWireNNs
		Developing RandWireNNs using PyTorch
			Defining a training routine and loading data
			Defining the randomly wired graph
			Defining RandWireNN model modules
			Transforming a random graph into a neural network
			Training the RandWireNN model
			Evaluating and visualizing the RandWireNN model
	Summary
	References
Chapter 6: Graph Neural Networks
	Introduction to GNNs
		Understanding the intuition behind GNNs
		Using regular NNs on graph data – a thought experiment
		Understanding the power of GNNs with computational graphs
	Types of graph learning tasks
		Understanding node-level tasks
		Understanding edge-level tasks
		Understanding graph-level tasks
	Reviewing prominent GNN models
		Understanding graph convolutions with GCNs
		Using attention in graphs with GAT
		Performing graph sampling with GraphSAGE
		Building a GCN model using PyTorch Geometric
		Loading and exploring the citation networks dataset
		Building a simple NN-based node classifier
		Building a GCN model for node classification
	Training a GAT model with PyTorch Geometric
	Summary
	Reference list
Chapter 7: Music and Text Generation with PyTorch
	Building a transformer-based text generator with PyTorch
		Training the transformer-based language model
		Saving and loading the language model
		Using the language model to generate text
	Using GPT models as text generators
		Out-of-the-box text generation with GPT-2
		Text generation strategies using PyTorch
			Greedy search
			Beam search
			Top-k and top-p sampling
		Text generation with GPT-3
	Generating MIDI music with LSTMs using PyTorch
		Loading the MIDI music data
		Defining the LSTM model and training routine
		Training and testing the music generation model
	Summary
	References
Chapter 8: Neural Style Transfer
	Understanding how to transfer style between images
	Implementing neural style transfer using PyTorch
		Loading the content and style images
		Loading and trimming the pretrained VGG19 model
		Building the neural style transfer model
		Training the style transfer model
		Experimenting with the style transfer system
	Summary
	References
Chapter 9: Deep Convolutional GANs
	Defining the generator and discriminator networks
		Understanding the DCGAN generator and discriminator
	Training a DCGAN using PyTorch
		Defining the generator
		Defining the discriminator
		Loading the image dataset
		Training loops for DCGANs
	Using GANs for style transfer
		Understanding the pix2pix architecture
			Exploring the pix2pix generator
			Exploring the pix2pix discriminator
	Summary
	References
Chapter 10: Image Generation Using Diffusion
	Understanding image generation using diffusion
		Understanding how diffusion works
		Training a forward diffusion model
		Performing reverse diffusion or denoising
	Training a diffusion model for image generation
		Loading the dataset using Hugging Face datasets
		Processing the dataset using torchvision transforms
		Adding noise to images using diffusers
		Defining the UNet model
		Training the UNet model
			Defining the optimizer and learning schedule
			Using Hugging Face Accelerate to accelerate training
			Running the model training loop
			Generating realistic anime images using (reverse) diffusion
	Understanding text-to-image generation using diffusion
		Encoding text input into an embedding vector
		Ingesting additional text data in the (conditional) UNet model
	Using the Stable Diffusion model to generate images from text
	Summary
	Reference list
Chapter 11: Deep Reinforcement Learning
	Reviewing RL concepts
		Types of RL algorithms
			Model-based
			Model-Free
	Discussing Q-learning
	Understanding deep Q-learning
		Using two separate DNNs
		Experience replay buffer
	Building a DQN model in PyTorch
		Initializing the main and target CNN models
		Defining the experience replay buffer
		Setting up the environment
		Defining the CNN optimization function
		Managing and running episodes
		Training the DQN model to learn Pong
	Summary
	Reference list
Chapter 12: Model Training Optimizations
	Distributed training with PyTorch
		Training the MNIST model in a regular fashion
		Training the MNIST model in a distributed fashion
	Distributed training on GPUs with CUDA
		Automatic mixed precision training
		Regular model training on a GPU
		Mixed precision training on a GPU
	Summary
	Reference list
Chapter 13: Operationalizing PyTorch Models into Production
	Model serving in PyTorch
		Creating a PyTorch model inference pipeline
		Saving and loading a trained model
		Building the inference pipeline
	Building a basic model server
		Writing a basic app using Flask
		Using Flask to build our model server
			Setting up model inference for Flask serving
			Building a Flask app to serve model
			Using a Flask server to run predictions
	Creating a model microservice
	Serving a PyTorch model using TorchServe
		Installing TorchServe
		Launching and using a TorchServe server
	Exporting universal PyTorch models using TorchScript and ONNX
		Understanding the utility of TorchScript
		Model tracing with TorchScript
		Model scripting with TorchScript
	Running a PyTorch model in C++
	Using ONNX to export PyTorch models
	Serving PyTorch models in the cloud
		Using PyTorch with AWS
		Serving a PyTorch model using an AWS instance
		Using TorchServe with Amazon SageMaker
		Serving PyTorch models on Google Cloud
		Serving PyTorch models with Azure
		Working with Azure’s DSVMs
		Discussing Azure Machine Learning Service
	Summary
		Reference list
Chapter 14: PyTorch on Mobile Devices
	Deploying a PyTorch model on Android
		Converting the PyTorch model to a mobile-friendly format
		Setting up the Android app development environment
	Using the phone camera in the Android app to capture images
		Enabling the camera during app startup
			Handling camera permissions in Android
			Opening the camera for image capture
			Capturing images using the phone camera
		Running ML model inference on camera-captured images
			Validating the ML model binary path
			Performing image classification on camera-captured images
		Launching the app on an Android mobile device
	Building PyTorch apps on iOS
		Setting up the iOS development environment
		Using a phone camera in the iOS app to capture images
		Running ML model inference on camera-captured images
	Summary
	Reference list
Chapter 15: Rapid Prototyping with PyTorch
	Using fastai to set up model training in a few minutes
		Setting up fastai and loading data
		Training an MNIST model using fastai
		Evaluating and interpreting the model using fastai
	Training models on any hardware using PyTorch Lightning
		Defining the model components in PyTorch Lightning
		Training and evaluating the model using PyTorch Lightning
	Profiling MNIST model inference using PyTorch Profiler
		Profiling on a CPU
		Profiling model inference on the GPU
			Visualizing model profiling results
	Summary
	Reference list
Chapter 16: PyTorch and AutoML
	Finding the best neural architectures with AutoML
		Using Auto-PyTorch for optimal MNIST model search
			Loading the MNIST dataset
			Running a neural architecture search with Auto-PyTorch
			Visualizing the optimal AutoML model
	Using Optuna for hyperparameter search
		Defining the model architecture and loading the dataset
		Defining the model training routine and optimization schedule
		Running Optuna’s hyperparameter search
	Summary
	Reference list
Chapter 17: PyTorch and Explainable AI
	Model interpretability in PyTorch
		Training the handwritten digits classifier – a recap
		Visualizing the convolutional filters of the model
		Visualizing the feature maps of the model
	Using Captum to interpret models
		Setting up Captum
		Exploring Captum’s interpretability tools
	Summary
	Reference List
Chapter 18: Recommendation Systems with PyTorch
	Using deep learning for recommendation systems
		Understanding a movie recommendation system dataset
		Understanding embedding-based recommender systems
	Understanding and processing the MovieLens dataset
		Downloading the MovieLens dataset
		Loading and analyzing the MovieLens dataset
		Processing the MovieLens dataset
		Creating the MovieLens dataloader
	Training and evaluating a recommendation system model
		Defining the EmbeddingNet architecture
		Training EmbeddingNet
		Evaluating the trained EmbeddingNet model
	Building a recommendation system using the trained model
	Summary
	Reference list
Chapter 19: PyTorch and Hugging Face
	Understanding Hugging Face within the PyTorch context
		Exploring Hugging Face components relevant to PyTorch
		Integrating Hugging Face with PyTorch
	Using the Hugging Face Hub for pre-trained models
	Using the Hugging Face Datasets library with PyTorch
	Using Accelerate to speed up PyTorch model training
	Using Optimum to optimize PyTorch model deployment
	Summary
	Reference list
Other Books You May Enjoy
Index