Искусственный интеллект и компьютерное зрение. Реальные проекты на Python, Keras и TensorFlow

Preface
	To the Backend/Frontend/Mobile Software Developer
	To the Data Scientist
	To the Student
	To the Teacher
	To the Robotics Enthusiast
	What to Expect in Each Chapter
	Conventions Used in This Book
	Using Code Examples
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
		Group Acknowledgments
		Personal Acknowledgments
1. Exploring the Landscape of Artificial Intelligence
	An Apology
	The Real Introduction
	What Is AI?
		Motivating Examples
	A Brief History of AI
		Exciting Beginnings
		The Cold and Dark Days
		A Glimmer of Hope
		How Deep Learning Became a Thing
	Recipe for the Perfect Deep Learning Solution
		Datasets
		Model Architecture
		Frameworks
			TensorFlow
			Keras
			PyTorch
			A continuously evolving landscape
		Hardware
	Responsible AI
		Bias
		Accountability and Explainability
		Reproducibility
		Robustness
		Privacy
	Summary
	Frequently Asked Questions
2. What’s in the Picture: Image Classification with Keras
	Introducing Keras
	Predicting an Image’s Category
	Investigating the Model
		ImageNet Dataset
		Model Zoos
		Class Activation Maps
	Summary
3. Cats Versus Dogs: Transfer Learning in 30 Lines with Keras
	Adapting Pretrained Models to New Tasks
		A Shallow Dive into Convolutional Neural Networks
		Transfer Learning
		Fine Tuning
		How Much to Fine Tune
	Building a Custom Classifier in Keras with Transfer Learning
	Organize the Data
	Build the Data Pipeline
		Number of Classes
			Binary classification
			Multiclass classification
		Batch Size
	Data Augmentation
	Model Definition
	Train the Model
		Set Training Parameters
		Start Training
	Test the Model
	Analyzing the Results
	Further Reading
	Summary
4. Building a Reverse Image Search Engine: Understanding Embeddings
	Image Similarity
	Feature Extraction
	Similarity Search
	Visualizing Image Clusters with t-SNE
	Improving the Speed of Similarity Search
		Length of Feature Vectors
		Reducing Feature-Length with PCA
	Scaling Similarity Search with Approximate Nearest Neighbors
		Approximate Nearest-Neighbor Benchmark
		Which Library Should I Use?
		Creating a Synthetic Dataset
		Brute Force
		Annoy
		NGT
		Faiss
	Improving Accuracy with Fine Tuning
		Fine Tuning Without Fully Connected Layers
	Siamese Networks for One-Shot Face Verification
	Case Studies
		Flickr
		Pinterest
		Celebrity Doppelgangers
		Spotify
		Image Captioning
	Summary
5. From Novice to Master Predictor: Maximizing Convolutional Neural Network Accuracy
	Tools of the Trade
		TensorFlow Datasets
		TensorBoard
		What-If Tool
		tf-explain
	Common Techniques for Machine Learning Experimentation
		Data Inspection
		Breaking the Data: Train, Validation, Test
		Early Stopping
		Reproducible Experiments
	End-to-End Deep Learning Example Pipeline
		Basic Transfer Learning Pipeline
		Basic Custom Network Pipeline
	How Hyperparameters Affect Accuracy
		Transfer Learning Versus Training from Scratch
		Effect of Number of Layers Fine-Tuned in Transfer Learning
		Effect of Data Size on Transfer Learning
		Effect of Learning Rate
		Effect of Optimizers
		Effect of Batch Size
		Effect of Resizing
		Effect of Change in Aspect Ratio on Transfer Learning
	Tools to Automate Tuning for Maximum Accuracy
		Keras Tuner
		AutoAugment
		AutoKeras
	Summary
6. Maximizing Speed and Performance of TensorFlow: A Handy Checklist
	GPU Starvation
		nvidia-smi
		TensorFlow Profiler + TensorBoard
	How to Use This Checklist
	Performance Checklist
		Data Preparation
		Data Reading
		Data Augmentation
		Training
		Inference
	Data Preparation
		Store as TFRecords
		Reduce Size of Input Data
		Use TensorFlow Datasets
	Data Reading
		Use tf.data
		Prefetch Data
		Parallelize CPU Processing
		Parallelize I/O and Processing
		Enable Nondeterministic Ordering
		Cache Data
		Turn on Experimental Optimizations
			Filter fusion
			Map and filter fusion
			Map fusion
		Autotune Parameter Values
	Data Augmentation
		Use GPU for Augmentation
			tf.image built-in augmentations
			NVIDIA DALI
	Training
		Use Automatic Mixed Precision
		Use Larger Batch Size
		Use Multiples of Eight
		Find the Optimal Learning Rate
		Use tf.function
		Overtrain, and Then Generalize
			Use progressive sampling
			Use progressive augmentation
			Use progressive resizing
		Install an Optimized Stack for the Hardware
		Optimize the Number of Parallel CPU Threads
		Use Better Hardware
		Distribute Training
		Examine Industry Benchmarks
	Inference
		Use an Efficient Model
		Quantize the Model
		Prune the Model
		Use Fused Operations
		Enable GPU Persistence
	Summary
7. Practical Tools, Tips, and Tricks
	Installation
	Training
	Model
	Data
	Privacy
	Education and Exploration
	One Last Question
8. Cloud APIs for Computer Vision: Up and Running in 15 Minutes
	The Landscape of Visual Recognition APIs
		Clarifai
			What’s unique about this API?
		Microsoft Cognitive Services
			What’s unique about this API?
		Google Cloud Vision
			What’s unique about this API?
		Amazon Rekognition
			What’s unique about this API?
		IBM Watson Visual Recognition
		Algorithmia
			What’s unique about this API?
	Comparing Visual Recognition APIs
		Service Offerings
		Cost
		Accuracy
		Bias
	Getting Up and Running with Cloud APIs
	Training Our Own Custom Classifier
		Top Reasons Why Our Classifier Does Not Work Satisfactorily
	Comparing Custom Classification APIs
	Performance Tuning for Cloud APIs
		Effect of Resizing on Image Labeling APIs
		Effect of Compression on Image Labeling APIs
		Effect of Compression on OCR APIs
		Effect of Resizing on OCR APIs
	Case Studies
		The New York Times
		Uber
		Giphy
		OmniEarth
		Photobucket
		Staples
		InDro Robotics
	Summary
9. Scalable Inference Serving on Cloud with TensorFlow Serving and KubeFlow
	Landscape of Serving AI Predictions
	Flask: Build Your Own Server
		Making a REST API with Flask
		Deploying a Keras Model to Flask
		Pros of Using Flask
		Cons of Using Flask
	Desirable Qualities in a Production-Level Serving System
		High Availability
		Scalability
		Low Latency
		Geographic Availability
		Failure Handling
		Monitoring
		Model Versioning
		A/B Testing
		Support for Multiple Machine Learning Libraries
	Google Cloud ML Engine: A Managed Cloud AI Serving Stack
		Pros of Using Cloud ML Engine
		Cons of Using Cloud ML Engine
		Building a Classification API
	TensorFlow Serving
		Installation
	KubeFlow
		Pipelines
		Fairing
		Installation
	Price Versus Performance Considerations
		Cost Analysis of Inference-as-a-Service
		Cost Analysis of Building Your Own Stack
	Summary
10. AI in the Browser with TensorFlow.js and ml5.js
	JavaScript-Based Machine Learning Libraries: A Brief History
		ConvNetJS
		Keras.js
		ONNX.js
		TensorFlow.js
	TensorFlow.js Architecture
	Running Pretrained Models Using TensorFlow.js
	Model Conversion for the Browser
	Training in the Browser
		Feature Extraction
		Data Collection
		Training
		GPU Utilization
	ml5.js
	PoseNet
	pix2pix
	Benchmarking and Practical Considerations
		Model Size
		Inference Time
	Case Studies
		Semi-Conductor
		TensorSpace
		Metacar
		Airbnb’s Photo Classification
		GAN Lab
	Summary
11. Real-Time Object Classification on iOS with Core ML
	The Development Life Cycle for Artificial Intelligence on Mobile
	A Brief History of Core ML
	Alternatives to Core ML
		TensorFlow Lite
		ML Kit
		Fritz
	Apple’s Machine Learning Architecture
		Domain-Based Frameworks
		ML Framework
		ML Performance Primitives
	Building a Real-Time Object Recognition App
	Conversion to Core ML
		Conversion from Keras
		Conversion from TensorFlow
	Dynamic Model Deployment
	On-Device Training
		Federated Learning
	Performance Analysis
		Benchmarking Models on iPhones
	Measuring Energy Impact
		Benchmarking Load
	Reducing App Size
		Avoid Bundling the Model
		Use Quantization
		Use Create ML
	Case Studies
		Magic Sudoku
		Seeing AI
		HomeCourt
		InstaSaber + YoPuppet
	Summary
12. Not Hotdog on iOS with Core ML and Create ML
	Collecting Data
		Approach 1: Find or Collect a Dataset
		Approach 2: Fatkun Chrome Browser Extension
		Approach 3: Web Scraper Using Bing Image Search API
	Training Our Model
		Approach 1: Use Web UI-based Tools
		Approach 2: Use Create ML
		Approach 3: Fine Tuning Using Keras
	Model Conversion Using Core ML Tools
	Building the iOS App
	Further Exploration
	Summary
13. Shazam for Food: Developing Android Apps with TensorFlow Lite and ML Kit
	The Life Cycle of a Food Classifier App
	An Overview of TensorFlow Lite
		TensorFlow Lite Architecture
	Model Conversion to TensorFlow Lite
	Building a Real-Time Object Recognition App
	ML Kit + Firebase
		Object Classification in ML Kit
		Custom Models in ML Kit
		Hosted Models
			Accessing a hosted model
			Uploading a hosted model
		A/B Testing Hosted Models
			Measuring an experiment
		Using the Experiment in Code
	TensorFlow Lite on iOS
	Performance Optimizations
		Quantizing with TensorFlow Lite Converter
		TensorFlow Model Optimization Toolkit
	Fritz
	A Holistic Look at the Mobile AI App Development Cycle
		How Do I Collect Initial Data?
		How Do I Label My Data?
		How Do I Train My Model?
		How Do I Convert the Model to a Mobile-Friendly Format?
		How Do I Make my Model Performant?
		How Do I Build a Great UX for My Users?
		How Do I Make the Model Available to My Users?
		How Do I Measure the Success of My Model?
		How Do I Improve My Model?
		How Do I Update the Model on My Users’ Phones?
	The Self-Evolving Model
	Case Studies
		Lose It!
		Portrait Mode on Pixel 3 Phones
		Speaker Recognition by Alibaba
		Face Contours in ML Kit
		Real-Time Video Segmentation in YouTube Stories
	Summary
14. Building the Purrfect Cat Locator App with TensorFlow Object Detection API
	Types of Computer-Vision Tasks
		Classification
		Localization
		Detection
		Segmentation
			Semantic segmentation
			Instance-level segmentation
	Approaches to Object Detection
	Invoking Prebuilt Cloud-Based Object Detection APIs
	Reusing a Pretrained Model
		Obtaining the Model
		Test Driving Our Model
		Deploying to a Device
	Building a Custom Detector Without Any Code
	The Evolution of Object Detection
		Performance Considerations
	Key Terms in Object Detection
		Intersection over Union
		Mean Average Precision
		Non-Maximum Suppression
	Using the TensorFlow Object Detection API to Build Custom Models
		Data Collection
		Labeling the Data
		Preprocessing the Data
	Inspecting the Model
		Training
		Model Conversion
	Image Segmentation
	Case Studies
		Smart Refrigerator
		Crowd Counting
			Wildlife conservation
			Kumbh Mela
		Face Detection in Seeing AI
		Autonomous Cars
	Summary
15. Becoming a Maker: Exploring Embedded AI at the Edge
	Exploring the Landscape of Embedded AI Devices
		Raspberry Pi
		Intel Movidius Neural Compute Stick
		Google Coral USB Accelerator
		NVIDIA Jetson Nano
		FPGA + PYNQ
			FPGAs
			PYNQ platform
		Arduino
	A Qualitative Comparison of Embedded AI Devices
	Hands-On with the Raspberry Pi
	Speeding Up with the Google Coral USB Accelerator
	Port to NVIDIA Jetson Nano
	Comparing the Performance of Edge Devices
	Case Studies
		JetBot
		Squatting for Metro Tickets
		Cucumber Sorter
	Further Exploration
	Summary
16. Simulating a Self-Driving Car Using End-to-End Deep Learning with Keras
	A Brief History of Autonomous Driving
	Deep Learning, Autonomous Driving, and the Data Problem
	The “Hello, World!” of Autonomous Driving: Steering Through a Simulated Environment
		Setup and Requirements
	Data Exploration and Preparation
		Identifying the Region of Interest
		Data Augmentation
		Dataset Imbalance and Driving Strategies
	Training Our Autonomous Driving Model
		Drive Data Generator
		Model Definition
			Callbacks
	Deploying Our Autonomous Driving Model
	Further Exploration
		Expanding Our Dataset
		Training on Sequential Data
		Reinforcement Learning
	Summary
17. Building an Autonomous Car in Under an Hour: Reinforcement Learning with AWS DeepRacer
	A Brief Introduction to Reinforcement Learning
	Why Learn Reinforcement Learning with an Autonomous Car?
	Practical Deep Reinforcement Learning with DeepRacer
		Building Our First Reinforcement Learning
		Step 1: Create Model
		Step 2: Configure Training
			Configure the simulation environment
			Configure the action space
			Configure reward function
			Configure stop conditions
		Step 3: Model Training
		Step 4: Evaluating the Performance of the Model
	Reinforcement Learning in Action
		How Does a Reinforcement Learning System Learn?
		Reinforcement Learning Theory
			The Markov decision process
			Model free versus model based
			Value based
			Policy based
			Policy based or value based—why not both?
			Delayed rewards and discount factor (γ)
		Reinforcement Learning Algorithm in AWS DeepRacer
		Deep Reinforcement Learning Summary with DeepRacer as an Example
		Step 5: Improving Reinforcement Learning Models
			Algorithm settings
			Hyperparameters for the neural network
			Insights into model training
			Heatmap visualization
			Improving the speed of our model
	Racing the AWS DeepRacer Car
		Building the Track
		AWS DeepRacer Single-Turn Track Template
		Running the Model on AWS DeepRacer
		Driving the AWS DeepRacer Vehicle Autonomously
			Sim2Real transfer
	Further Exploration
		DeepRacer League
		Advanced AWS DeepRacer
		AI Driving Olympics
		DIY Robocars
		Roborace
	Summary
A. A Crash Course in Convolutional Neural Networks
	Machine Learning
	Perceptron
	Activation Functions
	Neural Networks
	Backpropagation
	Shortcoming of Neural Networks
	Desired Properties of an Image Classifier
	Convolution
	Pooling
	Structure of a CNN
	Further Exploration
Index