Engineering Deep Learning Systems Version 4

Engineering Deep Learning Systems MEAP V04
Copyright
welcome
brief contents
Chapter 1: An Introduction to Deep Learning Systems
	1.1 Deep learning system overview
	1.2 Deep learning development cycle
		1.2.1 Different roles in the cycle
		1.2.2 Walkthrough of a typical deep learning development cycle
	1.3 Differences between building a deep learning system and developing a model
	1.4 Summary
Chapter 2: A Hello World Deep Learning System
	2.1 Introducing the Helloworld deep learning system
		2.1.1 Personas
		2.1.2 System overview
		2.1.3 User workflows
	2.2 Lab demo
		2.2.1 Demo steps:
		2.2.2 Further exercise
	2.3 Summary
Chapter 3: Deep Learning System Design
	3.1 High level architecture
		3.1.1 Gathering goals and requirements
		3.1.2 Reference architecture
	3.2 Key components
		3.2.1 Dataset management
		3.2.2 Model training
		3.2.3 Model serving
		3.2.4 Metadata & artifacts store
		3.2.5 Workflow management
		3.2.6 Experimentation
		3.2.7 Why do we recommend building components on top of Kubernetes?
	3.3 Shipping to production
		3.3.1 Updating models
		3.3.2 Updating services
	3.4 Survey of existing solutions
		3.4.1 Amazon SageMaker
		3.4.2 Google Vertex AI
		3.4.3 Microsoft Azure Machine Learning
		3.4.4 Kubeflow
		3.4.5 Side-by-side comparison
	3.5 Summary
Chapter 4: Dataset Management Service
	4.1 Understanding dataset management service
		4.1.1 Why deep learning systems need dataset management
		4.1.2 Dataset management (DM) design principles
	4.2 Touring sample dataset management (DM) service
		4.2.1 Play with the sample service
		4.2.2 Users, User scenarios and the big picture
		4.2.3 Data Ingestion API
		4.2.4 Fetch Training Dataset API
		4.2.5 Internal dataset storage
		4.2.6 Data schemas
		4.2.7 Adding new dataset type (MAGE_CLASS)
		4.2.8 Service design recap
	4.3 Open source approaches
		4.3.1 Delta Lake and Petaform with Apache Spark family
		4.3.2 Pachyderm with cloud object storage
	4.4 Summary
Chapter 5: Model Training Service
	5.1 Model training service: design overview
		5.1.1 Why do model training via a service?
		5.1.2 Training service design principles
	5.2 Deep learning training code pattern
		5.2.1 Model training workflow
		5.2.2 Dockerize model training code as a blackbox
	5.3 A sample model training service
		5.3.1 Play with the service
		5.3.2 Service design overview
		5.3.3 Training service API
		5.3.4 Launching a new training job
		5.3.5 Updating and fetching job status
		5.3.6 The intent classification model training code
		5.3.7 Training job management
		5.3.8 Troubleshooting metrics
		5.3.9 Supporting new algorithm or new version
	5.4 Kubeflow training operators, an open source approach
		5.4.1 Kubeflow training operators
		5.4.2 Kubernetes operator/controller pattern
		5.4.3 Kubeflow training operator design
		5.4.4 How to use Kubeflow training operators?
		5.4.5 How to integrate these operators to an existing system?
	5.5 When to use Public Cloud and when not?
		5.5.1 When to use a public cloud solution?
		5.5.2 When to build our own training service?
	5.6 Summary
Chapter 6: Distributed Training
	6.1 Types of distributed training methods
	6.2 Data Parallelism
		6.2.1 Understanding Data Parallelism
		6.2.2 Multi-worker training challenges
		6.2.3 Write Distributed training (data parallelism) code in different training frameworks
		6.2.4 Engineering effort in data parallel distributed training
	6.3 A sample service supports data parallel distributed training
		6.3.1 Service overview
		6.3.2 Play with the service
		6.3.3 Launching training jobs
		6.3.4 Updating and fetching job status
		6.3.5 Convert the training code to run distributedly
		6.3.6 Improvements
	6.4 Training large models that can’t load on one GPU
		6.4.1 Traditional methods - Memory saving
		6.4.2 Pipeline model parallelism
		6.4.3 How can software engineers support pipeline parallelism?
	6.5 Summary
Chapter 7: Hyper parameter optimization service
	7.1 Understanding Hyper Parameters
		7.1.1 What is a hyper parameter?
		7.1.2 Why are hyper parameters important?
	7.2 Understanding Hyper Parameter Optimization (HPO)
		7.2.1 What is HPO?
		7.2.2 Popular HPO algorithms
		7.2.3 Common automatic HPO approaches
	7.3 Opensource HPO Libraries
		7.3.1 Hyperopt
		7.3.2 Optuna
		7.3.3 Ray Tune
	7.4 Designing HPO service
		7.4.1 HPO Design principles
		7.4.2 A general HPO service design
	7.5 Creating a HPO service with Kubeflow Katib
		7.5.1 Katib overview
		7.5.2 Getting started with Katib
		7.5.3 Expedite HPO
		7.5.4 Katib system design
		7.5.5 Adding a new algorithm.
		7.5.6 Further reading
		7.5.7 When to use?
	7.6 Summary