Build an Orchestrator in Go (From Scratch) (Final Release)

inside front cover
Build an Orchestrator in Go (From Scratch)
Copyright
dedication
contents
Front matter
   preface
   acknowledgments
   about this book
      Who should read this book
      How this book is organized: A road map
      About the code
      liveBook discussion forum
   about the author
   about the cover illustration
Part 1 Introduction
1 What is an orchestrator?
   1.1 Why implement an orchestrator from scratch?
   1.2 The (not so) good ol’ days
   1.3 What is a container, and how is it different from a virtual machine?
   1.4 What is an orchestrator?
   1.5 The components of an orchestration system
      1.5.1 The task
      1.5.2 The job
      1.5.3 The scheduler
      1.5.4 The manager
      1.5.5 The worker
      1.5.6 The cluster
      1.5.7 Command-line interface
   1.6 Meet Cube
   1.7 What tools will we use?
   1.8 A word about hardware
   1.9 What we won’t be implementing or discussing
      1.9.1 Distributed computing
      1.9.2 Service discovery
      1.9.3 High availability
      1.9.4 Load balancing
      1.9.5 Security
   Summary
2 From mental model to skeleton code
   2.1 The task skeleton
   2.2 The worker skeleton
   2.3 The manager skeleton
   2.4 The scheduler skeleton
   2.5 Other skeletons
   2.6 Taking our skeletons for a spin
   Summary
3 Hanging some flesh on the task skeleton
   3.1 Docker: Starting, stopping, and inspecting containers from the command line
   3.2 Docker: Starting, stopping, and inspecting containers from the API
   3.3 Task configuration
   3.4 Starting and stopping tasks
   Summary
Part 2 Worker
4 Workers of the Cube, unite!
   4.1 The Cube worker
   4.2 Tasks and Docker
   4.3 The role of the queue
   4.4 The role of the DB
   4.5 Counting tasks
   4.6 Implementing the worker’s methods
      4.6.1 Implementing the StopTask method
      4.6.2 Implementing the StartTask method
      4.6.3 An interlude on task state
      4.6.4 Implementing the RunTask method
   4.7 Putting it all together
   Summary
5 An API for the worker
   5.1 Overview of the worker API
   5.2 Data format, requests, and responses
   5.3 The API struct
   5.4 Handling requests
   5.5 Serving the API
   5.6 Putting it all together
   Summary
6 Metrics
   6.1 What metrics should we collect?
   6.2 Metrics available from the /proc filesystem
   6.3 Collecting metrics with goprocinfo
   6.4 Exposing the metrics on the API
   6.5 Putting it all together
   Summary
Part 3 Manager
7 The manager enters the room
   7.1 The Cube manager
      7.1.1 The components that make up the manager
   7.2 The Manager struct
   7.3 Implementing the manager’s methods
      7.3.1 Implementing the SelectWorker method
      7.3.2 Implementing the SendWork method
      7.3.3 Implementing the UpdateTasks method
      7.3.4 Adding a task to the manager
      7.3.5 Creating a manager
   7.4 An interlude on failures and resiliency
   7.5 Putting it all together
   Summary
8 An API for the manager
   8.1 Overview of the manager API
   8.2 Routes
   8.3 Data format, requests, and responses
   8.4 The API struct
   8.5 Handling requests
   8.6 Serving the API
   8.7 A few refactorings to make our lives easier
   8.8 Putting it all together
   Summary
9 What could possibly go wrong?
   9.1 Overview of our new scenario
   9.2 Failure scenarios
      9.2.1 Application startup failure
      9.2.2 Application bugs
      9.2.3 Task startup failures due to resource problems
      9.2.4 Task failures due to Docker daemon crashes and restarts
      9.2.5 Task failures due to machine crashes and restarts
      9.2.6 Worker failures
      9.2.7 Manager failures
   9.3 Recovery options
      9.3.1 Recovery from application failures
      9.3.2 Recovering from environmental failures
      9.3.3 Recovering from task-level failures
      9.3.4 Recovering from worker failures
      9.3.5 Recovering from manager failures
   9.4 Implementing health checks
      9.4.1 Inspecting a task on the worker
      9.4.2 Implementing task updates on the worker
      9.4.3 Healthchecks and restarts
   9.5 Putting it all together
   Summary
Part 4 Refactorings
10 Implementing a more sophisticated scheduler
   10.1 The scheduling problem
   10.2 Scheduling considerations
   10.3 Scheduler interface
   10.4 Adapting the round-robin scheduler to the scheduler interface
   10.5 Using the new scheduler interface
      10.5.1 Adding new fields to the Manager struct
      10.5.2 Modifying the New helper function
   10.6 Did you notice the bug?
   10.7 Putting it all together
   10.8 The E-PVM scheduler
      10.8.1 The theory
      10.8.2 In practice
   10.9 Completing the Node implementation
   10.10 Using the E-PVM scheduler
   Summary
11 Implementing persistent storage for tasks
   11.1 The storage problem
   11.2 The Store interface
   11.3 Implementing an in-memory store for tasks
   11.4 Implementing an in-memory store for task events
   11.5 Refactoring the manager to use the new in-memory stores
   11.6 Refactoring the worker
   11.7 Putting it all together
   11.8 Introducing BoltDB
   11.9 Implementing a persistent task store
   11.10 Implementing a persistent task event store
   11.11 Switching out the in-memory stores for permanent ones
   Summary
Part 5 CLI
12 Building a command-line interface
   12.1 The core components of CLIs
   12.2 Introducing the Cobra framework
   12.3 Setting up our Cobra application
   12.4 Understanding the new main.go
   12.5 Understanding root.go
   12.6 Implementing the worker command
   12.7 Implementing the manager command
   12.8 Implementing the run command
   12.9 Implementing the stop command
   12.10 Implementing the status command
   12.11 Implementing the node command
   Summary
13 Now what?
   13.1 Working on Kubernetes and related tooling
   13.2 Manager-worker pattern and workflow systems
   13.3 Manager-worker pattern and integration systems
   13.4 In closing
Appendix. Environment setup
   A.1 Installing Go
      A.1.1 Installing on Linux
   A.2 Project structure and initialization
index
inside back cover