Designing Distributed Systems

Cover
Copyright
Table of Contents
Preface
	Who Should Read This Book
	Why I Wrote This Book
	The World of Distributed Systems Today
	Navigating This Book
	Conventions Used in This Book
	Online Resources
	Using Code Examples
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
Part I. Foundational Concepts
	Chapter 1. Introduction
		A Brief History of Systems Development
		A Brief History of Patterns in Software Development
			Formalization of Algorithmic Programming
			Patterns for Object-Oriented Programming
			The Rise of Open Source Software
		The Value of Patterns, Practices, and Components
			Standing on the Shoulders of Giants
			A Shared Language for Discussing Our Practice
			Shared Components for Easy Reuse
		Summary
	Chapter 2. Important Distributed System Concepts
		APIs and RPCs
		Latency
		Reliability
		Percentiles
		Idempotency
		Delivery Semantics
		Relational Integrity
		Data Consistency
		Orchestration and Kubernetes
		Health Checks
		Summary
Part II. Single-Node Patterns
	Chapter 3. The Sidecar Pattern
		An Example Sidecar: Adding HTTPS to a Legacy Service
		Dynamic Configuration with Sidecars
		Modular Application Containers
			Hands On: Deploying the topz Container
		Building a Simple PaaS with Sidecars
		Designing Sidecars for Modularity and Reusability
			Parameterized Containers
			Define Each Container’s API
			Documenting Your Containers
		Summary
	Chapter 4. Ambassadors
		Using an Ambassador to Shard a Service
			Hands On: Implementing a Sharded Redis
		Using an Ambassador for Service Brokering
		Using an Ambassador to Do Experimentation or Request Splitting
			Hands On: Implementing 10% Experiments
		Summary
	Chapter 5. Adapters
		Monitoring
			Hands On: Using Prometheus for Monitoring
		Logging
			Hands On: Normalizing Different Logging Formats with fluentd
		Adding a Health Monitor
			Hands On: Adding Rich Health Monitoring for MySQL
		Summary
Part III. Serving Patterns
	Chapter 6. Replicated Load-Balanced Services
		Stateless Services
			Readiness Probes for Load Balancing
			Hands On: Creating a Replicated Service in Kubernetes
		Session Tracked Services
		Application-Layer Replicated Services
		Introducing a Caching Layer
			Deploying Your Cache
			Hands On: Deploying the Caching Layer
		Expanding the Caching Layer
			Rate Limiting and Denial-of-Service Defense
			SSL Termination
			Hands On: Deploying nginx and SSL Termination
		Summary
	Chapter 7. Sharded Services
		Sharded Caching
			Why You Might Need a Sharded Cache
			The Role of the Cache in System Performance
			Replicated Sharded Caches
			Hands On: Deploying an Ambassador and Memcache for a Sharded Cache
		An Examination of Sharding Functions
			Selecting a Key
			Consistent Hashing Functions
			Hands On: Building a Consistent HTTP Sharding Proxy
		Sharded Replicated Serving
		Hot Sharding Systems
		Summary
	Chapter 8. Scatter/Gather
		Scatter/Gather with Root Distribution
			Hands On: Distributed Document Search
		Scatter/Gather with Leaf Sharding
			Hands On: Sharded Document Search
			Choosing the Right Number of Leaves
		Scaling Scatter/Gather for Reliability and Scale
		Summary
	Chapter 9. Functions and Event-Driven Processing
		Determining When FaaS Makes Sense
			The Benefits of FaaS
			The Challenges of FaaS
			The Need for Background Processing
			The Need to Hold Data in Memory
			The Costs of Sustained Request-Based Processing
		Patterns for FaaS
			The Decorator Pattern: Request or Response Transformation
			Hands On: Adding Request Defaulting Prior to Request Processing
			Handling Events
			Hands On: Implementing Two-Factor Authentication
			Event-Based Pipelines
			Hands On: Implementing a Pipeline for New User Signup
		Summary
	Chapter 10. Ownership Election
		Determining If You Even Need Leader Election
		The Basics of Leader Election
			Hands On: Deploying etcd
			Implementing Locks
			Hands On: Implementing Locks in etcd
			Implementing Ownership
			Hands On: Implementing Leases in etcd
		Handling Concurrent Data Manipulation
		Summary
Part IV. Batch Computational Patterns
	Chapter 11. Work Queue Systems
		A Generic Work Queue System
			The Source Container Interface
			Work Queue API
			The Worker Container Interface
			The Shared Work Queue Infrastructure
		Hands On: Implementing a Video Thumbnailer
		Dynamic Scaling of the Workers
		The Multiworker Pattern
		Summary
	Chapter 12. Event-Driven Batch Processing
		Patterns of Event-Driven Processing
			Copier
			Filter
			Splitter
			Sharder
			Merger
		Hands On: Building an Event-Driven Flow for New User Signup
		Publisher/Subscriber Infrastructure
		Hands On: Deploying Kafka
		Resiliency and Performance in Work Queues
			Work Stealing
			Errors, Priority, and Retry
		Summary
	Chapter 13. Coordinated Batch Processing
		Join (or Barrier Synchronization)
		Reduce
			Hands On: Count
			Sum
			Histogram
			Hands On: An Image Tagging and Processing Pipeline
		Summary
Part V. Universal Concepts
	Chapter 14. Monitoring and Observability Patterns
		Monitoring and Observability Basics
			Logging
			Metrics
			Basic Request Monitoring
			Advanced Request Monitoring
			Alerting
			Tracing
		Aggregating Information
		Summary
	Chapter 15. AI Inference and Serving
		The Basics of AI Systems
		Hosting a Model
		Distributing a Model
		Development with Models
		Retrieval-Augmented Generation
		Testing and Deployment
		Summary
	Chapter 16. Common Failure Patterns
		The Thundering Herd
		The Absence of Errors Is an Error
		“Client” and “Expected” Errors
		Versioning Errors
		The Myth of Optional Components
		Oops, We “Cleaned Up” Everything
		Challenges with the Breadth of Inputs
		Processing Obsolete Work
		The “Second System” Problem
		Summary
Conclusion: A New Beginning?
Index
About the Author
Colophon