Practical Event-Driven Microservices Architecture

Table of Contents
About the Author
About the Technical Reviewer
Acknowledgments
Introduction
Chapter 1: Embracing Event-Driven Architectures
	1.1	 The Truth About Monoliths
		1.1.1 Anatomy of a Typical Monolith
			Patchwork Monoliths
				Modular Monoliths
		1.1.2 What They Don’t Tell You About Monoliths; It’s Not All Bad
			Business Flow Is Visible
			No Network Overhead and Limited External Dependencies
			Local Validation
			Code Reuse
			Monitoring and Troubleshooting
			End-to-End Testing
			Simpler Deployment Strategy
			Data Is Centralized
			Possible to Scale
			Consistency
			Concurrency
		1.1.3 When Monoliths Become the Business Constrictor Knot
			Coupling and Lack of Boundaries
			Team’s Autonomy
			Release Cycle
			Scaling
			Outdated Technology Stack
			Reliability
		1.1.4 Using Event-Driven Architectures to Move Away from a Monolith
	1.2	 What Are Microservices and How Do They Relate to Event-Driven
		1.2.1 Deployment
		1.2.2 Decoupled and Autonomously Developed
		1.2.3 Data Ownership
	1.3	 SOA, Microservice, and Event-Driven Architectures
		1.3.1 SOA
		1.3.2 Microservice Architecture
		1.3.3 Event-Driven Microservice Architecture
	1.4	 The Promise of Event-Driven Microservices
		1.4.1 Evolving Architecture
		1.4.2 Deployment
		1.4.3 Team’s Autonomy
		1.4.4 Flexible Technology Stack
		1.4.5 Resilience and Availability
		1.4.6 Tunable Scaling
		1.4.7 The Past on Demand
	1.5	 When Should You Use Event-Driven Microservices?
	1.6	 Overview of the Challenges in Event-Driven Architectures
	1.7	 Summary
Chapter 2: Moving from a Monolith to an Event-Driven Architecture
	2.1 Is Migrating to an Event-Driven Architecture Your Best Option?
	2.2 Moving to an Event-Driven Architecture, How to Decide Where to Start
	2.3 Using an Event-Driven Approach to Move Data from a Monolith
	2.4 Using Change Data Capture (CDC) to Move Data from a Monolith
		2.4.1 Event-Driven and Change Data Capture (CDC), a  Real-World Example
			CDC Example Using SQL Server and Kafka
	2.5 Migrating Data from a Monolith: Event-Driven As a Source of Truth for Both Systems
	2.6 Incremental Migration from a Monolith to an  Event-Driven Architecture: Managing Dependencies
		2.6.1 Managing a Dependency from a New Event-Driven Service to a Legacy Monolith
		2.6.2 Managing a Dependency from a Legacy Application to a New Event-Driven Service
	2.7 Gradually Moving Traffic to New Microservices
	2.8 Migrating from a Monolith: Two-Way Synchronization and Living with Two Sources of Truth
	2.9 Summary
Chapter 3: Defining an Event-Driven Microservice and Its Boundaries
	3.1 Building Event-Driven Microservices
		3.1.1 N-Tier Architectures
		3.1.2 Clean Architecture
		3.1.3 Event-Driven Microservices: Durable vs. Ephemeral Message Brokers and GDPR
			The Right to Be Forgotten and Other Security Concerns
		3.1.4 Event-Driven Message Types
		3.1.5 Event-Driven Microservices: When to Use Documents over Events
		3.1.6 Common Event-Driven Messaging Patterns
			Send/Receive Pattern
			Publish/Subscribe Pattern
			Request/Response Pattern
		3.1.7 Event-Driven Service Topologies
		3.1.8 Common Event-Driven Pitfalls and Anti-patterns
			Faking Synchronous Responses
			Command Publishing
			Passive-Aggressive Events
	3.2 Organizing Event-Driven Microservice Boundaries
		3.2.1 Organizational Composition
		3.2.2 Likelihood of Changes
		3.2.3 Type of Data
	3.3 Brief and Practical Introduction to Domain-Driven Design and Bounded Contexts
		3.3.1 How We Can Apply It in Practice
	3.4 Event-Driven Microservices: The Impact of Aggregate Size and Common Pitfalls
	3.5 Request-Driven vs. Event-Driven Services
	3.6 Deciding When to Create a New Microservice or Add Functionality to an Existing One
	3.7 Summary
Chapter 4: Structural Patterns and Chaining Processes
	4.1 The Challenges of Transactional Consistency in Distributed Systems
		4.1.1 Why Move from a Monolithic Database in the First Place?
		4.1.2 The Limitations of Distributed Transactions
		4.1.3 Managing Multi-step Processes with Sagas
	4.2 Event-Driven Orchestration Pattern
	4.3 Event-Driven Choreography Pattern
	4.4 Event-Driven Microservices: Orchestration, Choreography, or Both?
	4.5 Data Retrieval in Event-Driven Architectures and Associated Patterns
		4.5.1 CQS, CQRS, and When to Use Them
			When to Use CQRS?
		4.5.2 The Different Flavors of CQRS
		4.5.3 When and How to Use Event Sourcing
		4.5.4 Concerns and When to Use Event Sourcing
		4.5.5 Using Command Sourcing and Its Applicability
	4.6 Building Multiple Read Models in Event-Driven Microservice Architectures
	4.7 The Pitfall of Microservice Spaghetti Architectures and How to Avoid It
		4.7.1 Domain Segregation and Clear Boundaries
		4.7.2 Context Maps
		4.7.3 Distributed Tracing
	4.8 Summary
Chapter 5: How to Manage Eventual Consistency
	5.1 The Impacts of Eventual Consistency and the Need for Alignment with the Business
		5.1.1 Safety
		5.1.2 Liveliness
		5.1.3 The CAP Theorem in the Real World
	5.2 Using Event Schema in Event-Driven Microservices to Leverage Eventual Consistency
	5.3 Applying Microservice Domain Boundaries to Leverage Eventual Consistency
	5.4 Handling Eventual Consistency Delays with Event Versioning
	5.5 Saving State in Event-Driven Microservices to Avoid Eventual Consistency
		5.5.1 Buffering State As an Alternative to Persistence
	5.6 Tackling Eventual Consistency with the End-to-End Argument: A Real-World Use Case
	5.7 For Most Use Cases, It’s Not Eventual If Nobody Notices
		5.7.1 Event-Driven Autoscaling Use Case with Prometheus and Kafka
	5.8 Discussing the Tradeoffs of Typical Eventual Consistency Handling Strategies
	5.9 Summary
Chapter 6: Dealing with Concurrency and Out-of-Order Messages
	6.1 Why Is Concurrency Different in a Monolith from an Event-Driven Architecture?
	6.2 Pessimistic vs. Optimistic Concurrency, When and When Not to Use
		6.2.1 Pessimistic vs. Optimistic Approaches
		6.2.2 Solving Concurrency by Implementation and by Design
	6.3 Using Optimistic Concurrency
	6.4 Using Pessimistic Concurrency
		6.4.1 Distributed Locks in Event-Driven Microservices
		6.4.2 Database Transactions As a Concurrency Approach in Distributed Microservices
	6.5 Dealing with Out-of-Order Events
		6.5.1 How Can Events Lose Their Order?
		6.5.2 Solving Out-of-Order Events with Versioning
	6.6 Using End-to-End Message Partitioning to Handle Concurrency and Guarantee Message Ordering
		6.6.1 Real-World Example of Event-Driven Message Routing Using Kafka
		6.6.2 The Relevance of Message Routing and Partitioning in Event-Driven Microservices
		6.6.3 Using End-to-End Partitioning to Handle Concurrency and Ordering
		6.6.4 Limitations of End-to-End Partitioning in Event-Driven Microservices
			Hotspotting
			Momentary Hotspots
			The Mix of Event-Driven and Synchronous APIs
	6.7 Summary
Chapter 7: Achieving Resilience and Event Processing Reliability in Event-Driven Microservices
	7.1 Common Failures in Microservice Architectures and How They Relate to Event-Driven Architectures
		7.1.1 Cascading Failures and Event-Driven Services
		7.1.2 Load Balancing and Rate Limiters in Event-Driven Services
	7.2 Understanding Message Delivery Semantics
		7.2.1 Exactly-Once Delivery Semantics in Kafka
	7.3 Avoiding Inconsistencies When Saving State and Publishing Events in Event-Driven Microservices
		7.3.1 Event Stream As the Only Source of Truth
		7.3.2 Outbox Pattern in Event-Driven Microservices
		7.3.3 Transactions and Compensating Actions to Avoid Inconsistencies in Event-Driven Microservices
	7.4 Applying ACID 2.0 As a Resilience Strategy in Event-Driven Microservices
	7.5 Avoiding Message Leak in Event-Driven Microservices
		7.5.1 Poison Events
	7.6 Applying Common Resilience Patterns in Event-Driven Microservices
		7.6.1 Retries As a Resilience Approach in Event-Driven Microservices
		7.6.2 Circuit Breakers in Event-Driven Microservices
	7.7 Recovering Data and Repairing State in Event-Driven Microservices
	7.8 Bulkhead Pattern in Event-Driven Microservices
		7.8.1 Priority Queues
	7.9 Conclusion
	7.10 Summary
Chapter 8: Choosing the Correct Event Schema Design in Event-Driven Microservices
	8.1 Event Storming and Event-Driven Microservices
		8.1.1 What Are the Limitations of Event Storming?
	8.2 Event Schema: Headers and Envelopes
		8.2.1 Headers vs. Envelopes in Event Schema
		8.2.2 Relevant Contextual Information in Events
	8.3 Town Crier Events Pattern
	8.4 Bee Events Pattern
	8.5 The Event Schema Goldilocks Principle
	8.6 Denormalized Event Schema
	8.7 Schema Evolution in Event-Driven Microservices
		8.7.1 Backward Compatibility
		8.7.2 Forward Compatibility
		8.7.3 Full Compatibility
		8.7.4 No Compatibility
		8.7.5 Managing Changes
		8.7.6 Event Stream Versioning
		8.7.7 Using a Downscaler/Upscaler When Evolving Event Schema
	8.8 Summary
Chapter 9: How to Leverage the User Interface in Event-Driven Microservice Architectures
	9.1 Using an Aggregating Layer to Build a UI in a Distributed Microservice Architecture
	9.2 Backends for Frontends (BFFs)
	9.3 UI Decomposition Pattern in Microservice Architectures
		9.3.1 UI Application Decomposition Pattern
		9.3.2 UI Page Decomposition Pattern
		9.3.3 UI Section Decomposition Pattern
	9.4 The Limitations of API Composition
	9.5 Task-Based UIs
	9.6 Event-Driven APIs
		9.6.1 Event-Driven Combined with WebSockets
		9.6.2 Event-Driven Combined with Server-Sent Events
		9.6.3 Event-Driven Combined with WebHooks
	9.7 Summary
Chapter 10: Overcoming the Challenges in Quality Assurance
	10.1 Microservice Testing Approaches and How They Relate to Event-Driven
		10.1.1 Unit Tests
		10.1.2 Component Tests
		10.1.3 Extended Component Tests
		10.1.4 Integration Tests
		10.1.5 Half-Breed Integration Tests
		10.1.6 End-to-End Tests
			Releases
			Data and Environments
			Governance
			Approaches to Mitigate the End-to-End Tests Challenges
	10.2 Applying Contract Tests and Consumer-Driven Contracts to Event-Driven
	10.3 Test Categorization and Purpose
	10.4 End-to-End Quality Without End-to-End Tests
	10.5 Testing in Production
		10.5.1 Shadowing
		10.5.2 Canaries
		10.5.3 Feature Flagging
		10.5.4 Production Automated Testing
	10.6 Summary
Index