Apache Pulsar in Action

Apache Pulsar in Action
brief contents
contents
foreword
preface
acknowledgments
about this book
	Who should read this book
	How this book is organized: A roadmap
	About the code
	Other online resources
	liveBook discussion forum
about the author
about the cover illustration
Part 1: Getting started with Apache Pulsar
	Chapter 1: Introduction to Apache Pulsar
		1.1 Enterprise messaging systems
			1.1.1 Key capabilities
		1.2 Message consumption patterns
			1.2.1 Publish–subscribe messaging
			1.2.2 Message queuing
		1.3 The evolution of messaging systems
			1.3.1 Generic messaging systems
			1.3.2 Message-oriented middleware
			1.3.3 Enterprise service bus
			1.3.4 Distributed messaging systems
		1.4 Comparison to Apache Kafka
			1.4.1 Multilayered architecture
			1.4.2 Message consumption
			1.4.3 Data durability
			1.4.4 Message acknowledgment
			1.4.5 Message retention
		1.5 Why do I need Pulsar?
			1.5.1 Guaranteed message delivery
			1.5.2 Infinite scalability
			1.5.3 Resilient to failure
			1.5.4 Support for millions of topics
			1.5.5 Geo-replication and active failover
		1.6 Real-world use cases
			1.6.1 Unified messaging systems
			1.6.2 Microservices platforms
			1.6.3 Connected cars
			1.6.4 Fraud detection
	Chapter 2: Pulsar concepts and architecture
		2.1 Pulsar’s physical architecture
			2.1.1 Pulsar’s layered architecture
			2.1.2 Stateless serving layer
			2.1.3 Stream storage layer
			2.1.4 Metadata storage
		2.2 Pulsar’s logical architecture
			2.2.1 Tenants, namespaces, and topics
			2.2.2 Addressing topics in Pulsar
			2.2.3 Producers, consumers, and subscriptions
			2.2.4 Subscription types
		2.3 Message retention and expiration
			2.3.1 Data retention
			2.3.2 Backlog quotas
			2.3.3 Message expiration
			2.3.4 Message backlog vs. message expiration
		2.4 Tiered storage
	Chapter 3: Interacting with Pulsar
		3.1 Getting started with Pulsar
		3.2 Administering Pulsar
			3.2.1 Creating a tenant, namespace, and topic
			3.2.2 Java Admin API
		3.3 Pulsar clients
			3.3.1 The Pulsar Java client
			3.3.2 The Pulsar Python client
			3.3.3 The Pulsar Go client
		3.4 Advanced administration
			3.4.1 Persistent topic metrics
			3.4.2 Message inspection
Part 2: Apache Pulsar development essentials
	Chapter 4: Pulsar functions
		4.1 Stream processing
			4.1.1 Traditional batching
			4.1.2 Micro-batching
			4.1.3 Stream native processing
		4.2 What is Pulsar Functions?
			4.2.1 Programming model
		4.3 Developing Pulsar functions
			4.3.1 Language native functions
			4.3.2 The Pulsar SDK
			4.3.3 Stateful functions
		4.4 Testing Pulsar functions
			4.4.1 Unit testing
			4.4.2 Integration testing
		4.5 Deploying Pulsar functions
			4.5.1 Generating a deployment artifact
			4.5.2 Function configuration
			4.5.3 Function deployment
			4.5.4 The function deployment life cycle
			4.5.5 Deployment modes
			4.5.6 Pulsar function data flow
	Chapter 5: Pulsar IO connectors
		5.1 What are Pulsar IO connectors?
			5.1.1 Sink connectors
			5.1.2 Source connectors
			5.1.3 PushSource connectors
		5.2 Developing Pulsar IO connectors
			5.2.1 Developing a sink connector
			5.2.2 Developing a PushSource connector
		5.3 Testing Pulsar IO connectors
			5.3.1 Unit testing
			5.3.2 Integration testing
			5.3.3 Packaging Pulsar IO connectors
		5.4 Deploying Pulsar IO connectors
			5.4.1 Creating and deleting connectors
			5.4.2 Debugging deployed connectors
		5.5 Pulsar’s built-in connectors
			5.5.1 Launching the MongoDB cluster
			5.5.2 Link the Pulsar and MongoDB containers
			5.5.3 Configure and create the MongoDB sink
		5.6 Administering Pulsar IO connectors
			5.6.1 Listing connectors
			5.6.2 Monitoring connectors
	Chapter 6: Pulsar security
		6.1 Transport encryption
		6.2 Authentication
			6.2.1 TLS authentication
			6.2.2 JSON Web Token authentication
		6.3 Authorization
			6.3.1 Roles
			6.3.2 An example scenario
		6.4 Message encryption
	Chapter 7: Schema registry
		7.1 Microservice communication
			7.1.1 Microservice APIs
			7.1.2 The need for a schema registry
		7.2 The Pulsar schema registry
			7.2.1 Architecture
			7.2.2 Schema versioning
			7.2.3 Schema compatibility
			7.2.4 Schema compatibility check strategies
		7.3 Using the schema registry
			7.3.1 Modelling the food order event in Avro
			7.3.2 Producing food order events
			7.3.3 Consuming the food order events
			7.3.4 Complete example
		7.4 Evolving the schema
Part 3: Hands-on application development with Apache Pulsar
	Chapter 8: Pulsar Functions patterns
		8.1 Data pipelines
			8.1.1 Procedural programming
			8.1.2 DataFlow programming
		8.2 Message routing patterns
			8.2.1 Splitter pattern
			8.2.2 Dynamic router pattern
			8.2.3 Content-based router pattern
		8.3 Message transformation patterns
			8.3.1 Message translator pattern
			8.3.2 Content enricher pattern
			8.3.3 Content filter pattern
	Chapter 9: Resiliency patterns
		9.1 Pulsar Functions resiliency
			9.1.1 Adverse events
			9.1.2 Fault detection
		9.2 Resiliency design patterns
			9.2.1 Retry pattern
			9.2.2 Circuit breaker pattern
			9.2.3 Rate limiter pattern
			9.2.4 Time limiter pattern
			9.2.5 Cache pattern
			9.2.6 Fallback pattern
			9.2.7 Credential refresh pattern
		9.3 Multiple layers of resiliency
	Chapter 10: Data access
		10.1 Data sources
		10.2 Data access use cases
			10.2.1 Device validation
			10.2.2 Driver location data
	Chapter 11: Machine learning in Pulsar
		11.1 Deploying ML models
			11.1.1 Batch processing
			11.1.2 Near real-time
		11.2 Near real-time model deployment
		11.3 Feature vectors
			11.3.1 Feature stores
			11.3.2 Feature calculation
		11.4 Delivery time estimation
			11.4.1 ML model export
			11.4.2 Feature vector mapping
			11.4.3 Model deployment
		11.5 Neural nets
			11.5.1 Neural net training
			11.5.2 Neural net deployment in Java
	Chapter 12: Edge analytics
		12.1 IIoT architecture
			12.1.1 The perception and reaction layer
			12.1.2 The transportation layer
			12.1.3 The data processing layer
		12.2 A Pulsar-based processing layer
		12.3 Edge analytics
			12.3.1 Telemetric data
			12.3.2 Univariate and multivariate
		12.4 Univariate analysis
			12.4.1 Noise reduction
			12.4.2 Statistical analysis
			12.4.3 Approximation
		12.5 Multivariate analysis
			12.5.1 Creating a bidirectional messaging mesh
			12.5.2 Multivariate dataset construction
		12.6 Beyond the book
appendix A: Running Pulsar on Kubernetes
	A.1 Create a Kubernetes cluster
		A.1.1 Install prerequisites
		A.1.2 Minikube
	A.2 The Pulsar Helm chart
		A.2.1 What is Helm?
		A.2.2 The Pulsar Helm chart
	A.3 Using the Pulsar Helm chart
		A.3.1 Administering Pulsar on Kubernetes
		A.3.2 Configuring clients
appendix B: Geo-replication
	B.1 Synchronous geo-replication
	B.2 Asynchronous geo-replication
		B.2.1 Configuring asynchronous geo-replication
	B.3 Asynchronous geo-replication patterns
		B.3.1 Multi-active geo-replication
		B.3.2 Active-standby geo-replication
		B.3.3 Aggregation geo-replication
index
	A
	B
	C
	D
	E
	F
	G
	H
	I
	J
	K
	L
	M
	N
	O
	P
	Q
	R
	S
	T
	U
	V
	W
	Y
	Z