Driving Data Quality with Data Contracts: A comprehensive guide to building reliable, trusted, and effective data platforms

Cover
Title Page
Copyright and Credits
Dedication
Foreword
Contributors
Table of Contents
Preface
Part 1: Why Data Contracts?
Chapter 1: A Brief History of Data Platforms
	The enterprise data warehouse
	The big data platform
	The modern data stack
	The state of today’s data platforms
		The lack of expectations
		The lack of reliability
		The lack of autonomy
	The ever-increasing use of data in business-critical applications
	Summary
	Further reading
Chapter 2: Introducing Data Contracts
	What is a data contract?
		An agreed interface between the generators of data, and its consumers
		Setting expectations around that data
		Defining how the data should be governed
		Facilitating the explicit generation of quality data
		The four principles of data contracts
	When to use data contracts
	Data contracts and the data mesh
		Domain ownership
		Data as a product
		Self-serve data platform
		Federated computational governance
		Data contracts enable a data mesh
	Summary
	Further reading
Part 2: Driving Data Culture Change with Data Contracts
Chapter 3: How to Get Adoption in Your Organization
	Using data contracts to change an organization
	Articulating the value of your data
	Building data products
		What is a data product?
		Adopting a data product mindset
		Designing a data product
	Walking through an example of a data product
	Summary
	Further reading
Chapter 4: Bringing Data Consumers and Generators Closer Together
	Who is a consumer, and who is a generator?
		Data consumers
		Data generators
	Assigning responsibility and accountability
	Feeding data back to the product teams
	Managing the evolution of data
	Summary
	Further reading
Chapter 5: Embedding Data Governance
	Why we need data governance
		The requirements of data governance
		How data governance programs are typically applied
	Promoting data governance through data contracts
	Assigning responsibility for data governance
		Responsibilities of the data generators
		Introducing the data architecture council
		Working together to implement federated data governance
	Summary
	Further reading
Part 3: Designing and Implementing a Data Architecture Based on Data Contracts
Chapter 6: What Makes Up a Data Contract
	The schema of a data contract
		Defining a schema
		Using a schema registry as the source of truth
	Evolving your data over time
		Evolving your schemas
		Migrating your consumers
	Defining the governance and controls
	Summary
	Further reading
Chapter 7: A Contract-Driven Data Architecture
	A step-change in building data platforms
		Building generic data tooling
		Introducing a data infrastructure team
		A case study from GoCardless in promoting autonomy
		Promoting autonomy through decentralization
	Introducing the principles of a contract-driven data architecture
		Automation
		Guidelines and guardrails
		Consistency
	Providing self-served data infrastructure
	Summary
	Further reading
Chapter 8: A Sample Implementation
	Technical requirements
	Creating a data contract
	Providing the interfaces to the data
		Introducing IaC
		Creating the interfaces from the data contract
	Creating libraries for data generators
	Populating a central schema registry
		Registering a schema with the Confluent schema registry
		Managing schema evolution
	Implementing contract-driven tooling
	Summary
	Further reading
Chapter 9: Implementing Data Contracts in Your Organization
	Getting started with data contracts
		The ability to define a data contract
		The ability to provision an interface for the data for consumers to query
		The ability of generators to write data to the interface
	Migrating to data contracts
	Discovering data contracts
		What is a data catalog?
		Why are data catalogs important for discovering data contracts?
		What is data lineage?
		Why is data lineage important for data contracts?
	Building a mature data contracts-backed data culture
	Summary
	Further reading
Chapter 10: Data Contracts in Practice
	Designing a data contract
		Identifying the purpose
		Considering the trade-offs
		Defining the data contract
		Deploying the data contract
	Monitoring and enforcing data contracts
		The data contract’s definition
		The quality of the data
		The performance and dependability of the data
	Data contract publishing patterns
		Writing directly to the interface
		Materialized views on CDC
		The transactional outbox pattern
		The listen-to-yourself pattern
	Summary
	Further reading
Index
About Packt
Other Books You May Enjoy