The Enterprise Big Data Lake: Delivering the Promise of Big Data and Data Science

Copyright
Table of Contents
Preface
	Who Should Read This Book?
	Conventions Used in This Book
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
Chapter 1. Introduction to Data Lakes
	Data Lake Maturity
		Data Puddles
		Data Ponds
	Creating a Successful Data Lake
		The Right Platform
		The Right Data
		The Right Interface
		The Data Swamp
	Roadmap to Data Lake Success
		Standing Up a Data Lake
		Organizing the Data Lake
		Setting Up the Data Lake for Self-Service
	Data Lake Architectures
		Data Lakes in the Public Cloud
		Logical Data Lakes
	Conclusion
Chapter 2. Historical Perspective
	The Drive for Self-Service Data—The Birth of Databases
	The Analytics Imperative—The Birth of Data Warehousing
	The Data Warehouse Ecosystem
		Storing and Querying the Data
		Loading the Data—Data Integration Tools
		Organizing and Managing the Data
		Consuming the Data
	Conclusion
Chapter 3. Introduction to Big Data and Data Science
	Hadoop Leads the Historic Shift to Big Data
		The Hadoop File System
		How Processing and Storage Interact in a MapReduce Job
		Schema on Read
		Hadoop Projects
	Data Science
	What Should Your Analytics Organization Focus On?
	Machine Learning
		Explainability
		Change Management
	Conclusion
Chapter 4. Starting a Data Lake
	The What and Why of Hadoop
	Preventing Proliferation of Data Puddles
	Taking Advantage of Big Data
		Leading with Data Science
		Strategy 1: Offload Existing Functionality
		Strategy 2: Data Lakes for New Projects
		Strategy 3: Establish a Central Point of Governance
		Which Way Is Right for You?
	Conclusion
Chapter 5. From Data Ponds/Big Data Warehouses to Data Lakes
	Essential Functions of a Data Warehouse
		Dimensional Modeling for Analytics
		Integrating Data from Disparate Sources
		Preserving History Using Slowly Changing Dimensions
		Limitations of the Data Warehouse as a Historical Repository
	Moving to a Data Pond
		Keeping History in a Data Pond
		Implementing Slowly Changing Dimensions in a Data Pond
	Growing Data Ponds into a Data Lake—Loading Data That’s Not in the Data Warehouse
		Raw Data
		External Data
		Internet of Things (IoT) and Other Streaming Data
	Real-Time Data Lakes
	The Lambda Architecture
	Data Transformations
	Target Systems
		Data Warehouses
		Operational Data Stores
		Real-Time Applications and Data Products
	Conclusion
Chapter 6. Optimizing for Self-Service
	The Beginnings of Self-Service
	Business Analysts
		Finding and Understanding Data—Documenting the Enterprise
		Establishing Trust
		Provisioning
		Preparing Data for Analysis
	Data Wrangling in the Data Lake
		Situating Data Preparation in Hadoop
		Common Use Cases for Data Preparation
	Analyzing and Visualizing
	The New World of Self-Service Business Intelligence
		The New Analytic Workflow
		Gatekeepers to Shopkeepers
		Governing Self-Service
	Conclusion
Chapter 7. Architecting the Data Lake
	Organizing the Data Lake
		Landing or Raw Zone
		Gold Zone
		Work Zone
		Sensitive Zone
	Multiple Data Lakes
		Advantages of Keeping Data Lakes Separate
		Advantages of Merging the Data Lakes
	Cloud Data Lakes
	Virtual Data Lakes
		Data Federation
		Big Data Virtualization
		Eliminating Redundancy
	Conclusion
Chapter 8. Cataloging the Data Lake
	Organizing the Data
		Technical Metadata
		Business Metadata
	Tagging
		Automated Cataloging
	Logical Data Management
		Sensitive Data Management and Access Control
		Data Quality
	Relating Disparate Data
	Establishing Lineage
	Data Provisioning
	Tools for Building a Catalog
		Tool Comparison
	The Data Ocean
	Conclusion
Chapter 9. Governing Data Access
	Authorization or Access Control
	Tag-Based Data Access Policies
	Deidentifying Sensitive Data
		Data Sovereignty and Regulatory Compliance
	Self-Service Access Management
		Provisioning Data
	Conclusion
Chapter 10. Industry-Specific Perspectives
	Big Data in Financial Services
		Consumers, Digitization, and Data Are Changing Finance as We Know It
		Saving the Bank
		New Opportunities Offered by New Data
		Key Processes in Making Use of the Data Lake
	Value Added by Data Lakes in Financial Services
	Data Lakes in the Insurance Industry
	Smart Cities
	Big Data in Medicine
Index
About the Author
Colophon