Understanding ETL: Data Pipelines for Modern Data Architectures by Matt Palmer

Introduction
   The Brave New World of AI
   A Changing Data Landscape
   What About ELT (and Other Flavors)?
   O’Reilly Online Learning
   How to Contact Us
   Acknowledgments
1. Data Ingestion
   Data Ingestion—Now Versus Then
   Sources and Targets
      The Source
         Examining sources
         Questions to ask
         Source checklist
      The Destination
         Examining destinations
         Staging ingested data
         Change data capture
         Destination checklist
   Ingestion Considerations
      Frequency
         Batch
         Micro-batch
         Streaming
            Methods
            Message services
            Stream processing engines
            Simplifying stream processing
      Payload
         Volume
         Structure and shape
            Unstructured
            Semi-structured
            Structured
         Format
         Variety
   Choosing a Solution
      Declarative Solutions
         Cost to build/maintain
         Extensibility
         Cost to switch
      Imperative Solutions
         Extensibility
         Cost to build/maintain
         Cost to switch
      Hybrid Solutions
2. Data Transformation
   What Is Data Transformation?
      Where Are We Now?
      How Do We Transform Data?
         Environments
         Data staging
         Languages
         Frameworks
         Other approaches
   Building a Transformation Solution
      Data Transformation Patterns
      Data Update Patterns
      Best Practices
      Real-Time Data Transformation
   The Future of Data Transformation
3. Data Orchestration
   What Is Data Orchestration?
      Why Orchestrate?
      The DAG
   Data Orchestration Tools
      Choosing an Orchestrator
         Characteristics
         Orchestrator options
      Orchestrating SQL
   Design Patterns and Best Practices
   The Future of Data Orchestration
4. Pipeline Issues and Troubleshooting
   Maintainability
   Monitoring and Benchmarking
      Metrics
      Methods
   Errors
      Error Handling
      Recovery
   Improving Workflows
      Start with Relationships
       Align Incentives
      Improve Outcomes
5. Efficiency and Scalability
   Efficiency and Scalability Defined
   Understand Your Environment
      Frameworks
      Resource Allocation
         Parallelization and concurrency
         Clusters
         Spot versus on-demand cluster instances
         Pooling
         Cluster sharing
         Autoscaling
         Serverless
      Data Processing Techniques
         Incremental processing
         Column-oriented data stores
         Data partitioning
         Materialization
   Process Efficiency
      Data (Engineering) Democratization
      Developer Experience
      Collaboration
   Conclusion
Conclusion