Data Engineering Teams

Introduction
About This Book
Warnings and Success Stories
Who Should Read This
Navigating the Book Chapters
Conventions Used in This Book
Big Data
Why Is Big Data So Much More Complicated?
Distributed Systems Are Hard
What Does It All Mean, Basil?
What Does It Mean for Software Engineering Teams?
What Does It Mean for Data Warehousing Teams?
What Is a Data Engineering Team?
Skills Needed in a Team
Skills Gap Analysis
Skill Gap Analysis Results
What I Look for in Data Engineering Teams
Operations
Quality Assurance
What Is a Data Engineer?
What I Look for in Data Engineers
Qualified Data Engineers
Not Just Data Warehousing and DBAs
Ability Gap
Themes and Thoughts of a Data Engineering Team
Hub of the Wheel
How to Work with a Data Science Team
How to Work with a Data Warehousing Team
How to Work with an Analytics and/or Business Intelligence Team
“How I Evaluate Teams
Equipment and Resources
Thought Frameworks
Building Data Pipelines
Knowledge of Use Case
Right Tool for the Job
Crawl, Walk, Run
Technologies
Why Do Teams Fail?
Why Do Teams Succeed?
Paying the Piper
Some Technologies Are Just Dead Ends
What if You Have Gaps and Still Have to Do It?
Pre-project Steps
Use Case
Evaluate the Team
Choose the Technologies
Write the Code
Evaluate
Repeat
Probability of Success
Conclusion
Best Efforts
About the Author