Software Engineering at Google: Lessons Learned from Programming Over Time

Cover
Copyright
Table of Contents
Foreword
Preface
	Programming Over Time
	Google’s Perspective
	What This Book Isn’t
	Parting Remarks
	Conventions Used in This Book
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
Part I. Thesis
	Chapter 1. What Is Software Engineering?
		Time and Change
			Hyrum’s Law
			Example: Hash Ordering
			Why Not Just Aim for “Nothing Changes”?
		Scale and Efficiency
			Policies That Don’t Scale
			Policies That Scale Well
			Example: Compiler Upgrade
			Shifting Left
		Trade-offs and Costs
			Example: Markers
			Inputs to Decision Making
			Example: Distributed Builds
			Example: Deciding Between Time and Scale
			Revisiting Decisions, Making Mistakes
		Software Engineering Versus Programming
		Conclusion
		TL;DRs
Part II. Culture
	Chapter 2. How to Work Well on Teams
		Help Me Hide My Code
		The Genius Myth
		Hiding Considered Harmful
			Early Detection
			The Bus Factor
			Pace of Progress
			In Short, Don’t Hide
		It’s All About the Team
			The Three Pillars of Social Interaction
			Why Do These Pillars Matter?
			Humility, Respect, and Trust in Practice
			Blameless Post-Mortem Culture
			Being Googley
		Conclusion
		TL;DRs
	Chapter 3. Knowledge Sharing
		Challenges to Learning
		Philosophy
		Setting the Stage: Psychological Safety
			Mentorship
			Psychological Safety in Large Groups
		Growing Your Knowledge
			Ask Questions
			Understand Context
		Scaling Your Questions: Ask the Community
			Group Chats
			Mailing Lists
			YAQS: Question-and-Answer Platform
		Scaling Your Knowledge: You Always Have Something to Teach
			Office Hours
			Tech Talks and Classes
			Documentation
			Code
		Scaling Your Organization’s Knowledge
			Cultivating a Knowledge-Sharing Culture
			Establishing Canonical Sources of Information
			Staying in the Loop
		Readability: Standardized Mentorship Through Code Review
			What Is the Readability Process?
			Why Have This Process?
		Conclusion
		TL;DRs
	Chapter 4. Engineering for Equity
		Bias Is the Default
		Understanding the Need for Diversity
		Building Multicultural Capacity
		Making Diversity Actionable
		Reject Singular Approaches
		Challenge Established Processes
		Values Versus Outcomes
		Stay Curious, Push Forward
		Conclusion
		TL;DRs
	Chapter 5. How to Lead a Team
		Managers and Tech Leads (and Both)
			The Engineering Manager
			The Tech Lead
			The Tech Lead Manager
		Moving from an Individual Contributor Role to a Leadership Role
			The Only Thing to Fear Is…Well, Everything
			Servant Leadership
		The Engineering Manager
			Manager Is a Four-Letter Word
			Today’s Engineering Manager
		Antipatterns
			Antipattern: Hire Pushovers
			Antipattern: Ignore Low Performers
			Antipattern: Ignore Human Issues
			Antipattern: Be Everyone’s Friend
			Antipattern: Compromise the Hiring Bar
			Antipattern: Treat Your Team Like Children
		Positive Patterns
			Lose the Ego
			Be a Zen Master
			Be a Catalyst
			Remove Roadblocks
			Be a Teacher and a Mentor
			Set Clear Goals
			Be Honest
			Track Happiness
		The Unexpected Question
		Other Tips and Tricks
		People Are Like Plants
			Intrinsic Versus Extrinsic Motivation
		Conclusion
		TL;DRs
	Chapter 6. Leading at Scale
		Always Be Deciding
			The Parable of the Airplane
			Identify the Blinders
			Identify the Key Trade-Offs
			Decide, Then Iterate
		Always Be Leaving
			Your Mission: Build a “Self-Driving” Team
			Dividing the Problem Space
		Always Be Scaling
			The Cycle of Success
			Important Versus Urgent
			Learn to Drop Balls
			Protecting Your Energy
		Conclusion
		TL;DRs
	Chapter 7. Measuring Engineering Productivity
		Why Should We Measure Engineering Productivity?
		Triage: Is It Even Worth Measuring?
		Selecting Meaningful Metrics with Goals and Signals
		Goals
		Signals
		Metrics
		Using Data to Validate Metrics
		Taking Action and Tracking Results
		Conclusion
		TL;DRs
Part III. Processes
	Chapter 8. Style Guides and Rules
		Why Have Rules?
		Creating the Rules
			Guiding Principles
			The Style Guide
		Changing the Rules
			The Process
			The Style Arbiters
			Exceptions
		Guidance
		Applying the Rules
			Error Checkers
			Code Formatters
		Conclusion
		TL;DRs
	Chapter 9. Code Review
		Code Review Flow
		How Code Review Works at Google
		Code Review Benefits
			Code Correctness
			Comprehension of Code
			Code Consistency
			Psychological and Cultural Benefits
			Knowledge Sharing
		Code Review Best Practices
			Be Polite and Professional
			Write Small Changes
			Write Good Change Descriptions
			Keep Reviewers to a Minimum
			Automate Where Possible
		Types of Code Reviews
			Greenfield Code Reviews
			Behavioral Changes, Improvements, and Optimizations
			Bug Fixes and Rollbacks
			Refactorings and Large-Scale Changes
		Conclusion
		TL;DRs
	Chapter 10. Documentation
		What Qualifies as Documentation?
		Why Is Documentation Needed?
		Documentation Is Like Code
		Know Your Audience
			Types of Audiences
		Documentation Types
			Reference Documentation
			Design Docs
			Tutorials
			Conceptual Documentation
			Landing Pages
		Documentation Reviews
		Documentation Philosophy
			WHO, WHAT, WHEN, WHERE, and WHY
			The Beginning, Middle, and End
			The Parameters of Good Documentation
			Deprecating Documents
		When Do You Need Technical Writers?
		Conclusion
		TL;DRs
	Chapter 11. Testing Overview
		Why Do We Write Tests?
			The Story of Google Web Server
			Testing at the Speed of Modern Development
			Write, Run, React
			Benefits of Testing Code
		Designing a Test Suite
			Test Size
			Test Scope
			The Beyoncé Rule
			A Note on Code Coverage
		Testing at Google Scale
			The Pitfalls of a Large Test Suite
		History of Testing at Google
			Orientation Classes
			Test Certified
			Testing on the Toilet
			Testing Culture Today
		The Limits of Automated Testing
		Conclusion
		TL;DRs
	Chapter 12. Unit Testing
		The Importance of Maintainability
		Preventing Brittle Tests
			Strive for Unchanging Tests
			Test via Public APIs
			Test State, Not Interactions
		Writing Clear Tests
			Make Your Tests Complete and Concise
			Test Behaviors, Not Methods
			Don’t Put Logic in Tests
			Write Clear Failure Messages
		Tests and Code Sharing: DAMP, Not DRY
			Shared Values
			Shared Setup
			Shared Helpers and Validation
			Defining Test Infrastructure
		Conclusion
		TL;DRs
	Chapter 13. Test Doubles
		The Impact of Test Doubles on Software Development
		Test Doubles at Google
		Basic Concepts
			An Example Test Double
			Seams
			Mocking Frameworks
		Techniques for Using Test Doubles
			Faking
			Stubbing
			Interaction Testing
		Real Implementations
			Prefer Realism Over Isolation
			How to Decide When to Use a Real Implementation
		Faking
			Why Are Fakes Important?
			When Should Fakes Be Written?
			The Fidelity of Fakes
			Fakes Should Be Tested
			What to Do If a Fake Is Not Available
		Stubbing
			The Dangers of Overusing Stubbing
			When Is Stubbing Appropriate?
		Interaction Testing
			Prefer State Testing Over Interaction Testing
			When Is Interaction Testing Appropriate?
			Best Practices for Interaction Testing
		Conclusion
		TL;DRs
	Chapter 14. Larger Testing
		What Are Larger Tests?
			Fidelity
			Common Gaps in Unit Tests
			Why Not Have Larger Tests?
		Larger Tests at Google
			Larger Tests and Time
			Larger Tests at Google Scale
		Structure of a Large Test
			The System Under Test
			Test Data
			Verification
		Types of Larger Tests
			Functional Testing of One or More Interacting Binaries
			Browser and Device Testing
			Performance, Load, and Stress testing
			Deployment Configuration Testing
			Exploratory Testing
			A/B Diff Regression Testing
			UAT
			Probers and Canary Analysis
			Disaster Recovery and Chaos Engineering
			User Evaluation
		Large Tests and the Developer Workflow
			Authoring Large Tests
			Running Large Tests
			Owning Large Tests
		Conclusion
		TL;DRs
	Chapter 15. Deprecation
		Why Deprecate?
		Why Is Deprecation So Hard?
			Deprecation During Design
		Types of Deprecation
			Advisory Deprecation
			Compulsory Deprecation
			Deprecation Warnings
		Managing the Deprecation Process
			Process Owners
			Milestones
			Deprecation Tooling
		Conclusion
		TL;DRs
Part IV. Tools
	Chapter 16. Version Control and Branch Management
		What Is Version Control?
			Why Is Version Control Important?
			Centralized VCS Versus Distributed VCS
			Source of Truth
			Version Control Versus Dependency Management
		Branch Management
			Work in Progress Is Akin to a Branch
			Dev Branches
			Release Branches
		Version Control at Google
			One Version
			Scenario: Multiple Available Versions
			The “One-Version” Rule
			(Nearly) No Long-Lived Branches
			What About Release Branches?
		Monorepos
		Future of Version Control
		Conclusion
		TL;DRs
	Chapter 17. Code Search
		The Code Search UI
		How Do Googlers Use Code Search?
			Where?
			What?
			How?
			Why?
			Who and When?
		Why a Separate Web Tool?
			Scale
			Zero Setup Global Code View
			Specialization
			Integration with Other Developer Tools
			API Exposure
		Impact of Scale on Design
			Search Query Latency
			Index Latency
		Google’s Implementation
			Search Index
			Ranking
		Selected Trade-Offs
			Completeness: Repository at Head
			Completeness: All Versus Most-Relevant Results
			Completeness: Head Versus Branches Versus All History Versus Workspaces
			Expressiveness: Token Versus Substring Versus Regex
		Conclusion
		TL;DRs
	Chapter 18. Build Systems and Build Philosophy
		Purpose of a Build System
		What Happens Without a Build System?
			But All I Need Is a Compiler!
			Shell Scripts to the Rescue?
		Modern Build Systems
			It’s All About Dependencies
			Task-Based Build Systems
			Artifact-Based Build Systems
			Distributed Builds
			Time, Scale, Trade-Offs
		Dealing with Modules and Dependencies
			Using Fine-Grained Modules and the 1:1:1 Rule
			Minimizing Module Visibility
			Managing Dependencies
		Conclusion
		TL;DRs
	Chapter 19. Critique: Google’s Code Review Tool
		Code Review Tooling Principles
		Code Review Flow
			Notifications
		Stage 1: Create a Change
			Diffing
			Analysis Results
			Tight Tool Integration
		Stage 2: Request Review
		Stages 3 and 4: Understanding and Commenting on a Change
			Commenting
			Understanding the State of a Change
		Stage 5: Change Approvals (Scoring a Change)
		Stage 6: Commiting a Change
			After Commit: Tracking History
		Conclusion
		TL;DRs
	Chapter 20. Static Analysis
		Characteristics of Effective Static Analysis
			Scalability
			Usability
		Key Lessons in Making Static Analysis Work
			Focus on Developer Happiness
			Make Static Analysis a Part of the Core Developer Workflow
			Empower Users to Contribute
		Tricorder: Google’s Static Analysis Platform
			Integrated Tools
			Integrated Feedback Channels
			Suggested Fixes
			Per-Project Customization
			Presubmits
			Compiler Integration
			Analysis While Editing and Browsing Code
		Conclusion
		TL;DRs
	Chapter 21. Dependency Management
		Why Is Dependency Management So Difficult?
			Conflicting Requirements and Diamond Dependencies
		Importing Dependencies
			Compatibility Promises
			Considerations When Importing
			How Google Handles Importing Dependencies
		Dependency Management, In Theory
			Nothing Changes (aka The Static Dependency Model)
			Semantic Versioning
			Bundled Distribution Models
			Live at Head
		The Limitations of SemVer
			SemVer Might Overconstrain
			SemVer Might Overpromise
			Motivations
			Minimum Version Selection
			So, Does SemVer Work?
		Dependency Management with Infinite Resources
			Exporting Dependencies
		Conclusion
		TL;DRs
	Chapter 22. Large-Scale Changes
		What Is a Large-Scale Change?
		Who Deals with LSCs?
		Barriers to Atomic Changes
			Technical Limitations
			Merge Conflicts
			No Haunted Graveyards
			Heterogeneity
			Testing
			Code Review
		LSC Infrastructure
			Policies and Culture
			Codebase Insight
			Change Management
			Testing
			Language Support
		The LSC Process
			Authorization
			Change Creation
			Sharding and Submitting
			Cleanup
		Conclusion
		TL;DRs
	Chapter 23. Continuous Integration
		CI Concepts
			Fast Feedback Loops
			Automation
			Continuous Testing
			CI Challenges
			Hermetic Testing
		CI at Google
			CI Case Study: Google Takeout
			But I Can’t Afford CI
		Conclusion
		TL;DRs
	Chapter 24. Continuous Delivery
		Idioms of Continuous Delivery at Google
		Velocity Is a Team Sport: How to Break Up a Deployment into Manageable Pieces
		Evaluating Changes in Isolation: Flag-Guarding Features
		Striving for Agility: Setting Up a Release Train
			No Binary Is Perfect
			Meet Your Release Deadline
		Quality and User-Focus: Ship Only What Gets Used
		Shifting Left: Making Data-Driven Decisions Earlier
		Changing Team Culture: Building Discipline into Deployment
		Conclusion
		TL;DRs
	Chapter 25. Compute as a Service
		Taming the Compute Environment
			Automation of Toil
			Containerization and Multitenancy
			Summary
		Writing Software for Managed Compute
			Architecting for Failure
			Batch Versus Serving
			Managing State
			Connecting to a Service
			One-Off Code
		CaaS Over Time and Scale
			Containers as an Abstraction
			One Service to Rule Them All
			Submitted Configuration
		Choosing a Compute Service
			Centralization Versus Customization
			Level of Abstraction: Serverless
			Public Versus Private
		Conclusion
		TL;DRs
Part V. Conclusion
Afterword
Index
About the Authors
Colophon