Continuous Delivery 2.0: Business-leading DevOps Essentials

Cover
Half Title
Title Page
Copyright Page
Contents
Foreword
Target Readers
Content
How to Read This book?
Preface 1
Preface 2
Acknowledgement
Author’s Note
1. Continuous Delivery 2.0
	1.1. Overview of Software Engineering Development
		1.1.1. Waterfall Model
		1.1.2. Agile Method
		1.1.3. The DevOps Movement
		1.1.4. Continuous Delivery 1.0
	1.2. Continuous Delivery 2.0.
		1.2.1. Lean Thinking
		1.2.2. Double-Flywheels Model
		1.2.3. Four Core Principles
			1.2.3.1. Doing less
			1.2.3.2. Continuous decomposition of problems
			1.2.3.3. Insistence on fast feedback (FFB)
			1.2.3.4. Continuous improvement and measurement
		1.2.4. Tangram of Continuous Delivery
	1.3. Summary
2. Value Discovery Loop
	2.1. Significance of Discovery Loop
	2.2. Four Key Steps In Discovery Loop
		2.2.1. Questioning
		2.2.2. Targeting
		2.2.3. Co-creation
			2.2.3.1. Measurable Impact Mapping
			2.2.3.2. User Journey Mapping
		2.2.4. Refining
	2.3. Working Principles
		2.3.1. Decomposition and Quick Trial and Error
		2.3.2. Verify Only One Point at a Time
		2.3.3. Allow Failure
	2.4. Common Methods For Co-creation And Refining
		2.4.1. Decorative Window
		2.4.2. Minimum Viable Feature
		2.4.3. Special Zone
		2.4.4. Directional Explorer
		2.4.5. Corn Dolly
		2.4.6. Minimum Viable Product
	2.5. Matters Needing Attention
		2.5.1. Multi-Role Participation
		2.5.2. A Cyclic Process
		2.5.3. Risk Is Not Equivalent
		2.5.4. Be Aware of God’s Perspective
		2.5.5. All Is Number
		2.5.6. Snake Crawler Effect
	2.6. Summary
3. Fast Verification Loop
	3.1. Goal Of Verification Loop
	3.2. Four Key Steps In Verification Loop
		3.2.1. Building
			3.2.1.1. Timeboxing
			3.2.1.2. Task decomposition
			3.2.1.3. Continuous Verification
		3.2.2. Running
		3.2.3. Monitoring
		3.2.4. Decision-Making
	3.3. Working Principles
		3.3.1. Building Quality in
		3.3.2. Eliminating Waiting
			3.3.2.1. Make value flow through “pull”
			3.3.2.2. Self-service tasks
		3.3.3. Automating Routine Work
		3.3.4. Monitoring Everything
	3.4. Summary
4. A Suitable Organizational Culture for Continuous Delivery 2.0
	4.1. Security, Mutual Trust, And Continuous Improvement
		4.1.1. Don’t Be Afraid of Failure
		4.1.2. Mutual Trust
		4.1.3. Continuous Improvement
	4.2. Four Steps In Culture Shaping
		4.2.1. Behavior Determines Culture
		4.2.2. Google’s Engineer Culture
		4.2.3. Etsy’s Continuous Testing Culture
	4.3. Action Guide
		4.3.1. Value Orientation
		4.3.2. Fast Verification
		4.3.3. Continuous Learning
			4.3.3.1. Regular retrospective
			4.3.3.2. Event-replay mechanism
	4.4. Importance Of Measurement
		4.4.1. Four Attributes of Measurement Indicators
			4.4.1.1. Leading and lagging indicators
			4.4.1.2. Observable and actionable indicators
		4.4.2. The Goal of Measurement Is to Improve
	4.5. The “improvement Kata” For Continuous Improvement
	4.6. Summary
5. Software Architecture for Continuous Delivery
	5.1. Dividing The System Into Modules
		5.1.1. Requirements for a Continuous Delivery Architecture
		5.1.2. Principles for System Decomposition
	5.2. Common Architectural Patterns
		5.2.1. Microcore Architecture
		5.2.2. Microservice Architecture
		5.2.3. Monolithic Architecture
	5.3. Architecture Transformation Patterns
		5.3.1. Demolisher Pattern
		5.3.2. Strangler Pattern
		5.3.3. Decorator Pattern
		5.3.4. Database Sharding
	5.4. Summary
6. Collaborative Management around Business Requirements
	6.1. Overview of Software Release Life Cycle
		6.1.1. Inception Period
		6.1.2. Delivery Period
	6.2. Pros And Cons of Requirements Decomposition
		6.2.1. Benefits of Requirements Decomposition
			6.2.1.1. Build consensus and coordinate work
			6.2.1.2. Small-batch delivery to speed up the flow of value
			6.2.1.3. Embrace change at a low cost
			6.2.1.4. Multiple integrations and timely feedback on quality
			6.2.1.5. Boost the morale of team members
		6.2.2. Costs of Requirements Decomposition
			6.2.2.1. Explicit cost arising from requirements decomposition
			6.2.2.2. Iterative cost arising from small-batch development, testing, and deployment
	6.3. Way Of Requirements Decomposition
		6.3.1. Sources of Requirements
		6.3.2. Technical Debts Are Another Requirement
		6.3.3. Roles Involved in Requirements Decomposition
		6.3.4. The Unequal “INVEST” Principle
		6.3.5. Five Ways to Split User Stories
			6.3.5.1. Splitting by path
			6.3.5.2. Splitting by contact point
			6.3.5.3. Splitting by data type or format
			6.3.5.4. Splitting by rule
			6.3.5.5. Splitting by exploration
		6.3.6. Seven Components of Each User Story
	6.4. Requirements Analysis And Management Tools
		6.4.1. User Story Mapping
		6.4.2. User Story Tree
		6.4.3. Dependency Graph
		6.4.4. Digital Management Platform
	6.5. Tools For Team Collaboration
		6.5.1. Shared Calendar
		6.5.2. Retrospective Meeting
		6.5.3. Visualized Story Wall
		6.5.4. Definition of “Done”
		6.5.5. Continuous Integration
		6.5.6. Story Verification
	6.6. Summary
7. Principles for Deployment Pipeline and Tool Design
	7.1. Simple Deployment Pipeline
		7.1.1. Simple Development Process
		7.1.2. Initial Deployment Pipeline
		7.1.3. Execution of the Deployment Pipeline
	7.2. Design And Use Of Deployment Pipeline
		7.2.1. Principles for Designing a Deployment Pipeline
			7.2.1.1. Build once, run anywhere
			7.2.1.2. Loosely coupled with business logic
			7.2.1.3. Parallelization
			7.2.1.4. Quick feedback to take precedence
			7.2.1.5. Important feedback to take precedence
		7.2.2. Principles for Teamwork
			7.2.2.1. Stop the line
			7.2.2.2. Security audit
	7.3. Composition of A Deployment Pipeline Platform
		7.3.1. Overall Architecture of the Platform
			7.3.1.1. True north source
			7.3.1.2. Deployment pipeline platform
			7.3.1.3. Infrastructure service layer
		7.3.2. Basic Capabilities of the Platform
		7.3.3. Strategy for Tool Chain Construction
	7.4. Cloudification of Infrastructure Services
		7.4.1. Interpretation of the Collaboration Process of Infrastructure Services
		7.4.2. Build System
		7.4.3. Automation Test System
		7.4.4. Deployment System
		7.4.5. Environment Management System
	7.5. Management of Artifact Repository
		7.5.1. Classification of Artifact Repository
			7.5.1.1. Internal artifact repository A (temporary)
			7.5.1.2. Internal artifact repository B (formal)
			7.5.1.3. Artifact repository X (external)
			7.5.1.4. Temporary mirror C, formal mirror D, and external mirror Y
		7.5.2. Principles for Management of Artifact Repository
	7.6. A Variety Of Deployment Pipelines
		7.6.1. Multi-Component Deployment Pipeline
		7.6.2. Individual Deployment Pipeline
		7.6.3. Evolution of Deployment Pipeline
	7.7. Build Self-service Tools For Engineers
	7.8. Summary
8. Branch Strategy Conducive to Integration
	8.1. Purpose of The Version Control System
		8.1.1. CVCS
		8.1.2. DVCS
		8.1.3. Basic Concepts in VCS
	8.2. Common Branching Modes
		8.2.1. Trunk-Based Development and Release
		8.2.2. Trunk-Based Development & Branch-Based Release
		8.2.3. Branch-Based Development & Trunk-Based Release
			8.2.3.1. Feature-based branch
			8.2.3.2. Team branch
	8.3. Evolution of Branch Modes
		8.3.1. Troika
		8.3.2. Git Flow
		8.3.3. GitHub Flow
	8.4. How to Choose a Suitable Branching Strategy?
		8.4.1. Release Pattern
			8.4.1.1. Project release pattern
			8.4.1.2. Release train pattern
			8.4.1.3. Intercity express pattern
		8.4.2. Relationship Between Branch Strategy and Release Cycle
	8.5. Summary
9. Continuous Integration
	9.1. Origin and Definition
		9.1.1. Original Definition
		9.1.2. Integration Process
	9.2. Six-step Check-in Dance
		9.2.1. Four Key Points of Check-In Dance
			9.2.1.1. Effect of the three times of verification
			9.2.1.2. Twice personal verifications are a necessity
			9.2.1.3. Ensure the performance of a personal build before code submission
			9.2.1.4. The contents of quality verification to be included in each build
		9.2.2. Synchronous and Asynchronous Modes
		9.2.3. Self-Check Sheet
			9.2.3.1. Trunk-based Development and frequent submission
			9.2.3.2. Content of each submission is a complete task
			9.2.3.3. Finish check-in build within 10 minutes
			9.2.3.4. After a failed check-in build, do not submit new code or check out the problematic code
			9.2.3.5. Fix the check-in build or roll back within ten minutes
			9.2.3.6. Passing the automation build will boost up the confidence in software quality
	9.3. Trade-off Between Speed And Quality
		9.3.1. Staging Build
		9.3.2. Builds Submitted by Multiple People at the Same Time
		9.3.3. The Power of Cloud Platform
	9.4. Implement Ci Practices In The Team
		9.4.1. Quickly Establish a CI Practice for the Team
			9.4.1.1. Scripted build and setup CI tool
			9.4.1.2. Add an existing automation verification script to the build
			9.4.1.3. Choose a branch strategy that is conducive to CI
			9.4.1.4. Adopt the six-step check-in dance
			9.4.1.5. Continuous optimization
			9.4.1.6. Engineers change habits and improve skills
		9.4.2. Branch Strategy and Deployment Pipeline
			9.4.2.1. Trunk-based Development & Release
			9.4.2.2. Trunk-based Development & Branch-based Release
			9.4.2.3. Branch-based Development & Trunk-based Release
			9.4.2.4. Multi-component integration
	9.5. Common Problems In Implementation
		9.5.1. Work Habits of Engineers
		9.5.2. Code Scanning is Often Ignored
		9.5.3. Lack of Automation Testcases
	9.6. Summary
10. Automation Test Strategies and Methods
	10.1. Self-positioning of Automation Test
		10.1.1. Advantages of Automation Test
		10.1.2. Investment Required for Automation Test
	10.2. Break Through The Dilemma of Traditional Automation Test
		10.2.1. Characteristics of Traditional Automation Test
		10.2.2. Layering of Automation Test
		10.2.3. Different Types of Test Pyramid
			10.2.3.1. Test Pyramid of Microcore Architecture
			10.2.3.2. Test Pyramid of Microservice Architecture
	10.3. Implementation Strategy Of Automation Testing
		10.3.1. Add Starting Points for Automation Testcases
			10.3.1.1. Write tests for code hotspots
			10.3.1.2. Write tests side by side with new features
			10.3.1.3. Expand up and down from the middle layer of the test pyramid
			10.3.1.4. Quality of automation tests is more important than quantity
		10.3.2. Increase the Execution Frequency of Automation Testing
			10.3.2.1. Share automation testcases
			10.3.2.2. Developers are the first users of automation tests
		10.3.3. Characteristics of Good Automation Testing
			10.3.3.1. Testcases must be independent of each other
			10.3.3.2. The testcases must be stable
			10.3.3.3. The testcases must run fast
			10.3.3.4. The testing environments should be unified
		10.3.4. Share Responsibilities for Maintenance
		10.3.5. Test Coverage
	10.4. Key Points of User Acceptance of Automation Tests
		10.4.1. Build a Layered Framework in the First Place
		10.4.2. The Total Number of UAT should be as Small as Possible
		10.4.3. Reserve API for Automation Testcases
		10.4.4. Prepare for Debugging
		10.4.5. Prepare Test Data
	10.5. Other Quality Inspection Methods
		10.5.1. Diff-Approval Testing
		10.5.2. Code Style Check and Code Dynamic/Static Analysis
		10.5.3. Application of Artificial Intelligence (AI) in the Field of Testing
	10.6. Summary
11. Software Configuration Management
	11.1. Put Everything Under SCM
		11.1.1. Goals of SCM
		11.1.2. Scope of SCM
		11.1.3. Principles of SCM
			11.1.3.1. Everything has a unique identifier
			11.1.3.2. Sharing the True North source
			11.1.3.3. Standardization and automation
	11.2. Versioning of Software Artifacts
		11.2.1. Anti-Pattern of Package Management
		11.2.2. Centralized Package Management Service
		11.2.3. Meta Information of Packages
			11.2.3.1. Unique ID
			11.2.3.2. Source
			11.2.3.3. Dependency
	11.3. Management Of Package Dependency
		11.3.1. Explicit Declaration of Dependencies
		11.3.2. Automatic Management of Dependencies
		11.3.3. Reduction of Complex Dependencies
			11.3.3.1. Too many dependencies or over-long chains
			11.3.3.2. Dependency conflict
			11.3.3.3. Circular dependency
	11.4. Management of Environmental Infrastructure Management
		11.4.1. Four States of Environment Previsioning
			11.4.1.1. “Wilderness” featured by “human brain + handwork”
			11.4.1.2. “Normalization” characterized by “document + private script”
			11.4.1.3. “Standardization” characterized by “paperless officing”
			11.4.1.4. “Automatic state” characterized by “controlled automation script”
		11.4.2. DSL Application
		11.4.3. Infrastructure as Code
	11.5. Management of Software Configuration Items
		11.5.1. Separation of Binary and Configuration
		11.5.2. Version Management of Configuration
		11.5.3. Storage of Configuration Items
		11.5.4. Configuration Drift and Governance
	11.6. Immutable Infrastructure and Cloud Applications
		11.6.1. Implement Immutable Infrastructure
			11.6.1.1. Physical machine mirroring technology and virtual machine mirroring technology
			11.6.1.2. Docker container technology
		11.6.2. Cloud Native Application
		11.6.3. Advantages and Challenges
	11.7. Data Version Management
		11.7.1. Changes of Database Structure
		11.7.2. Data File
	11.8. Association Between Requirements And Source Code
	11.9. Summary
12. Low-Risk Release
	12.1. High-frequency Release is a Trend
		12.1.1. High-Frequency Release by Dot-Com Companies
		12.1.2. Coexistence of Benefits and Costs
	12.2. Ways To Mitigate Release Risk
		12.2.1. Blue-Green Deployment
		12.2.2. Rolling Deployment
		12.2.3. Canary Release
		12.2.4. Dark Launch
	12.3. High-Frequency Release Support Technology
		12.3.1. Feature Toggle
		12.3.2. Data Migration
			12.3.2.1. Fields only to be added instead of being deleted
			12.3.2.2. Data migration
		12.3.3. Branch by Abstraction
		12.3.4. Fix-Forward Instead of Rollback
	12.4. Factors Affecting Release Frequency
	12.5. Summary
13. Monitoring and Decision-Making
	13.1. Scope of Production Monitoring
		13.1.1. Monitoring of Backend Services
		13.1.2. Monitoring of Consumer-Host Software
	13.2. Data Monitoring System
		13.2.1. Collection and Processing
		13.2.2. Standardization
		13.2.3. Monitoring Data System and Its Ability Measurement
	13.3. Issue Handling System
		13.3.1. Alarm Storm and Intelligent Management
		13.3.2. Problem Solving Is a Learning Process
	13.4. Test In Production
		13.4.1. Flattening Trend of Testing Activities
		13.4.2. Testing in Production
		13.4.3. Chaos Engineering
	13.5. Eastward Or Westward
	13.6. Summary
14. Large Internet Teams to Become Feature Teams
	14.1. Case Profile
		14.1.1. State before Improvement
			14.1.1.1. Team organization structure
			14.1.1.2. Development process and rhythm
		14.1.2. State after Improvement
			14.1.2.1. Team organization structure
			14.1.2.2. Workflow and rhythm
	14.2. Improvement Methodology
		14.2.1. Guiding Ideology
		14.2.2. Improvement Steps
	14.3. The Journey
		14.3.1. Architecture Decoupling
		14.3.2. Organization Decoupling
		14.3.3. R&D Workflow Reengineering
			14.3.3.1. The purpose and principles of multi-version release
			14.3.3.2. Hybrid branch of virtual trunk
			14.3.3.3. Create a unified product library
			14.3.3.4. Two-level release
			14.3.3.5. Quality assurance mechanism
			14.3.3.6. Multipartite collaborative release
			14.3.3.7. A build cloud for compilation
		14.3.4. Optimization for Automation Efficiency
	14.4. Summary
15. How Can Small Teams Implement a “Counter Attack”?
	15.1. Background
		15.1.1. A “Death March” before Improvement
		15.1.2. Zero-Defect Delivery after Improvement
	15.2. Improvement Methodology
		15.2.1. Guiding Ideology
		15.2.2. Selection of the Pilot Team
	15.3. Stage 1: Preparation
		15.3.1. Feature Introduction and Requirements Decomposition
		15.3.2. Architecture Design and Requirements Dependency Identification
		15.3.3. Workload Estimation and Scheduling
	15.4. Stage 2: Delivery
		15.4.1. Improve Workflow with Kanban
			15.4.1.1. Identify bad smells
			15.4.1.2. Let the value flow
			15.4.1.3. Standard definition of done in explicit declaration
			15.4.1.4. Graffiti design, eliminate waste
			15.4.1.5. Clarify state and care about rapid flow of requirements
			15.4.1.6. Avoid unnecessary task switching
			15.4.1.7. No feedback is a “risk”
			15.4.1.8. Limit the quantity of WIP
		15.4.2. Bug-Free Delivery
		15.4.3. Trunk-Based Development and CI
		15.4.4. Shift-Left Testing
		15.4.5. Code Review
		15.4.6. Care More about Process in Addition to Results
	15.5. Summary
16. DevOps Driven by Developers
	16.1. Background
	16.2. Original Working Mode
	16.3. Key Points Of Improvement
		16.3.1. Improvement Methodology
		16.3.2. Define Goals of Improvement
			16.3.2.1. Expectations of department director
			16.3.2.2. Delivery pressure of team manager
			16.3.2.3. Annoyances of project leaders
	16.4. Stage 1: Agile 101.
		16.4.1. Make a Trustable Plan
			16.4.1.1. Requirements decomposition
			16.4.1.2. Relative estimation
			16.4.1.3. Initial plan
		16.4.2. Start the Development Stage
			16.4.2.1. Selection of iteration interval
			16.4.2.2. Team collaboration process
		16.4.3. Constraints on Process Quality
			16.4.3.1. How to consciously abide by CI discipline?
			16.4.3.2. How to shorten the build time?
			16.4.3.3. Developers cannot run automated tests
			16.4.3.4. Automation testing strategy
			16.4.3.5. What to do in case of insufficient environments for automation tests?
			16.4.3.6. How to determine a requirement can be tested?
			16.4.3.7. How to do performance and stress testing?
		16.4.4. Summary of Stage One
	16.5. Stage 2: Devops Transformation
		16.5.1. “Conflict” with Ops Engineers
		16.5.2. Specific Obstacles in High-Frequency Deployment and Release
		16.5.3. Overall Solution Design
			16.5.3.1. Adjustment of automation testing strategy
			16.5.3.2. Convenience of running tests
			16.5.3.3. Put test code with product code
			16.5.3.4. SCM optimization
			16.5.3.5. Package management
			16.5.3.6. Optimization of deployment and monitoring
		16.5.4. Team Changes in the DevOps Stage
	16.6. Summary
Appendix A: Three Evolutions of Software Engineering
Appendix B: User Story Estimation by Relative Size
Index