Continuous Testing, Quality, Security, and Feedback: Essential strategies and secure practices for DevOps, DevSecOps

Continuous Testing, Quality, Security, and Feedback
Foreword
Contributors
About the author
About the reviewers
Preface
   Who this book is for
   What this book covers
   To get the most out of this book
   Conventions used
   Get in touch
   Share Your Thoughts
   Download a free PDF copy of this book
Part 1: Understanding Continuous Testing, Quality, Security, and Feedback
1
Principles of Continuous Testing, Quality, Security, and Feedback
   Introducing continuous testing, quality, security, and feedback
      Foundations for testing, quality, security, and feedback
      Evolution toward continuous testing, quality, security, and feedback
   Defining continuous testing, quality, security, and feedback
      The need for definitions of testing, quality, security, and feedback
      The challenges of defining continuous testing, quality, security, and feedback
      A definition of continuous testing, quality, security, and feedback
   The guiding principles and pillars of continuous testing
   The guiding principles and pillars of continuous quality
   The guiding principles and pillars of continuous security
   The guiding principles and pillars of continuous feedback
   Summary
2
The Importance of Continuous Testing, Quality, Security, and Feedback
   Why continuous strategies are important for DevOps and DevSecOps
      Principles and pillars of DevOps, and DevSecOps
      DevOps and DevSecOps dependencies on continuous testing, quality, security, and feedback
      Principles and pillars of SRE
      SRE dependencies on continuous testing, quality, security, and feedback
   Consequences of implementing DevOps, DevSecOps, and SRE without properly implementing continuous practices
   Summary
3
Experiences and Pitfalls with Continuous Testing, Quality, Security, and Feedback
   A lifetime of studying testing, quality, security, and feedback for DevOps, DevSecOps, and SRE
   BNR – World-class university
   Testing as a commercial enterprise
   Consulting and teaching
   Lessons learned, pitfalls, and strategies to overcome pitfalls
      The importance of quality
      Building testing tools into systems
      Test automation for efficiency and competitiveness
      Standards accelerate collaboration
      Security requires a comprehensive approach
      Without feedback, you are running blind
   Summary
Part 2: Determining Solutions Priorities
4
Engineering Approach to Continuous Testing, Quality, Security, and Feedback
   Why is an engineering approach needed?
   Understanding the Seven-Step Transformation Engineering Blueprint
      Expert and AI-accelerated transformations
   Capability maturity models guide transformations
   Capability maturity levels – Continuous testing
   Capability maturity levels – Continuous quality
   Capability maturity levels – Continuous security
   Capability maturity levels – Continuous feedback
   Summary
5
Determining Transformation Goals
   Transformation goal classifications
   The importance of transformation goals alignment
      Negative consequences of misalignment in each classification
   Determining specific goals for a transformation
      Using AI chatbots to help determine transformation goals
      Determining how many applications to transform at a time
      Model applications
   Determining model applications
   Determining goals for continuous testing
   Determining goals for continuous quality
   Determining goals for continuous security
   Determining goals for continuous feedback
   Summary
6
Discovery and Benchmarking
   Technical requirements
   Methodology for discovery and benchmarks
   Understanding current state discovery
      Surveys
      Example survey
      Interviews
      Example interview questions
   Understanding gap assessments
      Why gap assessments are important
      How gap assessments are conducted
      How gap assessment results are used
      Known good practices for continuous testing
      Known good practices for continuous quality
      Known good practices for continuous security
      Known good practices for continuous feedback
   Understanding CSVSM
      Steps to creating a CSVSM
      Challenges to overcome with value stream mapping
   How generative AI can be used to accelerate discovery and benchmarking
   Summary
7
Selecting Tool Platforms and Tools
   Tool platforms and tools concepts
      Tool platforms
      Tools
      Relationship between tool platforms and tools
   Platforms and tools for continuous testing, quality, security, and feedback
      Continuous testing platforms and tools
      Continuous quality platforms and tools
      Continuous security platforms and tools
      Continuous feedback platforms and tools
      Overlap and integration
   Source of platforms and tools
      Open-source tools
      Vendor product tools
      DIY or home-grown tools
   Factors for comparing tool platforms and tools
      Example tool platforms and tools
   Methodology for selecting tool platforms and tools
      Determining how many tools are enough
      Balancing act
   Summary
8
Applying AL/ML to Continuous Testing, Quality, Security, and Feedback
   AI/ML applications
   AI/ML for continuous testing
      Real-world use case for AI/ML-assisted continuous testing
   AI/ML for continuous quality
      Real-world use case for AI/ML-assisted continuous quality
   AI/ML for continuous security
      Real-world use case for AI/ML-assisted continuous security
   AI/ML for continuous feedback
      Real-world use case for AI/ML-assisted continuous feedback
   Methodology for selecting AI/ML tools
   Summary
Part 3: Deep Dive into Roadmaps, Implementation Patterns, and Measurements
9
Use Cases for Integrating with DevOps, DevSecOps, and SRE
   Use cases for DevOps
      Requirements stage
      Development stage
      Continuous integration stage
      Continuous delivery stage
      Continuous deployment stage
      Continuous operations stage
      Real-world use case for DevOps
   Use cases for DevSecOps
      Requirements stage
      Development stage
      Continuous integration stage
      Continuous delivery stage
      Continuous deployment stage
      Continuous operations stage
      Real-world use case for DevSecOps
   Use cases for SRE
      Requirements stage
      Development stage
      Continuous integration stage
      Continuous delivery stage
      Continuous deployment stage
      Continuous operations stage
      Real-world use case for SRE
   Sustaining integrations
   Summary
10
Building Roadmaps for Implementation
   Introduction to strategic roadmaps
      The difference between a roadmap and a plan
      The benefits of roadmaps
      The importance of a roadmap
      The perils of proceeding without a roadmap
      Best formats to represent the roadmap
   Creating a roadmap
      Steps to creating a roadmap
      Who should be involved
      Evaluating roadmap alternatives
      Determining an acceptable roadmap
   Creating a future state value stream map (FSVSM)
      The importance of FSVSMs in establishing transformation roadmaps
      FSVSM workshop
   Roadmap for continuous testing
   Roadmap for continuous quality
   Roadmap for continuous security
   Roadmap for continuous feedback
   Alignment on the roadmap
      Identifying risks and mitigation strategies
      Allocating budget and resources
      Defining success metrics and a change management plan
   Summary
11
Understanding Transformation Implementation Patterns
   What is a transformation implementation pattern?
      Key components of effective implementation patterns
      Choosing the right pattern
   Understanding transformation implementation patterns
      Dedicated platform team
      Embedded teams
      Outsourced teams
      Hybrid dedicated/outsourced teams
   Patterns to avoid during implementation
   Selecting an implementation pattern
   Summary
12
Measuring Progress and Outcomes
   Measures of progress and outcomes
      Why measures of progress and outcomes are important
      Linking measures to capability maturity
      Examples of outcome metrics
      Examples of progress metrics
   Selecting measures
      Leadership and teams for selecting outcome and progress metrics
   Practices for designing metrics and dashboards
      Designing an outcome and progress metrics
      Architectures for dashboards displaying metrics
   Sustaining measures of progress and outcomes
      Evaluating and deprecating metrics
      Introducing new metrics
      Validating metric implementations
   Summary
Part 4: Exploring Future Trends and Continuous Learning
13
Emerging Trends
   Macro trends in DevOps, DevSecOps, and SRE
   Testability and observability trends
   Platform engineering trends
   VSM trends
   AI/ML trends
   Summary
14
Exploring Continuous Learning and Improvement
   The Third Way of DevOps
      Continuous improvement in DevOps
      Learning in DevOps
      Continuous testing, quality, and security
   Learning from sharing
      Building a culture of open communication
      Sharing best practices and tools
      Cross-team collaboration and external engagement
      Leveraging feedback for continuous improvement
   Learning from outreach
      The role of external engagement in continuous improvement
      The benefits of industry collaboration
      Implementing outreach learnings in DevOps practices
   Learning from experimentation
      The importance of experimentation in DevOps
      Conducting safe experiments in DevOps
      Learning from experimentation outcomes
   Learning from failure
      Embracing a no-blame culture
      Practical steps to analyze failures
      Integrating failures into continuous improvement cycles
      The benefits of learning from failures
   Learning from chaos engineering
      Implementing chaos engineering
      Learning and improvement from chaos engineering
      Integrating chaos engineering into continuous feedback loops
   Summary
Glossary and References
   Glossary of terms
      A
      B
      C
      D
      F
      I
      M
      P
      Q
      R
      S
      T
      U
      V
   Book references
   Internet references
Index
   Why subscribe?
Other Books You May Enjoy
   Packt is searching for authors like you
   Share Your Thoughts
   Download a free PDF copy of this book