Software Testing Strategies: A testing guide for the 2020s

Software Testing Strategies
Contributors
About the authors
About the reviewers:
Preface
   Our scope - beyond button-pushing
      What is Testing?
   Who this book is for
   What this book covers
   To get the most out of this book
   Download the example code files
   Download the color images
   Conventions used
   Get in touch
   Share your thoughts
   Download a free PDF copy of this book
Part 1:The Practice of Software Testing
1
Testing and Designing Tests
   Jumping into testing
   The impossibility of complete testing
      What is the EVP asking for?
      Toward a theory of error
      Testing software – an example
   Testing software – an analysis
      Quick attacks – the airdropped tester
      Test design – input space coverage
      Equivalence classes and boundaries
      Decision tables
      Decision trees
      All-pairs and pairwise testing
      High volume automated approaches
      Other approaches
   Data and predictability – the oracle problem
   Summary
   Further reading
2
Fundamental Issues in Tooling and Automation
   Technical requirements
   No silver bullets – you can’t radically fix the test cycle
   The minefield regression problem
      Coverage model
   The Battleships problem – testing versus checking
      Comparing Battleships to bugs
      Automation rarely beats human intuition
   The maintenance problem
      The dangers of comprehensive tests
   The money problem
   Lessons from the fundamental issues
   Summary
3
Programmer-Facing Testing
   Technical requirements
   The programmer’s view
      Testing and reliability
      The hexagonal architecture
   Introducing FizzBuzz
   Unit tests
   TDD
      Consequences of TDD
   Unit test and unit code design
      Using test doubles to create seams
   Mutation testing
   Web APIs from a test perspective
      Web API testing strategy
   Testing functional and legacy code
   A Roman Numerals Kata
   Summary
4
Customer-Facing Tests
   Technical requirements
   A word of warning
   Human Or Tooling—is it either/or?
   GUI test automation patterns
      Eliminating redundancy with domain-specific libraries
      Eliminating redundancy through object locators
      Do you need conditionals, looping structures, and variables?
      The tradeoff between assertion and image recognition
   Designing your own system
      Toward specification by example
      Specification by example
   Low-code and no-code test automation
   Batch- and model-driven test automation
   This chapter is completely wrong
   Summary
5
Specialized Testing
   Technical requirements
   Understanding load and performance testing
      Getting to know the basics of load testing
      Setting up a load test
   Exploring security testing
      Security concepts
      Checking out common security vulnerabilities
      Learning about industry standards
      Investigating some security testing tools
   Delving into accessibility testing
      What is accessibility?
      Advocating for accessibility
      Investigating the distinctions between accessibility and inclusive design
      Learning about the WCAG standard
      Investigating some accessibility testing tools
   Internationalization and localization
      Preparing for internationalization and localization
      Investigating tools for internationalization and localization
   CI
      CI and the pipeline
      Getting involved with build management as a tester
      Investigating CI tools
   Regulated testing
      Navigating regulatory requirements
   Summary
6
Testing Related Skills
   Technical requirements
   Finding bugs
      Oracles in software testing
      Inattentional blindness and Oracles
      About the word bug
   Writing bug reports
      Effective bug reports
      Effective reproduction steps
   Planning testing – cases and plans
      Test cases in practice
   Metrics and measurement
      Metric dysfunction
   Project projections
   Influencing change
      Summarizing information
   Summary
7
Test Data Management
   Technical requirements
   The test data problem
      Test data and database apps
   The standard data seed
      Table-driven – when test data drives behavior
   Scriptable users and structure
   Exploring synthetic users
   Leveraging production refreshes
   Exploring development, test, and production environments
   Understanding the regulatory issues in test data management
      The user spy feature
   Summary
Part 2:Testing and Software Delivery
8
Delivery Models and Testing
   Technical requirements
   Waterfall
      The advantages of and place for the waterfall
      The V-Model
      Iterative, incremental, and mini waterfalls
   Extreme Programming (XP)
      The context of XP
   Scrum and SAFe
      The context of Scrum
      SAFe and its context
   House-rules software development
      The Agile Manifesto
      Context-driven testing
      Illustrations of the principles in action
      Kanban as a house rule
   Continuous delivery and deployment
   DevOps, Platform Engineering, SRE
   Summary
   Further reading
9
The Puzzle Pieces of Good Testing
   Technical requirements
   Recipes – how to do hard things
      Defining recipes
      Shouldn’t recipes just be automated?
      Do recipes overlap with technical documentation?
   Coverage – did we test the right things well enough?
      Precise code coverage measures
      Closing out coverage
   Defects – what is the status of the software?
   Schedule and risk – too many test ideas, not enough time
      Iterative testing
   Strategy – what are our risks and priorities?
   Dashboard – how do we communicate what we know?
   Summary
10
Putting Your Test Strategy Together
   What are we doing now?
      Getting the form filled out
      The elevator pitch
   A census of risk
      Defining a real risk census
   Setting priorities, time management, and scope
   Today’s strategy versus tomorrow’s goals
   Summary
11
Lean Software Testing
   Lean software testing defined
      From ideas in practice to the term “Lean”
   The seven wastes
      Waste #1 – transport
      Waste #2 – inventory
      Waste #3 – motion
      Waste #4 – waiting
      Waste #5 – overprocessing
      Waste #6 – overproduction
      Waste #7 – defects
      (New) waste #8 – ability
      Removing waste in testing
   Flow
      Visualizing flow – an example
      Multitasking
      Measurement – lead time versus cycle time
      Efficiency and congestion
      Metric – touch time
      Batch size
      Queues and efficiency
      Arrival time pacing
      Limiting work in progress to create a pull system
      Release cadence
   One-piece flow and CD
   Summary
Part 3:Practicing Politics
12
Case Studies and Experience Reports
   RCRCRC at scale
      The 1-day test plan
      RCRCRC in the enterprise
   A coverage dashboard
      Test coverage blinders
   Pair and tri-programming
      Discovering the expertise paradox
      Making expertise transparent
   The evolution of the test strategy
      The alternative to evolution – information hiding
   Professional pushback – dealing with bullies
      Power in the workplace
      One way to say it
      Boundaries in the enterprise
      Narcissistic communication
      Boundary enforcement – the power move
      If you choose to stay
   Summary
13
Testing Activities or a Testing Role?
   Technical requirements
   The cultural conflict with a testing role
      How we got here – the bad news
      How we got here – developing at internet speed
   Building a risk mitigation team
      The purpose of the risk exercise
   Faith-based versus empirical test automation
      The math behind faith-based test automation
      Possible outcomes
   Shift left and shift right
   (Actually) continuous testing
   Summary
14
Philosophy and Ethics in Software Testing
   Philosophy and why it matters in testing
      Sprint Length: it depends
      Shu Ha Ri: learning by osmosis
      A tough question: what do you want?
   Ethics and ethical reasoning in testing
      Ethical frameworks
      Classic ethical challenges
   Practical ethical issues in testing
      Skipping test steps and the good news
      Decisions are not made in meetings
   Scientific thinking and logical fallacies
      Logical fallacies
   How we wind up in hell and how to escape
      Put the responsibility in the right place
   Summary
   Additional readings for the serious philosopher
15
Words and Language About Work
   Context-driven testing and the other schools
      The Agile school
      The DevOps or CD school
      The analytical school
      The factory school
      The quality school
      The context-driven school
   Precise language
   Wordsmatter
      The benefits of being imprecise
   Process versus skill
      Testing and checking
      Yes, we can assure quality
   Summary
   Further reading
16
Testing Strategy Applied
   A mobile test strategy example
      Our sample mobile app
      Designing a test program
      Mobile system coordination and deployment
      The human element
   AI in software testing
      The state of AI tools
      Possible ways to use AI for testing
      Other forms of AI in testing
      The bottom line for AI testing
   A few thoughts to leave with
      Thoughts on documentation
   Summary
Index
   Why subscribe?
Other Books You May Enjoy
   Packt is searching for authors like you
   Share your thoughts
   Download a free PDF copy of this book