Advanced Software Testing - Vol. 3: Guide to the ISTQB Advanced Certification as an Advanced Technical Test Analyst

Cover Page......Page 1
About the Authors......Page 3
Title Page......Page 4
Copyright......Page 5
Rex Black's Acknowledgements......Page 6
Jamie Mitchell's Acknowledgements......Page 8
Contents......Page 10
Introduction......Page 20
1.1 Introduction......Page 24
1.2 Testing in the Software Lifecycle......Page 25
1.3 Specific Systems......Page 30
1.4 Metrics and Measurement......Page 34
1.5 Ethics......Page 37
1.6 Sample Exam Questions......Page 39
2.1 Introduction......Page 42
2.2 Test Process Models......Page 43
2.4 Test Analysis and Design......Page 44
2.4.1 Non-functional Test Objectives......Page 46
2.4.2 Identifying and Documenting Test Conditions......Page 48
2.4.3 Test Oracles......Page 52
2.4.4 Standards......Page 54
2.4.5 Static Tests......Page 57
2.4.6 Metrics......Page 58
2.5 Test Implementation and Execution......Page 59
2.5.1 Test Procedure Readiness......Page 60
2.5.2 Test Environment Readiness......Page 62
2.5.3 Blended Test Strategies......Page 64
2.5.4 Starting Test Execution......Page 65
2.5.5 Running a Single Test Procedure......Page 67
2.5.6 Logging Test Results......Page 68
2.5.7 Use of Amateur Testers......Page 70
2.5.8 Standards......Page 71
2.6 Evaluating Exit Criteria and Reporting......Page 76
2.6.1 Test Suite Summary......Page 77
2.6.2 Defect Breakdown......Page 79
2.6.3 Confirmation Test Failure Rate......Page 80
2.6.4 System Test Exit Review......Page 81
2.6.5 Standards......Page 82
2.6.7 System Test Exit Review......Page 83
2.6.8 Evaluating Exit Criteria and Reporting Exercise Debrief......Page 86
2.8 Sample Exam Questions......Page 90
3. Test Management......Page 92
3.2 Test Management Documentation......Page 93
3.3 Test Plan Documentation Templates......Page 94
3.4 Test Estimation......Page 95
3.6 Test Progress Monitoring and Control......Page 96
3.8 Distributed, Outsourced, and Insourced Testing......Page 97
3.9 Risk-Based Testing......Page 98
3.9.1 Risk Management......Page 101
3.9.2 Risk Identification......Page 103
3.9.3 Risk Analysis or Risk Assessment......Page 105
3.9.4 Risk Mitigation or Risk Control......Page 107
3.9.5 An Example of Risk Identification and Assessment Results......Page 110
3.9.6 Risk-Based Testing throughout the Lifecycle......Page 112
3.9.7 Risk-Aware Testing Standards......Page 113
3.9.8 Risk-Based Testing Exercise 1......Page 115
3.9.9 Risk-Based Testing Exercise Debrief 1......Page 116
3.9.11 Requirements Defect By-Products......Page 118
3.9.13 Risk-Based Testing Exercise Debrief 2......Page 119
3.10 Failure Mode and Effects Analysis......Page 120
3.12 Sample Exam Questions......Page 121
4. Test Techniques......Page 124
4.1 Introduction......Page 125
4.2 Specification-Based......Page 127
4.2.1 Equivalence Partitioning......Page 130
4.2.1.1 Avoiding Equivalence Partitioning Errors......Page 133
4.2.1.2 Composing Test Cases with Equivalence Partitioning......Page 134
4.2.1.3 Equivalence Partitioning Exercise......Page 138
4.2.1.4 Equivalence Partitioning Exercise Debrief......Page 139
4.2.2 Boundary Value Analysis......Page 142
4.2.2.1 Examples of Equivalence Partitioning and Boundary Values......Page 143
4.2.2.2 Non-functional Boundaries......Page 146
4.2.2.3 A Closer Look at Functional Boundaries......Page 147
4.2.2.4 Integers......Page 148
4.2.2.5 Floating Point Numbers......Page 151
4.2.2.6 Testing Floating Point Numbers......Page 153
4.2.2.7 How Many Boundaries?......Page 155
4.2.2.8 Boundary Value Exercise......Page 157
4.2.2.9 Boundary Value Exercise Debrief......Page 158
4.2.3 Decision Tables......Page 163
4.2.3.1 Collapsing Columns in the Table......Page 166
4.2.3.2 Combining Decision Table Testing with Other Techniques......Page 168
4.2.3.3 Nonexclusive Rules in Decision Tables......Page 170
4.2.3.4 Decision Table Exercise......Page 171
4.2.3.5 Decision Table Exercise Debrief......Page 172
4.2.4 State-Based Testing and State Transition Diagrams......Page 177
4.2.4.1 Superstates and Substates......Page 184
4.2.4.2 State Transition Tables......Page 185
4.2.4.3 Switch Coverage......Page 189
4.2.4.4 State Testing with Other Techniques......Page 192
4.2.4.5 State Testing Exercise......Page 193
4.2.4.6 State Testing Exercise Debrief......Page 195
4.2.6 Requirements-Based Testing Exercise Debrief......Page 198
4.3 Structure-Based......Page 200
4.3.1 Control-Flow Testing......Page 202
4.3.1.1 Building Control-Flow Graphs......Page 203
4.3.1.2 Statement Coverage......Page 206
4.3.1.3 Decision Coverage......Page 211
4.3.1.4 Loop Coverage......Page 214
4.3.1.5 Hexadecimal Converter Exercise......Page 218
4.3.1.7 Condition Coverage......Page 220
4.3.1.8 Decision/Condition Coverage......Page 223
4.3.1.9 Modified Condition/Decision Coverage (MC/DC)......Page 224
4.3.1.10 Multiple Condition Coverage......Page 228
4.3.1.11 Control-Flow Exercise......Page 232
4.3.1.12 Control-Flow Exercise Debrief......Page 233
4.3.2 Path Testing......Page 237
4.3.2.1 LCSAJ......Page 238
4.3.2.2 Basis Path/Cyclomatic Complexity Testing......Page 243
4.3.2.4 Cyclomatic Complexity Exercise Debrief......Page 248
4.3.3 A Final Word on Structural Testing......Page 250
4.3.4 Structure-Based Testing Exercise......Page 251
4.3.5 Structure-Based Testing Exercise Debrief......Page 252
4.4 Defect- and Experience-Based......Page 259
4.4.1 Defect Taxonomies......Page 260
4.4.2 Error Guessing......Page 265
4.4.3 Checklist Testing......Page 266
4.4.4 Exploratory Testing......Page 268
4.4.4.1 Test Charters......Page 270
4.4.4.3 Exploratory Testing Exercise Debrief......Page 272
4.4.5 Software Attacks......Page 275
4.4.5.1 An Example of Effective Attacks......Page 279
4.4.5.2 Other Attacks......Page 280
4.4.5.4 Software Attack Exercise Debrief......Page 282
4.4.7 Specification-, Defect-, and Experience-Based Exercise Debrief......Page 283
4.4.8 Common Themes......Page 284
4.5 Static Analysis......Page 287
4.5.1 Complexity Analysis......Page 288
4.5.2 Code Parsing Tools......Page 291
4.5.3 Standards and Guidelines......Page 293
4.5.4 Data-Flow Analysis......Page 296
4.5.5 Set-Use Pairs......Page 298
4.5.6 Set-Use Pair Example......Page 301
4.5.8 Data-Flow Exercise Debrief......Page 307
4.5.9 Data-Flow Strategies......Page 308
4.5.10 Static Analysis for Integration Testing......Page 311
4.5.11 Call-Graph Based Integration Testing......Page 313
4.5.12 McCabe Design Predicate Approach to Integration Testing......Page 315
4.5.13 Hex Converter Example......Page 319
4.5.15 McCabe Design Predicate Exercise Debrief......Page 324
4.6 Dynamic Analysis......Page 325
4.6.1 Memory Leak Detection......Page 328
4.6.2 Wild Pointer Detection......Page 330
4.6.3 API Misuse Detection......Page 331
4.7 Sample Exam Questions......Page 332
5.1 Introduction......Page 346
5.2 Quality Attributes for Domain Testing......Page 348
5.2.1 Accuracy......Page 349
5.2.2 Suitability......Page 352
5.2.3 Interoperability......Page 353
5.2.4 Usability......Page 354
5.2.6 Usability Test Exercise Debrief......Page 358
5.3 Quality Attributes for Technical Testing......Page 360
5.3.1 Technical Security......Page 361
5.3.2 Security Issues......Page 362
5.3.3 Timely Information......Page 367
5.3.4 Reliability......Page 372
5.3.5 Efficiency......Page 378
5.3.6 Multiple Flavors of Efficiency Testing......Page 380
5.3.7 Modeling the System......Page 384
5.3.8 Efficiency Measurements......Page 389
5.3.9 Examples of Efficiency Bugs......Page 391
5.3.11 Exercise: Security, Reliability, and Efficiency Debrief......Page 395
5.3.12 Maintainability......Page 398
5.3.13 Subcharacteristics of Maintainability......Page 402
5.3.14 Portability......Page 409
5.3.16 Maintainability and Portability Exercise Debrief......Page 416
5.4 Sample Exam Questions......Page 419
6.1 Introduction......Page 422
6.2 The Principles of Reviews......Page 426
6.3 Types of Reviews......Page 430
6.4 Introducing Reviews......Page 435
6.5 Success Factors for Reviews......Page 436
6.5.1 Deutsch's Design Review Checklist......Page 440
6.5.2 Marick's Code Review Checklist......Page 442
6.5.3 The OpenLaszlo Code Review Checklist......Page 445
6.6 Code Review Exercise......Page 446
6.7 Code Review Exercise Debrief......Page 447
6.8 Deutsch Checklist Review Exercise......Page 452
6.9 Deutsch Checklist Review Exercise Debrief......Page 453
6.10 Sample Exam Questions......Page 455
7.1 Introduction......Page 458
7.2 When Can a Defect Be Detected?......Page 459
7.3 Defect Lifecycle......Page 460
7.4 Defect Fields......Page 468
7.5 Metrics and Incident Management......Page 472
7.6 Communicating Incidents......Page 473
7.7 Incident Management Exercise......Page 474
7.8 Incident Management Exercise Debrief......Page 475
7.9 Sample Exam Questions......Page 477
8. Standards and Test Process Improvement......Page 480
9.1 Introduction......Page 482
9.2 Test Tool Concepts......Page 483
9.2.1 The Business Case for Automation......Page 484
9.2.2 General Test Automation Strategies......Page 489
9.2.3 An Integrated Test System Example......Page 494
9.3 Test Tool Categories......Page 496
9.3.1 Test Management Tools......Page 497
9.3.2 Test Execution Tools......Page 498
9.3.3 Debugging, Troubleshooting, Fault Seeding, and Injection Tools......Page 502
9.3.4 Static and Dynamic Analysis Tools......Page 503
9.3.5 Performance Testing Tools......Page 506
9.3.6 Monitoring Tools......Page 508
9.3.7 Web Testing Tools......Page 509
9.3.8 Simulators and Emulators......Page 511
9.4 Keyword-Driven Test Automation......Page 512
9.4.2 Capture/Replay Exercise Debrief......Page 518
9.4.3 Evolving from Capture/Replay......Page 520
9.4.4 The Simple Framework Architecture......Page 522
9.4.5 Data-Driven Architecture......Page 525
9.4.6 Keyword-Driven Architecture......Page 527
9.4.7 Keyword Exercise......Page 534
9.4.8 Keyword Exercise Debrief......Page 535
9.5 Performance Testing......Page 537
9.5.1 Performance Testing Exercise......Page 543
9.5.2 Performance Testing Exercise Debrief......Page 544
9.6 Sample Exam Questions......Page 546
10. People Skills and Team Composition......Page 550
10.3 Test Team Dynamics......Page 551
10.5 Motivation......Page 552
10.6 Communication......Page 553
10.7 Sample Exam Questions......Page 555
11.1 Learning Objectives......Page 558
11.1.2 Level 2: Understand (K2)......Page 559
11.1.3 Level 3: Apply (K3)......Page 560
11.1.5 Where Did These Levels of Learning Objectives Come From?......Page 561
11.2 ISTQB Advanced Exams......Page 562
11.2.1 Scenario-Based Questions......Page 564
11.2.2 On the Evolution of the Exams......Page 566
11.2.4 Advanced Syllabus Referenced Books......Page 568
11.2.6 Other References......Page 570
System Requirements Document......Page 572
I: Table of Contents......Page 574
II: Versioning......Page 576
III: Glossary......Page 578
000 Introduction......Page 580
001 Informal Use Case......Page 581
003 Scope......Page 583
004 System Business Benefits......Page 584
010 Functional System Requirements......Page 585
020 Reliability System Requirements......Page 589
030 Usability System Requirements......Page 590
040 Efficiency System Requirements......Page 591
050 Maintainability System Requirements......Page 593
060 Portability System Requirements......Page 594
A: Acknowledgement......Page 596
Appendix C. Answers to Sample Questions......Page 598
B......Page 600
C......Page 601
E......Page 602
F......Page 603
K......Page 604
P......Page 605
R......Page 606
S......Page 607
T......Page 608
W......Page 609