Testing IT: An Off-the-Shelf Software Testing Process

Half-title......Page 3
Title......Page 5
Copyright......Page 6
Dedication......Page 7
Contents......Page 9
Foreword to the Second Edition......Page 15
Foreword to the First Edition......Page 17
Second Edition Acknowledgments......Page 19
First Edition Acknowledgments......Page 20
1.1 Purpose of the Book......Page 21
1.3 How to Read This Book......Page 22
1.4 Structure and Content of This Book......Page 23
Part One The Traditional Testing Process......Page 27
2.2 The Challenge of Testing......Page 29
2.3 What Is Testing?......Page 30
2.4 Verification and Validation......Page 32
2.6 Testing – the Bottom Line......Page 33
2.7 Additional Information......Page 34
3.1 Introduction......Page 36
3.2.1 Positive and Negative Testing......Page 37
3.2.2 White Box and Black Box Testing......Page 38
3.2.4 Automated Software Testing......Page 39
3.3.2 Boundary Value Analysis......Page 41
3.3.4 Random Testing......Page 42
3.3.7 Thread Testing......Page 43
3.4.1 Configuration/Installation Testing......Page 44
3.4.5 Performance Testing......Page 45
3.4.7 Security Testing......Page 46
3.4.9 Usability Testing......Page 47
3.5 Further Reading on Testing Techniques......Page 48
4.1 Introduction......Page 50
4.2 The Organization of Testing......Page 51
4.3.1 Overview......Page 52
4.3.3 Test Team Leader......Page 53
4.3.4 Test Analyst......Page 54
4.3.6 Independent Test Observer......Page 55
Test Automation Architect......Page 56
4.4.1 Overview......Page 57
4.4.2 Unit Testing......Page 58
4.4.3 Integration Testing......Page 59
4.4.5 Systems Integration Testing......Page 60
4.4.7 Regression Testing......Page 61
4.4.8 When to Stop Testing......Page 62
4.5 Role of the V Model in Planning......Page 63
4.6 The Management of Test Requirements......Page 64
4.8 The Role and Use of Defect Tracking......Page 65
4.9 The Role of Risk in Test Planning and Management......Page 66
5.1 Overview......Page 68
5.3 Unit Test Data Requirements......Page 70
5.4 Roles and Responsibilities......Page 71
5.5 Planning and Resources......Page 72
5.8 Outputs......Page 73
6.1 Overview......Page 76
6.2 Integration Test Approach......Page 77
6.3 Integration Test Data Requirements......Page 78
6.5 Planning and Resources......Page 79
6.8 Outputs......Page 80
7.1 Overview......Page 83
7.3 System Test Data Requirements......Page 84
7.4 Roles and Responsibilities......Page 85
7.5 Planning and Resources......Page 86
7.8 Outputs......Page 87
8.1 Overview......Page 90
8.2 Systems Integration Test Approach......Page 91
8.4 Roles and Responsibilities......Page 92
8.5 Planning and Resources......Page 93
8.8 Outputs......Page 94
9.1 Overview......Page 97
9.2 User Acceptance Test Approach......Page 98
9.4 Roles and Responsibilities......Page 99
9.5 Planning and Resources......Page 100
9.6 Inputs......Page 101
9.8 Outputs......Page 102
10.1 Overview......Page 104
10.2 Operations Acceptance Test Approach......Page 105
10.4 Roles and Responsibilities......Page 106
10.5 Planning and Resources......Page 107
10.6 Inputs......Page 108
10.8 Outputs......Page 109
11.1 Overview......Page 111
11.3 Regression Test Data Requirements......Page 113
11.4 Roles and Responsibilities......Page 114
11.5 Planning and Resources......Page 115
11.8 Outputs......Page 116
12.1 Introduction......Page 119
12.2 Overview of the Role and Use of Metrics......Page 120
12.3 Metrics Typically Used Within the Testing Process......Page 121
Defect Detection Effectiveness Percentage or DDE......Page 123
12.4 Setting Up and Administering a Metrics Program......Page 124
12.5 A Proposal for a Simple and Effective Metrics Set......Page 126
12.6 Further Reading......Page 128
Establishing the Requirement......Page 129
Strategy for Introduction and Adoption......Page 130
13.3 Maintenance of the Testing Process......Page 133
13.4 A Proposal for a Quick Start Test Process......Page 134
14.2 Overview of Agile Testing......Page 137
14.3 Agile Quality Management Practices......Page 138
14.4 A Proposal for an Effective Subset of Agile Best Practices......Page 142
14.5 Conclusion......Page 143
Part Two The Testing Process in the Real World: Illustrative Case Studies......Page 145
Reuters Product Acceptance Group......Page 146
The Wine Society......Page 147
Confirmit......Page 148
15.2 Characteristics of the Testing Requirement......Page 150
15.3 The Management and Planning of Testing......Page 151
Testing Manager......Page 153
Tester......Page 154
15.5 Testing Phases......Page 155
15.6 Artifacts......Page 156
15.7 Process Improvement......Page 157
16.1 Overview of the Organization......Page 158
16.2 Testing Requirements......Page 159
16.3 The Management and Planning of Testing......Page 160
PAG Manager......Page 162
RDBGUI Acceptance Managers......Page 163
Tester......Page 164
16.5 Testing Phases......Page 165
Acceptance Testing......Page 166
Regression Testing......Page 167
16.6 Artifacts......Page 168
16.7 Process Improvement......Page 170
17.1 Overview of the Organization......Page 171
17.2 Testing Requirements......Page 172
17.3 The Management and Planning of Testing......Page 173
QA Team Manager......Page 175
Test Analyst/Tester......Page 176
Client Representative......Page 177
System Testing......Page 178
Crash Testing......Page 179
17.6 Artifacts......Page 180
17.7 Process Improvement......Page 181
18.1 Overview of the Organization......Page 183
18.2 Testing Requirements......Page 184
18.3 The Management and Planning of Testing......Page 186
Project Manager......Page 187
User Representative Tester......Page 188
Supplier Representative......Page 189
Regression Testing......Page 190
18.6 Artifacts......Page 191
18.7 Process Improvement......Page 192
19.1 Overview of the Organization......Page 194
19.2 Characteristics of the Testing Requirement......Page 195
19.3 The Management and Planning of Testing......Page 196
Product Development Manager......Page 198
Senior Quality Assurance Analyst......Page 199
Quality Assurance Analyst......Page 200
19.5 Testing Phases......Page 201
Acceptance Testing......Page 202
Regression Testing......Page 203
19.6 Artifacts......Page 204
19.7 Process Improvement......Page 206
20.1 Overview of the Organization......Page 207
20.2 Characteristics of the Testing Requirement......Page 208
20.3 The Management and Planning of Testing......Page 210
Confirmit Head of Research and Development......Page 213
Development Team......Page 214
Quality Assurance Team......Page 215
20.5 Testing Phases......Page 216
20.6 Artifacts......Page 218
20.7 Process Improvement......Page 219
20.8 Acknowledgment......Page 220
Part Three The Appendices......Page 221
Responsibilities......Page 223
References......Page 224
Characteristics of a Test Team Leader......Page 225
Characteristics of a Test Analyst......Page 226
Reporting and Liaison......Page 227
Reporting and Liaison......Page 228
Responsibilities......Page 229
Reference......Page 230
B1 Introduction......Page 231
Outputs......Page 232
Approach......Page 233
References......Page 234
Outputs......Page 235
Inputs......Page 236
References......Page 237
Outputs......Page 238
Inputs......Page 239
References......Page 240
C1 Introduction......Page 241
Test Plan Document Front Sheet......Page 242
Contents......Page 243
1.3 References......Page 244
3.2 Risks......Page 245
4.1 Introduction......Page 246
5.2 Entry Criteria......Page 247
6.2 Roles and Responsibilities......Page 248
6.5 Progress Reporting......Page 249
A Test Plan......Page 250
D1 Introduction......Page 251
Test Specification Document Front Sheet......Page 252
Contents......Page 253
1.4 References......Page 255
2.2.2 Functional Areas......Page 256
2.3.2 Hardware......Page 257
2.5.1 Test Scripts......Page 258
2.5.3 Acceptance Criteria......Page 259
3.2 Pre-test Activities......Page 260
3.3.1 Test Execution Procedure......Page 261
Annex A Test Cases......Page 262
E1 Introduction......Page 263
F1 Introduction......Page 268
G1 Introduction......Page 272
H1 Introduction......Page 276
I2 Creating a Reuse Pack......Page 279
I4 How to Use the Reuse Pack......Page 280
J1 Introduction......Page 281
Test Summary Report Front Sheet......Page 282
Contents......Page 283
1.3 References......Page 284
4 Assessment......Page 285
6 Evaluation......Page 286
7 Summary of Activities......Page 287
K4 Designing the Test Cases......Page 288
L4 Designing the Test Cases......Page 290
M4 Designing the Test Cases......Page 292
N2 The Testing Problem......Page 294
N3 The Pairwise Testing Solution......Page 295
N3 Pairwise Testing in Practice......Page 296
O2 Scoring Scheme......Page 297
O4 Support for Testing Management......Page 298
O6 Tool Defect Analysis Facilities......Page 299
O8 Tool Integration Issues......Page 300
O9 Tool Usability Issues......Page 301
O10 Supplier Issues......Page 302
O11 Training and Consultancy Issues......Page 303
O12 Contractual and Financial Issues......Page 304
O13 Miscellaneous Issues......Page 305
P2 Roles and Responsibilities......Page 309
P4 Usability Testing......Page 310
P5 Post-test Activities......Page 311
Q2 Terminology......Page 312
Q4 Testing Process......Page 313
Q5 Roles and Responsibilities......Page 314
Q7 Optional Evaluation Scheme......Page 315
R2 Object-Oriented Development......Page 318
Iterative Development......Page 320
Rapid Prototyping......Page 321
R4 Impact of OO on Test Design......Page 322
Integration Testing......Page 323
Acceptance Testing......Page 324
Introduction......Page 325
The Testing Challenge......Page 326
Making a Start......Page 327
Test Planning......Page 329
Bugs, Defects, and Problems......Page 330
Managing Stakeholder Expectations......Page 332
Summary......Page 333
References......Page 335
Introduction......Page 339
Index......Page 351