System Verification: Proving the Design Solution Satisfies the Requirements

Front Cover......Page 1
System Verification......Page 4
Copyright Page......Page 5
Table of Contents......Page 6
LIST OF ILLUSTRATIONS......Page 16
LIST OF TABLES......Page 20
PREFACE......Page 22
ACKNOWLEDGMENTS......Page 24
LIST OF ABBREVIATIONS......Page 26
Part 1 SETTING THE STAGE......Page 30
1.1.1 The Meaning of the Word Verification......Page 32
1.1.2 Verification Classes......Page 33
1.1.2.1 Item Qualification......Page 34
1.1.2.3 System Test and Evaluation......Page 35
1.1.5 Process Verification......Page 36
1.1.6.3 One-of-a-Kind Production Program......Page 37
1.1.7 Verification Documentation Intensity......Page 38
1.1.8 In the Aggregate......Page 40
1.2.1 What Is a System?......Page 42
1.2.3 Three Steps on the Way to Great Systems......Page 43
1.2.4 Organizational Structure......Page 44
1.2.5 The Systems Approach......Page 46
1.2.6 The Two Vs......Page 48
1.2.7 The Foundation of System Engineering......Page 50
1.2.8 System Development Phasing Overview......Page 53
1.2.9 Toward a Standard Process......Page 56
1.2.10 Development Environments......Page 58
1.2.10.2 The Spiral Development Model......Page 59
1.2.10.3 The V Development Model......Page 61
1.2.10.4 The N Development Model......Page 62
1.2.10.5 Development Environment Integration......Page 63
1.3.1 Requirements......Page 68
1.3.2 The Need and Its Initial Expansion Using Traditional Structured Analysis......Page 72
1.3.3 Structured Decomposition Using Traditional Structured Analysis......Page 76
1.3.3.1 Functional Analysis......Page 79
1.3.3.2 Performance Requirements Analysis......Page 82
1.3.3.3.1 Interface Requirements Analysis......Page 84
1.3.3.3.2 Environmental Requirements Analysis......Page 85
1.3.4 Computer Software Approaches......Page 88
1.3.5 Verification Requirements......Page 91
1.3.6 Applicable Documents......Page 92
1.3.7 Process Requirements Analysis......Page 93
Part 2 ITEM QUALIFICATION VERIFICATION......Page 96
2.1.1 Verification Documentation......Page 98
2.1.2 Item Planning Fundamentals......Page 100
2.1.2.1 Traceability Matrix......Page 101
2.1.2.2 Verification Methods......Page 103
2.1.2.3 Product and Verification Levels......Page 105
2.1.2.4 Verification Classes......Page 106
 2.1.2.5 Items Subject to Qualification and Acceptance......Page 108
2.1.2.7 Product Verification Layering......Page 109
2.1.3.1 Selecting the Method......Page 110
2.1.3.3 Writing the Verification Paragraph......Page 111
2.1.4 Verification Planning, Data Capture, and Documentation......Page 113
2.1.5.1 MIL-STD-961E Structure......Page 116
2.1.5.2 An Alternate Structure......Page 118
2.1.6 Verification Computer Databases......Page 119
2.2.1 A Matter of Scale......Page 122
2.2.2 Expansion of Function F44......Page 123
2.2.3 Item Qualification Process......Page 125
2.2.4 The Planning Transform......Page 126
2.2.5 Qualification Process Implementation......Page 133
2.2.6.2 Generic Analysis Task......Page 137
2.2.7 Program Task Structuring......Page 138
2.3.1 Overview......Page 140
2.3.2.1 Test Philosophy......Page 141
2.3.2.2 Top-Down and Bottom-Up Planning......Page 142
2.3.2.3 Qualification Test Scheduling......Page 143
2.3.2.4 Qualification Test Item Selection......Page 144
2.3.2.5 Item to Task Transformation......Page 146
2.3.2.6 Verification Information Organization......Page 147
2.3.3.1 Verification Matrix Test Data Fusion......Page 150
2.3.3.2.2 Process Analysis......Page 153
2.3.3.2.3 Resource Identification......Page 155
2.3.3.3 Test Procedure Development......Page 156
2.3.3.4 Validate the Plan......Page 158
2.3.4 Integrated Verification Plan Test Integration......Page 159
2.3.5.1 Life Cycle Requirements......Page 162
2.3.5.2 Rare Environmental Conditions......Page 165
2.3.6.1 Hardware......Page 166
2.3.6.2 Application Specific Integrated Circuit (ASIC)......Page 168
2.3.6.4 Computer Software......Page 170
2.3.6.4.2 Software Test Methods......Page 172
2.3.7 Commercial Subset......Page 175
2.4.1 Implementation Responsibility and Organizational Structure......Page 178
2.4.2 General Coordination Activities......Page 181
2.4.3 Individual Test Coordination......Page 182
2.4.3.1 Authorization to Proceed......Page 183
2.4.3.3 Test Readiness Review......Page 184
2.4.3.4 Test Task Go-Ahead......Page 185
2.4.3.6 Resolving Test Problems......Page 186
2.4.3.7 Test Results Documentation and Review......Page 188
2.4.3.8 Residual Disposition......Page 189
2.4.4 Intertest and Intermethod Coordination......Page 190
2.4.5 Qualification Test Sequencing......Page 191
2.4.6 First Production Article Integration......Page 194
2.5.1 Other Methods and Meanings......Page 196
2.5.3 Nontest Documentation......Page 197
2.5.4.1 Specialty Engineering Requirements......Page 198
2.5.4.1.1 System Safety......Page 199
2.5.4.1.2 Reliability......Page 200
2.5.4.1.3 Maintainability......Page 201
 2.5.4.2 Interface Requirements Verification Analysis......Page 202
2.5.4.4 Analysis Strings......Page 203
2.5.5.3 Environmental Influences......Page 204
2.5.6.1 Software Applications......Page 205
2.5.6.2 Hardware Too......Page 206
2.6.2 Engineering Ethics......Page 208
2.6.3 Report Structures......Page 209
2.6.3.2 Independent Reports......Page 210
2.6.5 Report Review, Approval, and Release......Page 211
2.6.7.1 Supplier Reporting and Access through the SDRL......Page 213
2.6.7.2 Associate Reporting and Distribution......Page 214
2.7.1 Setting the Management Stage......Page 216
2.7.2 The Management Agent......Page 217
2.7.3 Documentation Overview......Page 219
2.7.5 Task Readiness Review and Task Authorization......Page 221
2.7.6.2 In-House Review of Supplier Verification Reports......Page 222
2.7.6.3 Associate Interaction and Interface Verification Resolution......Page 224
2.7.6.5 Customer Coordination......Page 225
2.7.7.2.1 Completion Criteria......Page 227
2.7.7.2.3 Agenda Responsibilities Assignment......Page 228
2.7.7.2.6 Audit Dry Run......Page 230
2.7.8 Presentation of the Qualification Results: The Audit......Page 231
2.7.10 Information Resources......Page 232
2.8.1 System Verification Overview......Page 236
2.8.2.2 The Audit......Page 238
2.8.2.2.3 Interface Verification Audit......Page 239
2.8.3 Audit Follow-Up and Closeout......Page 240
Part 3 ITEM ACCEPTANCE VERIFICATION......Page 242
3.1.1 The Notion of Acceptance......Page 244
3.1.4 Conversion to Verification Requirements......Page 245
3.1.6 Associate Contractor Relationships......Page 249
3.1.8 Information Management......Page 250
3.1.10 Relationship between Technical Data and Acceptance......Page 251
3.1.12 Intercontinental Acceptance......Page 253
3.2.1 Organizational Responsibilities......Page 254
3.2.2 The Coordination Task......Page 255
 3.2.4.1 Quality Acceptance Examinations......Page 256
3.2.4.3 Software Acceptance Examination......Page 258
3.2.5.1 Logistics Demonstrations......Page 259
3.2.6 Analysis Cases......Page 260
3.3.2 The Two Steps of Acceptance......Page 262
3.3.4 Reporting Structures......Page 263
3.3.6 Recurring Acceptance Verification......Page 264
3.4.1 The Second Stage of Verification......Page 266
3.4.3 The Basis of Acceptance......Page 267
3.4.5 Management of the Work......Page 269
3.4.6.2 FRACAS Implementation......Page 272
3.4.7.1 PCA Planning and Preparation......Page 274
3.4.7.3 Post-PCA Activity......Page 275
3.4.8 Software Acceptance......Page 276
Part 4 SYSTEM TEST AND EVALUATION......Page 278
4.1.1 The Beginning......Page 280
 4.1.2.1 Product Cases......Page 281
4.1.2.2.1 Development Test and Evaluation (DT&E)......Page 283
4.1.2.2.2 Operational Test and Evaluation (OT&E)......Page 284
4.1.2.2.3 Initial or Interim Operational Test and Evaluation (IOT&E)......Page 285
4.1.2.2.4 Follow-On Operational Test and Evaluation (FOT&E)......Page 286
4.1.3.2 System Compatibility Test......Page 287
4.1.3.3 Certification......Page 290
4.1.4 The Grand Plan......Page 291
4.1.5 The Limits of Intellectual Thought......Page 292
4.1.6 The Details......Page 293
4.2.2 Instrumentation Management......Page 294
4.2.3 Reaching a State of DT&E Readiness......Page 296
4.2.4 A Normal Evolution......Page 297
4.2.5 Flexibility in the Face of Obstacles......Page 299
4.2.6 DT&E Reporting......Page 300
4.2.8 OT&E......Page 301
4.2.9 Test Results Applications......Page 302
4.2.10 System Test and Evaluation Closure......Page 303
4.3.1 Reverification Exposed......Page 304
4.3.2.1 Preproduction Development......Page 305
4.3.2.2 Manufacturing......Page 306
4.3.3 Temporal Considerations......Page 307
4.3.4 Process Redesign......Page 309
Part 5 PROCESS VALIDATION AND VERIFICATION......Page 312
5.1.1 Is There a Difference?......Page 314
5.1.2.1 Completeness......Page 315
5.1.2.2 Accounting for Planning Risks......Page 320
5.1.4 Expanding Our Mental Boundaries......Page 322
5.2.2 Program and Functional Metrics......Page 324
5.2.3 Use of C/SCS in Process Verification......Page 325
5.2.5 Organizational Possibilities......Page 326
5.2.7 Total Quality Management......Page 330
Part 6 POSTSCRIPT......Page 332
6.1.2 Closure on Meanings......Page 334
6.1.3 Hopes for Balanced Treatment......Page 335
6.1.4 Information Opportunities......Page 336
6.1.5 The Role of the System Engineer......Page 337
6.1.6 Singling Up the WBS Lines......Page 340
6.1.7 Focusing on the Whole......Page 346
6.2.1 Introduction......Page 350
6.2.2 The Contractual Difference......Page 351
6.2.3.1 Concept Validation......Page 352
6.2.3.3 Verification......Page 353
6.2.3.3.3 System Test and Evaluation Verification......Page 354
6.2.3.5.2 Representations Verification......Page 355
6.2.4 Toward a Generic Process......Page 356
6.2.5 Final Closure on Meanings......Page 361
6.2.6 Final Closure......Page 362
INDEX......Page 364