Competing with Data Quality: Relevance and Importance in Industry

COMPETING WITH HIGH QUALITY DATA......Page 3
Contents......Page 9
Foreword......Page 15
Prelude......Page 17
Preface......Page 19
Acknowledgments......Page 21
 1.1 UNDERSTANDING THE IMPLICATIONS OF DATA QUALITY......Page 25
 1.2 THE DATA MANAGEMENT FUNCTION......Page 28
 1.4 GUIDE TO THIS BOOK......Page 30
Section I Building a Data Quality Program......Page 35
  2.1 DATA QUALITY FOUNDATIONAL CAPABILITIES......Page 37
   2.1.2 Skilled Data Quality Resources......Page 38
   2.1.5 Data Integration......Page 39
   2.1.8 Data Quality Monitoring and Control......Page 40
   2.2.2 Conduct a Current-State Analysis......Page 41
   2 .2.5 Detailed Discussion on Establishing the Data Quality Program......Page 42
   2.2.6 Assess the Current State of Data Quality......Page 45
  2.3 CONCLUSIONS......Page 46
  3.1 SIX SIGMA METHODOLOGIES......Page 47
   3.1.1 Development of Six Sigma Methodologies......Page 49
   3.2.1 The Define Phase......Page 52
   3.2.2 The Assess Phase......Page 55
   3.2.3 The Improve Phase......Page 60
   3.2.4 The Control Phase (Monitor and Measure)......Page 61
  3.3 CONCLUSIONS......Page 64
Section II Executing a Data Quality Program......Page 65
  4.1 BUILDING A DATA QUALITY COST QUANTIFICATION FRAMEWORK......Page 67
   4.1.1 The Cost Waterfall......Page 68
   4.1.2 Prioritization Matrix......Page 70
   4.1.3 Remediation and Return on Investment......Page 74
  4.2 A TRADING OFFICE ILLUSTRATIVE EXAMPLE......Page 75
  4.3 CONCLUSIONS......Page 78
  5.1 WHAT IS STATISTICAL PROCESS CONTROL?......Page 79
   5.1.1 Common Causes and Special Causes......Page 81
   5.2.1 Different Types of Data......Page 83
   5.2.2 Sample and Sample Parameters......Page 84
   5.2.3 Construction of Attribute Control Charts......Page 86
   5.2.4 Construction of Variable Control Charts......Page 89
   5.2.5 Other Control Charts......Page 91
  5.3 RELEVANCE OF STATISTICAL PROCESS CONTROL IN DATA QUALITY MONITORING AND REPORTING......Page 93
  5.4 CONCLUSIONS......Page 94
   6.1.1 Data Elements and Critical Data Elements......Page 95
   6.1.2 CDE Rationalization Matrix......Page 96
  6.2 ASSESSMENT OF CRITICAL DATA ELEMENTS......Page 99
   6.2.1 Data Quality Dimensions......Page 100
   6.2.2 Data Quality Business Rules......Page 102
   6.2.3 Data Profi ling......Page 103
   6.2.5 Results Recording and Reporting (Scorecard)......Page 104
  6.3 CONCLUSIONS......Page 106
  7.1 THE FUNNEL METHODOLOGY (STATISTICAL ANALYSIS FOR CDE REDUCTION)......Page 107
   7.1.1 Correlation and Regression Analysis for Continuous CDEs......Page 109
   7.1.2 Association Analysis for Discrete CDEs......Page 112
   7.1.3 Signal-to-Noise Ratios Analysis......Page 114
   7.2.1 Basel II: CDE Rationalization Matrix......Page 115
   7.2.2 Basel II: Correlation and Regression Analysis......Page 118
   7.2.3 Basel II: Signal-to-Noise (S/N) Ratios......Page 120
  7.3 CONCLUSIONS......Page 123
  8.0 INTRODUCTION......Page 125
  8.2 ANALYTICAL FRAMEWORK (ANOVA, SPCs, THRESHOLDS, HEAT MAPS)......Page 126
   8.2.1 Thresholds and Heat Maps......Page 127
   8.2.2 Analysis of Variance (ANOVA) and SPC Charts......Page 131
  8.3 APPLICATION OF THE FRAMEWORK......Page 133
  8.4 CONCLUSIONS......Page 136
  9.1 DESCRIPTION OF THE METHODOLOGY1......Page 137
  9.2 DATA QUALITY METHODOLOGY......Page 138
  9.3 PROCESS QUALITY/SIX SIGMA APPROACH......Page 139
  9.4 CASE STUDY: ISSUE RESOLUTION PROCESS REENGINEERING......Page 141
  9.5 CONCLUSIONS......Page 143
  10.0 INTRODUCTION......Page 145
  10.1 TYPICAL SYSTEM ARRANGEMENT......Page 146
   10.2.1 Study of Two-Factor Combinations......Page 147
   10.2.2 Construction of Combination Tables......Page 148
  10.3 MTS SOFTWARE TESTING......Page 150
  10.4 CASE STUDY: A JAPANESE SOFTWARE COMPANY......Page 154
  10.5 CASE STUDY: A FINANCE COMPANY......Page 157
  10.6 CONCLUSIONS......Page 162
  11.1 DATA TRACING METHODOLOGY......Page 163
   11.1.1 Statistical Sampling......Page 166
   11.2.1 Analysis of Test Cases and CDE Prioritization......Page 168
  11.3 DATA LINEAGE THROUGH DATA TRACING......Page 173
  11.4 CONCLUSIONS......Page 175
  12.1 THE MAHALANOBIS-TAGUCHI STRATEGY......Page 177
  12.2 STAGES IN MTS......Page 182
   12.3.1 The Role of Orthogonal Arrays......Page 183
   12.3.2 The Role of S/N Ratios in MTS......Page 185
   12.3.3 Types of S/N Ratios......Page 186
   12.3.4 Direction of Abnormals......Page 188
  12.4 A MEDICAL DIAGNOSIS EXAMPLE......Page 196
  12.5 CASE STUDY: IMPROVING CLIENT EXPERIENCE......Page 199
   12.5.1 Improvements Made Based on Recommendations from MTS Analysis......Page 201
  12.6 CASE STUDY: UNDERSTANDING THE BEHAVIOR PATTERNS OF DEFAULTING CUSTOMERS......Page 202
  12.7 CASE STUDY: MARKETING......Page 204
   12.7.3 Identification of Useful Variables......Page 205
  12.8 CASE STUDY: GEAR MOTOR ASSEMBLY......Page 206
   12.8.1 Apparatus......Page 207
   12.8.3 High-Resolution Encoder......Page 208
   12.8.5 Characterization......Page 209
   12.8.6 Construction of the Reference Group or Mahalanobis Space......Page 210
   12.8.7 Validation of the MTS Scale......Page 211
   12.8.8 Selection of Useful Variables......Page 212
  12.9 CONCLUSIONS......Page 213
  13.1 DATA AND ANALYTICS AS KEY RESOURCES......Page 215
   13.1.1 Different Types of Analytics......Page 217
   13.1.2 Requirements for Executing Analytics......Page 219
   13.1.3 Process of Executing Analytics......Page 220
  13.2 DATA INNOVATION......Page 221
   13.2.1 Big Data......Page 222
   13.2.2 Big Data Analytics......Page 223
   13.2.3 Big Data Analytics Operating Model......Page 230
   13.2.4 Big Data Analytics Projects: Examples......Page 231
  13.3 CONCLUSIONS......Page 232
  14.1 BUILDING A DQPC......Page 233
  14.2 CONCLUSIONS......Page 235
Appendix A EQUATIONS FOR SIGNAL-TO-NOISE (S/N) RATIOS......Page 237
Appendix B MATRIX THEORY: RELATED TOPICS......Page 241
Appendix C SOME USEFUL ORTHOGONAL ARRAYS......Page 245
Index of Terms and Symbols......Page 283
References......Page 285
Index......Page 291