Meeting the Challenges of Data Quality Management

Meeting the Challenges of Data Quality Management
Copyright
Dedication
Contents
Foreword
Glossary
In praise of Meeting the Challenges of Data Quality Management
Introduction: The Challenges of Managing Data Quality
	Why Focus on Data Quality Management?
	Data Quality Management Goals
	Data Quality and the Context of the Organization
	The Five Challenges
	The Structure of This Book
		Section 1: Data in Today’s Organizations
		Section 2: The Five Challenges in Depth
		Section 3: Data Quality Management Practices
	Why I Wrote This Book
About the Author
Acknowledgments
Bibliography
Data in Today’s Organizations
1 The Importance of Data Quality Management
	Introduction
	Data and Value
	The More Things Change, the More They Stay the Same
	Every Organization Is Data-Dependent
	Big Data Is Here
	The Dream of Fully Integrated Data
	The Focus on Volume Distracts from Value
	New Data Opportunities Conflict with Each Other
	The Drive to Improve Data Quality Has Faded
	Organizational Responsibility for the Quality of Data Remains Ambiguous
	Poor-Quality Data Is Costly, Dangerous, and Tolerated
		Costly
		Dangerous
		Tolerated
	Meeting the Challenges
2 Organizational Data and the Five Challenges of Managing Data Quality
	Introduction
	The Five Challenges of Managing Data Quality
		The Data Challenge
		The Process Challenge
		The Technical Challenge
		The People Challenge
		The Culture Challenge
		The Sixth Challenge: Vocabulary
	Organizational Data
	Organizational Data and Systems Thinking
	The Data Challenge: The Mechanics of Meaning
		Vocabulary
	The Process Challenge: Managing for Quality
		Vocabulary
	The Technical Challenge: Data-Technology Balance
	The People Challenge: Knowledge and Data Literacy
	The Culture Challenge: Organizational Responsibility for Data
		Vocabulary
	Meeting the Challenges
3 Data Quality and Strategy
	Introduction
	Thinking Strategically
	Data Strategy
	Strategic Alignment: People, Process, and Technology
	Assessing Strategic Readiness for Data Quality Management
		Understanding the Business Strategy
		Assessing the Current State
			The Data Challenge: Organizational Data and Knowledge About Organizational Data
			The Process Challenge
			The Technology Challenge
			The People Challenge: Data Knowledge, Skills, and Experience
			The Culture Challenge: Data Governance Practices
			The Culture Challenge: Organizational Culture Change
	Defining the Future State
	Making a Plan
The Five Challenges in Depth
4 The Data Challenge: The Mechanics of Meaning
	Introduction
	Data: A Short History
		Scientific Data
		Statistical Data
		Commerce-Based Organizational Data
		Data Since the Introduction of the Computer
		Big Data and Data Quality
	What History Teaches Us About Data Quality
		More Information Is Now “Data”
		Much About Data Remains the Same
		Data’s Function Is Semiotic
		People Make Data
		Data Influences the Shape of Reality
		Knowledge of Data Quality Equals Knowledge of Data
	Meeting the Challenge of Understanding Data
		Teach the Organization About Its Own Data
		Manage Metadata
		Manage the Processes That Create Data
		Cultivate Data Literacy
		Formalize Data Quality Management Practices
		Develop Appropriate Data Governance Practices
5 The Process Challenge: Managing for Quality
	Introduction
	Quality Is Not an Accident
	Definitions of Quality
	Quality Data
	Data as a Product
	The Juran Trilogy: Quality Management Processes
	Dimensions of Product Quality
	Quality Management Principles
		Establish Organizational Commitment to Quality
		Focus on the Customer
		Manage the Production Process
		Manage the Supply Chain
		Measure and Monitor the Process Against Quality Goals
		Build Quality into the Product Life Cycle
		Continuously Improve
	Data Is Different from Other Resources
	Limitations of the Product Model for Data Quality
	Meeting the Process Challenge: Apply Quality Management Principles to Data
		Establish Organizational Commitment to High-Quality Data
		Focus Data Quality Improvement on the Data Consumer
		Manage the Data Production Process
		Manage the Data Supply Chain
		Monitor the Data Production Process
		Manage the Data Life Cycle
		Continuously Improve
	Coda: Build Quality In
6 The Technical Challenge: Data/Technology Balance
	Introduction
	Technology and Data
	Data Is Everywhere
	Information Technology Is Evolving Rapidly
	The Dangers of Technology Hype
	The Tension Between Data and Information Technology
	Codd, the Relational Model, and Data Independence
	Accounting for the Imprint of Technology
		Example: Format Differences in Tax Identification Numbers
		Example: Differences in Querying Tools
	IT Funding Models Contribute to the Tension
	Meeting the Challenges
		Put Data First
		Design Quality In
		Remember That Businesspeople Are IT Customers
7 The People Challenge: Building Data Literacy
	Introduction
	A Few Assumptions
		Data Literacy and General Literacy
		Literacy as a Continuum
		Models of Data Literacy
		Data Literacy and Organizational Data
		Data Literacy and Thinking Skills
	Data About Data Literacy: An Experiment in Observation
	Data Literacy: The Extended Definition
	Data Literacy Skills, Knowledge, Experience
		Skills
		Knowledge
			Metadata: Managing the Organization’s Explicit Knowledge
			Data Knowledge: An Example
		Experience
	The Data-Literate Organization
	The Alternative: Data Illiteracy
	Data Literacy and a Growth Mindset
	Meeting the People/Knowledge Challenge: Build Data Literacy
	Coda: Books for the Journey
8 The Culture Challenge: Organizational Accountability for Data
	Introduction
	Accountability, Responsibility, and Good Faith
	Data Requires Oversight
	The Politics of Data Within Organizations
	The Chief Data Officer
	Data Stewardship
	Data Governance
	What’s Wrong with Data Governance?
		Bad Faith
		Too Much, Too Soon
		Unclear Scope
		The Lure of Shiny Objects
		Failure to Achieve the Main Mission
	Status of the Oversight Problem: Not Solved
	Meeting the Challenges: Improving Data Governance
		Focus Data Governance on Oversight and Changing Behaviors Toward Data
		Focus Data Governance on the Most Important Data
		Align Accountability/Responsibility for Data with Process Ownership
		Put Process Before Tools
		Focus Stewardship on the Organization’s Data Requirements
		Formally Cultivate Better Data Behavior
		Formalize Data Quality Management Practices
Data Quality Management Practices
9 Core Data Quality Management Capabilities
	Introduction
	Data Quality in the Context of Data Management
		Data Architecture
		Data Modeling and Design
		Data Storage and Operations
		Data Integration and Interoperability
		Data Warehousing and Business Intelligence
		Document and Content Management
		Reference Data and Master Data Management
		Data Security
		Metadata Management
		Data Governance
		Other Components of Data Management
		DAMA and Data Quality Management
	ISO 8000 Part 61: Data Quality Management: Process Reference Model
	The Ten Steps Process: Accounting for Data Quality in Projects
	Core Data Quality Management Capabilities
		A Word of Caution: SAIL
	Define Data Quality Standards
	Assess Data Quality
	Monitor Data Quality
		Data Quality Monitoring Principles
	Report on Data Quality
		Practices Around Summarized/Aggregated Results
		Presenting Summarized Issue Management Data
		Presenting the Data Quality Report
	Data Quality Issue Management Overview
		The Issue Management Elephant in the Room
		Issue Management Principles
		Issue Management Phases
			Identification
			Definition
			Root Cause Analysis
			Quantification
			Prioritization
			Remediation
			Tracking and Reporting
	Improve Data Quality
	Applying Core Data Quality Management Capabilities
	Conclusion
10 Dimensions of Data Quality
	Introduction
	Perspectives on the Dimensions of Quality
		Wang and Strong
		Redman
		English
		Loshin
		McGilvray
		ISO 8000
		Common Threads
	Categorizing Dimensions of Quality
	The Meaning Challenge: Choices About Representation
		Data Modeling Terminology
		The Quality of Data Structure: Data Model Quality
			A Note About Unstructured Data
			Example: Uniqueness and Provider Data Granularity
		The Quality of Data Values
	The Process Challenge: Capturing Metadata
		Metadata Requirements to Support Data Quality
		Metadata Quality
			A Note About Unstructured Data
		Reference Data Quality
		Data Governance Policies and Metadata
	The Technical Challenge: Technical Processes Affect the Quality of Data
		System Reliability Characteristics
		Data Quality Dimensions Dependent on System Reliability
	The People Challenge: Data Consumers Are the Arbiters of Quality
	Concluding Thoughts
11 Data Life Cycle Processes
	Introduction
	The Data Life Cycle and the Asset/Resource Life Cycle
	Managing Quality Throughout the Data Life Cycle
		Plan/Prepare
		Create/Obtain
		Design and Enable
		Store and Share
		Maintain
		Use/Apply
		Gather and Utilize Feedback
		Correct, Enhance, and Improve
		Dispose Of
		Benefits of Understanding the Data Life Cycle
	The Data Supply Chain: Moving Data Into and Within an Organization
		Supply Chain Management Defined
		Data Movement as a Supply Chain
		The Supplier/Purchaser Relationship
	The Value Chain: Finding Efficiencies and Adding Value
	The Systems Development Life Cycle
	Concluding Thoughts
12 Tying It Together
Index