Unifying Business, Data, and Code: Designing Data Products with JSON Schema

Copyright
Table of Contents
Preface
	What You Can’t See Can Kill You, and the Same Is True for Data
		Hidden Threats to Organizations: A Modern Parallel
		Your AI Is Only as Good as Your Data
		Aligning Problem-Solving Strategies, Data, and AI
	A New Paradigm to Optimize Data Management and Business Strategy for the Age of AI
	The Origin Story of Unifying
	Orchestrating Alignment at Organizational Scale
	Conventions Used in This Book
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
Chapter 1. The Need for a Unifying Data Strategy
	Your Quest for Data-Driven Breakthroughs Begins
		There Are Usually Multiple, Conflicting North Stars
		The Good, the Bad, and the Ugly of Data Problems
		The Problem with Problems
		Unifying Concepts: The Key to Innovation
	What a Unifying Data Strategy Will Do for Agile
		Defining Being Agile
		Agile Theater
		Agile, Waterfall, and Unifying
		Defining a Unifying Data Strategy Approach
	Understanding the Phrase Being Data Driven
		To Be Data Driven, Be Data Centric
		Bottlenecks Preventing Teams from Being Data Driven
	This Book’s Project: Intelligence.AI Coffee Beans
	Summary
Chapter 2. The Lingua Franca of Data: JSON
	Introducing JSON
		A Simple JSON Example
		JSON Viewing and Authoring Tools
	Overview of JSON Grammar
		Booleans
		Numbers
		Strings
		Arrays
		Objects
		Null
		Learning More
		Minification
		Alternative Representations
	Creating a JSON Document
		A Product Entry
		A Store Order
	Summary
Chapter 3. Data-Centric Innovation: A Guide for Data Champions
	Data Transformations Require Data Champions
	The Rise of the Data Product Manager
	Alignment Is a Journey, Not a Destination
		Evaluating Alignment from a Holistic Perspective
		The Goal Isn’t Alignment, It’s Effective Alignment
		Strategies for Setting Up Teams for Success
	Incorporating a Product Management Mindset
		Defining Data Users’ Needs
		Defining Product Features
		Defining and Measuring Success
	Unifying Versus Aligning
	Summary
Chapter 4. Concept-First Design for Data Products
	Packaging and Products: An Example Using Coffee
	The Four Facets of a Data Product
	Getting Started with Concept-First Design
	A Blueprint for Unifying
	Mapping the Conceptual Terrain: Assessing Concepts
	Facilitating Assessments of Conceptual Alignment Across Technical and Nontechnical Teams
	Smooth Is Slow, Slow Is Fast
	Summary
Chapter 5. A Universal Language for Data
	What Is JSON Schema?
		What Is a Schema?
	The Building Blocks of JSON Schema
		Vocabularies and Dialects
		Meta-Schemas: Schemas That Describe Other Schemas
	Understanding JSON Schemas
		Step 1: Determining the Schema Dialect: The $schema Keyword
		Step 2: Determining the Schema Vocabularies
		Step 3: Understanding Schema Vocabularies
		Step 4: Understanding Schema Keywords
	JSON Schema as a Recursive Data Structure
	Referencing Schemas
		What does duplication look like?
		Local referencing
		Remote referencing
	Your First JSON Schema Project
		Writing a Schema: Step by Step
		Generating a Web Form
	Summary
Chapter 6. The Art of Alignment
	Enemies of Alignment: Ambiguity and Assumptions
		Ambiguity: The Culprit in the Illusion of Communication
		Assumptions: Ambiguity’s Best Friend
	Defining Success: Symmetry Between Concepts and JSON Schema Equals Minimal Ambiguity
	Illuminating Misalignment with a Concept Compass
		Step 1: Harmonizing the What
		Step 2: Harmonizing the Way
		Step 3: Harmonizing the How
		Harmonized Concepts
	Validating Concepts: Belief Scoring and Hypotheticals
		Counterfactuals
		Belief Scoring
	Summary
Chapter 7. The Science of Synchronization
	An Introduction to Thinking in Networks
		Example of Thinking in Networks: Athletes Versus Artists
		Graphs: The Visual Language of Networks
	Networks of Entities: Knowledge Graphs
		A Simple Knowledge Graph
		Challenges with Knowledge Graphs
		Aligning Knowledge for the 99%
	Fundamentals of CLEAN Data Governance
		Collaboration
		Knowledge
		Business Logic
		Activity
	CLEAN Data Governance in Practice
	The Four Facets of Data Products and CLEAN
	The Four Horsemen of Data Death
		Ignorance
		Siloed Incentives
		Shortsightedness
		Indecisiveness
	The Power of Design in Collaborative Networks
	Summary
Chapter 8. The Two Fundamental Operations of Schemas
	Validating the Structure of Data
		Using an Online Validator
		Validation Example
		JSON Schema as a Constraints Language
		Boolean Schemas
		Heterogeneous Data Structures
		The format Keyword
	Using Annotations to Define Meaning
		Annotation Extraction Example
		A Simple Use Case: Deprecations
		Runtime Extraction
		Standard Output Formats
		Revisiting the format Keyword
		Using an Online Validator
	Thinking in Schemas
	Summary
Chapter 9. Illuminating Pathways of Acceleration
	How Ambiguity, Knowledge Gaps, and Blind Spots Influence Decisions and Progress Toward Goals
	Which Is Bigger: Greenland or the US?
	Mapping Pathways of Processes and Progress
		Measuring Progress Toward Goals
		Defining Decisions and Steps with Process Maps
		How Process Maps Reveal Ambiguity
	Visualizing and Removing Ambiguity in Processes
		Enriching Process Maps with Annotations
		Process Maps Reveal Innovation Opportunities
	Summary
Chapter 10. Spectrums of Success
	An Introduction to Knowledge Frameworks
		Knowledge Experiences and Pathways
		A Tool for Designing Knowledge Experiences
		From Structured Knowledge to Computational Knowledge
	Success Spectrums
		Mapping Progress and Value
		Visualizing and Adding “Next Best States”
		Removing Blind Spots
		Embracing Multiperspective Design and Road Maps
		Defining KPIs for Success Measures and Metrics (Assessments)
		Using Demons and Magical Thinking for Innovation
		Faster Horses
		Imagining Magical Possibilities
		Problem Landscapes: Quantifying Pain Points Threatening Value
	Nudges: The Right Information at the Right Time
	A Real-Life Problem Landscape and Demon Example That Led to a Unified Data Product Model
		Understanding the Problem Landscape
		The Staggering Impact
		A Meeting of Minds and the Birth of a Solution
	Beyond Data Products: Data Product Management
	The Circular Nature of Unifying
	Summary
Chapter 11. Deploying a JSON Schema Registry
	Schemas Over HTTP
	Step 1: Setting Up a GitHub Repository
		Creating a GitHub Repository
		Uploading Your First Schema
	Step 2: Deploying to Cloudflare Pages
		Creating a New Cloudflare Pages Website Project
	Step 3: Configuring HTTP Headers
		Inspecting the Current HTTP Headers
		Declaring Custom HTTP Headers on Cloudflare Pages
		Checking the Results
	Step 4: Creating a Landing Page
		Adding an HTML Entry Point
	Step 5: Adding a Custom Domain
		Configuring a Custom Domain in Cloudflare Pages
		Setting Up a CNAME DNS Record
		Checking the Results
	Best Practices
		Schemas Are Immutable
		Adopt a Versioning Strategy
	Summary
Chapter 12. Designing Data Products Using JSON Schema
	First Facet: Data
		An Example CSV Dataset
		A JSON Row Representation
	Second Facet: Structure
		General-Purpose Concepts
		Application-Specific Concepts
		Dataset Entries
		The Dataset Schema
	Third Facet: Meaning
		Timestamp
		IP Address
		Email
		US State
		Currency
		Price
		Milestone
		Analytics Entry
	Fourth Facet: Context
		The Signup Analytics Schema
	Summary
		Automated Schema Extraction
		Next Steps
Chapter 13. Extending JSON Schema
	Simple Case: Unknown Keywords
		Extracting Unknown Keywords as Annotations
		Pros and Cons of This Approach
	Complex Case: Authoring Vocabularies
		The JSON Schema Vocabulary System
		Step 1: Writing a Specification
		Step 2: Writing a Vocabulary Meta-Schema
		Step 3: Extending an Implementation
	Consuming Vocabularies
		Defining a Dialect
		Making Use of the Dialect
		Example: Extracting Annotations with Hyperjump
	Summary
Chapter 14. Introducing JSON Unify
	Introducing the Dataset Vocabulary
		Revisiting the Signup Analytics Example
	JSON Schema Bundling
		The Bundling Process
		Bundling Our Example Data Product
	Referencing Remote Data
		The Problem of Streaming JSON
		Introducing JSON Lines
	Extracting Meaning
		A Simple Example
		Using Logic Operators
		The Signup Analytics Example
	Dataset Lineage
		Filtering
		Transforming
		Aggregation
	Summary
Chapter 15. Principles of Designing Intelligence
	Your Unifying Journey So Far
	A Constellation of Deeper Principles Guides Unifying
	1. The Principle of Alignment
		Transforming the Abstract to Concrete
		What You See Can Kill You, and the Same Is True in Data
	2. The Principle of Information
		Understanding Uncertainty
	3. The Principle of Learning
		Defining Learning
		Defining Errors
	4. The Principle of Integrated Simplicity
		Complexity Reduction
		Decomposition
		Compression
		Memoization
		Integrating in Communication Networks
	5. The Principle of Continuums
		Making Things Measurable
		The Dangers of Misusing Measurements
		A Continuum Example for a Control Strategy Problem
	6. The Principle of State Transitions
		A Simple State Machine
		Simplifying State Transitions
	7. The Principle of Decidability
		What Is Decidability?
		Two Key Approaches to Problem Solving
		Making Informed Decisions
		Real-World Decidability to Reduce Misalignment in Teams
	8. The Principle of Heuristics
		Awareness and Ethical Considerations
		Connection to Decision Making in Business
	9. The Principle of Mastery
		Levels of Mastery in Knowledge
		Strategies for Mastery
	10. The Principle of Wisdom
	Summary
Chapter 16. Toward Unified Intelligence
	Functional Artificial Intelligence
		Your AI Is Only as Good as Your Data
		Beware Illusions Within Vetting Processes
		Question Assumptions
	Collective Intelligence
	Collaborative Intelligence
	Unified Intelligence
		Collaborative Learning Networks
		Personalized Knowledge
		Anticipatory Design: Personalization and Digital Twins
	Codifying Principles of Intelligence
		Continuous Human–Machine Learning Loops
		Applying Wisdom in Practice
		Conceptual Zoomability
	Wisdom Graphs: Connecting Concepts, Actions, and Outcomes
		Cognitive Primitives: Standardizing Cognitive Experience Design
	The Value of Unifying
		Prioritize Knowledge Before AI
		A Tale of Simple Knowledge Versus Complex Intelligence
		Follow the Principle of Integrated Simplicity
	Summary
	Going Beyond This Book
Index
About the Authors
Colophon