Unifying Business, Data, and Code: Designing Data Products With Json Schema

Preface
   What You Can’t See Can Kill You, and the Same Is True for Data
      Hidden Threats to Organizations: A Modern Parallel
      Your AI Is Only as Good as Your Data
      Aligning Problem-Solving Strategies, Data, and AI
   A New Paradigm to Optimize Data Management and Business Strategy for the Age of AI
   The Origin Story of Unifying
   Orchestrating Alignment at Organizational Scale
   Conventions Used in This Book
   O’Reilly Online Learning
   How to Contact Us
   Acknowledgments
1. The Need for a Unifying Data Strategy
   Your Quest for Data-Driven Breakthroughs Begins
      There Are Usually Multiple, Conflicting North Stars
      The Good, the Bad, and the Ugly of Data Problems
      The Problem with Problems
      Unifying Concepts: The Key to Innovation
   What a Unifying Data Strategy Will Do for Agile
      Defining Being Agile
      Agile Theater
      Agile, Waterfall, and Unifying
      Defining a Unifying Data Strategy Approach
   Understanding the Phrase Being Data Driven
      To Be Data Driven, Be Data Centric
      Bottlenecks Preventing Teams from Being Data Driven
   This Book’s Project: Intelligence.AI Coffee Beans
   Summary
2. The Lingua Franca of Data: JSON
   Introducing JSON
      A Simple JSON Example
      JSON Viewing and Authoring Tools
         JSON Hero
         OK JSON
         Visual Studio Code
   Overview of JSON Grammar
      Booleans
      Numbers
      Strings
      Arrays
      Objects
      Null
      Learning More
      Minification
      Alternative Representations
         Textual alternatives
         Binary alternatives
         JSON versus the JSON data model
   Creating a JSON Document
      A Product Entry
      A Store Order
   Summary
3. Data-Centric Innovation: A Guide for Data Champions
   Data Transformations Require Data Champions
   The Rise of the Data Product Manager
   Alignment Is a Journey, Not a Destination
      Evaluating Alignment from a Holistic Perspective
      The Goal Isn’t Alignment, It’s Effective Alignment
      Strategies for Setting Up Teams for Success
   Incorporating a Product Management Mindset
      Defining Data Users’ Needs
      Defining Product Features
      Defining and Measuring Success
   Unifying Versus Aligning
   Summary
4. Concept-First Design for Data Products
   Packaging and Products: An Example Using Coffee
   The Four Facets of a Data Product
   Getting Started with Concept-First Design
   A Blueprint for Unifying
   Mapping the Conceptual Terrain: Assessing Concepts
   Facilitating Assessments of Conceptual Alignment Across Technical and Nontechnical Teams
   Smooth Is Slow, Slow Is Fast
   Summary
5. A Universal Language for Data
   What Is JSON Schema?
      What Is a Schema?
   The Building Blocks of JSON Schema
      Vocabularies and Dialects
      Meta-Schemas: Schemas That Describe Other Schemas
   Understanding JSON Schemas
      Step 1: Determining the Schema Dialect: The $schema Keyword
      Step 2: Determining the Schema Vocabularies
         Inspecting meta-schemas
         The $vocabulary keyword
      Step 3: Understanding Schema Vocabularies
         Using the reference documentation
         Keyword namespacing (or lack thereof)
      Step 4: Understanding Schema Keywords
         A first pass on top-level keywords
         The validation vocabulary
         The applicator vocabulary
   JSON Schema as a Recursive Data Structure
   Referencing Schemas
      What does duplication look like?
      Local referencing
      Remote referencing
   Your First JSON Schema Project
      Writing a Schema: Step by Step
      Generating a Web Form
   Summary
6. The Art of Alignment
   Enemies of Alignment: Ambiguity and Assumptions
      Ambiguity: The Culprit in the Illusion of Communication
      Assumptions: Ambiguity’s Best Friend
   Defining Success: Symmetry Between Concepts and JSON Schema Equals Minimal Ambiguity
   Illuminating Misalignment with a Concept Compass
      Step 1: Harmonizing the What
      Step 2: Harmonizing the Way
      Step 3: Harmonizing the How
      Harmonized Concepts
   Validating Concepts: Belief Scoring and Hypotheticals
      Counterfactuals
      Belief Scoring
   Summary
7. The Science of Synchronization
   An Introduction to Thinking in Networks
      Example of Thinking in Networks: Athletes Versus Artists
      Graphs: The Visual Language of Networks
   Networks of Entities: Knowledge Graphs
      A Simple Knowledge Graph
      Challenges with Knowledge Graphs
      Aligning Knowledge for the 99%
   Fundamentals of CLEAN Data Governance
      Collaboration
      Knowledge
      Business Logic
      Activity
   CLEAN Data Governance in Practice
   The Four Facets of Data Products and CLEAN
   The Four Horsemen of Data Death
      Ignorance
      Siloed Incentives
      Shortsightedness
      Indecisiveness
   The Power of Design in Collaborative Networks
   Summary
8. The Two Fundamental Operations of Schemas
   Validating the Structure of Data
      Using an Online Validator
      Validation Example
      JSON Schema as a Constraints Language
      Boolean Schemas
      Heterogeneous Data Structures
      The format Keyword
   Using Annotations to Define Meaning
      Annotation Extraction Example
      A Simple Use Case: Deprecations
      Runtime Extraction
      Standard Output Formats
      Revisiting the format Keyword
      Using an Online Validator
   Thinking in Schemas
   Summary
9. Illuminating Pathways of Acceleration
   How Ambiguity, Knowledge Gaps, and Blind Spots Influence Decisions and Progress Toward Goals
   Which Is Bigger: Greenland or the US?
   Mapping Pathways of Processes and Progress
      Measuring Progress Toward Goals
      Defining Decisions and Steps with Process Maps
      How Process Maps Reveal Ambiguity
   Visualizing and Removing Ambiguity in Processes
      Enriching Process Maps with Annotations
      Process Maps Reveal Innovation Opportunities
   Summary
10. Spectrums of Success
   An Introduction to Knowledge Frameworks
      Knowledge Experiences and Pathways
      A Tool for Designing Knowledge Experiences
      From Structured Knowledge to Computational Knowledge
   Success Spectrums
      Mapping Progress and Value
      Visualizing and Adding “Next Best States”
      Removing Blind Spots
      Embracing Multiperspective Design and Road Maps
      Defining KPIs for Success Measures and Metrics (Assessments)
      Using Demons and Magical Thinking for Innovation
      Faster Horses
      Imagining Magical Possibilities
      Problem Landscapes: Quantifying Pain Points Threatening Value
   Nudges: The Right Information at the Right Time
   A Real-Life Problem Landscape and Demon Example That Led to a Unified Data Product Model
      Understanding the Problem Landscape
      The Staggering Impact
      A Meeting of Minds and the Birth of a Solution
   Beyond Data Products: Data Product Management
   The Circular Nature of Unifying
   Summary
11. Deploying a JSON Schema Registry
   Schemas Over HTTP
   Step 1: Setting Up a GitHub Repository
      Creating a GitHub Repository
      Uploading Your First Schema
   Step 2: Deploying to Cloudflare Pages
      Creating a New Cloudflare Pages Website Project
   Step 3: Configuring HTTP Headers
      Inspecting the Current HTTP Headers
      Declaring Custom HTTP Headers on Cloudflare Pages
      Checking the Results
   Step 4: Creating a Landing Page
      Adding an HTML Entry Point
   Step 5: Adding a Custom Domain
      Configuring a Custom Domain in Cloudflare Pages
      Setting Up a CNAME DNS Record
      Checking the Results
   Best Practices
      Schemas Are Immutable
      Adopt a Versioning Strategy
   Summary
12. Designing Data Products Using JSON Schema
   First Facet: Data
      An Example CSV Dataset
      A JSON Row Representation
   Second Facet: Structure
      General-Purpose Concepts
         Timestamp
         IP address
         Email
         US state
         Cost and currency
      Application-Specific Concepts
      Dataset Entries
      The Dataset Schema
   Third Facet: Meaning
      Timestamp
      IP Address
      Email
      US State
      Currency
      Price
      Milestone
      Analytics Entry
   Fourth Facet: Context
      The Signup Analytics Schema
   Summary
      Automated Schema Extraction
      Next Steps
13. Extending JSON Schema
   Simple Case: Unknown Keywords
      Extracting Unknown Keywords as Annotations
      Pros and Cons of This Approach
   Complex Case: Authoring Vocabularies
      The JSON Schema Vocabulary System
      Step 1: Writing a Specification
         Vocabulary identifiers
         The context vocabulary specification
      Step 2: Writing a Vocabulary Meta-Schema
         Official vocabularies meta-schemas
         SPDX licenses
         The context vocabulary meta-schema
            Setting schema identifiers
            Configuring schema extension
      Step 3: Extending an Implementation
         Diversity of JSON Schema implementations
         Extending Hyperjump
   Consuming Vocabularies
      Defining a Dialect
      Making Use of the Dialect
      Example: Extracting Annotations with Hyperjump
         Adding the dialect
         Getting annotations
   Summary
14. Introducing JSON Unify
   Introducing the Dataset Vocabulary
      Revisiting the Signup Analytics Example
   JSON Schema Bundling
      The Bundling Process
      Bundling Our Example Data Product
   Referencing Remote Data
      The Problem of Streaming JSON
      Introducing JSON Lines
   Extracting Meaning
      A Simple Example
      Using Logic Operators
      The Signup Analytics Example
   Dataset Lineage
      Filtering
      Transforming
      Aggregation
   Summary
15. Principles of Designing Intelligence
   Your Unifying Journey So Far
   A Constellation of Deeper Principles Guides Unifying
   1. The Principle of Alignment
      Transforming the Abstract to Concrete
      What You See Can Kill You, and the Same Is True in Data
   2. The Principle of Information
      Understanding Uncertainty
   3. The Principle of Learning
      Defining Learning
      Defining Errors
   4. The Principle of Integrated Simplicity
      Complexity Reduction
      Decomposition
      Compression
      Memoization
      Integrating in Communication Networks
   5. The Principle of Continuums
      Making Things Measurable
      The Dangers of Misusing Measurements
      A Continuum Example for a Control Strategy Problem
   6. The Principle of State Transitions
      A Simple State Machine
      Simplifying State Transitions
   7. The Principle of Decidability
      What Is Decidability?
      Two Key Approaches to Problem Solving
      Making Informed Decisions
      Real-World Decidability to Reduce Misalignment in Teams
   8. The Principle of Heuristics
      Awareness and Ethical Considerations
      Connection to Decision Making in Business
   9. The Principle of Mastery
      Levels of Mastery in Knowledge
      Strategies for Mastery
   10. The Principle of Wisdom
   Summary
16. Toward Unified Intelligence
   Functional Artificial Intelligence
      Your AI Is Only as Good as Your Data
      Beware Illusions Within Vetting Processes
      Question Assumptions
   Collective Intelligence
   Collaborative Intelligence
   Unified Intelligence
      Collaborative Learning Networks
      Personalized Knowledge
      Anticipatory Design: Personalization and Digital Twins
   Codifying Principles of Intelligence
      Continuous Human–Machine Learning Loops
      Applying Wisdom in Practice
      Conceptual Zoomability
   Wisdom Graphs: Connecting Concepts, Actions, and Outcomes
      Cognitive Primitives: Standardizing Cognitive Experience Design
   The Value of Unifying
      Prioritize Knowledge Before AI
      A Tale of Simple Knowledge Versus Complex Intelligence
      Follow the Principle of Integrated Simplicity
   Summary
   Going Beyond This Book
Index