Interpreting Machine Learning Models

Table of Contents
About the Authors
About the Technical Reviewers
Acknowledgments
Introduction
Chapter 1: The Evolution of Machine Learning
	Defining Machine Learning
	The Evolution of Machine Learning
	Learning a Machine Learning Algorithm
		Piece It Together
		Focus on Specific Algorithm Descriptions
		Design an Algorithm Description Template
		Start Small and Build It Up
	Investigating Machine Learning Algorithm Behavior
		Step 1. Select an Algorithm
		Step 2. Identify a Question
		Step 3. Design the Experiment
		Step 4. Execute the Experiment and Report Results
		Step 5. Repeat
	What Does Machine Learning Model Accuracy Mean?
	Why Model Accuracy Is Not Enough
	Summary
Chapter 2: Introduction to Model Interpretability
	Humans Are Explanation Hungry
	Explanations in Machine Learning
	What Are Black-Box Models?
	What Is Interpretability?
	The Motivation Behind Interpretability
		To Make Better Decisions
		To Eliminate Bias
		To Justify Processes
		To Reproduce Operations
		Displacement Strategy
		To Determine Practical Accuracy
		To Maintain Privacy
		To Understand Security Risks
	The Research Behind Interpretability
	Summary
Chapter 3: Machine Learning Interpretability Taxonomy
	Scope-related Types of Post hoc Model Interpretability
		Global Model Interpretability on a Holistic Level
		Local Model Interpretability
	A Group of Predictions
	Model-related Types of Post hoc Model Interpretability
		Result-related Types of Post hoc Model Interpretability
		Categorizing Common Classes of Explainability Methods
	Summary
Chapter 4: Common Properties of Explanations Generated by Interpretability Methods
	Explanation Defined
	Properties of Explanation Methods
		Template of Expression
		Transparency
		Mobility
		Algorithmic Feasibility
	Properties of Individual Explanations
		Correctness
		Loyalty
		Dependability
		Resoluteness
		Lucidness
		Reliability
		Significance
		Originality
		Representativeness
	Human-Friendly Explanations
		Contrastiveness
		Selectivity
		Social
		Focus on the Abnormal
		Truthful
		Consistent with Prior Beliefs
		General and Probable
	Summary
Chapter 5: Human Factors in Model Interpretability
	Interpretability Roles
		Technical Expertise Builders
		Domain Knowledge Reviewers
		Stakeholders or End Users
	Interpretability Stages
		Ideation and Conceptualization Stage
		Building and Validation Stage
		Deployment, Maintenance, and Use Stage
	Interpretability Goals
		Interpretability for Model Validation and Improvement
		Interpretability for Decision-Making and Knowledge Discovery
		Interpretability to Gain Confidence and Obtain Trust
	Human-Friendly Themes Characterizing Interpretability Work
		Interpretability Is Cooperative
		Interpretability Is Process
		Interpretability Is a Mental Model Comparison
		Interpretability Is Context-Dependent
	Design Opportunities for Interpretability Challenges
		Identifying, Representing, and Integrating Human Expectations
		Communicating and Summarizing Model Behavior
		Scalable and Integrable Interpretability Tools
		Post-Deployment Support
	Summary
Chapter 6: Explainability Facts: A Framework for Systematic Assessment of Explainable Approaches
	Explainability Facts List Dimensions
	Functional Requirements
		F1: Problem Supervision Level
		F2: Problem Type
		F3: Explanation Target
		F4: Explanation Breadth/Scope
		F5: Computational Complexity
		F6: Applicable Model Class
		F7: Relation to the Predictive System
		F8: Compatible Feature Types
		F9: Caveats and Assumptions
	Operational Requirements
		O1: Explanation Family
		O2: Explanatory Medium
		O3: System Interaction
		O4: Explanation Domain
		O5: Data and Model Transparency
		O6: Explanation Audience
		O7: Function of the Explanation
		O8: Causality vs. Actionability
		O9: Trust vs. Performance
	Usability Requirements
		U1: Soundness
		U2: Completeness
		U3: Contextfullness
		U4: Interactiveness
		U5: Actionability
		U6: Novelty
		U7: Complexity
		U8: Personalization
	Safety Requirements
		S1: Information Leakage
		S2: Explanation Misuse
		S3: Explanation Invariance
	Validation Requirements
	Summary
Chapter 7: Interpretable ML and Explainable ML Differences
	Interpretable ML and Explainable ML Basics
		Analyzing the Decision Tree
		Digging Deeper
	Key Issues with Explainable ML
		Trade-offs Between Accuracy and Interpretability
		Beware of the Unfaithful
		Not Enough Detail
	Key Issues with Interpretable ML
		Profits vs. Losses
		Efforts to Construct
		Hidden Patterns
	Explanatory and Predictive Modeling
		Explaining or Predicting: The Key Differences Between Two Choices
	Validation, Model Evaluation, and Model Selection
		Validation
		Model Selection
		Model Use and Reporting Explanatory Models
	Summary
Chapter 8: The Framework of Model Explanations
	Data Sets at a Glance
	Types of Frameworks for Tabular Data
		Feature Importance (FI)
		Predictive Power of Feature Subsets
		Additive Importance Measures
		Removal-based Explanations for Feature Importance
		Feature Removal
		Explaining Different Model Behaviors
		Summarizing Feature Influence
		Rule-based Explanations
		Prototypes
		Counterfactuals
		Explanations for Image Data
		Saliency Maps
	Concept Attribution
		Text Data
		Sentence Highlighting
		Attention-based Methods
	Summary
Chapter 9: Feature Importance Methods: Details and Usage Examples
	Data Set Name
	Abstract
	Sources
	Data Set Information
	Attribute Information
	Random Forest Feature Importance
		Accuracy-based Importance
		Gini-based Importance
	Permutation Feature Importance
		Advantages
		Disadvantages
		Code
	SHAP
		Property 1 (Local Accuracy)
		Property 2 (Missingness)
		Property 3 (Consistency)
	SAGE
		How SHAP and SAGE Are Related
	LIME
	FACET
		Model Inspection
		Model Simulation
		Enhanced Machine Learning Workflow
		Code
		Synergy
		Redundancy
	Partial Dependence Plots (PDP)
		Code
	Individual Conditional Expectation
	DALEX
		Introduction to Instance-level Exploration
		Breakdown Plots for Additive Attributions
		Breakdown Plots for Interactions
		Ceteris Paribus Profiles
		Local Diagnostics Plots
		Implementation Example of DALEX on the Titanic Data Set
		Create a Pipeline Model
		Predict-level Explanations
			predict
			predict_parts
			predict_profile
		Model-level Explanations
			model_performance
			model_parts
			model_profile
	Summary
Chapter 10: Detailing Rule-Based Methods
	MAGIE (Model-Agnostic Global Interpretable Explanations)
	MAGIE Algorithm Approach
		Preprocessing the Input Data
		Generating Instance Level Conditions
		Learning Rules from Conditions
		Postprocessing Rules
		Sorting Rules by Mutual Information
	GLocaLX
		Local to Global Explanation Problem
		Local to Global Hierarchy of Explanation Theories
		Finding Similar Theories
		Code
		Output
	Skope-Rules
		Methodology
		Implementation
	Anchors
		Finding Anchors
		Advantages
		Disadvantages
		Getting an Anchor
	Summary
Chapter 11: Detailing Counterfactual Methods
	Counterfactual Explanations
	Use Case 1: Banking Software
	Use Case 2: Continuous Outcome
	Counterfactual Explanations at a Glance
	Generating Counterfactual Explanations
		Counterfactual Guided by Prototypes
		DiCE
		MOC (Multi-Objective Counterfactuals)
	Comparison Between the Algorithms
	DiCE
		Diversity and Feasibility Constraints
		Proximity
		Sparsity
		Optimization
		Advantages
		Disadvantages
	Summary
Chapter 12: Detailing Image Interpretability Methods
	Image Interpretation Using LIME
		Step 1. Generate Random Perturbations for Input Image
		Step 2. Predict Class for Perturbations
		Step 3. Compute Weights (Importance) For the Perturbations
		Step 4. Fit an Explainable Linear Model Using the Perturbations, Predictions, and Weights
	Image Interpretation Using Pixel Attribution (Saliency Maps)
	Image Interpretation Using Class Activation Maps
		Step 1. Modify the Model
		Step 2. Retrain the Model with CAMLogger Callback
		Step 3. Use CAMLogger to See the Class Activation Map
		Step 4. Draw Conclusions from the CAM
	Image Interpretation Using Gradient-Weighted Class Activation Maps
	Summary
Chapter 13: Explaining Text Classification Models
	Data Preprocessing, Feature Engineering, and Logistic Regression Model on the Data
	Interpreting Text Predictions with LIME
		Interpreting Text Predictions with SHAP
		Explaining Text Models with Sentence Highlighting
	Summary
Chapter 14: The Role of Data in Interpretability
	Summary
Chapter 15: The Eight Pitfalls of Explainability Methods
	Assuming One-Fits-All Interpretability
	Bad Model Generalization
	Unnecessary Use of Complex Models
	Ignoring Feature Dependence
		Interpretation with Extrapolation
		Confusing Linear Correlation with General Dependence
		Misunderstanding Conditional Interpretation
	Misleading Interpretations Due to Feature Interactions
		Misleading Feature Effects Due to Aggregation
		Failing to Separate Main from Interaction Effects
	Ignoring Model and Approximation Uncertainty
	Failure to Scale to High-Dimensional Settings
		Human-Intelligibility of High-Dimensional IML Output
		Computational Effort
	Unjustified Causal Interpretation
	Summary
Conclusion
	Engage Interpretability with a Good Plan
		User Experience and Interpretability Go Hand in Hand
			Wherever Possible, Design the Model to Be Interpretable
			Choose Metrics to Reflect the End Goal and the End Task
			Understand the Trained Model
			Communicate Explanations to Model Users
References
Index