Explanatory Model Analysis: Explore, Explain, and Examine Predictive Models

Cover
Half Title
Title Page
Copyright Page
Dedication
Table of Contents
Part I Introduction
	1 Introduction
		1.1 The aim of the book
		1.2 A bit of philosophy: three laws of model explanation
		1.3 Terminology
		1.4 Black-box models and glass-box models
		1.5 Model-agnostic and model-specific approach
		1.6 The structure of the book
		1.7 What is included in this book and what is not
		1.8 Acknowledgements
	2 Model Development
		2.1 Introduction
		2.2 Model-development process
		2.3 Notation
		2.4 Data understanding
		2.5 Model assembly (fitting)
		2.6 Model audit
	3 Do-it-yourself
		3.1 Do-it-yourself with R
			3.1.1 What to install?
			3.1.2 How to work with DALEX?
			3.1.3 How to work with archivist?
		3.2 Do-it-yourself with Python
			3.2.1 What to install?
			3.2.2 How to work with dalex?
			3.2.3 Code snippets for Python
	4 Datasets and Models
		4.1 Sinking of the RMS Titanic
			4.1.1 Data exploration
		4.2 Models for RMS Titanic, snippets for R
			4.2.1 Logistic regression model
			4.2.2 Random forest model
			4.2.3 Gradient boosting model
			4.2.4 Support vector machine model
			4.2.5 Models’ predictions
			4.2.6 Models’ explainers
			4.2.7 List of model-objects
		4.3 Models for RMS Titanic, snippets for Python
			4.3.1 Logistic regression model
			4.3.2 Random forest model
			4.3.3 Gradient boosting model
			4.3.4 Support vector machine model
			4.3.5 Models’ predictions
			4.3.6 Models’ explainers
		4.4 Apartment prices
			4.4.1 Data exploration
		4.5 Models for apartment prices, snippets for R
			4.5.1 Linear regression model
			4.5.2 Random forest model
			4.5.3 Support vector machine model
			4.5.4 Models’ predictions
			4.5.5 Models’ explainers
			4.5.6 List of model-objects
		4.6 Models for apartment prices, snippets for Python
			4.6.1 Linear regression model
			4.6.2 Random forest model
			4.6.3 Support vector machine model
			4.6.4 Models’ predictions
			4.6.5 Models’ explainers
Part II Instance Level
	5 Introduction to Instance-level Exploration
	6 Break-down Plots for Additive Attributions
		6.1 Introduction
		6.2 Intuition
		6.3 Method
			6.3.1 Break-down for linear models
			6.3.2 Break-down for a general case
		6.4 Example: Titanic data
		6.5 Pros and cons
		6.6 Code snippets for R
			6.6.1 Basic use of the predict_parts() function
			6.6.2 Advanced use of the predict_parts() function
		6.7 Code snippets for Python
	7 Break-down Plots for Interactions
		7.1 Intuition
		7.2 Method
		7.3 Example: Titanic data
		7.4 Pros and cons
		7.5 Code snippets for R
		7.6 Code snippets for Python
	8 Shapley Additive Explanations (SHAP) for Average Attributions
		8.1 Intuition
		8.2 Method
		8.3 Example: Titanic data
		8.4 Pros and cons
		8.5 Code snippets for R
		8.6 Code snippets for Python
	9 Local Interpretable Model-agnostic Explanations (LIME)
		9.1 Introduction
		9.2 Intuition
		9.3 Method
			9.3.1 Interpretable data representation
			9.3.2 Sampling around the instance of interest
			9.3.3 Fitting the glass-box model
		9.4 Example: Titanic data
		9.5 Pros and cons
		9.6 Code snippets for R
			9.6.1 The lime package
			9.6.2 The localModel package
			9.6.3 The iml package
		9.7 Code snippets for Python
	10 Ceteris-paribus Profiles
		10.1 Introduction
		10.2 Intuition
		10.3 Method
		10.4 Example: Titanic data
		10.5 Pros and cons
		10.6 Code snippets for R
			10.6.1 Basic use of the predict_profile() function
			10.6.2 Advanced use of the predict_profile() function
			10.6.3 Comparison of models (champion-challenger)
		10.7 Code snippets for Python
	11 Ceteris-paribus Oscillations
		11.1 Introduction
		11.2 Intuition
		11.3 Method
		11.4 Example: Titanic data
		11.5 Pros and cons
		11.6 Code snippets for R
			11.6.1 Basic use of the predict_parts() function
			11.6.2 Advanced use of the predict_parts() function
		11.7 Code snippets for Python
	12 Local-diagnostics Plots
		12.1 Introduction
		12.2 Intuition
		12.3 Method
			12.3.1 Nearest neighbors
			12.3.2 Local-fidelity plot
			12.3.3 Local-stability plot
		12.4 Example: Titanic
		12.5 Pros and cons
		12.6 Code snippets for R
		12.7 Code snippets for Python
	13 Summary of Instance-level Exploration
		13.1 Introduction
		13.2 Number of explanatory variables in the model
			13.2.1 Low to medium number of explanatory variables
			13.2.2 Medium to a large number of explanatory variables
			13.2.3 Very large number of explanatory variables
		13.3 Correlated explanatory variables
		13.4 Models with interactions
		13.5 Sparse explanations
		13.6 Additional uses of model exploration and explanation
		13.7 Comparison of models (champion-challenger analysis)
Part III Dataset Level
	14 Introduction to Dataset-level Exploration
	15 Model-performance Measures
		15.1 Introduction
		15.2 Intuition
		15.3 Method
			15.3.1 Continuous dependent variable
				15.3.1.1 Goodness-of-fit
				15.3.1.2 Goodness-of-prediction
			15.3.2 Binary dependent variable
				15.3.2.1 Goodness-of-fit
				15.3.2.2 Goodness-of-prediction
			15.3.3 Categorical dependent variable
				15.3.3.1 Goodness-of-fit
				15.3.3.2 Goodness-of-prediction
			15.3.4 Count dependent variable
		15.4 Example
			15.4.1 Apartment prices
			15.4.2 Titanic data
		15.5 Pros and cons
		15.6 Code snippets for R
		15.7 Code snippets for Python
	16 Variable-importance Measures
		16.1 Introduction
		16.2 Intuition
		16.3 Method
		16.4 Example: Titanic data
		16.5 Pros and cons
		16.6 Code snippets for R
		16.7 Code snippets for Python
	17 Partial-dependence Profiles
		17.1 Introduction
		17.2 Intuition
		17.3 Method
			17.3.1 Partial-dependence profiles
			17.3.2 Clustered partial-dependence profiles
			17.3.3 Grouped partial-dependence profiles
			17.3.4 Contrastive partial-dependence profiles
		17.4 Example: apartment-prices data
			17.4.1 Partial-dependence profiles
			17.4.2 Clustered partial-dependence profiles
			17.4.3 Grouped partial-dependence profiles
			17.4.4 Contrastive partial-dependence profiles
		17.5 Pros and cons
		17.6 Code snippets for R
			17.6.1 Partial-dependence profiles
			17.6.2 Clustered partial-dependence profiles
			17.6.3 Grouped partial-dependence profiles
			17.6.4 Contrastive partial-dependence profiles
		17.7 Code snippets for Python
			17.7.1 Grouped partial-dependence profiles
			17.7.2 Contrastive partial-dependence profiles
	18 Local-dependence and Accumulated-local Profiles
		18.1 Introduction
		18.2 Intuition
		18.3 Method
			18.3.1 Local-dependence profile
			18.3.2 Accumulated-local profile
			18.3.3 Dependence profiles for a model with interaction and correlated explanatory variables: an example
		18.4 Example: apartment-prices data
		18.5 Pros and cons
		18.6 Code snippets for R
		18.7 Code snippets for Python
	19 Residual-diagnostics Plots
		19.1 Introduction
		19.2 Intuition
		19.3 Method
		19.4 Example: apartment-prices data
		19.5 Pros and cons
		19.6 Code snippets for R
		19.7 Code snippets for Python
	20 Summary of Dataset-level Exploration
		20.1 Introduction
		20.2 Exploration on training/testing data
		20.3 Correlated explanatory variables
		20.4 Comparison of models (champion-challenger analysis)
Part IV Use-cases
	21 FIFA 19
		21.1 Introduction
		21.2 Data preparation
			21.2.1 Code snippets for R
			21.2.2 Code snippets for Python
		21.3 Data understanding
		21.4 Model assembly
			21.4.1 Code snippets for R
			21.4.2 Code snippets for Python
		21.5 Model audit
			21.5.1 Code snippets for R
			21.5.2 Code snippets for Python
		21.6 Model understanding (dataset-level explanations)
			21.6.1 Code snippets for R
			21.6.2 Code snippets for Python
		21.7 Instance-level explanations
			21.7.1 Robert Lewandowski
			21.7.2 Code snippets for R
			21.7.3 Code snippets for Python
			21.7.4 CR7
			21.7.5 Wojciech Szczęsny
			21.7.6 Lionel Messi
	22 Reproducibility
		22.1 Package versions for R
		22.2 Package versions for Python
Bibliography
Index