Algebraic Foundations for Applied Topology and Data Analysis

Acknowledgments
Contents
1 Introduction
	1.1 The Statistical Modeling Cycle
	1.2 Preliminaries on Probability Theory
	1.3 Lab: Exploratory Data Analysis
	1.4 Outline of This Book
2 Exponential Dispersion Family
	2.1 Exponential Family
		2.1.1 Definition and Properties
		2.1.2 Single-Parameter Linear EF: Count Variable Examples
			Bernoulli Distribution as a Single-Parameter Linear EF
			Binomial Distribution as a Single-Parameter Linear EF
			Poisson Distribution as a Single-Parameter Linear EF
			Negative-Binomial (Pólya) Distribution as a Single-Parameter Linear EF
		2.1.3 Vector-Valued Parameter EF: Absolutely Continuous Examples
			Gaussian Distribution as a Vector-Valued Parameter EF
			Gamma Distribution as a Vector-Valued Parameter EF
			Inverse Gaussian Distribution as a Vector-Valued Parameter EF
			Generalized Inverse Gaussian Distribution as a Vector-Valued Parameter EF
		2.1.4 Vector-Valued Parameter EF: Count Variable Example
			Categorical Distribution as a Vector-Valued Parameter EF
	2.2 Exponential Dispersion Family
		2.2.1 Definition and Properties
		2.2.2 Exponential Dispersion Family Examples
			Binomial Distribution as a Single-Parameter EDF
			Poisson Distribution as a Single-Parameter EDF
			Gamma Distribution as a Single-Parameter EDF
			Inverse Gaussian Distribution as a Single-Parameter EDF
		2.2.3 Tweedie\'s Distributions
		2.2.4 Steepness of the Cumulant Function
		2.2.5 Lab: Large Claims Modeling
	2.3 Information Geometry in Exponential Families
		2.3.1 Kullback–Leibler Divergence
		2.3.2 Unit Deviance and Bregman Divergence
3 Estimation Theory
	3.1 Introduction to Decision Theory
	3.2 Parameter Estimation
	3.3 Unbiased Estimators
		3.3.1 Cramér–Rao Information Bound
		3.3.2 Information Bound in the Exponential Family Case
			Cramér–Rao Information Bound in the EF Case
			Cramér–Rao Information Bound in the EDF Case
	3.4 Asymptotic Behavior of Estimators
		3.4.1 Consistency
		3.4.2 Asymptotic Normality
4 Predictive Modeling and Forecast Evaluation
	4.1 Generalization Loss
		4.1.1 Mean Squared Error of Prediction
		4.1.2 Unit Deviances and Deviance Generalization Loss
		4.1.3 A Decision-Theoretic Approach to Forecast Evaluation
			Consistency and Proper Scoring Rules
			Forecast Dominance
	4.2 Cross-Validation
		4.2.1 In-Sample and Out-of-Sample Losses
		4.2.2 Cross-Validation Techniques
			Leave-One-Out Cross-Validation
			K-Fold Cross-Validation
			Stratified K-Fold Cross-Validation
		4.2.3 Akaike\'s Information Criterion
	4.3 Bootstrap
		4.3.1 Non-parametric Bootstrap Simulation
		4.3.2 Parametric Bootstrap Simulation
5 Generalized Linear Models
	5.1 Generalized Linear Models and Log-Likelihoods
		5.1.1 Regression Modeling
		5.1.2 Definition of Generalized Linear Models
		5.1.3 Link Functions and Feature Engineering
		5.1.4 Log-Likelihood Function and Maximum Likelihood Estimation
		5.1.5 Balance Property Under the Canonical Link Choice
		5.1.6 Asymptotic Normality
		5.1.7 Maximum Likelihood Estimation and Unit Deviances
	5.2 Actuarial Applications of Generalized Linear Models
		5.2.1 Selection of a Generalized Linear Model
		5.2.2 Feature Engineering
			Categorical Feature Components: Dummy Coding
			Binary Feature Components
			Continuous Feature Components
			Interactions
		5.2.3 Offsets
		5.2.4 Lab: Poisson GLM for Car Insurance Frequencies
			Feature Engineering
			Choice of Learning and Test Samples
			Maximum-Likelihood Estimation and Results
	5.3 Model Validation
		5.3.1 Residuals and Dispersion
		5.3.2 Hypothesis Testing
		5.3.3 Analysis of Variance
		5.3.4 Lab: Poisson GLM for Car Insurance Frequencies, Revisited
			Continuous Coding of Non-monotone Feature Components
			Under-Sampling and Over-Sampling
		5.3.5 Over-Dispersion in Claim Counts Modeling
			Mixed Poisson Distribution
			Negative-Binomial Model
		5.3.6 Zero-Inflated Poisson Model
		5.3.7 Lab: Gamma GLM for Claim Sizes
			Feature Engineering
			Gamma Generalized Linear Model
			Maximum Likelihood Estimation and Model Selection
		5.3.8 Lab: Inverse Gaussian GLM for Claim Sizes
			Infinite Divisibility
			Inverse Gaussian Generalized Linear Model
		5.3.9 Log-Normal Model for Claim Sizes: A Short Discussion
	5.4 Quasi-Likelihoods
	5.5 Double Generalized Linear Model
		5.5.1 The Dispersion Submodel
		5.5.2 Saddlepoint Approximation
		5.5.3 Residual Maximum Likelihood Estimation
		5.5.4 Lab: Double GLM Algorithm for Gamma Claim Sizes
		5.5.5 Tweedie\'s Compound Poisson GLM
	5.6 Diagnostic Tools
		5.6.1 The Hat Matrix
		5.6.2 Case Deletion and Generalized Cross-Validation
	5.7 Generalized Linear Models with Categorical Responses
		5.7.1 Logistic Categorical Generalized Linear Model
		5.7.2 Maximum Likelihood Estimation in Categorical Models
	5.8 Further Topics of Regression Modeling
		5.8.1 Longitudinal Data and Random Effects
		5.8.2 Regression Models Beyond the GLM Framework
			Siblings of Generalized Linear Regression Functions
			Other Distributional Models
		5.8.3 Quantile Regression
			Pinball Loss Function
			Quantile Regression
6 Bayesian Methods, Regularization and Expectation-Maximization
	6.1 Bayesian Parameter Estimation
	6.2 Regularization
		6.2.1 Maximal a Posterior Estimator
		6.2.2 Ridge vs. LASSO Regularization
		6.2.3 Ridge Regression
		6.2.4 LASSO Regularization
			Gaussian Case
			Gradient Descent Algorithm for LASSO Regularization
			Oracle Property
		6.2.5 Group LASSO Regularization
	6.3 Expectation-Maximization Algorithm
		6.3.1 Mixture Distributions
		6.3.2 Incomplete and Complete Log-Likelihoods
		6.3.3 Expectation-Maximization Algorithm for Mixtures
		6.3.4 Lab: Mixture Distribution Applications
	6.4 Truncated and Censored Data
		6.4.1 Lower-Truncation and Right-Censoring
		6.4.2 Parameter Estimation Under Right-Censoring
		6.4.3 Parameter Estimation Under Lower-Truncation
		6.4.4 Composite Models
7 Deep Learning
	7.1 Deep Learning and Representation Learning
	7.2 Generic Feed-Forward Neural Networks
		7.2.1 Construction of Feed-Forward Neural Networks
		7.2.2 Universality Theorems
		7.2.3 Gradient Descent Methods
			Plain Vanilla Gradient Descent Algorithm
			Gradient Calculation via Back-Propagation
			Stochastic Gradient Descent
			Momentum-Based Gradient Descent Methods
			Maximum Likelihood Estimation and Over-fitting
			Regularization Through Early Stopping
	7.3 Feed-Forward Neural Network Examples
		7.3.1 Feature Pre-processing
			Categorical Feature Components: One-Hot Encoding
			Continuous Feature Components
		7.3.2 Lab: Poisson FN Network for Car Insurance Frequencies
	7.4 Special Features in Networks
		7.4.1 Special Purpose Layers
			Embedding Layers for Categorical Feature Components
			Drop-Out Layers and Regularization
			Normalization Layers
		7.4.2 The Balance Property in Neural Networks
			Simple Bias Regularization
			Sophisticated Bias Regularization Under the Canonical Link Choice
			Auto-Calibration for Bias Regularization
		7.4.3 Boosting Regression Models with Network Features
		7.4.4 Network Ensemble Learning
			Stochastic Gradient Descent Fitting of Networks
			Nagging Predictor
			Meta Model
			Ensembling over Selected Networks vs. All Networks
			Analysis of Over-dispersion
		7.4.5 Identifiability in Feed-Forward Neural Networks
	7.5 Auto-encoders
		7.5.1 Standardization of the Data Matrix
		7.5.2 Introduction to Auto-encoders
		7.5.3 Principal Components Analysis
		7.5.4 Lab: Lee–Carter Mortality Model
		7.5.5 Bottleneck Neural Network
	7.6 Model-Agnostic Tools
		7.6.1 Variable Permutation Importance
		7.6.2 Partial Dependence Plots
			Individual Conditional Expectation
			Partial Dependence Plot
			Accumulated Local Effects Profile
		7.6.3 Interaction Strength
		7.6.4 Local Model-Agnostic Methods
		7.6.5 Marginal Attribution by Conditioning on Quantiles
	7.7 Lab: Analysis of the Fitted Networks
8 Recurrent Neural Networks
	8.1 Motivation for Recurrent Neural Networks
	8.2 Plain-Vanilla Recurrent Neural Network
		8.2.1 Recurrent Neural Network Layer
		8.2.2 Deep Recurrent Neural Network Architectures
		8.2.3 Designing the Network Output
		8.2.4 Time-Distributed Layer
	8.3 Special Recurrent Neural Networks
		8.3.1 Long Short-Term Memory Network
		8.3.2 Gated Recurrent Unit Network
	8.4 Lab: Mortality Forecasting with RN Networks
		8.4.1 Lee–Carter Model, Revisited
			Lee–Carter Mortality Model: Random Walk with Drift Extrapolation
			Lee–Carter Mortality Model: LSTM Extrapolation
		8.4.2 Direct LSTM Mortality Forecasting
9 Convolutional Neural Networks
	9.1 Plain-Vanilla Convolutional Neural Network Layer
		9.1.1 Input Tensors and Channels
		9.1.2 Generic Convolutional Neural Network Layer
		9.1.3 Example: Time-Series Analysis and Image Recognition
			Time-Series Analysis with CN Networks
			Image Recognition
	9.2 Special Purpose Tools for Convolutional Neural Networks
		9.2.1 Padding with Zeros
		9.2.2 Stride
		9.2.3 Dilation
		9.2.4 Pooling Layer
		9.2.5 Flatten Layer
	9.3 Convolutional Neural Network Architectures
		9.3.1 Illustrative Example of a CN Network Architecture
		9.3.2 Lab: Telematics Data
		9.3.3 Lab: Mortality Surface Modeling
10 Natural Language Processing
	10.1 Feature Pre-processing and Bag-of-Words
	10.2 Word Embeddings
		10.2.1 Word to Vector Algorithms
			Skip-gram Approach
			Continuous Bag-of-Words
			Negative Sampling
		10.2.2 Global Vectors Algorithm
	10.3 Lab: Predictive Modeling Using Word Embeddings
	10.4 Lab: Deep Word Representation Learning
	10.5 Outlook: Creating Attention
11 Selected Topics in Deep Learning
	11.1 Deep Learning Under Model Uncertainty
		11.1.1 Recap: Tweedie\'s Family
		11.1.2 Lab: Claim Size Modeling Under Model Uncertainty
			Generalized Linear Models
			Deep FN Networks
			Robustified Representation Learning
			Using Forecast Dominance to Deal with Model Uncertainty
			Nagging Predictor
		11.1.3 Lab: Deep Dispersion Modeling
		11.1.4 Pseudo Maximum Likelihood Estimator
	11.2 Deep Quantile Regression
		11.2.1 Deep Quantile Regression: Single Quantile
		11.2.2 Deep Quantile Regression: Multiple Quantiles
		11.2.3 Lab: Deep Quantile Regression
	11.3 Deep Composite Model Regression
		11.3.1 Joint Elicitability of Quantiles and Expected Shortfalls
		11.3.2 Lab: Deep Composite Model Regression
	11.4 Model Uncertainty: A Bootstrap Approach
	11.5 LocalGLMnet: An Interpretable Network Architecture
		11.5.1 Definition of the LocalGLMnet
		11.5.2 Variable Selection in LocalGLMnets
		11.5.3 Lab: LocalGLMnet for Claim Frequency Modeling
		11.5.4 Variable Selection Through Regularization of the LocalGLMnet
		11.5.5 Lab: LASSO Regularization of LocalGLMnet
	11.6 Selected Applications
		11.6.1 Mixture Density Networks
		11.6.2 Estimation of Conditional Expectations
		11.6.3 Bayesian Networks: An Outlook
12 Appendix A: Technical Results on Networks
	12.1 Universality Theorems
	12.2 Consistency and Asymptotic Normality
	12.3 Functional Limit Theorem
	12.4 Hypothesis Testing
13 Appendix B: Data and Examples
	13.1 French Motor Third Party Liability Data
	13.2 Swedish Motorcycle Data
	13.3 Wisconsin Local Government Property Insurance Fund
	13.4 Swiss Accident Insurance Data
Bibliography
Index