Hands-On Machine Learning with R

FUNDAMENTALS 
Introduction to Machine Learning 
Supervised learning 
Regression problems 
Classification problems 
Unsupervised learning 
Roadmap 
The data sets 
Modeling Process 
Prerequisites 
Data splitting 
Simple random sampling 
Stratified sampling 
Class imbalances 
Creating models in R 
Many formula interfaces 
Many engines
Resampling methods
Contents
k-fold cross validation 
Bootstrapping 
Alternatives 
Bias variance trade-off 
Bias 
Variance 
Hyperparameter tuning 
Model evaluation 
Regression models 
Classification models 
Putting the processes together 
Feature & Target Engineering 
Prerequisites 
Target engineering 
Dealing with missingness 
Visualizing missing values 
Imputation 
Feature filtering 
Numeric feature engineering 
Skewness 
Standardization 
Categorical feature engineering 
Lumping 
One-hot & dummy encoding 
Label encoding 
Alternatives 
Dimension reduction 
Proper implementation 
Sequential steps 
Data leakage 
Putting the process together 
Contents v
SUPERVISED LEARNING 
Linear Regression 
Prerequisites 
Simple linear regression 
Estimation 
Inference 
Multiple linear regression 
Assessing model accuracy 
Model concerns 
Principal component regression 
Partial least squares 
Feature interpretation 
Final thoughts 
Logistic Regression 
Prerequisites 
Why logistic regression 
Simple logistic regression 
Multiple logistic regression 
Assessing model accuracy 
Model concerns 
Feature interpretation 
Final thoughts 
Regularized Regression
Prerequisites 
Why regularize? 
Ridge penalty 
Lasso penalty 
Elastic nets 
Implementation 
vi Contents
Tuning 
Feature interpretation 
Attrition data 
Final thoughts 
Multivariate Adaptive Regression Splines 
Prerequisites 
The basic idea 
Multivariate regression splines 
Fitting a basic MARS model 
Tuning 
Feature interpretation 
Attrition data 
Final thoughts 
K-Nearest Neighbors 
Prerequisites 
Measuring similarity 
Distance measures 
Pre-processing 
Choosing k 
MNIST example 
Final thoughts 
Decision Trees 
Prerequisites 
Structure 
Partitioning 
How deep? 
Early stopping 
Pruning 
Ames housing example 
Contents vii
Feature interpretation 
Final thoughts 
Bagging 
Prerequisites 
Why and when bagging works 
Implementation 
Easily parallelize 
Feature interpretation 
Final thoughts 
Random Forests 
Prerequisites 
Extending bagging 
Out-of-the-box performance 
Hyperparameters 
Number of trees 
mtry 
Tree complexity 
Sampling scheme 
Split rule 
Tuning strategies 
Feature interpretation 
Final thoughts 
Gradient Boosting 
Prerequisites 
How boosting works 
A sequential ensemble approach 
Gradient descent 
Basic GBM 
Hyperparameters 
viii Contents
Implementation 
General tuning strategy 
Stochastic GBMs 
Stochastic hyperparameters 
Implementation 
XGBoost 
XGBoost hyperparameters 
Tuning strategy 
Feature interpretation 
Final thoughts 
Deep Learning 
Prerequisites 
Why deep learning 
Feedforward DNNs 
Network architecture 
Layers and nodes 
Activation 
Backpropagation 
Model training 
Model tuning 
Model capacity
Batch normalization 
Regularization 
Adjust learning rate 
Grid Search 
Final thoughts 
Contents ix
Support Vector Machines 
Prerequisites 
Optimal separating hyperplanes 
The hard margin classifier 
The soft margin classifier 
The support vector machine 
More than two classes 
Support vector regression 
Job attrition example 
Class weights 
Class probabilities 
Feature interpretation 
Final thoughts 
Stacked Models 
Prerequisites 
The Idea 
Common ensemble methods 
Super learner algorithm 
Available packages 
Stacking existing models 
Stacking a grid search 
Automated machine learning 
Final thoughts 
Interpretable Machine Learning 
Prerequisites 
The idea 
Global interpretation 
Local interpretation 
Model-specific vs. model-agnostic 
x Contents
Permutation-based feature importance 
Concept 
Implementation 
Partial dependence 
Concept 
Implementation 
Alternative uses 
Individual conditional expectation 
Concept 
Implementation 
Feature interactions 
Concept 
Implementation 
Alternatives 
Local interpretable model-agnostic explanations 
Concept 
Implementation 
Tuning 
Alternative uses 
Shapley values 
Concept 
Implementation 
XGBoost and built-in Shapley values 
Localized step-wise procedure 
Concept 
Implementation 
Final thoughts 
DIMENSION REDUCTION 
Contents xi
Principal Components Analysis 
Prerequisites 
The idea 
Finding principal components 
Performing PCA in R 
Selecting the number of principal components 
Eigenvalue criterion 
Proportion of variance explained criterion 
Scree plot criterion 
Final thoughts 
Generalized Low Rank Models 
Prerequisites 
The idea 
Finding the lower ranks 
Alternating minimization 
Loss functions
Regularization 
Selecting k 
Fitting GLRMs in R 
Basic GLRM model 
Tuning to optimize for unseen data 
Final thoughts 
Autoencoders 
Prerequisites 
Undercomplete autoencoders 
Comparing PCA to an autoencoder 
Stacked autoencoders 
Visualizing the reconstruction 
Sparse autoencoders 
xii Contents
Denoising autoencoders 
Anomaly detection 
Final thoughts 
CLUSTERING 
K-means Clustering 
Prerequisites 
Distance measures 
Defining clusters 
k-means algorithm 
Clustering digits 
How many clusters? 
Clustering with mixed data 
Alternative partitioning methods 
Final thoughts 
Hierarchical Clustering 
Prerequisites 
Hierarchical clustering algorithms 
Hierarchical clustering in R 
Agglomerative hierarchical clustering 
Divisive hierarchical clustering 
Determining optimal clusters 
Working with dendrograms 
Final thoughts 
Model-based Clustering 
Prerequisites 
Measuring probability and uncertainty 
Covariance types 
Model selection 
My basket example 
Final thoughts