Machine Learning A First Course for Engineers and Scientists [draft]

Notation
Introduction
	Machine learning exemplified
	About this book
	Further reading
Supervised learning: a first approach
	Supervised machine learning
	A distance-based method: k-NN
	A rule-based method: Decision trees
	Further reading
Basic parametric models and a statistical perspective on learning
	Linear regression
	Classification and logistic regression
	Polynomial regression and regularization
	Generalized linear models
	Further reading
	Derivation of the normal equations
Understanding, evaluating and improving the performance
	Expected new data error Enew: performance in production
	Estimating Enew
	The training error–generalization gap decomposition of Enew
	The bias-variance decomposition of Enew
	Additional tools for evaluating binary classifiers
	Further reading
Learning parametric models
	Principles of parametric modeling
	Loss functions and likelihood-based models
	Regularization
	Parameter optimization
	Optimization with large datasets
	Hyperparameter optimization
	Further reading
Neural networks and deep learning
	The neural network model
	Training a neural network
	Convolutional neural networks
	Dropout
	Further reading
	Derivation of the backpropagation equations
Ensemble methods: Bagging and boosting
	Bagging
	Random forests
	Boosting and AdaBoost
	Gradient boosting
	Further reading
Nonlinear input transformations and kernels
	Creating features by nonlinear input transformations
	Kernel ridge regression
	Support vector regression
	Kernel theory
	Support vector classification
	Further reading
	The representer theorem
	Derivation of support vector classification
The Bayesian approach and Gaussian processes
	The Bayesian idea
	Bayesian linear regression
	The Gaussian process
	Practical aspects of the Gaussian process
	Other Bayesian methods in machine learning
	Further reading
	The multivariate Gaussian distribution
Generative models and learning from unlabeled data
	The Gaussian mixture model and discriminant analysis
	Cluster analysis
	Deep generative models
	Representation learning and dimensionality reduction
	Further reading
User aspects of machine learning
	Defining the machine learning problem
	Improving a machine learning model
	What if we cannot collect more data?
	Practical data issues
	Can I trust my machine learning model?
	Further reading
Ethics in machine learning
	Fairness and error functions
	Misleading claims about performance
	Limitations of training data
	Further reading
Notation
Bibliography