Mastering Machine Learning Algorithms Expert techniques for implementing popular machine learning algorithms, fine-tuning your models, and understanding how they work

Copyright
Packt Page
Contributors
Table of Contents
Preface
Chapter 1: Machine Learning Model Fundamentals
	Models and data
		Structure and properties of the datasets
			Limited Sample Populations
			Scaling datasets
			Whitening
			Training, validation, and test sets
			Cross-validation
	Characteristics of a machine learning model
		Learnability
		Capacity of a model
			Vapnik-Chervonenkis capacity
		Bias of an estimator
			Underfitting
		Variance of an estimator
			Overfitting
			The Cramér-Rao bound
	Summary
	Further reading
Chapter 2: Loss Functions and Regularization
	Defining loss and cost functions
		Examples of cost functions
			Mean squared error
			Huber cost function
			Hinge cost function
			Categorical cross-entropy
	Regularization
		Examples of Regularization Techniques
			L2 or Ridge regularization
			L1 or Lasso regularization
			ElasticNet
			Early stopping
	Summary
	Further reading
Chapter 3: Introduction to Semi-Supervised Learning
	Semi-supervised scenario
		Causal scenarios
		Transductive learning
		Inductive learning
		Semi-supervised assumptions
			Smoothness assumption
			Cluster assumption
			Manifold assumption
	Generative Gaussian Mixture
		Generative Gaussian Mixture theory
		Example of a Generative Gaussian Mixture
		Generative Gaussian Mixtures summary
			Weighted log-likelihood
	Self-Training
		Self-Training theory
		Example of Self-Training with the Iris dataset
		Self-Training summary
	Co-Training
		Co-Training theory
		Example of Co-Training with the Wine dataset
		Co-Training summary
	Summary
	Further reading
Chapter 4: Advanced Semi-Supervised Classification
	Contrastive Pessimistic Likelihood Estimation
		CPLE Theory
		Example of contrastive pessimistic likelihood estimation
		CPLE Summary
	Semi-supervised Support Vector Machines (S3VM)
		S3VM Theory
		Example of S3VM
		S3VM Summary
	Transductive Support Vector Machines (TSVM)
		TSVM Theory
		Example of TSVM
			Analysis of different TSVM configurations
		TSVM Summary
	Summary
	Further reading
Chapter 5: Graph-Based Semi-Supervised Learning
	Label propagation
	Example of label propagation
		Label propagation in scikit-learn
	Label spreading
		Example of label spreading
		Increasing the smoothness with Laplacian regularization
	Label propagation based on Markov random walks
		Example of label propagation based on Markov random walks
	Manifold learning
		Isomap
			Example of Isomap
		Locally linear embedding
			Example of LLE
		Laplacian Spectral Embedding
			Example of Laplacian Spectral Embedding
		t-SNE
			Example of t-distributed stochastic neighbor embedding
	Summary
	Further reading
Chapter 6: Clustering and Unsupervised Models
	K-nearest neighbors
		K-d trees
		Ball trees
		Fitting a KNN model
		Example of KNN with scikit-learn
	K-means
		K-means++
		Example of K-means with scikit-learn
	Evaluation metrics
		Homogeneity score
		Completeness score
		Adjusted Rand index
		Silhouette score
	Summary
	Further reading
Chapter 7: Advanced Clustering and Unsupervised Models
	Fuzzy C-means
		Example of Fuzzy C-means with SciKit-Fuzzy
	Spectral clustering
		Example of spectral clustering with scikit-learn
	DBSCAN
		Example of DBSCAN with scikit-learn
			The Calinski-Harabasz score
			The Davies-Bouldin score
		Analysis of DBSCAN results
	Summary
	Further reading
Chapter 8: Clustering and Unsupervised Models for Marketing
	Biclustering
		Example of Spectral Biclustering with Scikit-Learn
	Introduction to Market Basket Analysis with the Apriori Algorithm
		Example of Apriori in Python
	Summary
	Further reading
Chapter 9: Generalized Linear Models and Regression
	GLMs
		Least Squares Estimation
		Bias and Variance of Least Squares Estimators
		Example of Linear regression with Python
		Computing Linear regression Confidence Intervals with Statsmodels
		Increasing the robustness to outliers with Huber loss
	Other regression techniques
		Ridge Regression
			Example of Ridge Regression with scikit-learn
		Risk modeling with Lasso and Logistic Regression
			Example of Risk modeling with Lasso and Logistic Regression
		Polynomial Regression
			Examples of Polynomial Regressions
		Isotonic Regression
			Example of Isotonic Regression
	Summary
	Further reading
Chapter 10: Introduction to Time-Series Analysis
	Time-series
		Smoothing
	Introduction to linear models for time-series
		Autocorrelation
		AR, MA, and ARMA processes
			Modeling non-stationary trend models with ARIMA
	Summary
	Further reading
Chapter 11: Bayesian Networks and Hidden Markov Models
	Conditional probabilities and Bayes\' theorem
		Conjugate priors
	Bayesian networks
		Sampling from a Bayesian network
			Direct sampling
			A gentle introduction to Markov Chains
			Gibbs sampling
			The Metropolis-Hastings algorithm
		Sampling using PyMC3
			Running the Sampling Process
		Sampling using PyStan
	Hidden Markov Models
		The Forward-Backward algorithm
			Forward phase
			Backward phase
			HMM parameter estimation
		The Viterbi algorithm
			Finding the most likely hidden state sequence using the Viterbi algorithm and hmmlearn
	Summary
	Further reading
Chapter 12: The EM Algorithm
	MLE and MAP Learning
	EM Algorithm
		Convex functions and the Jensen\'s inequality
		Application of the Jensen\'s inequality to the EM algorithm
		An example of parameter estimation
	Gaussian Mixture
		Example of Gaussian Mixture with scikit-learn
		Determining the optimal number of components using AIC and BIC
		Automatic component selection using Bayesian Gaussian Mixture
	Summary
	Further reading
Chapter 13: Component Analysis and Dimensionality Reduction
	Factor Analysis
		Linear relation analysis
		Example of Factor Analysis with scikit-learn
	Principal Component Analysis
		Component importance evaluation
		Example of PCA with scikit-learn
		Kernel PCA
		Sparse PCA
	Independent Component Analysis
		Example of FastICA with scikit-learn
	Addendum to Hidden Markov Models
	Summary
	Further reading
Chapter 14: Hebbian Learning
	Hebb\'s rule
		Analysis of the Covariance Rule
			Example of application of the covariance rule
		Weight vector stabilization and Oja\'s rule
	Sanger\'s network
		Example of Sanger\'s network
	Rubner-Tavan\'s network
		Example of Rubner-Tavan\'s Network
	Self-organizing maps
		Kohonen Maps
		Example of SOM
	Summary
	Further reading
Chapter 15: Fundamentals of Ensemble Learning
	Ensemble learning fundamentals
	Random forests
		Random forest fundamentals
		Why use Decision Trees?
		Random forests and the bias-variance trade-off
		Example of random forest with scikit-learn
			Feature importance
	AdaBoost
		AdaBoost.SAMME
		AdaBoost.SAMME.R
		AdaBoost.R2
		Example of AdaBoost with scikit-learn
	Summary
	Further reading
Chapter 16: Advanced Boosting Algorithms
	Gradient boosting
		Loss functions for gradient boosting
		Example of gradient tree boosting with scikit-learn
		Example of gradient boosting with XGBoost
			Evaluating the predictive power of the features
	Ensembles of voting classifiers
		Example of voting classifiers with scikit-learn
	Ensemble learning as model selection
	Summary
	Further reading
Chapter 17: Modeling Neural Networks
	The basic artificial neuron
	The perceptron
		Example of a Perceptron with scikit-learn
	Multilayer Perceptrons (MLPs)
		Activation functions
			Sigmoid and Hyperbolic Tangent
			Rectifier activation functions
			Softmax
	The back-propagation algorithm
		Stochastic gradient descent (SGD)
		Weight initialization
		Example of MLP with TensorFlow and Keras
	Summary
	Further reading
Chapter 18: Optimizing Neural Networks
	Optimization algorithms
		Gradient perturbation
		Momentum and Nesterov momentum
			SGD with Momentum in TensorFlow and Keras
		RMSProp
			RMSProp in TensorFlow and Keras
		Adam
			Adam in TensorFlow and Keras
		AdaGrad
			AdaGrad with TensorFlow and Keras
		AdaDelta
			AdaDelta in TensorFlow and Keras
	Regularization and Dropout
		Regularization
			Regularization in TensorFlow and Keras
		Dropout
			Dropout with TensorFlow and Keras
	Batch normalization
		Example of batch normalization with TensorFlow and Keras
	Summary
	Further reading
Chapter 19: Deep Convolutional Networks
	Deep convolutional networks
	Convolutional operators
		Bidimensional discrete convolutions
			Strides and Padding
		Atrous convolution
		Separable convolution
		Transpose convolution
	Pooling layers
		Other helpful layers
	Example of a deep convolutional network with TensorFlow and Keras
		Example of a deep convolutional network with TensorFlow/Keras and data augmentation
	Summary
	Further reading
Chapter 20: Recurrent Neural Networks
	Recurrent networks
		Backpropagation through time
		Limitations of BPTT
	Long Short-Term Memory (LSTM)
		Gated Recurrent Unit (GRU)
		Example of an LSTM with TensorFlow and Keras
	Transfer learning
	Summary
	Further reading
Chapter 21: Autoencoders
	Autoencoders
		Example of a deep convolutional autoencoder with TensorFlow
	Denoising autoencoders
		Example of a denoising autoencoder with TensorFlow
	Sparse autoencoders
		Adding sparseness to the Fashion MNIST deep convolutional autoencoder
	Variational autoencoders
		Example of a VAE with TensorFlow
	Summary
	Further reading
Chapter 22: Introduction to Generative Adversarial Networks
	Adversarial training
	Deep Convolutional GANs
		Example of DCGAN with TensorFlow
		Mode collapse
	Wasserstein GAN
		Example of WGAN with TensorFlow
	Summary
	Further reading
Chapter 23: Deep Belief Networks
	Introduction to Markov random fields
	Restricted Boltzmann Machines
		Contrastive Divergence
	Deep Belief Networks
		Example of an unsupervised DBN in Python
		Example of a supervised DBN in Python
	Summary
	Further reading
Introduction to Reinforcement Chapter 24: Learning
	Fundamental concepts of RL
		The Markov Decision Process
		Environment
			Rewards
			A checkerboard environment in Python
		Policy
	Policy iteration
		Policy iteration in the checkerboard environment
	Value iteration
		Value iteration in the checkerboard environment
	The TD(0) algorithm
		TD(0) in the checkerboard environment
	Summary
	Further reading
Chapter 25: Advanced Policy Estimation Algorithms
	TD() algorithm
		TD() in a more complex checkerboard environment
		Actor-Critic TD(0) in the checkerboard environment
	SARSA algorithm
		SARSA in the checkerboard environment
	Q-learning
		Q-learning in the checkerboard environment
		Q-learning modeling the policy with a neural network
	Direct policy search through policy gradient
		Example of policy gradient with OpenAI Gym Cartpole
	Summary
	Further reading
Other Books You May Enjoy
Index