Machine Learning With Python For Everyone

Cover
Title Page
Copyright Page
Contents
Foreword
Preface
About the Author
Part I: First Steps
	1 Let’s Discuss Learning
		1.1 Welcome
		1.2 Scope, Terminology, Prediction, and Data
			1.2.1 Features
			1.2.2 Target Values and Predictions
		1.3 Putting the Machine in Machine Learning
		1.4 Examples of Learning Systems
			1.4.1 Predicting Categories: Examples of Classifiers
			1.4.2 Predicting Values: Examples of Regressors
		1.5 Evaluating Learning Systems
			1.5.1 Correctness
			1.5.2 Resource Consumption
		1.6 A Process for Building Learning Systems
		1.7 Assumptions and Reality of Learning
		1.8 End-of-Chapter Material
			1.8.1 The Road Ahead
			1.8.2 Notes
	2 Some Technical Background
		2.1 About Our Setup
		2.2 The Need for Mathematical Language
		2.3 Our Software for Tackling Machine Learning
		2.4 Probability
			2.4.1 Primitive Events
			2.4.2 Independence
			2.4.3 Conditional Probability
			2.4.4 Distributions
		2.5 Linear Combinations, Weighted Sums, and Dot Products
			2.5.1 Weighted Average
			2.5.2 Sums of Squares
			2.5.3 Sum of Squared Errors
		2.6 A Geometric View: Points in Space
			2.6.1 Lines
			2.6.2 Beyond Lines
		2.7 Notation and the Plus-One Trick
		2.8 Getting Groovy, Breaking the Straight-Jacket, and Nonlinearity
		2.9 NumPy versus “All the Maths”
			2.9.1 Back to 1D versus 2D
		2.10 Floating-Point Issues
		2.11 EOC 53
			2.11.1 Summary
			2.11.2 Notes
	3 Predicting Categories: Getting Started with Classification
		3.1 Classification Tasks
		3.2 A Simple Classification Dataset
		3.3 Training and Testing: Don’t Teach to the Test
		3.4 Evaluation: Grading the Exam
		3.5 Simple Classifier #1: Nearest Neighbors, Long Distance Relationships, and Assumptions
			3.5.1 Defining Similarity
			3.5.2 The k in k-NN
			3.5.3 Answer Combination
			3.5.4 k-NN, Parameters, and Nonparametric Methods
			3.5.5 Building a k-NN Classification Model
		3.6 Simple Classifier #2: Naive Bayes, Probability, and Broken Promises
		3.7 Simplistic Evaluation of Classifiers
			3.7.1 Learning Performance
			3.7.2 Resource Utilization in Classification
			3.7.3 Stand-Alone Resource Evaluation
		3.8 EOC
			3.8.1 Sophomore Warning: Limitations and Open Issues
			3.8.2 Summary
			3.8.3 Notes
			3.8.4 Exercises
	4 Predicting Numerical Values: Getting Started with Regression
		4.1 A Simple Regression Dataset
		4.2 Nearest-Neighbors Regression and Summary Statistics
			4.2.1 Measures of Center: Median and Mean
			4.2.2 Building a k-NN Regression Model
		4.3 Linear Regression and Errors
			4.3.1 No Flat Earth: Why We Need Slope
			4.3.2 Tilting the Field
			4.3.3 Performing Linear Regression
		4.4 Optimization: Picking the Best Answer
			4.4.1 Random Guess
			4.4.2 Random Step
			4.4.3 Smart Step
			4.4.4 Calculated Shortcuts
			4.4.5 Application to Linear Regression
		4.5 Simple Evaluation and Comparison of Regressors
			4.5.1 Root Mean Squared Error
			4.5.2 Learning Performance
			4.5.3 Resource Utilization in Regression
		4.6 EOC
			4.6.1 Limitations and Open Issues
			4.6.2 Summary
			4.6.3 Notes
			4.6.4 Exercises
Part II: Evaluation
	5 Evaluating and Comparing Learners
		5.1 Evaluation and Why Less Is More
		5.2 Terminology for Learning Phases
			5.2.1 Back to the Machines
			5.2.2 More Technically Speaking . . .
		5.3 Major Tom, There’s Something Wrong: Overfitting and Underfitting
			5.3.1 Synthetic Data and Linear Regression
			5.3.2 Manually Manipulating Model Complexity
			5.3.3 Goldilocks: Visualizing Overfitting, Underfitting, and “Just Right”
			5.3.4 Simplicity
			5.3.5 Take-Home Notes on Overfitting
		5.4 From Errors to Costs
			5.4.1 Loss
			5.4.2 Cost
			5.4.3 Score
		5.5 (Re)Sampling: Making More from Less
			5.5.1 Cross-Validation
			5.5.2 Stratification
			5.5.3 Repeated Train-Test Splits
			5.5.4 A Better Way and Shuffling
			5.5.5 Leave-One-Out Cross-Validation
		5.6 Break-It-Down: Deconstructing Error into Bias and Variance
			5.6.1 Variance of the Data
			5.6.2 Variance of the Model
			5.6.3 Bias of the Model
			5.6.4 All Together Now
			5.6.5 Examples of Bias-Variance Tradeoffs
		5.7 Graphical Evaluation and Comparison
			5.7.1 Learning Curves: How Much Data Do We Need?
			5.7.2 Complexity Curves
		5.8 Comparing Learners with Cross-Validation
		5.9 EOC
			5.9.1 Summary
			5.9.2 Notes
			5.9.3 Exercises
	6 Evaluating Classifiers
		6.1 Baseline Classifiers
		6.2 Beyond Accuracy: Metrics for Classification
			6.2.1 Eliminating Confusion from the Confusion Matrix
			6.2.2 Ways of Being Wrong
			6.2.3 Metrics from the Confusion Matrix
			6.2.4 Coding the Confusion Matrix
			6.2.5 Dealing with Multiple Classes: Multiclass Averaging
			6.2.6 F1
		6.3 ROC Curves
			6.3.1 Patterns in the ROC
			6.3.2 Binary ROC
			6.3.3 AUC: Area-Under-the-(ROC)-Curve
			6.3.4 Multiclass Learners, One-versus-Rest, and ROC
		6.4 Another Take on Multiclass: One-versus-One
			6.4.1 Multiclass AUC Part Two: The Quest for a Single Value
		6.5 Precision-Recall Curves
			6.5.1 A Note on Precision-Recall Tradeoff
			6.5.2 Constructing a Precision-Recall Curve
		6.6 Cumulative Response and Lift Curves
		6.7 More Sophisticated Evaluation of Classifiers: Take Two
			6.7.1 Binary
			6.7.2 A Novel Multiclass Problem
		6.8 EOC
			6.8.1 Summary
			6.8.2 Notes
			6.8.3 Exercises
	7 Evaluating Regressors
		7.1 Baseline Regressors
		7.2 Additional Measures for Regression
			7.2.1 Creating Our Own Evaluation Metric
			7.2.2 Other Built-in Regression Metrics
			7.2.3 R2
		7.3 Residual Plots
			7.3.1 Error Plots
			7.3.2 Residual Plots
		7.4 A First Look at Standardization
		7.5 Evaluating Regressors in a More Sophisticated Way: Take Two
			7.5.1 Cross-Validated Results on Multiple Metrics
			7.5.2 Summarizing Cross-Validated Results
			7.5.3 Residuals
		7.6 EOC
			7.6.1 Summary
			7.6.2 Notes
			7.6.3 Exercises
Part III: More Methods and Fundamentals
	8 More Classification Methods
		8.1 Revisiting Classification
		8.2 Decision Trees
			8.2.1 Tree-Building Algorithms
			8.2.2 Let’s Go: Decision Tree Time
			8.2.3 Bias and Variance in Decision Trees
		8.3 Support Vector Classifiers
			8.3.1 Performing SVC
			8.3.2 Bias and Variance in SVCs
		8.4 Logistic Regression
			8.4.1 Betting Odds
			8.4.2 Probabilities, Odds, and Log-Odds
			8.4.3 Just Do It: Logistic Regression Edition
			8.4.4 A Logistic Regression: A Space Oddity
		8.5 Discriminant Analysis
			8.5.1 Covariance
			8.5.2 The Methods
			8.5.3 Performing DA
		8.6 Assumptions, Biases, and Classifiers
		8.7 Comparison of Classifiers: Take Three
			8.7.1 Digits
		8.8 EOC
			8.8.1 Summary
			8.8.2 Notes
			8.8.3 Exercises
	9 More Regression Methods
		9.1 Linear Regression in the Penalty Box: Regularization
			9.1.1 Performing Regularized Regression
		9.2 Support Vector Regression
			9.2.1 Hinge Loss
			9.2.2 From Linear Regression to Regularized Regression to Support Vector Regression
			9.2.3 Just Do It — SVR Style
		9.3 Piecewise Constant Regression
			9.3.1 Implementing a Piecewise Constant Regressor
			9.3.2 General Notes on Implementing Models
		9.4 Regression Trees
			9.4.1 Performing Regression with Trees
		9.5 Comparison of Regressors: Take Three
		9.6 EOC
			9.6.1 Summary
			9.6.2 Notes
			9.6.3 Exercises
	10 Manual Feature Engineering: Manipulating Data for Fun and Profit
		10.1 Feature Engineering Terminology and Motivation
			10.1.1 Why Engineer Features?
			10.1.2 When Does Engineering Happen?
			10.1.3 How Does Feature Engineering Occur?
		10.2 Feature Selection and Data Reduction: Taking out the Trash
		10.3 Feature Scaling
		10.4 Discretization
		10.5 Categorical Coding
			10.5.1 Another Way to Code and the Curious Case of the Missing Intercept
		10.6 Relationships and Interactions
			10.6.1 Manual Feature Construction
			10.6.2 Interactions
			10.6.3 Adding Features with Transformers
		10.7 Target Manipulations
			10.7.1 Manipulating the Input Space
			10.7.2 Manipulating the Target
		10.8 EOC
			10.8.1 Summary
			10.8.2 Notes
			10.8.3 Exercises
	11 Tuning Hyperparameters and Pipelines
		11.1 Models, Parameters, Hyperparameters
		11.2 Tuning Hyperparameters
			11.2.1 A Note on Computer Science and Learning Terminology
			11.2.2 An Example of Complete Search
			11.2.3 Using Randomness to Search for a Needle in a Haystack
		11.3 Down the Recursive Rabbit Hole: Nested Cross-Validation
			11.3.1 Cross-Validation, Redux
			11.3.2 GridSearch as a Model
			11.3.3 Cross-Validation Nested within Cross-Validation
			11.3.4 Comments on Nested CV
		11.4 Pipelines
			11.4.1 A Simple Pipeline
			11.4.2 A More Complex Pipeline
		11.5 Pipelines and Tuning Together
		11.6 EOC
			11.6.1 Summary
			11.6.2 Notes
			11.6.3 Exercises
Part IV: Adding Complexity
	12 Combining Learners
		12.1 Ensembles
		12.2 Voting Ensembles
		12.3 Bagging and Random Forests
			12.3.1 Bootstrapping
			12.3.2 From Bootstrapping to Bagging
			12.3.3 Through the Random Forest
		12.4 Boosting
			12.4.1 Boosting Details
		12.5 Comparing the Tree-Ensemble Methods
		12.6 EOC
			12.6.1 Summary
			12.6.2 Notes
			12.6.3 Exercises
	13 Models That Engineer Features for Us
		13.1 Feature Selection
			13.1.1 Single-Step Filtering with Metric-Based Feature Selection
			13.1.2 Model-Based Feature Selection
			13.1.3 Integrating Feature Selection with a Learning Pipeline
		13.2 Feature Construction with Kernels
			13.2.1 A Kernel Motivator
			13.2.2 Manual Kernel Methods
			13.2.3 Kernel Methods and Kernel Options
			13.2.4 Kernelized SVCs: SVMs
			13.2.5 Take-Home Notes on SVM and an Example
		13.3 Principal Components Analysis: An Unsupervised Technique
			13.3.1 A Warm Up: Centering
			13.3.2 Finding a Different Best Line
			13.3.3 A First PCA
			13.3.4 Under the Hood of PCA
			13.3.5 A Finale: Comments on General PCA
			13.3.6 Kernel PCA and Manifold Methods
		13.4 EOC
			13.4.1 Summary
			13.4.2 Notes
			13.4.3 Exercises
	14 Feature Engineering for Domains: Domain-Specific Learning
		14.1 Working with Text
			14.1.1 Encoding Text
			14.1.2 Example of Text Learning
		14.2 Clustering
			14.2.1 k-Means Clustering
		14.3 Working with Images
			14.3.1 Bag of Visual Words
			14.3.2 Our Image Data
			14.3.3 An End-to-End System
			14.3.4 Complete Code of BoVW Transformer
		14.4 EOC
			14.4.1 Summary
			14.4.2 Notes
			14.4.3 Exercises
	15 Connections, Extensions, and Further Directions
		15.1 Optimization
		15.2 Linear Regression from Raw Materials
			15.2.1 A Graphical View of Linear Regression
		15.3 Building Logistic Regression from Raw Materials
			15.3.1 Logistic Regression with Zero-One Coding
			15.3.2 Logistic Regression with Plus-One Minus-One Coding
			15.3.3 A Graphical View of Logistic Regression
		15.4 SVM from Raw Materials
		15.5 Neural Networks
			15.5.1 A NN View of Linear Regression
			15.5.2 A NN View of Logistic Regression
			15.5.3 Beyond Basic Neural Networks
		15.6 Probabilistic Graphical Models
			15.6.1 Sampling
			15.6.2 A PGM View of Linear Regression
			15.6.3 A PGM View of Logistic Regression
		15.7 EOC
			15.7.1 Summary
			15.7.2 Notes
			15.7.3 Exercises
Appendix A: mlwpy.py Listing
Index
	A
	B
	C
	D
	E
	F
	G
	I
	H
	J
	K
	L
	M
	N
	P
	O
	Q
	R
	S
	T
	U
	V
	W
	X
	Y
	Z