Data Science Projects with Python

Cover
FM
Copyright
Table of Contents
Preface
Chapter 1: Data Exploration and Cleaning
	Introduction
	Python and the Anaconda Package Management System
		Indexing and the Slice Operator
		Exercise 1.01: Examining Anaconda and Getting Familiar with Python
	Different Types of Data Science Problems
	Loading the Case Study Data with Jupyter and pandas
		Exercise 1.02: Loading the Case Study Data in a Jupyter Notebook
		Getting Familiar with Data and Performing Data Cleaning
		The Business Problem
		Data Exploration Steps
		Exercise 1.03: Verifying Basic Data Integrity
		Boolean Masks
		Exercise 1.04: Continuing Verification of Data Integrity
		Exercise 1.05: Exploring and Cleaning the Data
	Data Quality Assurance and Exploration
		Exercise 1.06: Exploring the Credit Limit and Demographic Features
		Deep Dive: Categorical Features
		Exercise 1.07: Implementing OHE for a Categorical Feature
	Exploring the Financial History Features in the Dataset
		Activity 1.01: Exploring the Remaining Financial Features in the Dataset
	Summary
Chapter 2: Introduction to Scikit-Learn and Model Evaluation
	Introduction
	Exploring the Response Variable and Concluding the Initial Exploration
	Introduction to Scikit-Learn
		Generating Synthetic Data
		Data for Linear Regression
		Exercise 2.01: Linear Regression in Scikit-Learn
	Model Performance Metrics for Binary Classification
		Splitting the Data: Training and Test Sets
		Classification Accuracy
		True Positive Rate, False Positive Rate, and Confusion Matrix
		Exercise 2.02: Calculating the True and False Positive and Negative Rates and Confusion Matrix in Python
		Discovering Predicted Probabilities: How Does Logistic Regression Make Predictions?
		Exercise 2.03: Obtaining Predicted Probabilities from a Trained Logistic Regression Model
		The Receiver Operating Characteristic (ROC) Curve
		Precision
		Activity 2.01: Performing Logistic Regression with a New Feature and Creating a Precision-Recall Curve
	Summary
Chapter 3: Details of Logistic Regression and Feature Exploration
	Introduction
	Examining the Relationships Between Features and the Response Variable
		Pearson Correlation
		Mathematics of Linear Correlation
		F-test
		Exercise 3.01: F-test and Univariate Feature Selection
		Finer Points of the F-test: Equivalence to the t-test for Two Classes and Cautions
		Hypotheses and Next Steps
		Exercise 3.02: Visualizing the Relationship Between the Features and Response Variable
	Univariate Feature Selection: What it Does and Doesn't Do
		Understanding Logistic Regression and the Sigmoid Function Using Function Syntax in Python
		Exercise 3.03: Plotting the Sigmoid Function
		Scope of Functions
		Why Is Logistic Regression Considered a Linear Model?
		Exercise 3.04: Examining the Appropriateness of Features for Logistic Regression
		From Logistic Regression Coefficients to Predictions Using Sigmoid
		Exercise 3.05: Linear Decision Boundary of Logistic Regression
		Activity 3.01: Fitting a Logistic Regression Model and Directly Using the Coefficients
	Summary
Chapter 4: The Bias-Variance Trade-Off
	Introduction
	Estimating the Coefficients and Intercepts of Logistic Regression
		Gradient Descent to Find Optimal Parameter Values
		Exercise 4.01: Using Gradient Descent to Minimize a Cost Function
		Assumptions of Logistic Regression
		The Motivation for Regularization: The Bias-Variance Trade-Off
		Exercise 4.02: Generating and Modeling Synthetic Classification Data
		Lasso (L1) and Ridge (L2) Regularization
	Cross-Validation: Choosing the Regularization Parameter
		Exercise 4.03: Reducing Overfitting on the Synthetic Data Classification Problem
		Options for Logistic Regression in Scikit-Learn
		Scaling Data, Pipelines, and Interaction Features in Scikit-Learn
		Activity 4.01: Cross-Validation and Feature Engineering with the Case Study Data
	Summary
Chapter 5: Decision Trees and Random Forests
	Introduction
	Decision Trees
		The Terminology of Decision Trees and Connections to Machine Learning
		Exercise 5.01: A Decision Tree in Scikit-Learn
		Training Decision Trees: Node Impurity
		Features Used for the First Splits: Connections to Univariate Feature Selection and Interactions
		Training Decision Trees: A Greedy Algorithm
		Training Decision Trees: Different Stopping Criteria and Other Options
		Using Decision Trees: Advantages and Predicted Probabilities
		A More Convenient Approach to Cross-Validation
		Exercise 5.02: Finding Optimal Hyperparameters for a Decision Tree
	Random Forests: Ensembles of Decision Trees
		Random Forest: Predictions and Interpretability
		Exercise 5.03: Fitting a Random Forest
		Checkerboard Graph
		Activity 5.01: Cross-Validation Grid Search with Random Forest
	Summary
Chapter 6: Gradient Boosting, XGBoost, and SHAP Values
	Introduction
	Gradient Boosting and XGBoost
		What Is Boosting?
		Gradient Boosting and XGBoost
	XGBoost Hyperparameters
		Early Stopping
		Tuning the Learning Rate
		Other Important Hyperparameters in XGBoost
		Exercise 6.01: Randomized Grid Search for Tuning XGBoost Hyperparameters
	Another Way of Growing Trees: XGBoost's grow_policy
	Explaining Model Predictions with SHAP Values
		Exercise 6.02: Plotting SHAP Interactions, Feature Importance, and Reconstructing Predicted Probabilities from SHAP Values
	Missing Data
		Saving Python Variables to a File
		Activity 6.01: Modeling the Case Study Data with XGBoost and Explaining the Model with SHAP
	Summary
Chapter 7: Test Set Analysis, Financial Insights, and Delivery to the Client
	Introduction
	Review of Modeling Results
		Feature Engineering
		Ensembling Multiple Models
		Different Modeling Techniques
		Balancing Classes
	Model Performance on the Test Set
		Distribution of Predicted Probability and Decile Chart
		Exercise 7.01: Equal-Interval Chart
		Calibration of Predicted Probabilities
	Financial Analysis
		Financial Conversation with the Client
		Exercise 7.02: Characterizing Costs and Savings
		Activity 7.01: Deriving Financial Insights
	Final Thoughts on Delivering a Predictive Model to the Client
		Model Monitoring
		Ethics in Predictive Modeling
	Summary
Appendix
Index