Practical Linear Algebra for Data Science

Cover
Copyright
Table of Contents
Preface
	Conventions Used in This Book
	Using Code Examples
	O’Reilly Online Learning
	How to Contact Us
	Acknowledgments
Chapter 1. Introduction
	What Is Linear Algebra and Why Learn It?
	About This Book
	Prerequisites
		Math
		Attitude
		Coding
	Mathematical Proofs Versus Intuition from Coding
	Code, Printed in the Book and Downloadable Online
	Code Exercises
	How to Use This Book (for Teachers and Self Learners)
Chapter 2. Vectors, Part 1
	Creating and Visualizing Vectors in NumPy
		Geometry of Vectors
	Operations on Vectors
		Adding Two Vectors
		Geometry of Vector Addition and Subtraction
		Vector-Scalar Multiplication
		Scalar-Vector Addition
		Transpose
		Vector Broadcasting in Python
	Vector Magnitude and Unit Vectors
	The Vector Dot Product
		The Dot Product Is Distributive
		Geometry of the Dot Product
	Other Vector Multiplications
		Hadamard Multiplication
		Outer Product
		Cross and Triple Products
	Orthogonal Vector Decomposition
	Summary
	Code Exercises
Chapter 3. Vectors, Part 2
	Vector Sets
	Linear Weighted Combination
	Linear Independence
		The Math of Linear Independence
		Independence and the Zeros Vector
	Subspace and Span
	Basis
		Definition of Basis
	Summary
	Code Exercises
Chapter 4. Vector Applications
	Correlation and Cosine Similarity
	Time Series Filtering and Feature Detection
	k-Means Clustering
	Code Exercises
		Correlation Exercises
		Filtering and Feature Detection Exercises
		k-Means Exercises
Chapter 5. Matrices, Part 1
	Creating and Visualizing Matrices in NumPy
		Visualizing, Indexing, and Slicing Matrices
		Special Matrices
	Matrix Math: Addition, Scalar Multiplication, Hadamard Multiplication
		Addition and Subtraction
		“Shifting” a Matrix
		Scalar and Hadamard Multiplications
	Standard Matrix Multiplication
		Rules for Matrix Multiplication Validity
		Matrix Multiplication
		Matrix-Vector Multiplication
	Matrix Operations: Transpose
		Dot and Outer Product Notation
	Matrix Operations: LIVE EVIL (Order of Operations)
	Symmetric Matrices
		Creating Symmetric Matrices from Nonsymmetric Matrices
	Summary
	Code Exercises
Chapter 6. Matrices, Part 2
	Matrix Norms
		Matrix Trace and Frobenius Norm
	Matrix Spaces (Column, Row, Nulls)
		Column Space
		Row Space
		Null Spaces
	Rank
		Ranks of Special Matrices
		Rank of Added and Multiplied Matrices
		Rank of Shifted Matrices
		Theory and Practice
	Rank Applications
		In the Column Space?
		Linear Independence of a Vector Set
	Determinant
		Computing the Determinant
		Determinant with Linear Dependencies
		The Characteristic Polynomial
	Summary
	Code Exercises
Chapter 7. Matrix Applications
	Multivariate Data Covariance Matrices
	Geometric Transformations via Matrix-Vector Multiplication
	Image Feature Detection
	Summary
	Code Exercises
		Covariance and Correlation Matrices Exercises
		Geometric Transformations Exercises
		Image Feature Detection Exercises
Chapter 8. Matrix Inverse
	The Matrix Inverse
	Types of Inverses and Conditions for Invertibility
	Computing the Inverse
		Inverse of a 2 × 2 Matrix
		Inverse of a Diagonal Matrix
		Inverting Any Square Full-Rank Matrix
		One-Sided Inverses
	The Inverse Is Unique
	Moore-Penrose Pseudoinverse
	Numerical Stability of the Inverse
	Geometric Interpretation of the Inverse
	Summary
	Code Exercises
Chapter 9. Orthogonal Matrices and QR Decomposition
	Orthogonal Matrices
	Gram-Schmidt
	QR Decomposition
		Sizes of Q and R
		QR and Inverses
	Summary
	Code Exercises
Chapter 10. Row Reduction and LU Decomposition
	Systems of Equations
		Converting Equations into Matrices
		Working with Matrix Equations
	Row Reduction
		Gaussian Elimination
		Gauss-Jordan Elimination
		Matrix Inverse via Gauss-Jordan Elimination
	LU Decomposition
		Row Swaps via Permutation Matrices
	Summary
	Code Exercises
Chapter 11. General Linear Models and Least Squares
	General Linear Models
		Terminology
		Setting Up a General Linear Model
	Solving GLMs
		Is the Solution Exact?
		A Geometric Perspective on Least Squares
		Why Does Least Squares Work?
	GLM in a Simple Example
	Least Squares via QR
	Summary
	Code Exercises
Chapter 12. Least Squares Applications
	Predicting Bike Rentals Based on Weather
		Regression Table Using statsmodels
		Multicollinearity
		Regularization
	Polynomial Regression
	Grid Search to Find Model Parameters
	Summary
	Code Exercises
		Bike Rental Exercises
		Multicollinearity Exercise
		Regularization Exercise
		Polynomial Regression Exercise
		Grid Search Exercises
Chapter 13. Eigendecomposition
	Interpretations of Eigenvalues and Eigenvectors
		Geometry
		Statistics (Principal Components Analysis)
		Noise Reduction
		Dimension Reduction (Data Compression)
	Finding Eigenvalues
	Finding Eigenvectors
		Sign and Scale Indeterminacy of Eigenvectors
	Diagonalizing a Square Matrix
	The Special Awesomeness of Symmetric Matrices
		Orthogonal Eigenvectors
		Real-Valued Eigenvalues
	Eigendecomposition of Singular Matrices
	Quadratic Form, Definiteness, and Eigenvalues
		The Quadratic Form of a Matrix
		Definiteness
		???? T     ????   Is Positive (Semi)definite
	Generalized Eigendecomposition
	Summary
	Code Exercises
Chapter 14. Singular Value Decomposition
	The Big Picture of the SVD
		Singular Values and Matrix Rank
	SVD in Python
	SVD and Rank-1 “Layers” of a Matrix
	SVD from EIG
		SVD of       ???? T     ????
		Converting Singular Values to Variance, Explained
		Condition Number
	SVD and the MP Pseudoinverse
	Summary
	Code Exercises
Chapter 15. Eigendecomposition and SVD Applications
	PCA Using Eigendecomposition and SVD
		The Math of PCA
		The Steps to Perform a PCA
		PCA via SVD
	Linear Discriminant Analysis
	Low-Rank Approximations via SVD
		SVD for Denoising
	Summary
	Exercises
		PCA
		Linear Discriminant Analyses
		SVD for Low-Rank Approximations
		SVD for Image Denoising
Chapter 16. Python Tutorial
	Why Python, and What Are the Alternatives?
	IDEs (Interactive Development Environments)
	Using Python Locally and Online
		Working with Code Files in Google Colab
	Variables
		Data Types
		Indexing
	Functions
		Methods as Functions
		Writing Your Own Functions
		Libraries
		NumPy
		Indexing and Slicing in NumPy
	Visualization
	Translating Formulas to Code
	Print Formatting and F-Strings
	Control Flow
		Comparators
		If Statements
		For Loops
		Nested Control Statements
	Measuring Computation Time
	Getting Help and Learning More
		What to Do When Things Go Awry
	Summary
Index
About the Author
Colophon