Linear Algebra. Theory, Intuition, Code

Cover 1
0 Front Matter
	0.1 Front matter
	0.2 Dedication
	0.3 Forward
1 Introduction
	1.1 What is linear algebra and why learn it?
	1.2 About this book
	1.3 Prerequisites
	1.4 Exercises and code challenges
	1.5 Online and other resources
2 Vectors
	2.1 Scalars
	2.2 Vectors: geometry and algebra
	2.3 Transpose operation
	2.4 Vector addition and subtraction
	2.5 Vector-scalar multiplication
	2.6 Exercises
	2.7 Answers
	2.8 Code challenges
	2.9 Code solutions
3 Vector multiplication
	3.1 Vector dot product: Algebra
	3.2 Dot product properties
	3.3 Vector dot product: Geometry
	3.4 Algebra and geometry
	3.5 Linear weighted combination
	3.6 The outer product
	3.7 Hadamard multiplication
	3.8 Cross product
	3.9 Unit vectors
	3.10 Exercises
	3.11 Answers
	3.12 Code challenges
	3.13 Code solutions
4 Vector spaces
	4.1 Dimensions and fields
	4.2 Vector spaces
	4.3 Subspaces and ambient spaces
	4.4 Subsets
	4.5 Span
	4.6 Linear independence
	4.7 Basis
	4.8 Exercises
	4.9 Answers
5 Matrices
	5.1 Interpretations and uses of matrices
	5.2 Matrix terms and notation
	5.3 Matrix dimensionalities
	5.4 The transpose operation
	5.5 Matrix zoology
	5.6 Matrix addition and subtraction
	5.7 Scalar-matrix mult.
	5.8 "Shifting" a matrix
	5.9 Diagonal and trace
	5.10 Exercises
	5.11 Answers
	5.12 Code challenges
	5.13 Code solutions
6 Matrix multiplication
	6.1 "Standard" multiplication
	6.2 Multiplication and eqns.
	6.3 Multiplication with diagonals
	6.4 LIVE EVIL
	6.5 Matrix-vector multiplication
	6.6 Creating symmetric matrices
	6.7 Multiply symmetric matrices
	6.8 Hadamard multiplication
	6.9 Frobenius dot product
	6.10 Matrix norms
	6.11 What about matrix division?
	6.12 Exercises
	6.13 Answers
	6.14 Code challenges
	6.15 Code solutions
7 Rank
	7.1 Six things about matrix rank
	7.2 Interpretations of matrix rank
	7.3 Computing matrix rank
	7.4 Rank and scalar multiplication
	7.5 Rank of added matrices
	7.6 Rank of multiplied matrices
	7.7 Rank of A, AT, ATA, and AAT
	7.8 Rank of random matrices
	7.9 Boosting rank by "shifting"
	7.10 Rank difficulties
	7.11 Rank and span
	7.12 Exercises
	7.13 Answers
	7.14 Code challenges
	7.15 Code solutions
8 Matrix spaces
	8.1 Column space of a matrix
	8.2 Column space: A and AAT
	8.3 Determining whether v ∈ C(A)
	8.4 Row space of a matrix
	8.5 Row spaces of A and ATA
	8.6 Null space of a matrix
	8.7 Geometry of the null space
	8.8 Orthogonal subspaces
	8.9 Matrix space orthogonalities
	8.10 Dimensionalities of matrix spaces
	8.11 More on Ax = b and Ay = 0
	8.12 Exercises
	8.13 Answers
	8.14 Code challenges
	8.15 Code solutions
9 Complex numbers 239
	9.1 Complex numbers and C
	9.2 What are complex numbers?
	9.3 The complex conjugate
	9.4 Complex arithmetic
	9.5 Complex dot product
	9.6 Special complex matrices
	9.7 Exercises
	9.8 Answers
	9.9 Code challenges
	9.10 Code solutions
10 Systems of equations
	10.1 Algebra and geometry of eqns.
	10.2 From systems to matrices
	10.3 Row reduction
	10.4 Gaussian elimination
	10.5 Row-reduced echelon form
	10.6 Gauss-Jordan elimination
	10.7 Possibilities for solutions
	10.8 Matrix spaces, row reduction
	10.9 Exercises
	10.10 Answers
	10.11 Coding challenges
	10.12 Code solutions
11 Determinant
	11.1 Features of determinants
	11.2 Determinant of a 2×2 matrix
	11.3 The characteristic polynomial
	11.4 3×3 matrix determinant
	11.5 The full procedure
	11.6 ∆ of triangles
	11.7 Determinant and row reduction
	11.8 ∆ and scalar multiplication
	11.9 Theory vs practice
	11.10 Exercises
	11.11 Answers
	11.12 Code challenges
	11.13 Code solutions
12 Matrix inverse
	12.1 Concepts and applications
	12.2 Inverse of a diagonal matrix
	12.3 Inverse of a 2×2 matrix
	12.4 The MCA algorithm
	12.5 Inverse via row reduction
	12.6 Left inverse
	12.7 Right inverse
	12.8 The pseudoinverse, part 1
	12.9 Exercises
	12.10 Answers
	12.11 Code challenges
	12.12 Code solutions
13 Projections
	13.1 Projections in R2
	13.2 Projections in RN
	13.3 Orth and par vect comps
	13.4 Orthogonal matrices
	13.5 Orthogonalization via GS
	13.6 QR decomposition
	13.7 Inverse via QR
	13.8 Exercises
	13.9 Answers
	13.10 Code challenges
	13.11 Code solutions
14 Least-squares
	14.1 Introduction
	14.2 5 steps of model-fitting
	14.3 Terminology
	14.4 Least-squares via left inverse
	14.5 Least-squares via projection
	14.6 Least-squares via row-reduction
	14.7 Predictions and residuals
	14.8 Least-squares example
	14.9 Code challenges
	14.10 Code solutions
15 Eigendecomposition
	15.1 Eigenwhatnow?
	15.2 Finding eigenvalues 421
	15.3 Finding eigenvectors
	15.4 Diagonalization
	15.5 Conditions for diagonalization
	15.6 Distinct, repeated eigenvalues
	15.7 Complex solutions
	15.8 Symmetric matrices
	15.9 Eigenvalues singular matrices
	15.10 Eigenlayers of a matrix
	15.11 Matrix powers and inverse
	15.12 Generalized eigendecomposition
	15.13 Exercises
	15.14 Answers
	15.15 Code challenges
	15.16 Code solutions
16 The SVD
	16.1 Singular value decomposition
	16.2 Computing the SVD
	16.3 Singular values and eigenvalues
	16.4 SVD of a symmetric matrix
	16.5 SVD and the four subspaces
	16.6 SVD and matrix rank
	16.7 SVD spectral theory
	16.8 Low-rank approximations
	16.9 Normalizing singular values
	16.10 Condition number of a matrix
	16.11 SVD and the matrix inverse
	16.12 MP Pseudoinverse, part 2
	16.13 Code challenges
	16.14 Code solutions
17 Quadratic form
	17.1 Algebraic perspective
	17.2 Geometric perspective
	17.3 The normalized quadratic form
	17.4 Evecs and the qf surface
	17.5 Matrix definiteness
	17.6 The definiteness of ATA
	17.7 λ and definiteness
	17.8 Code challenges
	17.9 Code solutions
18 Covariance matrices
	18.1 Correlation
	18.2 Variance and standard deviation
	18.3 Covariance
	18.4 Correlation coefficient
	18.5 Covariance matrices
	18.6 Correlation to covariance
	18.7 Code challenges
	18.8 Code solutions
19 PCA
	19.1 PCA: interps and apps
	19.2 How to perform a PCA
	19.3 The algebra of PCA
	19.4 Regularization
	19.5 Is PCA always the best?
	19.6 Code challenges
	19.7 Code solutions
20 The end.
	20.1 The end... of the beginning!
	20.2 Thanks!
Index