Linear Algebra and Matrix Analysis for Statistics

Content: Matrices, Vectors, and Their Operations Basic definitions and notations  Matrix addition and scalar-matrix multiplication  Matrix multiplication  Partitioned matrices The "trace" of a square matrix  Some special matrices        Systems of Linear Equations Introduction  Gaussian elimination  Gauss-Jordan elimination  Elementary matrices  Homogeneous linear systems  The inverse of a matrix        More on Linear Equations The LU decomposition Crout's Algorithm  LU decomposition with row interchanges  The LDU and Cholesky factorizations  Inverse of partitioned matrices  The LDU decomposition for partitioned matrices The Sherman-Woodbury-Morrison formula        Euclidean Spaces Introduction  Vector addition and scalar multiplication  Linear spaces and subspaces  Intersection and sum of subspaces  Linear combinations and spans  Four fundamental subspaces  Linear independence  Basis and dimension        The Rank of a Matrix Rank and nullity of a matrix  Bases for the four fundamental subspaces  Rank and inverse  Rank factorization  The rank-normal form  Rank of a partitioned matrix  Bases for the fundamental subspaces using the rank normal form        Complementary Subspaces Sum of subspaces  The dimension of the sum of subspaces Direct sums and complements  Projectors        Orthogonality, Orthogonal Subspaces, and Projections Inner product, norms, and orthogonality  Row rank = column rank: A proof using orthogonality  Orthogonal projections  Gram-Schmidt orthogonalization  Orthocomplementary subspaces  The fundamental theorem of linear algebra        More on Orthogonality Orthogonal matrices  The QR decomposition  Orthogonal projection and projector  Orthogonal projector: Alternative derivations  Sum of orthogonal projectors Orthogonal triangularization        Revisiting Linear Equations Introduction Null spaces and the general solution of linear systems  Rank and linear systems Generalized inverse of a matrix  Generalized inverses and linear systems  The Moore-Penrose inverse        Determinants Definitions  Some basic properties of determinants  Determinant of products  Computing determinants  The determinant of the transpose of a matrix - revisited  Determinants of partitioned matrices  Cofactors and expansion theorems  The minor and the rank of a matrix  The Cauchy-Binet formula  The Laplace expansion        Eigenvalues and Eigenvectors Characteristic polynomial and its roots  Spectral decomposition of real symmetric matrices Spectral decomposition of Hermitian and normal matrices  Further results on eigenvalues  Singular value decomposition    Singular Value and Jordan Decompositions  Singular value decomposition (SVD) The SVD and the four fundamental subspaces  SVD and linear systems  SVD, data compression and principal components  Computing the SVD  The Jordan canonical form  Implications of the Jordan canonical form        Quadratic Forms Introduction Quadratic forms  Matrices in quadratic forms  Positive and nonnegative definite matrices  Congruence and Sylvester's law of inertia Nonnegative definite matrices and minors Extrema of quadratic forms  Simultaneous diagonalization    The Kronecker Product and Related Operations  Bilinear interpolation and the Kronecker product  Basic properties of Kronecker products  Inverses, rank and nonsingularity of Kronecker products  Matrix factorizations for Kronecker products  Eigenvalues and determinant  The vec and commutator operators  Linear systems involving Kronecker products  Sylvester's equation and the Kronecker sum  The Hadamard product         Linear Iterative Systems, Norms, and Convergence  Linear iterative systems and convergence of matrix powers  Vector norms  Spectral radius and matrix convergence  Matrix norms and the Gerschgorin circles  SVD - revisited  Web page ranking and Markov chains  Iterative algorithms for solving linear equations         Abstract Linear Algebra  General vector spaces  General inner products Linear transformations, adjoint and rank The four fundamental subspaces - revisited  Inverses of linear transformations  Linear transformations and matrices  Change of bases, equivalence and similar matrices  Hilbert spaces             References        Exercises appear at the end of each chapter