Linear Algebra Tools for Data Mining

Contents
Preface
About the Author
1. Preliminaries
	1.1 Introduction
	1.2 Functions
	1.3 Sequences
	1.4 Permutations
	1.5 Combinatorics
	1.6 Groups, Rings, and Fields
	1.7 Closure and Interior Systems
	Bibliographical Comments
2. Linear Spaces
	2.1 Introduction
	2.2 Linear Spaces
	2.3 Linear Independence
	2.4 Linear Mappings
	2.5 Bases in Linear Spaces
	2.6 Isomorphisms of Linear Spaces
	2.7 Constructing Linear Spaces
	2.8 Dual Linear Spaces
	2.9 Topological Linear Spaces
	2.10 Isomorphism Theorems
	2.11 Multilinear Functions
	Exercises and Supplements
	Bibliographical Comments
3. Matrices
	3.1 Introduction
	3.2 Matrices with Arbitrary Elements
	3.3 Fields and Matrices
	3.4 Invertible Matrices
	3.5 Special Classes of Matrices
	3.6 Partitioned Matrices and Matrix Operations
	3.7 Change of Bases
	3.8 Matrices and Bilinear Forms
	3.9 Generalized Inverses of Matrices
	3.10 Matrices and Linear Transformations
	3.11 The Notion of Rank
	3.12 Matrix Similarity and Congruence
	3.13 Linear Systems and LU Decompositions
	3.14 The Row Echelon Form of Matrices
	3.15 The Kronecker and Other Matrix Products
	3.16 Outer Products
	3.17 Associative Algebras
	Exercises and Supplements
	Bibliographical Comments
4. MATLAB Environment
	4.1 Introduction
	4.2 The Interactive Environment of MATLAB
	4.3 Number Representation and Arithmetic Computations
	4.4 Matrices and Multidimensional Arrays
	4.5 Cell Arrays
	4.6 Solving Linear Systems
	4.7 Control Structures
	4.8 Indexing
	4.9 Functions
	4.10 Matrix Computations
	4.11 Matrices and Images in MATLAB
	Exercises and Supplements
	Bibliographical Comments
5. Determinants
	5.1 Introduction
	5.2 Determinants and Multilinear Forms
	5.3 Cramer’s Formula
	5.4 Partitioned Matrices and Determinants
	5.5 Resultants
	5.6 MATLAB Computations
	Exercises and Supplements
	Bibliographical Comments
6. Norms and Inner Products
	6.1 Introduction
	6.2 Basic Inequalities
	6.3 Metric Spaces
	6.4 Norms
	6.5 The Topology of Normed Linear Spaces
	6.6 Norms for Matrices
	6.7 Matrix Sequences and Matrix Series
	6.8 Conjugate Norms
	6.9 Inner Products
	6.10 Hyperplanes in Rn
	6.11 Unitary and Orthogonal Matrices
	6.12 Projection on Subspaces
	6.13 Positive Definite and Positive Semidefinite Matrices
	6.14 The Gram–Schmidt Orthogonalization Algorithm
	6.15 Change of Bases Revisited
	6.16 The QR Factorization of Matrices
	6.17 Matrix Groups
	6.18 Condition Numbers for Matrices
	6.19 Linear Space Orientation
	6.20 MATLAB Computations
	Exercises and Supplements
	Bibliographical Comments
7. Eigenvalues
	7.1 Introduction
	7.2 Eigenvalues and Eigenvectors
	7.3 The Characteristic Polynomial of a Matrix
	7.4 Spectra of Hermitian Matrices
	7.5 Spectra of Special Matrices
	7.6 Geometry of Eigenvalues
	7.7 Spectra of Kronecker Products and Sums
	7.8 The Power Method for Eigenvalues
	7.9 The QR Iterative Algorithm
	7.10 MATLAB Computations
	Exercises and Supplements
	Bibliographical Comments
8. Similarity and Spectra
	8.1 Introduction
	8.2 Diagonalizable Matrices
	8.3 Matrix Similarity and Spectra
	8.4 The Sylvester Operator
	8.5 Geometric versus Algebraic Multiplicity
	8.6 λ-Matrices
	8.7 The Jordan Canonical Form
	8.8 Matrix Norms and Eigenvalues
	8.9 Matrix Pencils and Generalized Eigenvalues
	8.10 Quadratic Forms and Quadrics
	8.11 Spectra of Positive Matrices
	8.12 Spectra of Positive Semidefinite Matrices
	8.13 MATLAB Computations
	Exercises and Supplements
	Bibliographical Comments
9. Singular Values
	9.1 Introduction
	9.2 Singular Values and Singular Vectors
	9.3 Numerical Rank of Matrices
	9.4 Updating SVDs
	9.5 Polar Form of Matrices
	9.6 CS Decomposition
	9.7 Geometry of Subspaces
	9.8 Spectral Resolution of a Matrix
	9.9 MATLAB Computations
	Exercises and Supplements
	Bibliographical Comments
10. The k-Means Clustering
	10.1 Introduction
	10.2 The k-Means Algorithm and Convexity
	10.3 Relaxation of the k-Means Problem
	10.4 SVD and Clustering
	10.5 Evaluation of Clusterings
	10.6 MATLAB Computations
	Exercises and Supplements
	Bibliographical Comments
11. Data Sample Matrices
	11.1 Introduction
	11.2 The Sample Matrix
	11.3 Biplots
	Exercises and Supplements
	Bibliographical Comments
12. Least Squares Approximations and Data Mining
	12.1 Introduction
	12.2 Linear Regression
	12.3 The Least Square Approximation and QR Decomposition
	12.4 Partial Least Square Regression
	12.5 Locally Linear Embedding
	12.6 MATLAB Computations
	Exercises and Supplements
	Bibliographical Comments
13. Dimensionality Reduction Techniques
	13.1 Introduction
	13.2 Principal Component Analysis
	13.3 Linear Discriminant Analysis
	13.4 Latent Semantic Indexing
	13.5 Recommender Systems and SVD
	13.6 Metric Multidimensional Scaling
	13.7 Procrustes Analysis
	13.8 Non-negative Matrix Factorization
	Exercises and Supplements
	Bibliographical Comments
14. Tensors and Exterior Algebras
	14.1 Introduction
	14.2 The Summation Convention
	14.3 Tensor Products of Linear Spaces
	14.4 Tensors on Inner Product Spaces
	14.5 Contractions
	14.6 Symmetric and Skew-Symmetric Tensors
	14.7 Exterior Algebras
	14.8 Linear Mappings between Spaces SKSV,k
	14.9 Determinants and Exterior Algebra
	Exercises and Supplements
	Bibliographical Comments
15. Multidimensional Array and Tensors
	15.1 Introduction
	15.2 Multidimensional Arrays
	15.3 Outer Products
	15.4 Tensor Rank
	15.5 Matricization and Vectorization
	15.6 Inner Product and Norms
	15.7 Evaluation of a Set of Bilinear Forms
	15.8 Matrix Multiplications and Arrays
	15.9 MATLAB Computations
	15.10 Hyperdeterminants
	15.11 Eigenvalues and Singular Values
	15.12 Decomposition of Tensors
	15.13 Approximation of mdas
	Exercises and Supplements
	Bibliographical Comments
Bibliography
Index