Data-Driven Science and Engineering: Machine Learning, Dynamical Systems, and Control

Contents
Preface
Common Optimization Techniques, Equations, Symbols, and Acronyms
Part I Dimensionality Reduction and Transforms
	1 Singular Value Decomposition (SVD)
		1.1 Overview
		1.2 Matrix Approximation
		1.3 Mathematical Properties and Manipulations
		1.4 Pseudo-Inverse, Least-Squares, and Regression
		1.5 Principal Component Analysis (PCA)
		1.6 Eigenfaces Example
		1.7 Truncation and Alignment
		1.8 Randomized Singular Value Decomposition
		1.9 Tensor Decompositions and N-Way Data Arrays
	2 Fourier and Wavelet Transforms
		2.1 Fourier Series and Fourier Transforms
		2.2 Discrete Fourier Transform (DFT) and Fast Fourier Transform (FFT)
		2.3 Transforming Partial Differential Equations
		2.4 Gabor Transform and the Spectrogram
		2.5 Wavelets and Multi-Resolution Analysis
		2.6 2D Transforms and Image Processing
	3 Sparsity and Compressed Sensing
		3.1 Sparsity and Compression
		3.2 Compressed Sensing
		3.3 Compressed Sensing Examples
		3.4 The Geometry of Compression
		3.5 Sparse Regression
		3.6 Sparse Representation
		3.7 Robust Principal Component Analysis (RPCA)
		3.8 Sparse Sensor Placement
Part II Machine Learning and Data Analysis
	4 Regression and Model Selection
		4.1 Classic Curve Fitting
		4.2 Nonlinear Regression and Gradient Descent
		4.3 Regression and Ax = b: Over- and Under-Determined Systems
		4.4 Optimization as the Cornerstone of Regression
		4.5 The Pareto Front and Lex Parsimoniae
		4.6 Model Selection: Cross-Validation
		4.7 Model Selection: Information Criteria
			
	5 Clustering and Classification
		5.1 Feature Selection and Data Mining
		5.2 Supervised versus Unsupervised Learning
		5.3 Unsupervised Learning: k-means Clustering
		5.4 Unsupervised Hierarchical Clustering: Dendrogram
		5.5 Mixture Models and the Expectation-Maximization Algorithm
		5.6 Supervised Learning and Linear Discriminants
		5.7 Support Vector Machines (SVM)
		5.8 Classification Trees and Random Forest
		5.9 Top 10 Algorithms in Data Mining 2008
	6 Neural Networks and Deep Learning
		6.1 Neural Networks: 1-Layer Networks
		6.2 Multi-Layer Networks and Activation Functions
		6.3 The Backpropagation Algorithm
		6.4 The Stochastic Gradient Descent Algorithm
		6.5 Deep Convolutional Neural Networks
		6.6 Neural Networks for Dynamical Systems
		6.7 The Diversity of Neural Networks
Part III Dynamics and Control
	7 Data-Driven Dynamical Systems
		7.1 Overview, Motivations, and Challenges
		7.2 Dynamic Mode Decomposition (DMD)
		7.3 Sparse Identification of Nonlinear Dynamics (SINDy)
		7.4 Koopman Operator Theory
		7.5 Data-Driven Koopman Analysis
	8 Linear Control Theory
		8.1 Closed-Loop Feedback Control
		8.2 Linear Time-Invariant Systems
		8.3 Controllability and Observability
		8.4 Optimal Full-State Control: Linear Quadratic Regulator (LQR)
		8.5 Optimal Full-State Estimation: The Kalman Filter
		8.6 Optimal Sensor-Based Control: Linear Quadratic Gaussian (LQG)
		8.7 Case Study: Inverted Pendulum on a Cart
		8.8 Robust Control and Frequency Domain Techniques
	9 Balanced Models for Control
		9.1 Model Reduction and System Identification
		9.2 Balanced Model Reduction
		9.3 System identification
	10 Data-Driven Control
		10.1 Nonlinear System Identification for Control
		10.2 Machine Learning Control
		10.3 Adaptive Extremum-Seeking Control
Part IV Reduced Order Models
	11 Reduced Order Models (ROMs)
		11.1 POD for Partial Differential Equations
		11.2 Optimal Basis Elements: The POD Expansion
		11.3 POD and Soliton Dynamics
		11.4 Continuous Formulation of POD
		11.5 POD with Symmetries: Rotations and Translations
	12 Interpolation for Parametric ROMs
		12.1 Gappy POD
		12.2 Error and Convergence of Gappy POD
		12.3 Gappy Measurements: Minimize Condition Number
		12.4 Gappy Measurements: Maximal Variance
		12.5 POD and the Discrete Empirical Interpolation Method (DEIM)
		12.6 DEIM Algorithm Implementation
		12.7 Machine Learning ROMs
Glossary
Bibliography
Index