Fundamentals of High-Dimensional Statistics. With Exercises and R Labs

Preface
Exercises, Labs, and Literature
Notation
Contents
1 Introduction
	Contents
	1.1 Embracing High-Dimensionality
	1.2 Statistical Limitations of Classical Estimators
	1.3 Incorporating Prior Information
	1.4 Regularization for Increasing the Numerical Stability*
	1.5 Outlook
	1.6 Exercises
		1.6.1 Exercises for ▶Sect. 1.1
		1.6.2 Exercises for ▶Sect. 1.2
		1.6.3 Exercises for ▶Sect. 1.3
		1.6.4 Exercises for ▶Sect. 1.4
	1.7 R Lab: Least-Squares vs. Ridge Estimation
		1.7.1 Generating Toy Data
		1.7.2 Implementing the Estimators
		1.7.3 Showing that the Ridge Estimator Approximates a Least-Squares Solution
		1.7.4 Comparing Estimation Errors
		1.7.5 Showing That the Ridge Estimator Is Continuous in the Data
		1.7.6 Showing That the Coordinates of the Ridge Estimator Are Not Necessarily Monotone in the Tuning Parameter
		1.7.7 Comparing Ridge and Least-Squares on Economic Data
		1.7.8 Bonus: Checking the Theoretical Formulae of ▶Sect. 1.4
		1.7.9 Bonus: A Technical Note
	1.8 Notes and References
2 Linear Regression
	Contents
	2.1 Overview
	2.2 Sparsity-Inducing Prior Functions
	2.3 Post-Processing Methods
	2.4 Hölder Inequality*
	2.5 Optimality Conditions*
	2.6 Exercises
		2.6.1 Exercises for ▶Sect. 2.2
		2.6.2 Exercises for ▶Sect. 2.4
		2.6.3 Exercises for ▶Sect. 2.5
	2.7 R Lab: Overfitting
		2.7.1 Generating Data
		2.7.2 Implementing the Estimators
		2.7.3 Computing and Visualizing the Results
		2.7.4 Further Illustrations
	2.8 Notes and References
3 Graphical Models
	Contents
	3.1 Overview
	3.2 Gaussian Graphical Models
	3.3 Maximum Regularized Likelihood Estimation
	3.4 Neighborhood Selection
	3.5 Exercises
		3.5.1 Exercises for ▶Sect. 3.1
		3.5.2 Exercises for ▶Sect. 3.3
		3.5.3 Exercises for ▶Sect. 3.4
	3.6 R Lab: Estimating a Gene–Gene Coactivation Network
		3.6.1 Tests on Synthetic Data
			3.6.1.1 Generating Data
			3.6.1.2 Parameter Estimation via Maximum Likelihood
			3.6.1.3 Parameter Estimation via Neighborhood Selection
		3.6.2 A Low-Dimensional Gene Network
			3.6.2.1 Loading and Pre-processing the Data
			3.6.2.2 Estimating the Inverse Covariance Matrix
			3.6.2.3 Estimating the Graph
		3.6.3 A High-Dimensional Gene Network
			3.6.3.1 Loading and Pre-processing the Data
			3.6.3.2 Estimating the Inverse Covariance Matrix
			3.6.3.3 Visualizing the Inverse Covariance Matrix
	3.7 Notes and References
4 Tuning-Parameter Calibration
	Contents
	4.1 Overview
	4.2 Bounds on the Lasso's Effective Noise
	4.3 Cross-Validation
	4.4 Adaptive Validation
	4.5 Exercises
		4.5.1 Exercises for ▶Sect. 4.2
		4.5.2 Exercises for ▶Sect. 4.3
		4.5.3 Exercises for ▶Sect. 4.4
	4.6 R Lab: Cross-Validation
		4.6.1 Data Generation
		4.6.2 Computing a Set of Estimators
		4.6.3 Implementing a Monte Carlo Cross-Validation
		4.6.4 The Double Role of the Training Sample Size
		4.6.5 The Role of the Number of Splits
	4.7 Notes and References
5 Inference
	Contents
	5.1 One-Step Estimators
	5.2 Confidence Intervals
	5.3 Exercises
		5.3.1 Exercises for ▶Sect. 5.1
		5.3.2 Exercises for ▶Sect. 5.2
	5.4 R Lab: Confidence Intervals in Low and High Dimensions
		5.4.1 Generating Synthetic Data
		5.4.2 Defining Initial Estimators
		5.4.3 Constructing Approximate Inverses
		5.4.4 Testing the Inversion Algorithm
		5.4.5 Bonus: A Scalable Optimization Algorithm
		5.4.6 Debiasing
		5.4.7 Constructing Confidence Intervals
		5.4.8 Validating the Pipeline
		5.4.9 Real Data Analysis
			5.4.9.1 Data Loading
			5.4.9.2 Estimating Confidence Intervals
			5.4.9.3 Plotting the Intervals
			5.4.9.4 Testing Against a Standard Function
	5.5 Notes and References
6 Theory I: Prediction
	Contents
	6.1 Overview
	6.2 Basic Inequalities
	6.3 Prediction Guarantees
	6.4 Prediction Guarantees for Sparse and Weakly Correlated Models
	6.5 Exercises
		6.5.1 Exercises for ▶Sect. 6.2
		6.5.2 Exercises for ▶Sect. 6.3
		6.5.3 Exercises for ▶Sect. 6.4
	6.6 Notes and References
7 Theory II: Estimation and Support Recovery
	Contents
	7.1 Overview
	7.2 Estimation Guarantees in Prior Loss
	7.3 Estimation Guarantees in Dual Loss
		7.3.1 Step 1: Construction of Primal-Dual Pairs
		7.3.2 Step 2: Establishing Conditions for a Successful Construction
		7.3.3 Step 3: Deriving Dual Bounds
	7.4 Support Recovery Guarantees
	7.5 Exercises
		7.5.1 Exercises for ▶Sect. 7.2
		7.5.2 Exercises for ▶Sect. 7.3
	7.6 Notes and References
Supplementary Information
A Solutions
	A.1 Solutions for ▶Chap. 1
		Solutions for ▶Sect. 1.1
		Solutions for ▶Sect. 1.2
		Solutions for ▶Sect. 1.3
		Solutions for ▶Sect. 1.4
	A.2 Solutions for ▶Chap. 2
		Solutions for ▶Sect. 2.2
		Solutions for ▶Sect. 2.4
		Solutions for ▶Sect. 2.5
	A.3 Solutions for ▶Chap. 3
		Solutions for ▶Sect. 3.1
		Solutions for ▶Sect. 3.3
		Solutions for ▶Sect. 3.4
	A.4 Solutions for ▶Chap. 4
		Solutions for ▶Sect. 4.2
		Solutions for ▶Sect. 4.3
	A.5 Solutions for ▶Chap. 5
		Solutions for ▶Sect. 5.1
		Solutions for ▶Sect. 5.2
	A.6 Solutions for ▶Chap. 6
		Solutions for ▶Sect. 6.2
		Solutions for ▶Sect. 6.4
	A.7 Solutions for ▶Chap. 7
		Solutions for ▶Sect. 7.2
		Solutions for ▶Sect. 7.3
B Mathematical Background
	B.1 Analysis
	B.2 Matrix Algebra
		Basic Definitions
		Transpose of Matrices and Symmetric Matrices
		Positive (Semi-)Definite Matrices
		Determinant of Matrices
		Inverse of Matrices
		Trace of Matrices
		Eigenvectors and Eigenvalues of Matrices
		Singular Value Decomposition
		Specific Results for Gaussian Graphical Models
Bibliography
Index