Statistical Modeling and Computation

Preface
Acknowledgments
Contents
Abbreviations and Acronyms
Mathematical Notation
Part I Fundamentals of Probability
	1 Probability Models
		1.1 Random Experiments
		1.2 Sample Space
		1.3 Events
		1.4 Probability
		1.5 Conditional Probability and Independence
			1.5.1 Product Rule
			1.5.2 Law of Total Probability and Bayes\' Rule
			1.5.3 Independence
		1.6 Problems
	2 Random Variables and Probability Distributions
		2.1 Random Variables
		2.2 Probability Distribution
			2.2.1 Discrete Distributions
			2.2.2 Continuous Distributions
		2.3 Expectation
		2.4 Transforms
		2.5 Common Discrete Distributions
			2.5.1 Bernoulli Distribution
			2.5.2 Binomial Distribution
			2.5.3 Geometric Distribution
			2.5.4 Poisson Distribution
		2.6 Common Continuous Distributions
			2.6.1 Uniform Distribution
			2.6.2 Exponential Distribution
			2.6.3 Normal (Gaussian) Distribution
			2.6.4 Gamma and chi2 Distribution
			2.6.5 F Distribution
			2.6.6 Student\'s t Distribution
		2.7 Generating Random Variables
			2.7.1 Generating Uniform Random Variables
			2.7.2 Inverse-Transform Method
			2.7.3 Acceptance–Rejection Method
		2.8 Problems
	3 Joint Distributions
		3.1 Discrete Joint Distributions
			3.1.1 Multinomial Distribution
		3.2 Continuous Joint Distributions
		3.3 Mixed Joint Distributions
		3.4 Expectations for Joint Distributions
		3.5 Functions of Random Variables
			3.5.1 Linear Transformations
			3.5.2 General Transformations
		3.6 Multivariate Normal Distribution
		3.7 Limit Theorems
		3.8 Problems
Part II Statistical Modeling andFrequentist and Bayesian Inference
	4 Common Statistical Models
		4.1 Independent Sampling from a Fixed Distribution
		4.2 Multiple Independent Samples
		4.3 Regression Models
			4.3.1 Simple Linear Regression
			4.3.2 Multiple Linear Regression
			4.3.3 Regression in General
		4.4 Analysis of Variance (ANOVA) Models
			4.4.1 Single-Factor ANOVA
			4.4.2 Two-Factor ANOVA
		4.5 Normal Linear Model
		4.6 Statistical Learning
			4.6.1 Training and Test Loss
			4.6.2 Trade-Offs in Statistical Learning
		4.7 Problems
	5 Statistical Inference
		5.1 Estimation
			5.1.1 Method of Moments
			5.1.2 Least-Squares Estimation
		5.2 Confidence Intervals
			5.2.1 Iid Data: Approximate Confidence Interval for mu
			5.2.2 Normal Data: Confidence Intervals for mu and sig
			5.2.3 Two Normal Samples: Confidence Intervals for mux-muy and sigxdsigy
			5.2.4 Binomial Data: Approximate Confidence Intervals for Proportions
			5.2.5 Confidence Intervals for the Normal Linear Model
		5.3 Hypothesis Testing
			5.3.1 ANOVA for the Normal Linear Model
		5.4 Cross-Validation
		5.5 Sufficiency and Exponential Families
		5.6 Problems
	6 Likelihood
		6.1 Log-Likelihood and Score Functions
		6.2 Fisher Information and Cramér–Rao Inequality
		6.3 Likelihood Methods for Estimation
			6.3.1 Score Intervals
			6.3.2 Properties of the ML Estimator
		6.4 Likelihood Methods in Statistical Tests
		6.5 Newton–Raphson Method
		6.6 Expectation–Maximization (EM) Algorithm
		6.7 Problems
	7 Monte Carlo Sampling
		7.1 Empirical Cdf
		7.2 Density Estimation
		7.3 Resampling and the Bootstrap Method
		7.4 Markov Chain Monte Carlo
		7.5 Metropolis–Hastings Algorithm
		7.6 Gibbs Sampler
		7.7 Problems
	8 Bayesian Inference
		8.1 Hierarchical Bayesian Models
		8.2 Common Bayesian Models
			8.2.1 Normal Model with Unknown mu and sig
			8.2.2 Bayesian Normal Linear Model
			8.2.3 Bayesian Multinomial Model
		8.3 Bayesian Networks
		8.4 Asymptotic Normality of the Posterior Distribution
		8.5 Priors and Conjugacy
		8.6 Bayesian Model Comparison
		8.7 Problems
Part III Advanced Models and Inference
	9 Shrinkage and Regularization
		9.1 James–Stein Estimator
		9.2 Ridge Regression
			9.2.1 Gram Matrix
			9.2.2 Not Penalizing the Constant Feature
		9.3 Lasso Regression
		9.4 False-Discovery Rate
		9.5 Problems
	10 Generalized Linear Models
		10.1 Generalized Linear Models
		10.2 Logit and Probit Models
			10.2.1 Logit Model
			10.2.2 Probit Model
			10.2.3 Latent Variable Representation
		10.3 Poisson Regression
		10.4 Problems
	11 Nonparametric Methods
		11.1 Order Statistics
		11.2 Nonparametric Statistical Tests
			11.2.1 One-Sample Nonparametric Tests
			11.2.2 Two-Sample Nonparametric Tests
		11.3 Gram Matrix and Kernel Functions
		11.4 Regression Splines and Smoothing Splines
		11.5 Gaussian Process Regression
		11.6 Problems
	12 Dependent Data Models
		12.1 Autoregressive and Moving Average Models
			12.1.1 Autoregressive Models
			12.1.2 Moving Average Models
			12.1.3 Autoregressive Moving Average Models
		12.2 Gaussian Models
			12.2.1 Gaussian Graphical Model
			12.2.2 Random Effects
			12.2.3 Gaussian Linear Mixed Models
		12.3 Problems
	13 State Space Models
		13.1 Unobserved Components Model
			13.1.1 Frequentist Inference
			13.1.2 Bayesian Estimation
		13.2 Time-Varying Parameter Model
			13.2.1 Bayesian Estimation
		13.3 Stochastic Volatility Model
			13.3.1 Auxiliary Mixture Sampling Approach
		13.4 Problems
Solutions
Julia Primer
	A.1 Getting Started
	A.2 Variables and Their Types
	A.3 Vectors, Matrices, and Arrays
	A.4 Functions
	A.5 Flow Control
	A.6 Graphics
	A.7 Optimization Routines
	A.8 Handling Sparse Matrices
	A.9 Distributions
	A.10 Input/Output
	A.11 Other Aspects of the Language and Caveats
	A.12 Further Reading and References
Mathematical Supplement
	B.1 Multivariate Differentiation
	B.2 Proof of Theorem 2.6 and Corollary 2.2
	B.3 Proof of Theorem 2.7
	B.4 Proof of Theorem 3.10
	B.5 Proof of Theorem 5.2
References
Index