Probability and Statistical Inference: From Basic Principles to Advanced Models (Chapman & Hall/CRC Texts in Statistical Science)

Cover
Half Title
Series Page
Title Page
Copyright Page
Dedication
Contents
Preface
About the authors
1. Introduction
2. Probability
	2.1. Intuitive probability
	2.2. Mathematical probability
		2.2.1. Measure
		2.2.2. Probability measure
	2.3. Methods for counting outcomes
		2.3.1. Permutations and combinations
		2.3.2. Number of combinations and multinomial coefficients
	2.4. Conditional probability and independence
		2.4.1. Conditional probability
		2.4.2. Law of total probability and Bayes’ theorem
		2.4.3. Independence
	2.5. Further exercises
	2.6. Chapter summary
3. Random variables and univariate
distributions
	3.1. Mapping outcomes to real numbers
	3.2. Cumulative distribution functions
	3.3. Discrete and continuous random variables
		3.3.1. Discrete random variables and mass functions
		3.3.2. Continuous random variables and density functions
		3.3.3. Parameters and families of distributions
	3.4. Expectation, variance, and higher moments
		3.4.1. Mean of a random variable
		3.4.2. Expectation operator
		3.4.3. Variance of a random variable
		3.4.4. Inequalities involving expectation
		3.4.5. Moments
	3.5. Generating functions
		3.5.1. Moment-generating functions
		3.5.2. Cumulant-generating functions and cumulants
	3.6. Functions of random variables
		3.6.1. Distribution and mass/density for g(X)
		3.6.2. Monotone functions of random variables
	3.7. Sequences of random variables and convergence
	3.8. A more thorough treatment of random variables
	3.9. Further exercises
	3.10. Chapter summary
4. Multivariate distributions
	4.1. Joint and marginal distributions
	4.2. Joint mass and joint density
		4.2.1. Mass for discrete distributions
		4.2.2. Density for continuous distributions
	4.3. Expectation and joint moments
		4.3.1. Expectation of a function of several variables
		4.3.2. Covariance and correlation
		4.3.3. Joint moments
		4.3.4. Joint moment-generating functions
	4.4. Independent random variables
		4.4.1. Independence for pairs of random variables
		4.4.2. Mutual independence
		4.4.3. Identical distributions
	4.5. Random vectors and random matrices
	4.6. Transformations of continuous random variables
		4.6.1. Bivariate transformations
		4.6.2. Multivariate transformations
	4.7. Sums of random variables
		4.7.1. Sum of two random variables
		4.7.2. Sum of n independent random variables
	4.8. Multivariate normal distribution
		4.8.1. Bivariate case
		4.8.2. n-dimensional multivariate case
	4.9. Further exercises
	4.10. Chapter summary
5. Conditional distributions
	5.1. Discrete conditional distributions
	5.2. Continuous conditional distributions
	5.3. Relationship between joint, marginal, and conditional
	5.4. Conditional expectation and conditional moments
		5.4.1. Conditional expectation
		5.4.2. Conditional moments
		5.4.3. Conditional moment-generating functions
	5.5. Hierarchies and mixtures
	5.6. Random sums
	5.7. Conditioning for random vectors
	5.8. Further exercises
	5.9. Chapter summary
6. Statistical models
	6.1. Modelling terminology, conventions, and assumptions
		6.1.1. Sample, observed sample, and parameters
		6.1.2. Structural and distributional assumptions
	6.2. Independent and identically distributed sequences
		6.2.1. Random sample
		6.2.2. Error sequences
	6.3. Linear models
		6.3.1. Simple linear regression
		6.3.2. Multiple linear regression
		6.3.3. Applications
	6.4. Generalised linear models
		6.4.1. Motivation
		6.4.2. Link function
	6.5. Time-to-event models
		6.5.1. Survival function and hazard function
		6.5.2. Censoring of time-to-event data
		6.5.3. Covariates in time-to-event models
	6.6. Time series models
		6.6.1. Autoregressive models
		6.6.2. Moving-average models
		6.6.3. Autocovariance, autocorrelation, and stationarity
	6.7. Poisson processes
		6.7.1. Stochastic processes and counting processes
		6.7.2. Definitions of the Poisson process
		6.7.3. Thinning and superposition
		6.7.4. Arrival and interarrival times
		6.7.5. Compound Poisson process
		6.7.6. Non-homogeneous Poisson process
	6.8. Markov chains
		6.8.1. Classification of states and chains
		6.8.2. Absorption
		6.8.3. Periodicity
		6.8.4. Limiting distribution
		6.8.5. Recurrence and transience
		6.8.6. Continuous-time Markov chains
	6.9. Further exercises
	6.10. Chapter summary
7. Sample moments and quantiles
	7.1. Sample mean
		7.1.1. Mean and variance of the sample mean
		7.1.2. Central limit theorem
	7.2. Higher-order sample moments
		7.2.1. Sample variance
		7.2.2. Joint sample moments
	7.3. Sample mean and variance for a normal population
	7.4. Sample quantiles and order statistics
		7.4.1. Sample minimum and sample maximum
		7.4.2. Distribution of ith order statistic
	7.5. Further exercises
	7.6. Chapter summary
8. Estimation, testing, and prediction
	8.1. Functions of a sample
		8.1.1. Statistics
		8.1.2. Pivotal functions
	8.2. Point estimation
		8.2.1. Bias, variance, and mean squared error
		8.2.2. Consistency
		8.2.3. The method of moments
		8.2.4. Ordinary least squares
	8.3. Interval estimation
		8.3.1. Coverage probability and length
		8.3.2. Constructing interval estimators using pivotal functions
		8.3.3. Constructing interval estimators using order statistics
		8.3.4. Confidence sets
	8.4. Hypothesis testing
		8.4.1. Statistical hypotheses
		8.4.2. Decision rules
		8.4.3. Types of error and the power function
		8.4.4. Basic ideas in constructing tests
		8.4.5. Conclusions and p-values from tests
	8.5. Prediction
	8.6. Further exercises
	8.7. Chapter summary
9. Likelihood-based inference
	9.1. Likelihood function and log-likelihood function
	9.2. Score and information
	9.3. Maximum-likelihood estimation
		9.3.1. Properties of maximum-likelihood estimates
		9.3.2. Numerical maximisation of likelihood
		9.3.3. EM algorithm
	9.4. Likelihood-ratio test
		9.4.1. Testing in the presence of nuisance parameters
		9.4.2. Properties of the likelihood ratio
		9.4.3. Approximate tests
	9.5. Further exercises
	9.6. Chapter summary
10. Inferential theory
	10.1. Sufficiency
		10.1.1. Sufficient statistics and the sufficiency principle
		10.1.2. Factorisation theorem
		10.1.3. Minimal sufficiency
		10.1.4. Application of sufficiency in point estimation
	10.2. Variance of unbiased estimators
	10.3. Most powerful tests
	10.4. Further exercises
	10.5. Chapter summary
11. Bayesian inference
	11.1. Prior and posterior distributions
	11.2. Choosing a prior
		11.2.1. Constructing reference priors
		11.2.2. Conjugate priors
	11.3. Bayesian estimation
		11.3.1. Point estimators
		11.3.2. Absolute loss
		11.3.3. 0-1 loss
		11.3.4. Interval estimates
	11.4. Hierarchical models and empirical Bayes
		11.4.1. Hierarchical models
		11.4.2. Empirical Bayes
		11.4.3. Predictive inference
	11.5. Further exercises
	11.6. Chapter summary
12. Simulation methods
	12.1. Simulating independent values from a distribution
		12.1.1. Table lookup
		12.1.2. Probability integral
		12.1.3. Box-Muller method
		12.1.4. Accept/reject method
		12.1.5. Composition
		12.1.6. Simulating model structure and the bootstrap
	12.2. Monte Carlo integration
		12.2.1. Averaging over simulated instances
		12.2.2. Univariate vs. multivariate integrals
		12.2.3. Importance sampling
		12.2.4. Antithetic variates
	12.3. Markov chain Monte Carlo
		12.3.1. Discrete Metropolis
		12.3.2. Continuous Metropolis
		12.3.3. Metropolis-Hastings algorithm
		12.3.4. Gibbs sampler
	12.4. Further exercises
	12.5. Chapter summary
A. Proof of Proposition 5.7.2
Index