Introduction to Probability, Statistics & R: Foundations for Data-Based Sciences

Preface
Contents
Part I Introduction to Basic Statistics and R
	1 Introduction to Basic Statistics
		1.1 What Is Statistics?
			1.1.1 Early and Modern Definitions
			1.1.2 Uncertainty: The Main Obstacle to DecisionMaking
			1.1.3 Statistics Tames Uncertainty
			1.1.4 Place of Statistics Among Other Disciplines
			1.1.5 Lies, Damned Lies and Statistics?
			1.1.6 Example: Harold Shipman Murder Enquiry
			1.1.7 Summary
		1.2 Example Data Sets
		1.3 Basic Statistics
			1.3.1 Measures of Location
				1.3.1.1 Mean, Median and Mode
				1.3.1.2 Which of the Three Measures to Use?
			1.3.2 Measures of Spread
			1.3.3 Boxplot
			1.3.4 Summary
		1.4 Exercises
	2 Getting Started with R
		2.1 What Is R?
			2.1.1 R Basics
			2.1.2 Script Files
			2.1.3 Working Directory in R
			2.1.4 Reading Data into R
			2.1.5 Summary Statistics from R
		2.2 R Data Types
			2.2.1 Vectors and Matrices
			2.2.2 Data Frames and Lists
			2.2.3 Factors and Logical Vectors
		2.3 Plotting in R
			2.3.1 Stem and Leaf Diagrams
			2.3.2 Bar Plot
			2.3.3 Histograms
			2.3.4 Scatter Plot
			2.3.5 Boxplots
		2.4 Appendix: Advanced Materials
			2.4.1 The Functions apply and tapply
			2.4.2 Graphing Using ggplot2
			2.4.3 Writing Functions: Drawing a Butterfly
		2.5 Summary
		2.6 Exercises
Part II Introduction to Probability
	3 Introduction to Probability
		3.1 Two Types of Probabilities: Subjective and Objective
		3.2 Union, Intersection, Mutually Exclusive and Complementary Events
		3.3 Axioms of Probability
		3.4 Exercises
		3.5 Using Combinatorics to Find Probability
			3.5.1 Multiplication Rule of Counting
			3.5.2 The Number of Permutations of k from n: nPk
			3.5.3 The Number of Combinations of k from n: nCk or n ()k
				3.5.3.1 Calculating nCk
			3.5.4 Calculation of Probabilities of Events Under Sampling `at Random'
			3.5.5 A General `urn Problem'
			3.5.6 Section Summary
		3.6 Exercises
	4 Conditional Probability and Independence
		4.1 Definition of Conditional Probability
		4.2 Exercises
		4.3 Multiplication Rule of Conditional Probability
		4.4 Total Probability Formula
		4.5 The Bayes Theorem
		4.6 Example: Monty Hall Problem
		4.7 Exercises
		4.8 Independent Events
			4.8.1 Definition
			4.8.2 Independence of Complementary Events
			4.8.3 Independence of More Than Two Events
			4.8.4 Bernoulli Trials
		4.9 Fun Examples of Independent Events
			4.9.1 System Reliability
				4.9.1.1 Two Components in Series
				4.9.1.2 Two Components in Parallel
				4.9.1.3 A General System
			4.9.2 The Randomised Response Technique
		4.10 Exercises
	5 Random Variables and Their Probability Distributions
		5.1 Definition of a Random Variable
		5.2 Probability Distribution of a Random Variable
			5.2.1 Continuous Random Variable
			5.2.2 Cumulative Distribution Function (cdf)
				5.2.2.1 cdf of a Discrete Random Variable
				5.2.2.2 cdf of a Continuous Random Variable
		5.3 Exercises
		5.4 Expectation and Variance of a Random Variable
			5.4.1 Variance of a Random Variable
		5.5 Quantile of a Random Variable
		5.6 Exercises
	6 Standard Discrete Distributions
		6.1 Bernoulli Distribution
		6.2 Binomial Distribution
			6.2.1 Probability Calculation Using R
			6.2.2 Mean of the Binomial Distribution
			6.2.3 Variance of the Binomial Distribution
		6.3 Geometric Distribution
			6.3.1 Probability Calculation Using R
			6.3.2 Negative Binomial Series
			6.3.3 Mean of the Geometric Distribution
			6.3.4 Variance of the Geometric Distribution
			6.3.5 Memoryless Property of the GeometricDistribution
		6.4 Hypergeometric Distribution
		6.5 Negative Binomial Distribution
			6.5.1 Probability Calculation Using R
			6.5.2 Mean of the Negative Binomial Distribution
			6.5.3 Variance of the Negative Binomial Distribution
		6.6 Poisson Distribution
			6.6.1 Probability Calculation Using R
			6.6.2 Mean of the Poisson Distribution
			6.6.3 Variance of the Poisson Distribution
			6.6.4 Poisson Distribution as a Limit of the Binomial Distribution
		6.7 Exercises
	7 Standard Continuous Distributions
		7.1 Exponential Distribution
			7.1.1 Definition
			7.1.2 Using R to Calculate Probabilities
			7.1.3 Gamma Function
			7.1.4 Mean and Variance of the ExponentialDistribution
			7.1.5 Memoryless Property
			7.1.6 Exercises
		7.2 The Normal Distribution
			7.2.1 The Mean and Variance of the NormalDistribution
			7.2.2 Mode of the Normal Distribution
			7.2.3 Symmetry of the Normal Distribution
			7.2.4 Standard Normal Distribution
			7.2.5 Using R for the Normal Distribution
				7.2.5.1 Probability Calculation Using R
				7.2.5.2 Quantile Calculation Using R
			7.2.6 Log-Normal Distribution
			7.2.7 Exercises
		7.3 Gamma and Beta Distributions
			7.3.1 Gamma Distribution
			7.3.2 Probability Calculation Using R
			7.3.3 Mean and Variance of the Gamma Distribution
			7.3.4 Beta Distribution
			7.3.5 Exercises
	8 Joint Distributions and the CLT
		8.1 Joint Distribution of Discrete Random Variables
		8.2 Continuous Bivariate Distributions
		8.3 Covariance and Correlation
		8.4 Independence
		8.5 Conditional Distribution
			8.5.1 Conditional Expectation
			8.5.2 Conditional Distribution Under Independence
			8.5.3 Some Further Remarks on ConditionalDistribution
		8.6 Exercises
		8.7 Properties of Sum of Independent Random Variables
		8.8 The Central Limit Theorem
			8.8.1 Statement of the Central Limit Theorem (CLT)
			8.8.2 Application of CLT to Binomial Distribution
		8.9 Exercises
Part III Introduction to Statistical Inference
	9 Introduction to Statistical Inference
		9.1 Drawing Random Samples
		9.2 Foundations of Statistical Inference
			9.2.1 Statistical Models
			9.2.2 A Fully Specified Model
			9.2.3 A Parametric Statistical Model
			9.2.4 A Nonparametric Statistical Model
			9.2.5 Modelling Cycle
		9.3 Estimation
			9.3.1 Population and Sample
			9.3.2 Statistic, Estimator and Sampling Distribution
			9.3.3 Bias and Mean Square Error
			9.3.4 Section Summary
		9.4 Estimation of Mean, Variance and Standard Error
			9.4.1 Estimation of a Population Mean
			9.4.2 Standard Deviation and Standard Error
			9.4.3 Section Summary
		9.5 Exercises
	10 Methods of Point Estimation
		10.1 Method of Moments
		10.2 Maximum Likelihood Estimation
			10.2.1 Examples of the Log-Likelihood Function
			10.2.2 Maximum Likelihood Estimates
			10.2.3 Examples of m.l.e
		10.3 Bayesian Estimation and Inference
			10.3.1 Prior and Posterior Distributions
				10.3.1.1 Bayesian Learning
			10.3.2 Bayes Estimators
			10.3.3 Squared Error Loss Function
			10.3.4 Absolute Error Loss Function
			10.3.5 Step Function Loss
			10.3.6 Credible Regions
		10.4 Exercises
	11 Interval Estimation
		11.1 Pivoting Method
		11.2 Interpreting Confidence Intervals
		11.3 Confidence Intervals Using the CLT
			11.3.1 Confidence Intervals for μ Using the CLT
			11.3.2 Confidence Interval for a Bernoulli p by Quadratic Inversion
			11.3.3 Confidence Interval for a Poisson λ by Quadratic Inversion
			11.3.4 Summary
		11.4 Exact Confidence Interval for the Normal Mean
		11.5 Exercises
	12 Hypothesis Testing
		12.1 Testing Procedure
		12.2 The Test Statistic
			12.2.1 The Significance Level
			12.2.2 Rejection Region for the t-Test
			12.2.3 t-Test Summary
			12.2.4 p-Values
		12.3 Power Function, Sensitivity and Specificity
		12.4 Equivalence of Testing and Interval Estimation
		12.5 Two Sample t-Tests
		12.6 Paired t-Test
		12.7 Design of Experiments
			12.7.1 Three Principles of Experimental Design
			12.7.2 Questions of Interest
			12.7.3 Main Effects
			12.7.4 Interaction Between Factors A and B
			12.7.5 Computation of Interaction Effect
			12.7.6 Summary
		12.8 Exercises
Part IV Advanced Distribution Theory and Probability
	13 Generating Functions
		13.1 Moments, Skewnees and Kurtosis
		13.2 Moment Generating Function
			13.2.1 Uniqueness of the Moment GeneratingFunctions
			13.2.2 Using mgf to Prove Distribution of SampleSum
		13.3 Cumulant Generating Function
		13.4 Probability Generating Function
		13.5 Exercises
	14 Transformation and Transformed Distributions
		14.1 Transformation of a Random Variable
			14.1.1 Transformation of Discrete Random Variables
			14.1.2 Transformation of Continuous RandomVariables
		14.2 Exercises
		14.3 Transformation Theorem for Joint Density
		14.4 Exercises
		14.5 Generated Distributions: χ2, t and F
			14.5.1 χ2 Distribution
			14.5.2 t Distribution
			14.5.3 F Distribution
		14.6 Sampling from the Normal Distribution
		14.7 Exercises
	15 Multivariate Distributions
		15.1 An Example of a Bivariate Continuous Distribution
		15.2 Joint cdf
		15.3 Iterated Expectations
		15.4 Exercises
		15.5 Bivariate Normal Distribution
			15.5.1 Derivation of the Joint Density
			15.5.2 Marginal Distributions
			15.5.3 Covariance and Correlation
			15.5.4 Independence
			15.5.5 Conditional Distributions
			15.5.6 Linear Combinations
			15.5.7 Exercises
		15.6 Multivariate Probability Distributions
			15.6.1 Expectation
			15.6.2 Variance and Covariance
			15.6.3 Independence
		15.7 Joint Moment Generating Function
			15.7.1 The Multivariate Normal Distribution
			15.7.2 Conditional Distribution
			15.7.3 Joint mgf of the Multivariate NormalDistribution
		15.8 Multinomial Distribution
			15.8.1 Relation Between the Multinomial and Binomial Distributions
			15.8.2 Means, Variances and Covariances
			15.8.3 Exercises
		15.9 Compound Distributions
	16 Convergence of Estimators
		16.1 Convergence in Distribution
		16.2 Convergence in Probability
		16.3 Consistent Estimator
		16.4 Extending the CLT Using Slutsky's Theorem
		16.5 Score Function
		16.6 Information
		16.7 Asymptotic Distribution of the Maximum LikelihoodEstimators
		16.8 Exercises
Part V Introduction to Statistical Modelling
	17 Simple Linear Regression Model
		17.1 Motivating Example: The Body Fat Data
		17.2 Writing Down the Model
		17.3 Fitting the Model
			17.3.1 Least Squares Estimation
			17.3.2 Estimating Fitted Values
			17.3.3 Defining Residuals
		17.4 Estimating the Variance
		17.5 Quantifying Uncertainty
		17.6 Obtaining Confidence Intervals
		17.7 Performing Hypothesis Testing
		17.8 Comparing Models
		17.9 Analysis of Variance (ANOVA) Decomposition
		17.10 Assessing Correlation and Judging Goodness-of-Fit
		17.11 Using the lm Command
		17.12 Estimating the Conditional Mean
		17.13 Predicting Observations
		17.14 Analysing Residuals
		17.15 Applying Diagnostics Techniques
			17.15.1 The Anscombe Residual Plot
			17.15.2 Normal Probability Plots
			17.15.3 Dependence Plots
			17.15.4 Discrepant Explanatory Variables
			17.15.5 Suggested Remedies
		17.16 Illustration of Diagnostics Plots for the Body Fat DataExample
		17.17 Remphasising the Paradigm of Regression Modelling
		17.18 Summary of Linear Regression Methods
		17.19 Exercises
	18 Multiple Linear Regression Model
		18.1 Motivating Example: Optimising Rice Yields
		18.2 Motivating Example: Cheese Testing
		18.3 Matrix Formulation of the Model
		18.4 Least Squares Estimation
		18.5 Simple Linear Regression: A Special Case of Multiple Regression
		18.6 Inference for the Multiple Regression Model
			18.6.1 Estimating σ2
			18.6.2 Cochran's Theorem
			18.6.3 The Sampling Distribution of S2
			18.6.4 Inference for Regression Coefficients
			18.6.5 Inference for Fitted Values and Residuals
			18.6.6 Prediction
			18.6.7 Model Comparison
		18.7 Illustrative Example: Puffin Nesting
		18.8 Model Selection
			18.8.1 Omitting a Variable Which Is Needed
			18.8.2 Including an Unnecessary Variable
			18.8.3 Backwards Elimination
			18.8.4 Forward Selection
			18.8.5 Criteria Based Model Selection
		18.9  Illustrative Example: Nitrous Oxide Emission
		18.10  Multicollinearity
		18.11  Summary of Multiple Regression Modelling
		18.12  Exercises
	19 Analysis of Variance
		19.1 The Problem
		19.2 Motivating Example: Possum Data
		19.3 Formulating a Multiple Regression Model
		19.4 Testing Significance of a Factor—One-Way Anova
		19.5 Modelling Interaction Effects
		19.6 Possum Data Example Revisited
		19.7 Exercises
	20 Solutions to Selected Exercises
		Problems of Chap.1
		Problems of Chap.2
		Problems of Chap.3
		Problems of Chap.4
		Problems of Chap.5
		Problems of Chap.6
		Problems of Chap.7
		Problems of Chap.8
		Problems of Chap.9
		Problems of Chap.10
		Problems of Chap.11
		Problems of Chap.12
		Problems of Chap.13
		Problems of Chap.14
		Problems of Chap.15
		Problems of Chap.16
		Problems of Chap.17
		Problems of Chap.18
		Problems of Chap.19
A Appendix: Table of Common Probability Distributions
	A.1 Discrete Distributions
	A.2 Continuous Distributions
	A.3 An Important Mathematical Result
	A.4 Revision of Some Useful Matrix Algebra Results
References
Index