Introduction to Probability, Statistics, and Random Processes

Preface
Chapter 1
	1.0 Introduction
	1.1.0 Introduction: What Is Probability?
	1.1.1 Example: Communication Systems
	1.2 Review of Set Theory
	1.2.1 Venn Diagrams
	1.2.2 Set Operations
	1.2.3 Cardinality: Countable and Uncountable Sets
	1.2.4 Functions
	1.2.5 Solved Problems:Review of Set Theory
	1.3.1 Random Experiments
	1.3.2 Probability
	1.3.3 Finding Probabilities
	1.3.4 Discrete Probability Models
	1.3.5 Continuous Probability Models
	1.3.6 Solved Problems:Random Experiments and Probabilities
	1.4.0 Conditional Probability
	1.4.1 Independence
	1.4.2 Law of Total Probability
	1.4.3 Bayes' Rule
	1.4.4 Conditional Independence
	1.4.5 Solved Problems:Conditional Probability
	1.5.0 End of Chapter Problems
Chapter 2
	2.1 Counting
	2.1.1 Ordered Sampling with Replacement
	2.1.2 Ordered Sampling without Replacement: Permutations
	2.1.3 Unordered Sampling without Replacement: Combinations
	2.1.4 Unordered Sampling with Replacement
	2.1.5 Solved Problems:Combinatorics
	2.2.0 End of Chapter Problems
Chapter 3
	3.1.1 Random Variables
	3.1.2 Discrete Random Variables
	3.1.3 Probability Mass Function (PMF)
	3.1.4 Independent Random Variables
	3.1.5 Special Distributions
	3.1.6 Solved Problems:Discrete Random Variables
	3.2.1 Cumulative Distribution Function
	3.2.2 Expectation
	3.2.3 Functions of Random Variables
	3.2.4 Variance
	3.2.5 Solved Problems:More about Discrete Random Variables
	3.3 End of Chapter Problems
Chapter 4
	4.0.0 Introduction
	4.1.0 Continuous Random Variables and their Distributions
	4.1.1 Probability Density Function (PDF)
	4.1.2 Expected Value and Variance
	4.1.3 Functions of Continuous Random Variables
	4.1.4 Solved Problems:  Continuous Random Variables
	4.2.1 Uniform Distribution
	4.2.2 Exponential Distribution
	4.2.3 Normal (Gaussian) Distribution
	4.2.4 Gamma Distribution
	4.2.5 Other Distributions
	4.2.6 Solved Problems: Special Continuous Distributions
	4.3.1 Mixed Random Variables
	4.3.2 Using the Delta Function
	4.3.3 Solved Problems:Mixed Random Variables
	4.4 End of Chapter Problems
Chapter 5
	5.1.0 Joint Distributions: Two Random Variables
	5.1.1 Joint Probability Mass Function (PMF)
	5.1.2 Joint Cumulative Distributive Function (CDF)
	5.1.3 Conditioning and Independence
	5.1.4 Functions of Two Random Variables
	5.1.5 Conditional Expectation (Revisited) and Conditional Variance
	5.1.6 Solved Problems
	5.2.0 Two Continuous Random Variables
	5.2.1 Joint Probability Density Function (PDF)
	5.2.2 Joint Cumulative Distribution Function (CDF)
	5.2.3 Conditioning and Independence
	5.2.4 Functions of Two Continuous Random Variables
	5.2.5 Solved Problems
	5.3.1 Covariance and Correlation
	5.3.2 Bivariate Normal Distribution
	5.3.3 Solved Problems
	5.4.0 End of Chapter Problems
Chapter 6
	6.0.0 Introduction
	6.1.1 Joint Distributions and Independence
	6.1.2 Sums of Random Variables
	6.1.3 Moment Generating Functions
	6.1.4 Characteristic Functions
	6.1.5 Random Vectors
	6.1.6 Solved Problems
	6.2.0 Probability Bounds
	6.2.1 The Union Bound and Extension
	6.2.2 Markov and Chebyshev Inequalities
	6.2.3 Chernoff Bounds
	6.2.4 Cauchy-Schwarz Inequality
	6.2.5 Jensen's Inequality
	6.2.6 Solved Problems
	6.3.0 Chapter Problems
Chapter 7
	7.0.0 Introduction
	7.1.0 Limit Theorems
	7.1.1 Law of Large Numbers
	7.1.2 Central Limit Theorem
	7.1.3 Solved Problems
	7.2.0 Convergence of Random Variables
	7.2.1 Convergence of a Sequence of Numbers
	7.2.2 Sequence of Random Variables
	7.2.3 Different Types of Convergence for Sequences of Random Variables
	7.2.4 Convergence in Distribution
	7.2.5 Convergence in Probability
	7.2.6 Convergence in Mean
	7.2.7 Almost Sure Convergence
	7.2.8 Solved Problems
	7.3.0 End of Chapter Problems
Chapter 8
	8.1.0 Introduction
	8.1.1 Random Sampling
	8.2.0 Point Estimation
	8.2.1 Evaluating Estimators
	8.2.2 Point Estimators for Mean and Variance
	8.2.3 Maximum Likelihood Estimation
	8.2.4 Asymptotic Properties of MLEs
	8.2.5 Solved Problems
	8.3.0 Interval Estimation (Confidence Intervals)
	8.3.1 The General Framework of Interval Estimation
	8.3.2 Finding Interval Estimators
	8.3.3 Confidence Intervals for Normal Samples
	8.3.4 Solved Problems
	8.4.1 Introduction
	8.4.2 General Setting and Definitions
	8.4.3 Hypothesis Testing for the Mean
	8.4.4 P-Values
	8.4.5 Likelihood Ratio Tests
	8.4.6 Solved Problems
	8.5.0 Linear Regression
	8.5.1 Simple Linear Regression Model
	8.5.2 The First Method for Finding $\beta_0$ and $\beta_1$
	8.5.3 The Method of Least Squares
	8.5.4 Extensions and Issues
	8.5.5 Solved Problems
	8.6.0 End of Chapter Problems
Chapter 9
	9.1.0 Bayesian Inference
	9.1.10 Solved Problems
	9.1.1 Prior and Posterior
	9.1.2 Maximum A Posteriori (MAP) Estimation
	9.1.3 Comparison to ML Estimation
	9.1.4 Conditional Expectation (MMSE)
	9.1.5 Mean Squared Error (MSE)
	9.1.6 Linear MMSE Estimation of Random Variables
	9.1.7 Estimation for Random Vectors
	9.1.8 Bayesian Hypothesis Testing
	9.1.9 Bayesian Interval Estimation
	9.2.0 End of Chapter Problems
Chapter 10
	10.1.0 Basic Concepts
	10.1.1 PDFs and CDFs
	10.1.2 Mean and Correlation Functions
	10.1.3 Multiple Random Processes
	10.1.4 Stationary Processes
	10.1.5 Gaussian Random Processes
	10.1.6 Solved Problems
	10.2.0 Processing of Random Signals
	10.2.1 Power Spectral Density
	10.2.2 Linear Time-Invariant (LTI) Systems with Random Inputs
	10.2.3 Power in a Frequency Band
	10.2.4 White Noise
	10.2.5 Solved Problems
	10.3.0 End of Chapter Problems
Chapter 11
	11.0.0 Introduction
	11.1.1 Counting Processes
	11.1.2 Basic Concepts of the Poisson Process
	11.1.3 Merging and Splitting Poisson Processes
	11.1.4 Nonhomogeneous Poisson Processes
	11.1.5 Solved Problems
	11.2.1 Introduction
	11.2.2 State Transition Matrix and Diagram
	11.2.3 Probability Distributions
	11.2.4 Classification of States
	11.2.5 Using the Law of Total Probability with Recursion
	11.2.6 Stationary and Limiting Distributions
	11.2.7 Solved Problems
	11.3.1 Introduction
	11.3.2 Stationary and Limiting Distributions
	11.3.3 The Generator Matrix
	11.3.4 Solved Problems
	11.4.0 Brownian Motion (Wiener Process)
	11.4.1 Brownian Motion as the Limit of a Symmetric Random Walk
	11.4.2 Definition and Some Properties
	11.4.3 Solved Problems
	11.5.0 End of Chapter Problems
Chapter 12
Chapter 13
Chapter 14
Appendix
	Review of Fourier Transform
	Some Important Distributions
Bibliography