Probability and Stochastic Processes: A Friendly Introduction for Electrical and Computer Engineers

Content: Features of this Text i Preface vii 1 Experiments, Models, and Probabilities 1 Getting Started with Probability 1 1.1 Set Theory 3 1.2 Applying Set Theory to Probability 7 1.3 Probability Axioms 11 1.4 Conditional Probability 15 1.5 Partitions and the Law of Total Probability 18 1.6 Independence 24 1.7 Matlab 27 Problems 29 2 Sequential Experiments 35 2.1 Tree Diagrams 35 2.2 Counting Methods 40 2.3 Independent Trials 49 2.4 Reliability Analysis 52 2.5 Matlab 55 Problems 57 3 Discrete Random Variables 62 3.1 Definitions 62 3.2 Probability Mass Function 65 3.3 Families of Discrete Random Variables 68 3.4 Cumulative Distribution Function (CDF) 77 3.5 Averages and Expected Value 80 3.6 Functions of a Random Variable 86 3.7 Expected Value of a Derived Random Variable 90 3.8 Variance and Standard Deviation 93 3.9 Matlab 99 Problems 106 4 Continuous Random Variables 118 4.1 Continuous Sample Space 118 4.2 The Cumulative Distribution Function 121 4.3 Probability Density Function 123 4.4 Expected Values 128 4.5 Families of Continuous Random Variables 132 4.6 Gaussian Random Variables 138 4.7 Delta Functions, Mixed Random Variables 145 4.8 Matlab 152 Problems 154 5 Multiple Random Variables 162 5.1 Joint Cumulative Distribution Function 163 5.2 Joint Probability Mass Function 166 5.3 Marginal PMF 169 5.4 Joint Probability Density Function 171 5.5 Marginal PDF 177 5.6 Independent Random Variables 178 5.7 Expected Value of a Function of Two Random Variables 181 5.8 Covariance, Correlation and Independence 184 5.9 Bivariate Gaussian Random Variables 191 5.10 Multivariate Probability Models 195 5.11 Matlab 201 Problems 206 6 Probability Models of Derived Random Variables 218 6.1 PMF of a Function of Two Discrete Random Variables 219 6.2 Functions Yielding Continuous Random Variables 220 6.3 Functions Yielding Discrete or Mixed Random Variables 226 6.4 Continuous Functions of Two Continuous Random Variables 229 6.5 PDF of the Sum of Two Random Variables 232 6.6 Matlab 234 Problems 236 7 Conditional Probability Models 242 7.1 Conditioning a Random Variable by an Event 242 7.2 Conditional Expected Value Given an Event 248 7.3 Conditioning Two Random Variables by an Event 252 7.4 Conditioning by a Random Variable 256 7.5 Conditional Expected Value Given a Random Variable 262 7.6 Bivariate Gaussian Random Variables: Conditional PDFs 265 7.7 Matlab 268 Problems 269 8 Random Vectors 277 8.1 Vector Notation 277 8.2 Independent Random Variables and Random Vectors 280 8.3 Functions of Random Vectors 281 8.4 Expected Value Vector and Correlation Matrix 285 8.5 Gaussian Random Vectors 291 8.6 Matlab 298 Problems 300 9 Sums of Random Variables 306 9.1 Expected Values of Sums 306 9.2 Moment Generating Functions 310 9.3 MGF of the Sum of Independent Random Variables 314 9.4 Random Sums of Independent Random Variables 317 9.5 Central Limit Theorem 321 9.6 Matlab 328 Problems 331 10 The Sample Mean 337 10.1 Sample Mean: Expected Value and Variance 337 10.2 Deviation of a Random Variable from the Expected Value 339 10.3 Laws of Large Numbers 343 10.4 Point Estimates of Model Parameters 345 10.5 Confidence Intervals 352 10.6 Matlab 358 Problems 360 11 Hypothesis Testing 366 11.1 Significance Testing 367 11.2 Binary Hypothesis Testing 370 11.3 Multiple Hypothesis Test 384 11.4 Matlab 387 Problems 389 12 Estimation of a Random Variable 399 12.1 Minimum Mean Square Error Estimation 400 12.2 Linear Estimation of X given Y 404 12.3 MAP and ML Estimation 409 12.4 Linear Estimation of Random Variables from Random Vectors 414 12.5 Matlab 421 Problems 423 13 Stochastic Processes 429 13.1 Definitions and Examples 430 13.2 Random Variables from Random Processes 435 13.3 Independent, Identically Distributed Random Sequences 437 13.4 The Poisson Process 439 13.5 Properties of the Poisson Process 443 13.6 The Brownian Motion Process 446 13.7 Expected Value and Correlation 448 13.8 Stationary Processes 452 13.9 Wide Sense Stationary Stochastic Processes 455 13.10 Cross-Correlation 459 13.11 Gaussian Processes 462 13.12 Matlab 464 Problems 468 Appendix A Families of Random Variables 477 A.1 Discrete Random Variables 477 A.2 Continuous Random Variables 479 Appendix B A Few Math Facts 483 References 489 Index 491