Probability and Conditional Expectation: Fundamentals for the Empirical Sciences

Content: Part I Measure-Theoretical Foundations of Probability Theory     1 Measure 3     1.1 Introductory Examples 3     1.2   -Algebra and Measurable Space 4     1.2.1   -Algebra Generated by a Set System 9     1.2.2   -Algebra of Borel Sets on Rn 12     1.2.3   -Algebra on a Cartesian Product 13     1.2.4    -Stable Set Systems That Generate a   -Algebra 15     1.3 Measure and Measure Space 16     1.3.1   -Additivity and Related Properties 17     1.3.2 Other Properties 18     1.4 Specific Measures 20     1.4.1 Dirac Measure and Counting Measure 21     1.4.2 Lebesgue Measure 22     1.4.3 Other Examples of a Measure 23     1.4.4 Finite and   -Finite Measures 23     1.4.5 Product Measure 24     1.5 Continuity of a Measure 25     1.6 Specifying a Measure via a Generating System 27     1.7   -Algebra That is Trivial With Respect to a Measure 28     1.8 Proofs 28     1.9 Exercises 31     2 Measurable Mapping 41     2.1 Image and Inverse Image 41     2.2 Introductory Examples 42     2.2.1 Example 1: Rectangles 42     2.2.2 Example 2: Flipping two Coins 44     2.3 Measurable Mapping 46     2.3.1 Measurable Mapping 46     2.3.2   -Algebra Generated by a Mapping 51     2.3.3 Final   -Algebra 54     2.3.4 Multivariate Mapping 54     2.3.5 Projection Mapping 56     2.3.6 Measurability With Respect to a Mapping 56     2.4 Theorems on Measurable Mappings 58     2.4.1 Measurability of a Composition 59     2.4.2 Theorems on Measurable Functions 61     2.5 Equivalence of Two Mappings With Respect to a Measure 64     2.6 Image Measure 67     2.7 Proofs 70     2.8 Exercises 75     3 Integral 83     3.1 Definition 83     3.1.1 Integral of a Nonnegative Step Function 83     3.1.2 Integral of a Nonnegative Measurable Function 88     3.1.3 Integral of a Measurable Function 93     3.2 Properties 96     3.2.1 Integral of   -Equivalent Functions 98     3.2.2 Integral With Respect to a Weighted Sum of Measures 100     3.2.3 Integral With Respect to an Image Measure 102     3.2.4 Convergence Theorems 103     3.3 Lebesgue and Riemann Integral 104     3.4 Density 106     3.5 Absolute Continuity and the Radon-Nikodym Theorem 108     3.6 Integral With Respect to a Product Measure 110     3.7 Proofs 111     3.8 Exercises 120     Part II Probability, Random Variable and its Distribution     4 Probability Measure 127     4.1 Probability Measure and Probability Space 127     4.1.1 Definition 127     4.1.2 Formal and Substantive Meaning of Probabilistic Terms 128     4.1.3 Properties of a Probability Measure 128     4.1.4 Examples 130     4.2 Conditional Probability 132     4.2.1 Definition 132     4.2.2 Filtration and Time Order Between Events and Sets of Events 133     4.2.3 Multiplication Rule 135     4.2.4 Examples 136     4.2.5 Theorem of Total Probability 137     4.2.6 Bayes    Theorem 138     4.2.7 Conditional-Probability Measure 139     4.3 Independence 143     4.3.1 Independence of Events 143     4.3.2 Independence of Set Systems 144     4.4 Conditional Independence Given an Event 145     4.4.1 Conditional Independence of Events Given an Event 145     4.4.2 Conditional Independence of Set Systems Given an Event 146     4.5 Proofs 148     4.6 Exercises 150     5 Random Variable, Distribution, Density, and Distribution Function 155     5.1 Random Variable and its Distribution 155     5.2 Equivalence of Two Random Variables With Respect to a Probability Measure 161     5.2.1 Identical and P-Equivalent Random Variables 161     5.2.2 P-Equivalence, PB-Equivalence, and Absolute Continuity 164     5.3 Multivariate Random Variable 167     5.4 Independence of Random Variables 169     5.5 Probability Function of a Discrete Random Variable 175     5.6 Probability Density With Respect to a Measure 178     5.6.1 General Concepts and Properties 178     5.6.2 Density of a Discrete Random Variable 180     5.6.3 Density of a Bivariate Random Variable 180     5.7 Uni- or Multivariate Real-Valued Random Variable 182     5.7.1 Distribution Function of a Univariate Real-Valued Random Variable 182     5.7.2 Distribution Function of a Multivariate Real-Valued Random Variable 184     5.7.3 Density of a Continuous Univariate Real-Valued Random Variable 185     5.7.4 Density of a Continuous Multivariate Real-Valued Random Variable 187     5.8 Proofs 188     5.9 Exercises 196     6 Expectation, Variance, and Other Moments 199     6.1 Expectation 199     6.1.1 Definition 199     6.1.2 Expectation of a Discrete Random Variable 200     6.1.3 Computing the Expectation Using a Density 202     6.1.4 Transformation Theorem 203     6.1.5 Rules of Computation 206     6.2 Moments, Variance, and Standard Deviation 207     6.3 Proofs 212     6.4 Exercises 213     7 Linear Quasi-Regression, Covariance, and Correlation 217     7.1 Linear Quasi-Regression 217     7.2 Covariance 220     7.3 Correlation 224     7.4 Expectation Vector and Covariance Matrix 227     7.4.1 Random Vector and Random Matrix 227     7.4.2 Expectation of a Random Vector and a Random Matrix 228     7.4.3 Covariance Matrix of two Multivariate Random Variables 229     7.5 Multiple Linear Quasi-Regression 231     7.6 Proofs 233     7.7 Exercises 237     8 Some Distributions 245     8.1 Some Distributions of Discrete Random Variables 245     8.1.1 Discrete Uniform Distribution 245     8.1.2 Bernoulli Distribution 246     8.1.3 Binomial Distribution 247     8.1.4 Poisson Distribution 250     8.1.5 Geometric Distribution 252     8.2 Some Distributions of Continuous Random Variables 254     8.2.1 Continuous Uniform Distribution 254     8.2.2 Normal Distribution 256     8.2.3 Multivariate Normal Distribution 259     8.2.4 Central   2-Distribution 262     8.2.5 Central t -Distribution 264     8.2.6 Central F-Distribution 266     8.3 Proofs 267     8.4 Exercises 271     Part III Conditional Expectation and Regression     9 Conditional Expectation Value and Discrete Conditional Expectation 277     9.1 Conditional Expectation Value 277     9.2 Transformation Theorem 280     9.3 Other Properties 282     9.4 Discrete Conditional Expectation 283     9.5 Discrete Regression 285     9.6 Examples 287     9.7 Proofs 291     9.8 Exercises 291     10 Conditional Expectation 295     10.1 Assumptions and Definitions 295     10.2 Existence and Uniqueness 297     10.2.1 Uniqueness With Respect to a Probability Measure 298     10.2.2 A Necessary and Sufficient Condition of Uniqueness 299     10.2.3 Examples 300     10.3 Rules of Computation and Other Properties 301     10.3.1 Rules of Computation 301     10.3.2 Monotonicity 302     10.3.3 Convergence Theorems 302     10.4 Factorization, Regression, and Conditional Expectation Value 306     10.4.1 Existence of a Factorization 306     10.4.2 Conditional Expectation and Mean-Squared Error 307     10.4.3 Uniqueness of a Factorization 308     10.4.4 Conditional Expectation Value 309     10.5 Characterizing a Conditional Expectation by the Joint Distribution 312     10.6 Conditional Mean Independence 313     10.7 Proofs 318     10.8 Exercises 321     11 Residual, Conditional Variance, and Conditional Covariance 329     11.1 Residual With Respect to a Conditional Expectation 329     11.2 Coefficient of Determination and Multiple Correlation 333     11.3 Conditional Variance and Covariance Given a   -Algebra 338     11.4 Conditional Variance and Covariance Given a Value of a Random Variable 339     11.5 Properties of Conditional Variances and Covariances 342     11.6 Partial Correlation 345     11.7 Proofs 347     11.8 Exercises 348     12 Linear Regression 357     12.1 Basic Ideas 357     12.2 Assumptions and Definitions 359     12.3 Examples 361     12.4 Linear Quasi-Regression 366     12.5 Uniqueness and Identification of Regression Coefficients 367     12.6 Linear Regression 369     12.7 Parametrizations of a Discrete Conditional Expectation 370     12.8 Invariance of Regression Coefficients 374     12.9 Proofs 375     12.10Exercises 377     13 Linear Logistic Regression 381     13.1 Logit Transformation of a Conditional Probability 381     13.2 Linear Logistic Parametrization 383     13.3 A Parametrization of a Discrete Conditional Probability 385     13.4 Identification of Coefficients of a Linear Logistic Parametrization 387     13.5 Linear Logistic Regression and Linear Logit Regression 388     13.6 Proofs 394     13.7 Exercises 396     14 Conditional Expectation With Respect to a Conditional-Probability Measure 399     14.1 Introductory Examples 399     14.2 Assumptions and Definitions 404     14.3 Properties 410     14.4 Partial Conditional Expectation 412     14.5 Factorization 413     14.5.1 Conditional Expectation Value With Respect to PB 414     14.5.2 Uniqueness of Factorizations 415     14.6 Uniqueness 415     14.6.1 A Necessary and Sufficient Condition of Uniqueness 415     14.6.2 Uniqueness w.r.t. P and Other Probability Measures 417     14.6.3 Necessary and Sufficient Conditions of P-Uniqueness 418     14.6.4 Properties Related to P-Uniqueness 420     14.7 Conditional Mean Independence With Respect to PZ=z 424     14.8 Proofs 426     14.9 Exercises 431     15 Conditional Effect Functions of a Discrete Regressor 437     15.1 Assumptions and Definitions 437     15.2 Conditional Intercept Function and Effect Functions 438     15.3 Implications of Independence of X and Z for Regression Coefficients 441     15.4 Adjusted Conditional Effect Functions 443     15.5 Conditional Logit Effect Functions 447     15.6 Implications of Independence of X and Z for the Logit Regression Coefficients 450     15.7 Proofs 452     15.8 Exercises 454     Part IV Conditional Independence and Conditional Distribution     16 Conditional Independence 459     16.1 Assumptions and Definitions 459     16.1.1 Two Events 459     16.1.2 Two Sets of Events 461     16.1.3 Two Random Variables 462     16.2 Properties 463     16.3 Conditional Independence and Conditional Mean Independence 470     16.4 Families of Events 473     16.5 Families of Set Systems 473     16.6 Families of Random Variables 475     16.7 Proofs 478     16.8 Exercises 486     17 Conditional Distribution 491     17.1 Conditional Distribution Given a   -Algebra or a Random Variable 491     17.2 Conditional Distribution Given a Value of a Random Variable 494     17.3 Existence and Uniqueness 497     17.3.1 Existence 497     17.3.2 Uniqueness of the Functions PY |C ( *, A   ) 498     17.3.3 Common Null Set (CNS) Uniqueness of a Conditional Distribution 499     17.4 Conditional-Probability Measure Given a Value of a Random Variable 502     17.5 Decomposing the Joint Distribution of Random Variables 504     17.6 Conditional Independence and Conditional Distributions 506     17.7 Expectations With Respect to a Conditional Distribution 511     17.8 Conditional Distribution Function and Probability Density 513     17.9 Conditional Distribution and Radon-Nikodym Density 516     17.10Proofs 520     17.11Exercises 536     References 541