Information Theory and Statistics

INFORMATION THEORY AND STATISTICS
......Page 1
Title Page......Page 2
Copyright Page......Page 3
Dedication......Page 4
Preface to the Dover Edition......Page 5
Preface to the First Edition......Page 6
Acknowledgment......Page 9
Contents......Page 10
1 Introduction......Page 15
2 Definition......Page 17
3 Divergence......Page 20
4 Examples......Page 21
5 Problems......Page 24
2 Additivity......Page 26
3 Convexity......Page 28
4 Invariance......Page 32
5 Divergence......Page 36
6 Fisher's information......Page 40
7 Information and sufficiency......Page 42
8 Problems......Page 45
2 Minimum discrimination information......Page 50
3 Sufficient statistics......Page 57
4 Exponential family......Page 59
5 Neighboring parameters......Page 69
6 Efficiency......Page 77
7 Problems......Page 80
2 Limiting properties......Page 84
3 Type I and type II errors......Page 88
4 Problems......Page 92
1 Estimate of I(*:2)......Page 95
2 Classification......Page 97
3 Testing hypotheses......Page 99
4 Discussion......Page 108
5 Asymptotic properties......Page 111
6 Estimate of J(*,2)......Page 120
7 Problems......Page 121
1 Introduction......Page 123
2 Background......Page 124
3 Conjugate distributions......Page 125
4.1 Basic problem......Page 126
4.2 Analysis of Î(*:2; O[sub(N)])......Page 128
4.3 Parametric case......Page 131
4.4 "One-sided" binomial hypothesis......Page 133
4.5 "One-sided" multinomial hypotheses......Page 135
4.5.2 Illustrative values......Page 139
5.1 Basic problem......Page 142
5.2 "One-sided" hypothesis for the binomial......Page 145
6.1 Basic problem......Page 148
6.2 Partition......Page 150
6.3 Parametric case......Page 153
7 Problems......Page 154
1 Background......Page 156
2 Conjugate distributions......Page 157
3.1 Basic problem......Page 158
3.2 Partition......Page 160
4 "One-sided" hypothesis, single sample......Page 162
5 "One-sided" hypothesis, two samples......Page 165
6 Problems......Page 167
2 Two-way tables......Page 169
3 Three-way tables......Page 173
3.1 Independence of the three classifications......Page 174
3.2 Row classification independent of the other classifications......Page 176
3.3 Independence hypotheses......Page 179
3.4 Conditional independence......Page 180
3.5 Further analysis......Page 181
4 Homogeneity of two-way tables......Page 182
5 Conditional homogeneity......Page 183
6 Homogeneity......Page 184
7 Interaction......Page 185
8 Negative interaction......Page 186
9 Partitions......Page 187
10 Parametric case......Page 190
11 Symmetry......Page 191
12 Examples......Page 193
13 Problems......Page 200
1 Introduction......Page 203
2 Components of information......Page 205
3 Canonical form......Page 208
5 Equal covariance matrices......Page 210
6 Principal components......Page 211
7 Canonical correlation......Page 214
8 Covariance variates......Page 218
9 General case......Page 219
10 Problems......Page 221
2 Background......Page 225
4 The minimum discrimination information statistic......Page 226
5.1 Two-partition subhypothesis......Page 228
5.2 Three-partition subhypothesis......Page 232
6 Analysis of regression: one-way classification, k categories......Page 233
7.1 One-way classification, k categories......Page 239
7.2 Carter's regression case......Page 243
8 Example......Page 245
9.1 Hypotheses not of full rank......Page 250
9.2 Partition......Page 252
10 Analysis of regression, two-way classification......Page 253
11 Problems......Page 265
3.1 Specification......Page 267
3.2 Linear discriminant function......Page 268
4 The minimum discrimination information statistic......Page 269
5.1 Two-partition subhypothesis......Page 271
5.2 Three-partition subhypothesis......Page 274
6.1 Hotelling's generalized Student ratio (Hotelling's T²)......Page 275
6.2 Centering......Page 276
6.3 Homogeneity of r samples......Page 278
6.4.1 Test of regression......Page 282
6.4.2 Test of homogeneity of means and regression......Page 286
6.4.3 Test of homogeneity, assuming regression......Page 287
7 Canonical correlation......Page 289
8.1 Homogeneity of r samples......Page 290
8.2 Canonical correlation......Page 291
9 Examples......Page 293
9.1 Homogeneity of sample means......Page 294
9.2 Canonical correlation......Page 295
9.3 Subhypothesis......Page 298
10.1 Hypotheses not of full rank......Page 303
11 Remark......Page 308
12 Problems......Page 309
2 Background......Page 311
3.1 Homogeneity of the sample......Page 313
3.2 The hypothesis that a k-variate normal population has a specified covariance matrix......Page 316
3.3 The hypothesis of independence......Page 317
3.5 Linear discriminant function......Page 318
3.6 Independence of sets of variates......Page 320
3.7 Independence and equality of variances......Page 321
4.1 Two samples......Page 323
4.3 r samples......Page 325
5.1 Two samples......Page 329
5.2 Linear discriminant function......Page 331
5.3 r samples......Page 332
5.4 Correlation matrices......Page 334
6.1 Homogeneity of covariance matrices......Page 338
6.2 Single sample......Page 342
6.3 The hypothesis of independence......Page 343
6.4 Roots of determinantal equations......Page 344
7.1 A multivariate normal hypothesis......Page 347
8 Problems......Page 348
2 Iteration......Page 356
3 Example......Page 358
4 Remark......Page 361
5 Other linear discriminant functions......Page 362
6 Comparison of the various linear discriminant functions......Page 364
7 Problems......Page 366
References......Page 367
Table I. Log_e (n) and n log_e (n) for Values of n from 1 through 1000......Page 381
Table II. F(p1, p2) = p1 log(p1/p2) + q1 log(q1/q2), p1 + q1 = 1 = p2 + q2......Page 392
Table III. Noncentral χ²......Page 394
Glossary......Page 395
Appendix......Page 403
Index......Page 407