Information Theory and the Central Limit Theorem

INFORMATION THEORY AND THE CENTRAL LIMIT THEOREM......Page 1
Half-title......Page 2
Title Page......Page 4
Copyright Page......Page 5
Dedication......Page 6
Preface......Page 8
Contents......Page 12
1.1.1 Discrete entropy......Page 16
1.1.2 Differential entropy......Page 20
1.1.3 Relative entropy......Page 23
1.1.4 Other entropy-like quantities......Page 28
1.1.5 Axiomatic definition of entropy......Page 31
1.2.1 Definition of thermodynamic entropy......Page 32
1.2.2 Maximum entropy and the Second Law......Page 34
1.3.1 Definition and properties......Page 36
1.3.2 Behaviour on convolution......Page 40
1.4.1 Rényi’s method......Page 42
1.4.2 Convergence of Fisher information......Page 45
2.1.1 Sandwich inequality......Page 48
2.1.2 Projections and adjoints......Page 51
2.1.3 Normal case......Page 53
2.1.4 Results of Brown and Barron......Page 56
2.2.1 Projection of functions in L²......Page 58
2.2.2 Restricted Poincaré constants......Page 59
2.2.3 Convergence of restricted Poincaré constants......Page 61
2.3.1 Proof of O(1/n) rate of convergence......Page 62
2.3.2 Comparison with other forms of convergence......Page 65
2.3.3 Extending the Cramér–Rao lower bound......Page 66
3.1.1 Previous results......Page 70
3.1.2 Improved projection inequalities......Page 72
3.2.1 Definitions......Page 79
3.2.2 Behaviour on convolution......Page 80
3.2.3 Projection inequalities......Page 81
4.1.1 Mixing coefficients......Page 84
4.1.2 Main results......Page 87
4.2 Fisher information and convolution......Page 89
4.3.1 Notation and definitions......Page 92
4.3.2 Bounds on densities......Page 94
4.3.3 Bounds on tails......Page 97
4.3.4 Control of the mixing coeficients......Page 98
5.1.1 Definitions......Page 102
5.1.2 Domains of attraction......Page 104
5.1.3 Entropy of stable laws......Page 106
5.2.1 Minimising relative entropy......Page 107
5.2.2 Minimising Fisher information distance......Page 109
5.2.3 Matching logarithm of density......Page 110
5.3.1 Partial differential equations......Page 111
5.3.2 Derivatives of relative entropy......Page 112
5.3.3 Integral form of the identities......Page 115
5.4 Relationship between forms of convergence......Page 117
5.5 Steps towards a Brown inequality......Page 120
6.1.1 Introduction to topological groups......Page 124
6.1.2 Convergence of convolutions......Page 126
6.1.3 Conditions for uniform convergence......Page 129
6.2.1 Introduction and results......Page 133
6.2.2 Entropy on compact groups......Page 134
6.3 Comparison of forms of convergence......Page 136
6.4.1 Explicit rate of convergence......Page 140
6.4.2 No explicit rate of convergence......Page 141
7.1.1 The law of small numbers......Page 144
7.1.2 Simplest bounds on relative entropy......Page 147
7.2.1 Standard Fisher information......Page 151
7.2.2 Scaled Fisher information......Page 153
7.2.3 Dependent variables......Page 155
7.3 Strength of bounds......Page 157
7.4 De Bruijn identity......Page 159
7.5.1 L² definitions......Page 161
7.5.2 Sums of Bernoulli variables......Page 162
7.5.3 Normal convergence......Page 165
8.1.1 Operators and algebras......Page 168
8.1.2 Expectations and Cauchy transforms......Page 169
8.1.3 Free interaction......Page 173
8.2.1 Derivations......Page 176
8.2.2 Fisher information and entropy......Page 178
8.3 Projection inequalities......Page 181
A.1 Gamma distribution......Page 186
A.2 Stable distributions......Page 188
B.1 Standard Poincaré inequalities......Page 192
B.2 Weighted Poincaré inequalities......Page 194
Appendix C. de Bruijn Identity......Page 198
Appendix D. Entropy Power Inequality......Page 202
E.1 Convergence in relative entropy to the Gaussian......Page 206
E.2 Convergence to other variables......Page 209
E.3 Convergence in Fisher information......Page 210
Bibliography......Page 214
Index......Page 222