A Mathematical Theory of Communication

Introduction......Page 1
The Discrete Noiseless Channel......Page 3
The Discrete Source of Information......Page 4
The Series of Approximations to English......Page 7
Ergodic and Mixed Sources......Page 8
Choice, Uncertainty and Entropy......Page 10
Representation of the Encoding and Decoding Operation......Page 15
The Fundamental Theorem of a Noiseless Channel......Page 16
Discussion and Examples......Page 17
Representation of a Noisy Discrete Channel......Page 19
The Fundamental Theorem for a Discrete Channel with Noise......Page 22
Discussion......Page 24
Example of a Discrete Channel and its Capacity......Page 25
The Channel Capacity in Certain Special Cases......Page 26
An Example of Efficient Coding......Page 27
A2. The Derivation of Entropy......Page 28
A3. Theorems on Ergodic Sources......Page 29
A4. Maximizing the Rate for a System of Constraints......Page 30
Sets and Ensembles of Functions......Page 32
Band Limited Ensembles of Functions......Page 34
Entropy of a Continuous Distribution......Page 35
Entropy of an Ensemble of Functions......Page 38
Entropy Loss in Linear Filters......Page 39
Entropy of a Sum of Two Ensembles......Page 40
The Capacity of a Continuous Channel......Page 41
Channel Capacity with an Average Power Limitation......Page 43
The Channel Capacity with a Peak Power Limitation......Page 45
Fidelity Evaluation Functions......Page 47
The Rate for a Source Relative to a Fidelity Evaluation......Page 49
The Calculation of Rates......Page 50
A6......Page 52
A7......Page 54