Exercises in Applied Mathematics. With a View toward Information Theory, Machine Learning, Wavelets, and Statistical Physics

Contents
1 Prologue
	1.1 A Guide Throughout the Book
	1.2 Introduction
	1.3 The Source Partition Theorem: Entropy
	1.4 Another Motivation for H
	1.5 Shannon\'s Theorem for the Binary Symmetric Channel (BSC)
	1.6 Repetition Code and Hamming Code
	1.7 Maximum Entropy and a First Connection with Statistical Physics
	1.8 The Case of Continuous Signals
	1.9 Some Questions Related to Statistical Physics
	1.10 Some Questions and Tools in Quantum Mechanics
	1.11 Some Questions and Tools in Quantum Information Theory
	1.12 Machine Learning
	1.13 Solutions of the Exercises
	1.14 Some Specific Exercises and Questions
Part I Algebra
	2 Complements in Linear Algebra
		2.1 Complex Numbers
		2.2 Beyond Complex Numbers
		2.3 Vector Spaces and Linear Transformations
		2.4 Hermitian Forms and Positive Operators
		2.5 The Perceptron Convergence Theorem
		2.6 The Scalar Case
		2.7 Solutions of the Exercises
	3 Positive Semi-Definite Matrices
		3.1 Matrices
		3.2 Schur Complement
		3.3 Newton\'s Formulas
		3.4 Hermitian Matrices
		3.5 Positive Semi-Definite Matrices
		3.6 Tensor Products and the Tensor Algebra
		3.7 Completely Positive Maps
		3.8 Maximum Entropy Analysis and the Yule-Walker Equation
		3.9 Solutions of the Exercises
	4 Algebra and Error-Correcting Codes
		4.1 Sets, Functions, and Groups
		4.2 Rings, Ideals, and Fields
		4.3 Finite Fields
		4.4 Splitting Field
		4.5 Error-Correcting Codes: Generalities
		4.6 Linear Block Codes
		4.7 A Code Which Corrects Two Errors
		4.8 Cyclic Codes
		4.9 Solutions of the Exercises
Part II Analysis
	5 Complements in Real and Complex Analysis
		5.1 Warm-Up Questions and Exercises
		5.2 Some Known Facts: Bolzano, Rolle, and Cauchy et les autres
		5.3 Convex and Concave Functions
		5.4 Stirling\'s Formula
		5.5 Ordinary Differential Equations
		5.6 Liouville\'s Theorem
		5.7 Multivariable Calculus and Differentials
		5.8 Optimization
		5.9 Analytic Functions
		5.10 Solution of the Exercises
	6 Complements in Functional Analysis
		6.1 Topologies and Metric Spaces
		6.2 Hilbert Spaces
		6.3 The Lebesgue Space bold upper L 2 left parenthesis double struck upper R comma d t right parenthesisL2(R,dt)
		6.4 Operators in Banach and Hilbert Space
		6.5 The Fourier Transform and Distributions
		6.6 Legendre and Zak
		6.7 Positive Definite Functions and Kernels
		6.8 Positive Operators
		6.9 Reproducing Kernel Hilbert Spaces
		6.10 Bochner\'s Theorem
		6.11 The Fock Space
		6.12 Hermitian Forms and Reproducing Kernel Krein Spaces
		6.13 Solutions of the Exercises
Part III Probability and Applications
	7 Probability Theory
		7.1 Review on Finite Probability Spaces
		7.2 Random Variables in Finite Probability Spaces
		7.3 Conditional Probabilities
		7.4 Probability Densities
		7.5 Sigma-Algebras and General Probability Spaces
		7.6 Markov Chains
		7.7 Second-Order Stationary Processes: Discrete Case
		7.8 Loève\'s Theorem
		7.9 Solutions of the Exercises
	8 Entropy: Discrete Case
		8.1 Proof of the Source Partition Theorem
		8.2 Properties of the Entropy
		8.3 Other Measures of Entropy
		8.4 Capacity
		8.5 Metrics on the Set of Random Variables and Entropy
		8.6 Boltzmann-Gibbs Distribution
		8.7 Solutions of the Exercises
	9 Thermodynamics
		9.1 Thermodynamics
		9.2 Fluids and the State Equation
		9.3 The Ideal Gas
		9.4 Nonideal Gas
		9.5 Solutions of the Exercises
References
Index
Name Index