The Joy of Fourier

Title page......Page 1
Preface......Page 3
Contents......Page 4
1.1  Ubiquitous Fourier Analysis......Page 17
1.2  Jean Baptiste Joseph Fourier......Page 18
1.3.1  Surveying the Contents......Page 19
2.1  Signal Classes......Page 25
2.2.1  The Continuous Time Fourier Transform......Page 29
2.2.2  The Fourier Series......Page 30
2.2.2.2  Convergence......Page 32
2.2.3  Relationship Between Fourier and Laplace Transforms......Page 33
2.2.4.2  The Convolution Theorem......Page 34
2.2.4.4  The Duality Theorem......Page 35
2.2.5.1  The Dirac Delta Function......Page 36
2.2.5.2  Trigonometric Functions......Page 40
2.2.5.3  The Sinc and Related Functions......Page 43
2.2.5.4  The Rectangle Function......Page 44
2.2.5.5  The Signum Function......Page 45
2.2.5.6  The Gaussian......Page 47
2.2.5.7  The Comb Function......Page 48
2.2.5.9  The Array Function......Page 49
2.2.5.10  The Gamma Function......Page 53
2.2.5.12  Bessel Functions......Page 54
2.2.5.14  Orthogonal Polynomials......Page 57
2.2.6.1  The Signal Integral Property......Page 62
2.2.6.2  Conjugate Symmetry of a Real Signal\'s Spectrum......Page 63
2.2.6.3  Hilbert Transform Relationships Within a Causal Signals\' Spectrum......Page 64
2.2.6.5  The Poisson Sum Formula......Page 65
2.3.1  Parseval\'s theorem for an Orthonormal Basis......Page 66
2.3.2.2  The Fourier Series......Page 67
2.4  The Discrete Time Fourier Transform......Page 68
2.4.1  Relation to the Continuous Time Fourier Transform......Page 69
2.5  The Discrete Fourier Transform......Page 71
2.5.1  Relation to the Continuous Time Fourier Transform......Page 72
2.6  Related Transforms......Page 74
2.6.1  The Cosine Transform......Page 75
2.6.4  The z Transforms......Page 76
2.7  Exercises......Page 79
2.8  Solutions for Chapter 2 Exercises......Page 92
3.1.1.3  Linear Systems......Page 125
3.1.1.4  Time-Invariant Systems......Page 126
3.1.1.6  Causal Systems......Page 127
3.1.1.7  Memoryless Systems......Page 128
3.1.2.1  The Magnifier......Page 129
3.1.2.2  The Fourier Transformer......Page 130
3.1.2.3  Convolution......Page 131
3.2  System Characterization......Page 133
3.3  Linear Systems......Page 134
3.4  Causal Linear Systems......Page 135
3.5  Linear Time Invariant (LTI) Systems......Page 136
3.5.1  Convolution Algebra......Page 137
3.5.2  Convolution Mechanics......Page 138
3.5.3  Step and Ramp Response Characterization......Page 140
3.5.4  Characterizing an LTI System......Page 141
3.5.5  Filters......Page 142
3.6  Goertzel\'s Algorithm for Computing the DFT......Page 143
3.7.1  Periodicity of the Fourier Transform Operator......Page 144
3.7.3  The Weighted Fractional Fourier Transform......Page 145
3.8  Approximating a Linear System with LTI Systems......Page 147
3.8.1.1  The Magni¯er......Page 148
3.8.1.2  The Fourier Transformer......Page 149
3.9  Exercises......Page 152
3.10  Solutions for Chapter 3 Exercises......Page 156
4.1.1  Probability Density Functions, Expectation, and Characteristic Functions......Page 164
4.1.1.2  Moments from the Characteristic Functions......Page 165
4.1.1.3  Chebyshev\'s inequality......Page 168
4.1.2.1  The Uniform Random Variable......Page 169
4.1.2.3  The Gaussian Random Variable......Page 173
4.1.2.5  The Laplace Random Variable......Page 174
4.1.2.6  The Hyperbolic Secant Random Variable......Page 175
4.1.2.7  The Gamma Random Variable......Page 176
4.1.2.8  The Beta Random Variable......Page 178
4.1.2.9  The Cauchy Random Variable......Page 179
4.1.2.10  The Deterministic Random Variable......Page 181
4.1.2.11  The Bernoulli Random Variable......Page 182
4.1.2.13  The Binomial Random Variable......Page 183
4.1.2.14  The Poisson Random Variable......Page 185
4.1.3.1  The Sum of Independent Random Variables......Page 187
4.1.3.2  Distributions Closed Under Independent Summation......Page 188
4.1.3.4  The Average of i.i.d. Random Variables......Page 190
4.1.4.1  Stochastic Resonance......Page 191
4.1.5  The Central Limit Theorem......Page 192
4.1.5.2  Random Variables That Are Asymptotically Gaussian......Page 194
4.2  Uncertainty in Wave Equations......Page 195
4.3  Stochastic Processes......Page 197
4.3.1  First and Second Order Statistics......Page 198
4.3.1.1  Stationary Processes......Page 199
4.3.2  Power Spectral Density......Page 200
4.3.3.2  Stationary Discrete White Noise......Page 201
4.3.4  Linear Systems with Stationary Stochastic Inputs......Page 202
4.3.5  Properties of the Power Spectral Density......Page 203
4.3.5.3  Nonnegativity......Page 204
4.4  Exercises......Page 216
4.5  Solutions for Chapter 4 Exercises......Page 220
5.1.2  History......Page 229
5.2  Interpretation......Page 232
5.3.1  Using Comb Functions......Page 233
5.3.2  Fourier Series Proof......Page 235
5.4.1.1  For Finite Energy Signals......Page 236
5.4.1.2  For Bandlimited Functions with Finite Area Spectra......Page 237
5.4.2.1  Of Bandlimited Functions......Page 238
5.4.2.3  Derivation of the Low Passed Kernel......Page 239
5.4.3  The Time-Bandwidth Product......Page 244
5.5  Application to Spectra Containing Distributions......Page 245
5.6  Application to Bandlimited Stochastic Processes......Page 246
5.7  Exercises......Page 248
5.8  Solutions for Chapter 5 Exercises......Page 251
6.1.1  Oversampling......Page 257
6.1.2.2  Restoring a Single Lost Sample......Page 258
6.1.2.3  Restoring M Lost Samples......Page 260
6.1.2.4  Direct Interpolation From M Lost Samples......Page 261
6.1.2.5  Relaxed Interpolation Formulae......Page 262
6.1.3  Criteria for Generalized Interpolation Functions......Page 263
6.1.3.1  Interpolation Functions......Page 264
6.1.4  Reconstruction from a Filtered Signal\'s Samples......Page 265
6.1.4.1  Restoration of Samples from an Integrating Detector......Page 266
6.2  Papoulis\' Generalization......Page 267
6.2.1  Derivation......Page 268
6.2.2  Interpolation Function Computation......Page 272
6.2.3.1  Recurrent Nonuniform Sampling......Page 273
6.2.3.2  Interlaced Signal{Derivative Sampling......Page 274
6.2.3.3  Higher Order Derivative Sampling......Page 275
6.2.3.4  E®ects of Oversampling......Page 277
6.3.1  Properties of the Derivative Kernel......Page 278
6.5  Sampling Trigonometric Polynomials......Page 282
6.6.1  Heterodyned Sampling......Page 284
6.6.2  Direct Bandpass Sampling......Page 286
6.8  Lagrangian Interpolation......Page 288
6.9  Kramer\'s Generalization......Page 291
6.10  Exercises......Page 292
6.11  Solutions for Chapter 6 Exercises......Page 296
7.1.1  On Cardinal Series Interpolation......Page 307
7.1.1.2  E®ects of Oversampling and Filtering......Page 308
7.1.2  Interpolation Noise Variance for Directly Sampled Signals......Page 312
7.1.2.1  Interpolation with Lost Samples......Page 313
7.1.2.2  Bandpass Functions......Page 320
7.1.3  On Papoulis\' Generalization......Page 321
7.1.3.1  Examples......Page 324
7.1.3.2  Notes......Page 325
7.1.4  On Derivative Interpolation......Page 327
7.1.5  A Lower Bound on the NINV......Page 329
7.1.5.1  Examples......Page 330
7.3  Filtered Cardinal Series Interpolation......Page 335
7.3.1  Unbiased Interpolation from Jittered Samples......Page 336
7.3.2  In Stochastic Bandlimited Signal Interpolation......Page 338
7.3.2.1  NINV of Unbiased Restoration......Page 339
7.3.2.2  Examples......Page 340
7.4.1  An Error Bound......Page 341
7.5  Exercises......Page 345
7.6  Solutions for Chapter 7 Exercises......Page 347
8  Multidimensional Signal Analysis......Page 350
8.1  Notation......Page 351
8.2.1  N Dimensional Tic Tac Toe......Page 352
8.2.2  Solving Simultaneous Linear Equations and Matrix Manipulation in Higher Dimensions......Page 353
8.3  Continuous Time Multidimensional Fourier Analysis......Page 354
8.3.2  Multidimensional Convolution......Page 356
8.3.2.1  The Mechanics of Two Dimensional Convolution......Page 357
8.3.3  Separability......Page 359
8.3.4.1  Transposition......Page 361
8.3.4.3  Rotation......Page 362
8.3.4.5  Rotation in Higher Dimensions......Page 363
8.3.4.6  E®ects of Rotation, Scale and Transposition on Fourier Transformation......Page 365
8.3.5  Fourier Transformation of Circularly Symmetric Functions......Page 367
8.3.5.1  The Two Dimensional Fourier Transform of a Circle......Page 368
8.3.5.2  Polar Representation of Two Dimensional Functions that are Not Circularly Symmetric......Page 369
8.4  Characterization of Signals from their Tomographic Projections......Page 370
8.4.1  The Abel Transform and Its Inverse......Page 371
8.4.2  The Central Slice Theorem......Page 373
8.4.3  The Radon Transform and Its Inverse......Page 375
8.5.1  Multidimensional Periodicity......Page 378
8.5.2  The Multidimensional Fourier Series Expansion......Page 381
8.5.3  Multidimensional Discrete Fourier Transforms......Page 385
8.5.3.2  The Role of Phase in Image Characterization......Page 386
8.6  Discrete Cosine Transform-Based Image Coding......Page 387
8.6.2  The DCT in image compression......Page 388
8.7  McClellan Transformation for Filter Design......Page 391
8.7.2  Implementation Issues......Page 395
8.8  The Multidimensional Sampling Theorem......Page 402
8.8.1  The Nyquist Density......Page 404
8.8.1.1  Circular and Rectangular Spectral Support......Page 405
8.8.1.2  Sampling Density Comparisons......Page 407
8.8.2.1  Tightening the Integration Region......Page 408
8.8.2.2  Allowing Slower Roll O®......Page 410
8.9.1  Restoration Formulae......Page 411
8.9.1.1  Lost Sample Restoration Theorem......Page 412
8.9.2  Noise Sensitivity......Page 413
8.9.2.2  Deleting Samples from Optical Images......Page 415
8.10  Periodic Sample Decimation and Restoration......Page 417
8.10.1  Preliminaries......Page 418
8.10.2  First Order Decimated Sample Restoration......Page 421
8.10.3.1  The Square Doughnut......Page 423
8.10.4  Higher Order Decimation......Page 424
8.11  Raster Sampling......Page 425
8.11.1  Bandwidth Equivalence of Line Samples......Page 427
8.12  Exercises......Page 431
8.13  Solutions for Chapter 8 Exercises......Page 439
9  Time-Frequency Representations......Page 447
9.1  Short Time Fourier Transforms and Spectrograms......Page 448
9.1.2  The Mechanics of Short Time Fourier Transformation......Page 451
9.1.2.1  The Time Resolution Versus Frequency Resolution Trade O®......Page 452
9.1.3  Computational Architectures......Page 453
9.1.3.1  Modulated Inputs......Page 454
9.1.3.2  Window Design Using Truncated IIR Filters......Page 455
9.1.3.3  Modulated Windows......Page 458
9.2  Generalized Time Frequency Representations......Page 460
9.2.2  Kernel Properties......Page 462
9.2.3  Marginals......Page 464
9.2.3.1  Kernel Constraints......Page 465
9.2.4.1  The Spectrogram......Page 469
9.2.4.3  Kernel Synthesis Using Spectrogram Superposition......Page 470
9.2.4.4  The Cone Shaped Kernel......Page 471
9.2.4.5  Cone Kernel GTFR Implementation Using Short Time Fourier Transforms......Page 472
9.2.4.6  Cone Kernel GTFR Implementation for Real Signals......Page 475
9.2.4.7  Kernel Synthesis Using POCS......Page 476
9.3  Exercises......Page 478
9.4  Solutions for Chapter 9 Exercises......Page 481
10.1  Continuous Sampling......Page 485
10.2  Interpolation From Periodic Continuous Samples......Page 487
10.2.1  The Restoration Algorithm......Page 489
10.2.1.1  Trigonometric Polynomials......Page 492
10.2.1.2  Noise Sensitivity......Page 495
10.2.1.3  Colored Noise......Page 497
10.2.2.1  Comparison with the NINV of the Cardinal Series......Page 500
10.2.3  Application to Interval Interpolation......Page 501
10.3.1  Problem Description......Page 506
10.3.1.2  Interpolation......Page 508
10.3.1.3  The A[N; P] Matrix......Page 511
10.3.3  Quadrature Version......Page 512
10.4  Prolate Spheroidal Wave Functions......Page 513
10.4.0.1  Properties......Page 514
10.4.1  Application to Extrapolation......Page 516
10.4.2  Application to Interval Interpolation......Page 518
10.5.1  The Basic Algorithm......Page 519
10.5.2  Proof of the PGA using PSWF\'s......Page 522
10.5.3  Remarks......Page 524
10.6  Exercises......Page 525
10.7  Solutions for Chapter 10 Exercises......Page 531
11.1  Geometical POCS......Page 541
11.1.1  Geometrical Convex Sets......Page 542
11.1.3  POCS......Page 543
11.2  Convex Sets of Signals......Page 546
11.2.1  The Hilbert Space......Page 547
11.2.1.2  Subspaces......Page 549
11.2.1.5  Cones......Page 550
11.2.2  Some Commonly Used Convex Sets of Signals......Page 551
11.2.2.1  Matrix equations......Page 552
11.2.2.3  Duration Limited......Page 554
11.2.2.5  Constant Area......Page 555
11.2.2.8  Bounded Signals......Page 558
11.3.2  Solution of Simultaneous Equations......Page 560
11.3.3  The Papoulis-Gerchberg Algorithm......Page 561
11.3.4  Howard\'s Minimum-Negativity-Constraint Algorithm......Page 563
11.3.5.1  Template Matching......Page 564
11.3.5.3  POCS Associative Memory Examples......Page 565
11.3.5.4  POCS Associative Memory Convergence......Page 567
11.3.5.6  Heteroassociative Memory......Page 570
11.3.6  Recovery of Lost Image Blocks......Page 571
11.3.7  Subpixel Resolution......Page 588
11.3.8  Reconstruction of Images from Tomographic Projections......Page 591
11.3.9  Correcting Quantization Error for Oversampled Bandlimited Signals......Page 592
11.3.10.1  GTFR Constraints as Convex Sets......Page 594
11.3.10.2  GTFR Kernel Synthesis Using POCS......Page 599
11.3.11  Application to Conformal Radiotherapy......Page 602
11.3.11.1  Convex Constraint Sets......Page 605
11.3.11.2  Example Applications......Page 614
11.4.2  Contractive and Nonexpansive Operators......Page 615
11.4.2.1  Contractive and Nonexpansive Functions......Page 616
11.5  Exercises......Page 623
11.6  Solutions for Chapter 11 Exercises......Page 628
12.1.1  Minkowski Arithmetic......Page 631
12.1.2  Relation of Convolution Support to the Operation of Dilation......Page 632
12.1.3  Other Morphological Operations......Page 634
12.1.4  Minkowski Algebra......Page 636
12.2  Fourier and Signal Analysis on Time Scales......Page 639
12.2.1  Background......Page 640
12.2.1.1  Fourier Transforms on a Time Scale......Page 641
12.2.2  The Additive Closure of Two Time Scales......Page 642
12.2.3  Convolution on a Time Scale......Page 643
12.2.3.1  Convolution on Discrete Time Scales......Page 644
12.2.3.2  Time Scales in Deconvolution......Page 661
12.2.4  Additively Idempotent Time Scales......Page 663
12.2.4.2  AITS Examples......Page 664
12.2.4.4  Asymptotic Graininess of A»´......Page 665
12.2.5.1  Applications......Page 669
12.2.6  Multidimensional Time Scales......Page 672
12.3  Exercises......Page 680
12.4  Solutions for Chapter 12 Exercises......Page 681
13.1.1  The Wave Equation......Page 682
13.1.2  The Fourier Series Solution......Page 683
13.1.3  The Fourier Series and Western Harmony......Page 684
13.1.4  Pythagorean Harmony......Page 686
13.1.4.1  Melodies of Harmonics: Bugle Tunes......Page 688
13.1.4.2  Pythagorean and Tempered String Vibrations......Page 689
13.1.5  Harmonics Expansions Produce Major Chords and the Major Scale......Page 690
13.1.6  Fret Calibration......Page 691
13.2.1  Derivation of the Di®raction Integral from a System\'s Approach......Page 692
13.2.2.1  Vector Description......Page 693
13.2.2.2  Solution of the Wave Equation......Page 695
13.2.2.4  The Fresnel Di®raction Integral......Page 697
13.2.2.5  The Fraunhofer Approximation......Page 698
13.2.3.1  Analysis......Page 701
13.2.3.2  Imaging System......Page 703
13.2.4  Fourier Transformation......Page 704
13.2.5  Implementation of the PGA Using Optics......Page 705
13.3  Heisenberg\'s Uncertainty Principle......Page 706
13.4  Elementary Deterministic Finance......Page 708
13.4.2  Compound Interest on a One Time Deposit......Page 709
13.4.2.2  The case of continuous compounding......Page 710
13.4.2.3  Di®erent rates and compounding periods - same yield......Page 711
13.4.2.5  E®ect of Annual Taxes......Page 712
13.4.2.6  E®ect of In°ation......Page 713
13.4.3  Compound Interest With Constant Periodic Deposits......Page 714
13.4.3.2  Be a Millionaire......Page 715
13.4.3.3  Starting With a Nest Egg......Page 716
13.4.4.1  Monthly Payment......Page 717
13.4.4.3  Monthly Payment Over Long Periods of Time......Page 718
13.5  Exercises......Page 727
13.6  Solutions for Chapter 13 Exercises......Page 728
14.1  Scharz\'s Inequality......Page 729
14.4.1  Binomial Series......Page 730
14.4.2.1  Derivation......Page 731
14.5  Ill-Conditioned Matrices......Page 732
14.6.1  The Pareto Random Variable......Page 734
14.6.3  The Chi Random Variable......Page 735
14.6.4  The Noncentral Chi-Squared Random Variable......Page 736
14.6.5  The Half Normal Random Variable......Page 737
14.6.7  The Maxwell Random Variable......Page 738
14.6.8  The Log Random Variable......Page 739
14.6.10  The Uniform Product Variable......Page 740
14.6.12  The Logistic Random Variable......Page 741
14.6.14  The F Random Variable......Page 742
14.6.15  The Noncentral F Random Variable......Page 743
14.6.17  The Gumbel Random Variable......Page 744
14.6.19  The Noncentral Student\'s t Random Variable......Page 745
14.6.21  The Planck\'s Radiation Random Variable......Page 746
14.6.23  The Generalized Cauchy Random Variable......Page 747
15  Reference......Page 750
15.1  Acronym List......Page 751
15.2  Notation......Page 752
15.3  References......Page 753
15.4  Index......Page 796