Quantitative Methods of Data Analysis for the Physical Sciences and Engineering

Contents......Page 6
Preface......Page 11
Acknowledgments......Page 13
Part I. Fundamentals......Page 14
1.2 Data Nomenclature......Page 15
1.3 Representing Discrete Data and Functions as Vectors......Page 17
1.4.2 Range......Page 18
1.4.3 Frequency......Page 19
1.5.1 Instrument Error......Page 20
1.5.2 Experimental/Observational Error......Page 21
1.5.3 Digital Representation and Computational Errors......Page 23
1.6 Practical Issues......Page 24
2.1 Overview......Page 27
2.2 Definitions......Page 28
2.3 Probability......Page 30
2.4.1 Discrete Probability Distributions......Page 31
2.4.2 Continuous Probability Distributions......Page 34
2.5.1 Discrete Joint Probability Distributions......Page 39
2.5.2 Continuous Joint Probability Distributions......Page 41
2.6.1 General......Page 43
2.6.2 Univariate Expectance......Page 45
2.6.3 Multivariate Expectance......Page 53
2.6.4 Moments of Nonlinear Functions of Random Variables......Page 57
2.7 Common Distributions and their Moments......Page 62
2.7.1 Normal Distribution......Page 63
2.7.2 Central Limit Theorem......Page 66
2.7.3 Binomial (or Bernoulli) Distribution......Page 69
2.7.4 Poisson Distribution......Page 70
2.8 Take-Home Points......Page 71
2.9 Questions......Page 72
3.2 Estimation......Page 74
3.3 Estimating the Distribution......Page 78
3.3.1 Outliers......Page 80
3.4.1 Estimating the Central Value of a Random Variable......Page 81
3.4.2 Estimating the Spread of a Random Variable......Page 83
3.5 Principle of Maximum Likelihood (An Important Principle)......Page 88
3.6 Interval Estimates......Page 92
3.6.1 Confidence Intervals......Page 93
3.7 Hypothesis Testing......Page 98
3.7.1 Level of Significance (α; or Alternatively, p)......Page 101
3.7.2 Testing Normal Distribution Means......Page 103
3.7.3 Degrees of Freedom......Page 105
3.7.4 Practical Considerations (Data Trolling)......Page 106
3.8.1 t-Distribution......Page 108
3.8.2 Chi-Squared (χ²) Distribution......Page 109
3.9 Take-Home Points......Page 112
3.10 Questions......Page 113
Part II. Fitting Curves to Data......Page 114
4.1.1 What Is Involved......Page 115
4.1.2 Interpolant Types......Page 117
4.1.3 Interpolation Schemes......Page 119
4.2 Piecewise Continuous Interpolants......Page 120
4.2.1 Piecewise Linear Interpolant......Page 121
4.2.2 Cubic Spline Interpolant......Page 124
4.2.3 Additional Types of Splines......Page 132
4.3.1 Continuous Polynomial Interpolation......Page 134
4.4 Take-Home Points......Page 136
4.5 Questions......Page 137
5.2 Introduction......Page 138
5.3 Functional Form of the Curve......Page 139
5.4.1 Nature of the Problem......Page 140
5.4.2 Defining Error......Page 141
5.4.3 Nature of Data: Influence on Defining Best Fit......Page 145
5.5.1 Standard Curve Fitting......Page 148
5.6 Orthogonal Fitting of a Straight Line......Page 168
5.7 Assessing Uncertainty in Optimal Parameter Values......Page 169
5.7.1 Significance of Best-Fit Parameters......Page 181
5.8.1 Appropriateness of the Curve......Page 182
5.8.2 Quality of Curve Fit......Page 183
5.10 Questions......Page 186
6.2 Weighted Curve Fits......Page 188
6.2.2 Matrix Form of Weighted Fits......Page 192
6.3 Constrained Fits......Page 195
6.3.1 Solution via Substitution......Page 196
6.3.2 Method of Lagrange Multipliers......Page 198
6.5 Regression/Calibration......Page 203
6.6 Correlation Coefficient......Page 205
6.6.1 Interpreting a Correlation Coefficient......Page 206
6.7 Take-Home Points......Page 210
6.8 Questions......Page 211
Part III. Sequential Data Fundamentals......Page 215
7.1 Overview......Page 216
7.2.1 Definitions and Assumptions......Page 218
7.2.2 Estimation......Page 223
7.3 Convolution......Page 231
7.4 Serial Correlation......Page 243
7.5 Take-Home Points......Page 258
7.6 Questions......Page 259
8.1 Overview......Page 261
8.3.1 Definitions and Concepts......Page 262
8.4.1 Interpolation with Fourier Sines and Cosines......Page 274
8.4.2 Interpreting the Fourier Series......Page 276
8.6 Questions......Page 279
9.2 Discrete Periodic Data......Page 280
9.2.1 Fourier Series in Summation Form......Page 282
9.2.2 A Most Excellent Form of the Discrete Fourier Series......Page 283
9.2.3 Fourier Frequencies......Page 284
9.2.4 Summation Properties and Orthogonality Conditions......Page 286
9.3 Discrete Sine and Cosine Transforms......Page 291
9.3.1 Interpretation of the Discrete Cosine and Sine Transforms......Page 294
9.4 Continuous Sine and Cosine Transforms......Page 297
9.5.1 Complex Numbers......Page 298
9.5.2 Orthogonality Conditions for Complex Form of Sines and Cosines......Page 299
9.5.3 Complex Discrete Fourier Transform......Page 300
9.5.4 Real-Valued Time Series......Page 304
9.6.1 General Continuous Fourier Transform......Page 305
9.6.2 Existence of the Fourier Integral (A Formality)......Page 308
9.7.1 Symmetry Property......Page 310
9.7.2 Linearity......Page 312
9.7.3 Scaling......Page 313
9.7.4 Time Shifting......Page 314
9.7.6 Symmetrical Function Transforms......Page 315
9.8.2 Integration Theorem......Page 320
9.8.3 Convolution Theorem (Major Importance)......Page 321
9.8.4 Autocovariance Theorem (Wiener–Khinchin Relationship)......Page 322
9.8.6 Parseval’s Theorem......Page 323
9.9 Fast Fourier Transform......Page 328
9.10 Take-Home Points......Page 329
9.11 Questions......Page 330
10.1 Overview......Page 331
10.2.1 A Sampling Theorem Derivation......Page 332
10.2.2 Aliasing......Page 339
10.3.1 Sampling the Time Series......Page 347
10.3.2 Truncating the Time Series......Page 348
10.3.3 Sampling the Spectrum......Page 349
10.3.4 Resulting Discrete Spectrum and Time Series......Page 351
10.3.5 Leakage......Page 353
10.4 Other Sampling Considerations......Page 356
10.6 Questions......Page 357
11.1 Overview......Page 359
11.2 Noise in the Spectrum......Page 360
11.3.1 Smoothing the Spectrum......Page 366
11.3.2 Confidence Intervals for Averaged Estimates......Page 376
11.3.3 Single-Realization Treatment......Page 384
11.3.4 Practical Considerations......Page 388
11.3.5 Least-Squares Spectral Estimates......Page 394
11.4 Spectral Estimation in Practice......Page 400
11.4.1 Sampling......Page 401
11.4.2 Smoothing......Page 402
11.4.3 Technique Used......Page 407
11.5.1 Generating Colored Noise Time Series......Page 409
11.6 Take-Home Points......Page 412
11.7 Questions......Page 413
12.2 Joint PDF Moments in the Time Domain......Page 415
12.2.1 Linear Causal Relationship......Page 417
12.3.1 Definitions and Interpretation......Page 423
12.4.1 Amplitude and Phase Spectrum Uncertainty, Unsmoothed......Page 428
12.4.2 Smoothing the Cross Spectrum......Page 429
12.5 Take-Home Points......Page 432
12.6 Questions......Page 433
13.1 Overview......Page 434
13.2.1 Transfer Function......Page 436
13.2.2 Phase Shifts and Causality......Page 438
13.3.1 Ideal Filters......Page 439
13.3.2 Cascaded Filters......Page 444
13.4 Practical Considerations......Page 445
13.6.1 Direct Solution......Page 446
13.6.2 Inverse Filtering......Page 450
13.7.1 Generalizing the Deconvolution Problem......Page 455
13.7.2 Truncated Deconvolution......Page 458
13.7.3 General Least-Squares Deconvolution......Page 459
13.8 Take-Home Points......Page 463
13.9 Questions......Page 464
14.1 Overview......Page 465
14.1.1 Two Fundamental Types of Parametric Model......Page 466
14.2.1 Definitions......Page 467
14.2.2 Statistical Moments of the General Linear Process......Page 469
14.2.3 Moving Average (MA) Process......Page 471
14.2.4 Autoregressive (AR) Process......Page 474
14.2.5 Mixed Autoregressive and Moving Average (ARMA) Process......Page 477
14.3 Model Identification and Solution......Page 478
14.3.1 Identifying a Moving Average Process......Page 480
14.3.2 Identifying an Autoregressive Process......Page 482
14.4.1 Parameters of an AR(p) Process......Page 485
14.4.2 Parameters of an MA(q) and ARMA(p,q) Process......Page 487
14.6.1 General......Page 488
14.6.2 Theoretical Parametric Spectral Representations......Page 489
14.6.3 Examples......Page 498
14.8 Questions......Page 500
14.9 Time Series References......Page 501
15.2 Introduction......Page 504
15.3.1 Fundamentals......Page 508
15.3.2 Orthogonality for Symmetrical Matrices......Page 512
15.4 Principal Components (PC)......Page 519
15.4.1 Definitions......Page 520
15.4.2 Solving for the PC Coefficients......Page 524
15.4.3 Interpretation and Use......Page 529
15.5 Singular Spectrum Analysis (SSA)......Page 533
15.7 Questions......Page 542
A1.2 Definitions......Page 544
A1.3 Basic Matrix Operations......Page 548
A1.4 Special Matrix Products......Page 554
A1.4.1 Determinant of a Matrix......Page 555
A1.5 Matrix \"Division\": Inverse Matrix......Page 557
A1.5.1 Solution of Simultaneous Equations......Page 558
A1.5.2 Additional Terms......Page 559
A1.6 Useful Properties......Page 562
A1.7.1 Vector Space and Basis......Page 563
A1.7.3 Quadratic Form......Page 564
A1.7.4 Ill-Conditioning......Page 567
A1.7.5 Orthogonal Decomposition......Page 569
A1.8.2 Statistical Moments of Random Matrices (Expectance Operations)......Page 576
A1.9 Matrix References......Page 579
A2.2.1 Instrument Error......Page 581
A2.2.2 Experimental/Observational Error......Page 582
A2.3.1 Variance of a Univariate Random Variable......Page 583
A2.3.2 Multivariate Expectance......Page 585
A2.3.3 Moments of Nonlinear Functions of Random Variables......Page 590
A2.3.4 Expectance with Random Vectors and Matrices......Page 594
A2.4 Bootstrap......Page 598
A2.4.1 Generating Your Random Data......Page 600
A2.5 Expectance Versus Bootstrap......Page 604
References......Page 605
Index......Page 608