Advanced Kalman Filtering, Least-Squares and Modeling: A Practical Handbook

ADVANCED KALMAN FILTERING, LEAST-SQUARES AND MODELING......Page 5
CONTENTS......Page 9
PREFACE......Page 17
CHAPTER 1: INTRODUCTION......Page 23
1.1 THE FORWARD AND INVERSE MODELING PROBLEM......Page 24
1.2 A BRIEF HISTORY OF ESTIMATION......Page 26
1.3 FILTERING, SMOOTHING , AND PREDICTION......Page 30
1.5 NOTATION......Page 31
1.6 SUMMARY......Page 33
CHAPTER 2: SYSTEM DYNAMICS AND MODELS......Page 35
2.1 DISCRETE - TIME MODELS......Page 36
2.2 CONTINUOUS - TIME DYNAMIC MODELS......Page 39
2.2.1 State Transition and Process Noise Covariance Matrices......Page 41
2.2.2 Dynamic Models Using Basic Function Expansions......Page 44
2.2.3 Dynamic Models Derived from First Principles......Page 47
2.2.4 Stochastic (Random) Process Models......Page 53
2.2.5 Linear Regression Models......Page 64
2.2.6 Reduced - Order Modeling......Page 66
2.3.1 Numeric Computation of Φ......Page 67
2.3.2 Numeric Computation of QD......Page 79
2.4 MEASUREMENT MODELS......Page 80
2.5 SIMULATING STOCHASTIC SYSTEMS......Page 82
2.6 COMMON MODELING ERRORS AND SYSTEM BIASES......Page 84
2.7 SUMMARY......Page 87
3.1 ANGLE - ONLY TRACKING OF LINEAR TARGET MOTION......Page 89
3.2.1 Maneuvering Tank Tracking Using Multiple Models......Page 91
3.2.2 Aircraft Tracking......Page 95
3.3 STRAPDOWN INERTIAL NAVIGATION SYSTEM ( INS ) ERROR MODEL......Page 96
3.4 SPACECRAFT ORBIT DETERMINATION ( OD )......Page 102
3.4.1 Geopotential Forces......Page 105
3.4.2 Other Gravitational Attractions......Page 108
3.4.3 Solar Radiation Pressure......Page 109
3.4.4 Aerodynamic Drag......Page 110
3.4.6 Earth Motion......Page 111
3.4.7 Numerical Integration and Computation of Φ......Page 112
3.4.8 Measurements......Page 114
3.4.9 GOES I - P Satellites......Page 118
3.4.10 Global Positioning System ( GPS )......Page 119
3.6 SUMMARY......Page 121
4.1 LEAST - SQUARES DATA FITTING......Page 123
4.2 WEIGHTED LEAST SQUARES......Page 130
4.3.1 Bayesian Least Squares......Page 137
4.3.2 Bayes ’ Theorem......Page 139
4.3.3 Minimum Variance or Minimum Mean-Squared Error (MMSE)......Page 143
4.3.4 Orthogonal Projections......Page 146
4.4 PROBABILISTIC APPROACHES — MAXIMUM LIKELIHOOD AND MAXIMUM A POSTERIORI......Page 147
4.4.1 Gaussian Random Variables......Page 148
4.4.2 Maximum Likelihood Estimation......Page 150
4.4.3 Maximum A Posteriori......Page 155
4.5 SUMMARY OF LINEAR ESTIMATION APPROACHES......Page 159
5.1.1 Vector - Matrix Norms......Page 161
5.1.3 Condition Number......Page 163
5.2.1 Computation of the Normal Equations......Page 167
5.2.2 Cholesky Decomposition of the Normal Equations......Page 171
5.3 ORTHOGONAL TRANSFORMATIONS AND THE QR METHOD......Page 178
5.3.1 Givens Rotations......Page 180
5.3.2 Householder Transformations......Page 181
5.3.3 Modified Gram - Schmidt ( MGS ) Orthogonalization......Page 184
5.4 LEAST - SQUARES SOLUTION USING THE SVD......Page 187
5.5.1 Sparse Array Storage......Page 189
5.5.2 Linear Iteration......Page 190
5.5.3 Least - Squares Solution for Large Sparse Problems Using Krylov Space Methods......Page 191
5.6.1 Solution Accuracy for Polynomial Problem......Page 197
5.6.2 Algorithm Timing......Page 203
5.7 SOLUTION UNIQUENESS, OBSERVABILITY, AND CONDITION NUMBER......Page 205
5.8 PSEUDO - INVERSES AND THE SINGULAR VALUE TRANSFORMATION ( SVD )......Page 207
5.9 SUMMARY......Page 212
CHAPTER 6: LEAST - SQUARES ESTIMATION: MODEL ERRORS AND MODEL ORDER......Page 215
6.1.1 Residual Sum - of - Squares (SOS)......Page 216
6.1.2 Residual Patterns......Page 217
6.1.4 Measurement Prediction......Page 218
6.1.5 Estimate Comparison......Page 219
6.2.1 State Error Covariance and Confi dence Bounds......Page 230
6.2.2 Model Error Analysis......Page 234
6.3 REGRESSION ANALYSIS FOR WEIGHTED LEAST SQUARES......Page 259
6.3.1 Analysis of Variance......Page 260
6.3.2 Stepwise Regression......Page 261
6.3.3 Prediction and Optimal Data Span......Page 266
6.4 SUMMARY......Page 267
7.1.1 Least - Squares with Linear Equality Constraints (Problem LSE )......Page 271
7.1.2 Least - Squares with Linear Inequality Constraints (Problem LSI )......Page 278
7.2 RECURSIVE LEAST SQUARES......Page 279
7.3 NONLINEAR LEAST SQUARES......Page 281
7.3.1 1 - D Nonlinear Least - Squares Solutions......Page 285
7.3.2 Optimization for Multidimensional Unconstrained Nonlinear Least Squares......Page 286
7.3.3 Stopping Criteria and Convergence Tests......Page 291
7.4.1 De - Weighting Large Residuals......Page 304
7.4.2 Data Editing......Page 305
7.5 MEASUREMENT PREPROCESSING......Page 307
7.6 SUMMARY......Page 308
CHAPTER 8: KALMAN FILTERING......Page 311
8.1.1 Truth Model......Page 312
8.1.2 Discrete - Time Kalman Filter Algorithm......Page 313
8.2.1 Correlation between Measurement and Process Noise......Page 325
8.2.2 Time - Correlated (Colored) Measurement Noise......Page 327
8.2.3 Innovations, Model Validation, and Editing......Page 333
8.3 CONTINOUS - TIME KALMAN - BUCY FILTER......Page 336
8.4.1 Friedland Bias - Free/Bias - Restoring Filter......Page 343
8.4.2 Kalman - Schmidt Consider Filter......Page 347
8.5 STEADY - STATE SOLUTION......Page 350
8.6 WIENER FILTER......Page 354
8.6.1 Wiener - Hopf Equation......Page 355
8.6.2 Solution for the Optimal Weighting Function......Page 357
8.6.3 Filter Input Covariances......Page 358
8.6.4 Equivalence of Weiner and Steady - State Kalman - Bucy Filters......Page 359
8.7 SUMMARY......Page 363
CHAPTER 9: FILTERING FOR NONLINEAR SYSTEMS, SMOOTHING, ERROR ANALYSIS/MODEL DESIGN, AND MEASUREMENT PREPROCESSING......Page 365
9.1.1 Linearized and Extended Kalman Filters......Page 366
9.1.2 Iterated Extended Kalman Filter......Page 371
9.2 SMOOTHING......Page 374
9.2.1 Fixed - Point Smoother......Page 375
9.2.2 Fixed - Lag Smoother......Page 378
9.2.3 Fixed - Interval Smoother......Page 379
9.3 FILTER ERROR ANALYSIS AND REDUCED - ORDER MODELING......Page 392
9.3.1 Linear Analysis of Independent Error Sources......Page 394
9.3.2 Error Analysis for ROM Defi ned as a Transformed Detailed Model......Page 402
9.3.3 Error Analysis for Different Truth and Filter Models......Page 404
9.5 SUMMARY......Page 407
CHAPTER 10: FACTORED (SQUARE - ROOT) FILTERING......Page 411
10.1 FILTER NUMERICAL ACCURACY......Page 412
10.2 U - D FILTER......Page 414
10.2.1 U - D Filter Measurement Update......Page 416
10.2.2 U - D Filter Time Update......Page 418
10.2.3 RTS Smoother for U - D Filter......Page 423
10.2.4 U - D Error Analysis......Page 425
10.3 SQUARE ROOT INFORMATION FILTER ( SRIF )......Page 426
10.3.1 SRIF Time Update......Page 427
10.3.2 SRIF Measurement Update......Page 429
10.3.3 Square Root Information Smoother ( SRIS )......Page 430
10.3.5 SRIF Error Analysis......Page 432
10.4 INERTIAL NAVIGATION SYSTEM ( INS ) EXAMPLE USING FACTORED FILTERS......Page 434
10.5 LARGE SPARSE SYSTEMS AND THE SRIF......Page 439
10.6 SPATIAL CONTINUITY CONSTRAINTS AND THE SRIF DATA EQUATION......Page 441
10.6.1 Flow Model......Page 443
10.6.2 Log Conductivity Spatial Continuity Model......Page 444
10.6.4 SRIF Processing......Page 446
10.6.5 Steady - State Flow Constrained Iterative Solution......Page 447
10.7 SUMMARY......Page 449
CHAPTER 11: ADVANCED FILTERING TOPICS......Page 453
11.1 MAXIMUM LIKELIHOOD PARAMETER ESTIMATION......Page 454
11.1.1 Calculation of the State Transition Partial Derivatives......Page 456
11.1.2 Derivatives of the Filter Time Update......Page 460
11.1.3 Derivatives of the Filter Measurement Update......Page 461
11.1.4 Partial Derivatives for Initial Condition Errors......Page 462
11.1.5 Computation of the Log Likelihood and Scoring Step......Page 463
11.2 ADAPTIVE FILTERING......Page 471
11.3 JUMP DETECTION AND ESTIMATION......Page 472
11.3.1 Jump - Free Filter Equations......Page 474
11.3.2 Stepwise Regression......Page 476
11.3.3 Correction of Jump - Free Filter State......Page 477
11.3.4 Real - Time Jump Detection Using Stepwise Regression......Page 478
11.4 ADAPTIVE TARGET TRACKING USING MULTIPLE MODEL HYPOTHESES......Page 483
11.4.1 Weighted Sum of Filter Estimates......Page 484
11.4.2 Maximum Likelihood Filter Selection......Page 485
11.4.3 Dynamic and Interactive Multiple Models......Page 486
11.6 ROBUST ESTIMATION: H - INFINITY FILTERS......Page 493
11.7 UNSCENTED KALMAN FILTER ( UKF )......Page 496
11.7.1 Unscented Transform......Page 497
11.7.2 UKF Algorithm......Page 500
11.8 PARTICLE FILTERS......Page 507
11.9 SUMMARY......Page 512
CHAPTER 12: EMPIRICAL MODELING......Page 515
12.1 EXPLORATORY TIME SERIES ANALYSIS AND SYSTEM IDENTIFICATION......Page 516
12.2 SPECTRAL ANALYSIS BASED ON THE FOURIER TRANSFORM......Page 517
12.2.1 Fourier Series for Periodic Functions......Page 519
12.2.2 Fourier Transform of Continuous Energy Signals......Page 520
12.2.3 Fourier Transform of Power Signals......Page 524
12.2.4 Power Spectrum of Stochastic Signals......Page 526
12.2.5 Time - Limiting Window Functions......Page 528
12.2.6 Discrete Fourier Transform......Page 531
12.2.7 Periodogram Computation of Power Spectra......Page 534
12.2.8 Blackman - Tukey (Correlogram) Computation of Power Spectra......Page 536
12.3 AUTOREGRESSIVE MODELING......Page 544
12.3.1 Maximum Entropy Method ( MEM )......Page 546
12.3.2 Burg MEM......Page 547
12.3.3 Final Prediction Error ( FPE ) and Akaike Information Criteria ( AIC )......Page 548
12.3.4 Marple AR Spectral Analysis......Page 550
12.3.5 Summary of MEM Modeling Approaches......Page 551
12.4 ARMA MODELING......Page 553
12.4.1 ARMA Parameter Estimation......Page 554
12.5 CANONICAL VARIATE ANALYSIS......Page 556
12.5.1 CVA Derivation and Overview......Page 558
12.5.2 Summary of CVA Steps......Page 561
12.5.3 Sample Correlation Matrices......Page 562
12.5.4 Order Selection Using the AIC......Page 563
12.5.5 State - Space Model......Page 565
12.5.6 Measurement Power Spectrum Using the State - Space Model......Page 566
12.6 CONVERSION FROM DISCRETE TO CONTINUOUS MODELS......Page 570
12.7 SUMMARY......Page 573
A.1.2 Matrices......Page 577
A.2.3 Inner (Dot) Product of Vectors......Page 579
A.3.1 Matrix Inverse......Page 580
A.3.2 Partitioned Matrix Inversion......Page 581
A.3.3 Matrix Inversion Identity......Page 582
A.3.4 Determinant......Page 583
A.3.5 Matrix Trace......Page 584
A.3.6 Derivatives of Matrix Functions......Page 585
A.3.7 Norms......Page 586
A.4.2 Cholesky Factorization......Page 587
A.4.5 Singular Value Decomposition ( SVD )......Page 588
A.4.6 Pseudo - Inverse......Page 589
A.4.7 Condition Number......Page 590
B.1.1 Definitions......Page 591
B.1.2 Joint and Conditional Probability, and Independence......Page 592
B.2.1 Distribution and Density Functions......Page 593
B.2.2 Bayes ’ Theorem for Density Functions......Page 594
B.2.3 Moments of Random Variables......Page 595
B.2.5 Chi - Squared Distribution......Page 596
B.3 STOCHASTIC PROCESSES......Page 597
B.3.2 Markov Process......Page 598
B.3.3 Differential and Integral Equations with White Noise Inputs......Page 599
BIBLIOGRAPHY......Page 601
INDEX......Page 621