System Identification: Theory for the User

Preface to the First Edition  xiv
Acknowledgments  xvi
Preface to the Second Edition  xviii
Operators and Notational Conventions  xix

•  1  Introduction  1
1.1  Dynamic Systems  1
1.2  Models  6
1.3  An Archetypical Problem—ARX Models and the Linear Least Squares
Method  8
1.4  The System Identification Procedure  13
1.5  Organization of the Book  14
1.6  Bibliography  16

part i:  systems and models

•  2  Time-Invariant Linear Systems  18
2.1  Impulse Responses, Disturbances, and Transfer Functions  18
2.2  Frequency-Domain Expressions  28
2.3  Signal Spectra  33
2.4  Single Realization Behavior and Ergodicity Results (*)  42
2.5  Multivariable Systems (*)  44
2.6  Summary  45
2.7  Bibliography  46
2.8  Problems  47
Appendix 2A: Proof of Theorem 2.2  52
Appendix 2B: Proof of Theorem 2.3  55
Appendix 2C: Covariance Formulas  61

•  3  Simulation and Prediction 63
3.1  Simulation  63
3.2  Prediction  64
3.3  Observers  72
3.4  Summary  75
3.5  Bibliography  75
3.6  Problems  76

•  4  Models of Linear Time-Invariant Systems  79
4.1  Linear Models and Sets of Linear Models  79
4.2  A Family of Transfer-Function Models  81
4.3  State-Space Models  93
4.4  Distributed Parameter Models (*)  103
4.5  Model Sets. Model Structures, and Identifiabilitv: Some Formal Aspects
(*)  105
4.6  Identifiabilitv of Some Model Structures  114
4.7  Summary  118
4.8  Bibliography  119
4.9  Problems  121
Appendix 4A: Identifiabilitv of Black-Box Multivariable Model
Structures  128

•  5  Models for Time-varying and Nonlinear Systems  140
5.1  Linear Time-Varying Models  140
5.2  Models with Nonlinearities  143
5.3  Nonlinear State-Space Models  146
5.4  Nonlinear Black-Box Models: Basic Principles  148
5.5  Nonlinear Black-Box Models: Neural Networks, Wavelets and Classical
Models  154
5.6  Fuzzy Models  156
5.7  Formal Characterization of Models (*)  161
5.8  Summary  164
5.9  Bibliography  165
5.10 Problems  165

part ii:  methods

•  6  Nonparametric Time- and Frequency-Domain Methods  168
6.1  Transient-Response Analysis and Correlation Analysis  168
6.2  Frequency-Response Analysis  170
6.3  Fourier Analysis  173
6.4  Spectral Analysis  178
6.5  Estimating the Disturbance Spectrum (*)  187
6.6  Summary  189
6.7  Bibliography  190
6.8  Problems  191
Appendix 6A: Derivation of the Asymptotic Properties of the Spectral
Analysis Estimate  194

•  7  Parameter Estimation Methods  197
7.1  Guiding Principles Behind Parameter Estimation Methods  197
7.2  Minimizing Prediction Errors  199
7.3  Linear Regressions and the Least-Squares Method  203
7.4  A Statistical Framework for Parameter Estimation and the Maximum
Likelihood Method  212
7.5  Correlating Prediction Errors with Past Data  222
7.6  Instrumental-Variable Methods  224
7.7  Using Frequency Domain Data to Fit Linear Models (*)  227
7.8  Summary  233
7.9  Bibliography  234
7.10 Problems  236
Appendix 7A: Proof of the Cramer-Rao Inequality  245

•  8  Convergence and Consistency  247
8.1  Introduction  247
8.2  Conditions on the Data Set  249
8.3  Prediction-Error Approach  253
8.4  Consistency and Identifiabilitv  258
8.5  Linear Time-Invariant Models: A Frequency-Domain Description of the
Limit Model  263
8.6  The Correlation Approach  269
8.7  Summary  273
8.8  Bibliography  274
8.9  Problems  275

•  9  Asymptotic Distribution of Parameter Estimates
9.1  Introduction  280
9.2  The Prediction-Error Approach: Basic Theorem  281
9.3  Expressions for the Asymptotic Variance  283
9.4  Frequency-Domain Expressions for the Asymptotic Variance  290
9.5  The Correlation Approach  296
9.6  Use and Relevance of Asymptotic Variance Expressions  302
9.7  Summary  304
9.8  Bibliography  305
9.9  Problems  305
Appendix 9A: Proof of Theorem 9.1  309
Appendix 9B: The Asymptotic Parameter Variance  313

•  10  Computing the Estimate  317
10.1  Linear Regressions and Least Squares  317
10.2  Numerical Solution by Iterative Search Methods  326
10.3  Computing Gradients  329
10.4  Two-Stage and Multistage Methods  333
10.5  Local Solutions and Initial Values  338
10.6  Subspace Methods for Estimating State Space Models  340
10.7  Summary  351
10.8  Bibliography  352
10.9  Problems  353

•  11  Recursive Estimation Methods  361
11.1  Introduction  361
11.2  The Recursive Least-Squares Algorithm  363
11.3  The Recursive IV Method  369
11.4  Recursive Prediction-Error Methods  370
11.5  Recursive Pseudolinear Regressions  374
11.6  The Choice of Updating Step  376
11.7  Implementation  382
11.8  Summary  386
11.9  Bibliography  387
11.10 Problems  388
Appendix 11 A: Techniques for Asymptotic Analysis of Recursive
Algorithms  389
11A Problems  398

•  12  Options and Objectives
12.1  Options  399
12.2  Objectives  400
12.3  Bias and Variance  404
12.4  Summary  406
12.5  Bibliography  406
12.6  Problems  406

•  13  Experiment Design  408
13.1  Some General Considerations  408
13.2  Informative Experiments  411
13.3  Input Design for Open Loop Experiments  415
13.4  Identification in Closed Loop: Identifiabilitv  428
13.5  Approaches to Closed Loop Identification  434
13.6  Optimal Experiment Design for High-Order Black-Box Models  441
13.7  Choice of Sampling Interval and Presampling Filters  444
13.8  Summary  452
13.9  Bibliography  453
13.10 Problems  454

•  14  Preprocessing Data  458
14.1  Drifts and Detrending  458
14.2  Outliers and Missing Data  461
14.3  Selecting Segments of Data and Merging Experiments  464
14.4  Prefiltering  466
14.5  Formal Design of Prefiltering and Input Properties  470
14.6  Summary  474
14.7  Bibliography  475
14.8  Problems  475

•  15  Choice of Identification Criterion  477
15.1  General Aspects  477
15.2  Choice of Norm: Robustness  479
15.3  Variance-Optimal Instruments  485
15.4  Summary  488

part iii:  user\'s choices

15.5  Bibliography  489
15.6  Problems  490

•  16  Model Structure Selection and Model Validation  491
16.1  General Aspects of the Choice of Model Structure  491
16.2  A Priori Considerations  493
16.3  Model Structure Selection Based on Preliminary Data Analysis  495
16.4  Comparing Model Structures  498
16.5  Model Validation  509
16.6  Residual Analysis  511
16.7  Summary  516
16.8  Bibliography  517
16.9  Problems  518

•  17  System Identification in Practice  520
17.1  The Tool: Interactive Software  520
17.2  The Practical Side of System Identification  522
17.3  Some Applications  525
17.4  What Does System Identification Have To Offer?  536

•  Appendix I  Some Concepts From Probability Theory  539

•  Appendix II  Some Statistical Techniques for Linear Regressions  543
II.1  Linear Regressions and the Least Squares Estimate  543
II.2  Statistical Properties of the Least-Squares Estimate  551
II.3  Some Further Topics in Least-Squares Estimation  559
II.4  Problems  564

References  565
Subject Index  596
Reference Index  603