Time Series Analysis, Fourth Edition

Title Page......Page 3
Copyright\r......Page 4
Contents......Page 6
Preface to the Fourth Edition......Page 20
Preface to the Third Edition......Page 22
CHAPTER ONE Introduction......Page 24
1.1.1 Forecasting Time Series......Page 25
1.1.2 Estimation of Transfer Functions......Page 26
1.1.3 Analysis of Effects of Unusual Intervention Events to a System......Page 27
1.1.5 Discrete Control Systems......Page 28
1.2.1 Stationary and Nonstationary Stochastic Models for Forecasting and Control......Page 30
1.2.2 Transfer Function Models......Page 35
1.2.3 Models for Discrete Control Systems......Page 37
1.3.1 Parsimony......Page 39
1.3.2 Iterative Stages in the Selection of a Model......Page 40
PART ONE Stochastic Models andTheir Forecasting\r......Page 42
Time Series......Page 43
2.1.2 Stationary Stochastic Processes......Page 46
2.1.3 Positive Definiteness and the Autocovariance Matrix......Page 47
2.1.4 Autocovariance and Autocorrelation Functions......Page 51
2.1.5 Estimation of Autocovariance and Autocorrelation Functions......Page 53
2.1.6 Standard Errors of Autocorrelation Estimates......Page 55
2.2.1 Periodogram of a Time Series......Page 57
2.2.2 Analysis of Variance......Page 59
2.2.3 Spectrum and Spectral Density Function......Page 60
2.2.4 Simple Examples of Autocorrelation and Spectral Density Functions......Page 65
APPENDIX A2.1 LINK BETWEEN THE SAMPLE SPECTRUM AND AUTOCOVARIANCE FUNCTION ESTIMATE......Page 67
3.1.1 Two Equivalent Forms for the Linear Process......Page 69
3.1.2 Autocovariance Generating Function of a Linear Process......Page 72
Stationarity......Page 73
3.1.4 Autoregressive and Moving Average Processes......Page 75
3.2.1 Stationarity Conditions for Autoregressive Processes......Page 77
Autocorrelation Function......Page 79
3.2.3 First-Order Autoregressive (Markov) Process......Page 81
Stationarity Condition......Page 83
3.2.5 Partial Autocorrelation Function......Page 88
3.2.6 Estimation of the Partial Autocorrelation Function......Page 91
3.2.7 Standard Errors of Partial Autocorrelation Estimates......Page 92
3.3.1 Invertibility Conditions for Moving Average Processes......Page 93
Autocorrelation Function......Page 94
3.3.3 First-Order Moving Average Process......Page 95
Invertibility Conditions......Page 97
3.3.5 Duality Between Autoregressive and Moving Average Processes......Page 100
3.4.1 Stationarity and Invertibility Properties......Page 101
Autocorrelation Function......Page 102
3.4.3 First-Order Autoregressive–First-Order Moving Average Process......Page 104
Autocovariances......Page 108
APPENDIX A3.2 RECURSIVE METHOD FOR CALCULATING ESTIMATES OF AUTOREGRESSIVE PARAMETERS......Page 111
4.1.1 Nonstationary First-Order Autoregressive Process......Page 114
4.1.2 General Model for a Nonstationary Process Exhibiting Homogeneity......Page 116
4.1.3 General Form of the Autoregressive Integrated Moving Average Model......Page 121
4.2.1 Difference Equation Form of the Model......Page 124
Model in Terms of Current and Previous Shocks......Page 125
Model in Terms of Previous z’s and the Current Shock at......Page 132
4.3 INTEGRATED MOVING AVERAGE PROCESSES......Page 135
4.3.1 Integrated Moving Average Process of Order (0, 1, 1)......Page 136
4.3.2 Integrated Moving Average Process of Order (0, 2, 2)......Page 140
4.3.3 General Integrated Moving Average Process of Order (0, d, q)......Page 144
APPENDIX A4.1 LINEAR DIFFERENCE EQUATIONS......Page 146
Evaluation of the Complementary Function......Page 148
APPENDIX A4.3 ARIMA PROCESSES WITH ADDED NOISE......Page 152
A4.3.1 Sum of Two Independent Moving Average Processes......Page 153
A4.3.2 Effect of Added Noise on the General Model......Page 154
A4.3.3 Example for an IMA(0, 1, 1) Process with Added White Noise......Page 155
A4.3.5 Autocovariance Function of the General Model with Added Correlated Noise......Page 156
5.1 MINIMUM MEAN SQUARE ERROR FORECASTS AND THEIR PROPERTIES......Page 158
5.1.1 Derivation of the Minimum Mean Square Error Forecasts......Page 160
5.1.2 Three Basic Forms for the Forecast......Page 162
5.2.1 Convenient Format for the Forecasts......Page 166
5.2.2 Calculation of the ψ Weights......Page 168
5.2.3 Use of the ψ Weights in Updating the Forecasts......Page 169
5.2.4 Calculation of the Probability Limits of the Forecasts at Any Lead Time......Page 171
5.3.1 Eventual Forecast Function Determined by the Autoregressive Operator......Page 173
5.3.2 Role of the Moving Average Operator in Fixing the Initial Values......Page 174
5.3.3 Lead l Forecast Weights......Page 175
5.4.1 Forecasting an IMA(0, 1, 1) Process......Page 178
5.4.2 Forecasting an IMA(0, 2, 2) Process......Page 181
5.4.3 Forecasting a General IMA(0, d, q) Process......Page 184
5.4.4 Forecasting Autoregressive Processes......Page 185
5.4.5 Forecasting a (1, 0, 1) Process......Page 188
5.4.6 Forecasting a (1, 1, 1) Process......Page 190
5.5.1 State-Space Model Representation for the ARIMA Process......Page 191
5.5.2 Kalman Filtering Relations for Use in Prediction......Page 192
5.5.3 Smoothing Relations in the State Variable Model......Page 196
5.6 SUMMARY......Page 198
A5.1.1 Autocorrelation Function of Forecast Errors at Different Origins......Page 201
APPENDIX A5.2 FORECAST WEIGHTS FOR ANY LEAD TIME......Page 203
A5.3.1 General Method of Obtaining the Integrated Form......Page 206
A5.3.3 Comparison with the Discounted Least Squares Method......Page 208
PART TWO Stochastic ModelBuilding\r......Page 213
6.1.1 Stages in the Identification Procedure......Page 215
6.2.1 Use of the Autocorrelation and Partial Autocorrelation Functions in Identification......Page 216
6.2.2 Standard Errors for Estimated Autocorrelations and Partial Autocorrelations......Page 218
6.2.3 Identification of Some Actual Time Series......Page 220
6.2.4 Some Additional Model Identification Tools......Page 228
6.3.2 Initial Estimates for Moving Average Processes......Page 233
6.3.3 Initial Estimates for Autoregressive Processes......Page 235
6.3.4 Initial Estimates for Mixed Autoregressive–Moving Average Processes......Page 236
6.3.6 Approximate Standard Error for w......Page 238
6.3.7 Choice Between Stationary and Nonstationary Models in Doubtful Cases......Page 240
6.4.1 Multiplicity of Autoregressive-Moving Average Models......Page 241
6.4.2 Multiple Moment Solutions for Moving Average Parameters......Page 244
APPENDIX A6.1 EXPECTED BEHAVIOR OF THE ESTIMATED AUTOCORRELATION FUNCTION FOR A NONSTATIONARY PROCESS......Page 245
APPENDIX A6.2 GENERAL METHOD FOR OBTAINING INITIAL ESTIMATES OF THE PARAMETERS OF A MIXED AUTOREGRESSIVE–MOVING AVERAGE PROCESS......Page 246
 7.1.1 Likelihood Function\r......Page 250
7.1.2 Conditional Likelihood for an ARIMA Process......Page 251
7.1.3 Choice of Starting Values for Conditional Calculation......Page 253
7.1.4 Unconditional Likelihood; Sum-of-Squares Function; Least Squares Estimates......Page 254
7.1.5 General Procedure for Calculating the Unconditional Sum of Squares......Page 259
7.1.6 Graphical Study of the Sum-of-Squares Function......Page 264
7.1.7 Description of “Well-Behaved” Estimation Situations; Confidence Regions......Page 267
7.2.1 General Method of Approach\r......Page 274
7.2.2 Numerical Estimates of the Derivatives......Page 276
7.2.3 Direct Evaluation of the Derivatives......Page 277
7.2.4 General Least Squares Algorithm for the Conditional Model......Page 279
7.2.5 Summary of Models Fitted to Series A to F......Page 282
7.2.6 Large-Sample Information Matrices and Covariance Estimates......Page 283
7.3.1 Autoregressive Processes......Page 287
7.3.2 Moving Average Processes......Page 289
7.3.4 Separation of Linear and Nonlinear Components in Estimation......Page 290
7.3.5 Parameter Redundancy......Page 292
7.4 LIKELIHOOD FUNCTION BASED ON THE STATE-SPACE MODEL......Page 294
7.5 UNIT ROOTS IN ARIMA MODELS......Page 299
7.5.1 Formal Tests for Unit Roots in AR Models......Page 300
7.5.2 Extensions of Unit-Root Testing to Mixed ARIMA Models......Page 305
7.6.1 Bayes’s Theorem\r......Page 306
7.6.2 Bayesian Estimation of Parameters......Page 308
7.6.3 Autoregressive Processes......Page 309
7.6.4 Moving Average Processes\r......Page 312
7.6.5 Mixed Processes......Page 313
A7.1.2 Two Useful Integrals......Page 315
A7.1.3 Normal Distribution......Page 316
A7.1.4 Student’s......Page 319
 A7.2.1 Normal Equations and Least Squares\r......Page 322
A7.2.2 Estimation of Error Variance......Page 323
A7.2.5 Correlated Errors......Page 324
APPENDIX A7.3 EXACT LIKELIHOOD FUNCTION FOR MOVING AVERAGE AND MIXED PROCESSES\r......Page 325
APPENDIX A7.4 EXACT LIKELIHOOD FUNCTION FOR AN AUTOREGRESSIVE PROCESS......Page 333
APPENDIX A7.5 ASYMPTOTIC DISTRIBUTION OF ESTIMATORS FOR AUTOREGRESSIVE MODELS......Page 342
APPENDIX A7.6 EXAMPLES OF THE EFFECT OF PARAMETER ESTIMATION ERRORS ON VARIANCES OF FORECAST ERRORS AND PROBABILITY LIMITS FOR FORECASTS\r......Page 346
APPENDIX A7.7 SPECIAL NOTE ON ESTIMATION OF MOVING AVERAGE PARAMETERS......Page 349
8.1.1 General Philosophy......Page 351
8.1.2 Overfitting......Page 352
8.2 DIAGNOSTIC CHECKS APPLIED TO RESIDUALS......Page 353
8.2.1 Autocorrelation Check......Page 355
8.2.2 Portmanteau Lack-of-Fit Test......Page 356
8.2.3 Model Inadequacy Arising from Changes in Parameter Values......Page 361
8.2.4 Score Tests for Model Checking......Page 362
8.2.5 Cumulative Periodogram Check......Page 365
8.3.1 Nature of the Correlations in the Residuals When an Incorrect Model Is Used......Page 368
8.3.2 Use of Residuals to Modify the Model......Page 370
9.1.1 Fitting versus Forecasting......Page 371
9.1.2 Seasonal Models Involving Adaptive Sines and Cosines......Page 372
9.1.3 General Multiplicative Seasonal Model......Page 374
9.2.1 Multiplicative (0, 1, 1) × (0, 1, 1)12 Model......Page 377
9.2.2 Forecasting......Page 378
9.2.3 Identification......Page 385
9.2.4 Estimation......Page 388
9.3.1 Multiplicative and Nonmultiplicative Models......Page 393
9.3.2 Identification......Page 397
9.3.3 Estimation......Page 398
9.3.4 Eventual Forecast Functions for Various Seasonal Models......Page 399
9.3.5 Choice of Transformation......Page 401
9.4.1 Structural Component Time Series Models......Page 402
9.4.2 Deterministic Seasonal and Trend Components and Common Factors......Page 406
9.4.3 Estimation of Unobserved Components in Structural Models......Page 408
9.5 REGRESSION MODELS WITH TIME SERIES ERROR TERMS......Page 415
9.5.1 Model Building, Estimation, and Forecasting Procedures forRegression Models......Page 417
9.5.2 Restricted Maximum Likelihood Estimation for Regression Models......Page 422
APPENDIX A9.1 AUTOCOVARIANCES FOR SOME SEASONAL MODELS......Page 425
10.1 AUTOREGRESSIVE CONDITIONAL HETEROSCEDASTIC(ARCH) MODELS......Page 430
10.1.1 First-Order ARCH Model......Page 432
10.1.2 Consideration for More General Models......Page 433
10.1.3 Model Building and Parameter Estimation......Page 434
10.2 NONLINEAR TIME SERIES MODELS......Page 437
10.2.1 Classes of Nonlinear Models......Page 438
10.2.2 Implications and Examples of Nonlinear Models......Page 441
10.3 LONG MEMORY TIME SERIES PROCESSES......Page 445
10.3.1 Fractionally Integrated Processes......Page 446
10.3.2 Estimation of  Parameters......Page 450
PART THREE Transfer Function andMultivariate ModelBuilding\r......Page 454
11.1.1 Discrete Transfer Function......Page 456
First-Order Dynamic System......Page 459
11.2.1 General Form of the Difference Equation......Page 464
11.2.2 Nature of the Transfer Function......Page 466
11.2.3 First- and Second-Order Discrete Transfer Function Models......Page 467
11.2.4 Recursive Computation of Output for Any Input......Page 473
11.3 RELATION BETWEEN DISCRETE AND CONTINUOUS MODELS......Page 475
11.3.1 Response to a Pulsed Input......Page 476
Undelayed First-Order System......Page 478
Delayed First-Order System......Page 479
Undelayed Second-Order System......Page 480
11.3.3 Approximating General Continuous Models by Discrete Models......Page 481
APPENDIX A11.1 CONTINUOUS MODELS WITH PULSED INPUTS......Page 482
APPENDIX A11.2 NONLINEAR TRANSFER FUNCTIONS AND LINEARIZATION......Page 487
CHAPTER TWELVE Identification, Fitting, and Checkingof Transfer Function Models......Page 490
Bivariate Stochastic Processes......Page 491
12.1.2 Estimation of the Cross-Covariance and Cross-Correlation Functions......Page 494
12.1.3 Approximate Standard Errors of Cross-Correlation Estimates......Page 495
12.2 IDENTIFICATION OF TRANSFER FUNCTION MODELS......Page 498
12.2.1 Identification of Transfer Function Models by Prewhitening the Input......Page 500
12.2.2 Example of the Identification of a Transfer Function Model......Page 501
12.2.3 Identification of the Noise Model......Page 505
12.2.4 Some General Considerations in Identifying Transfer Function Models......Page 507
12.3.1 Conditional Sum-of-Squares Function......Page 509
12.3.2 Nonlinear Estimation......Page 512
12.3.3 Use of Residuals for Diagnostic Checking......Page 514
12.3.4 Specific Checks Applied to the Residuals......Page 515
12.4.1 Fitting and Checking of the Gas Furnace Model......Page 518
12.4.2 Simulated Example with Two Inputs......Page 524
12.5 FORECASTING WITH TRANSFER FUNCTION MODELS USING LEADING INDICATORS......Page 526
12.5.1 Minimum Mean Square Error Forecast......Page 527
12.5.2 Forecast of CO2 Output from Gas Furnace......Page 531
12.5.3 Forecast of Nonstationary Sales Data Using a Leading Indicator......Page 534
12.6 SOME ASPECTS OF THE DESIGN OF EXPERIMENTS TO ESTIMATE TRANSFER FUNCTIONS......Page 536
A12.1.1 Identification of Single Input Transfer Function Models......Page 538
A12.1.2 Identification of Multiple Input Transfer Function Models......Page 540
A12.2.1 Design of Optimal Inputs for a Simple System......Page 541
A12.2.2 Numerical Example......Page 544
13.1.1 Models for Intervention Analysis......Page 546
13.1.2 Example of Intervention Analysis......Page 549
13.1.3 Nature of the MLE for a Simple Level Change Parameter Model......Page 550
13.2 OUTLIER ANALYSIS FOR TIME SERIES......Page 553
13.2.1 Models for Additive and Innovational Outliers......Page 554
13.2.2 Estimation of Outlier Effect for Known Timing of the Outlier......Page 555
13.2.3 Iterative Procedure for Outlier Detection......Page 557
13.2.4 Examples of Analysis of Outliers......Page 558
13.3 ESTIMATION FOR ARMA MODELS WITH MISSING VALUES......Page 560
13.3.1 State-Space Model and Kalman Filter with Missing Values......Page 561
13.3.2 Estimation of Missing Values of an ARMA Process......Page 563
CHAPTER FOURTEEN Multivariate Time Series Analysis......Page 568
14.1.1 Covariance Properties of Stationary Multivariate Time Series......Page 569
14.1.2 Spectral Characteristics for Stationary Multivariate Processes......Page 571
14.1.3 Linear Filtering Relations for Stationary Multivariate Processes......Page 572
14.2 LINEAR MODEL REPRESENTATIONS FOR STATIONARY MULTIVARIATE PROCESSES......Page 573
14.2.1 Vector Autoregressive–Moving Average (ARMA) Models and Representations......Page 574
14.2.2 Aspects of Nonuniqueness and Parameter Identifiability for Vector ARMA Models......Page 580
14.2.3 Echelon Canonical Form of the Vector ARMA Model......Page 582
14.2.4 Relation of Vector ARMA to Transfer Functionand ARMAX Model Forms......Page 586
14.3 NONSTATIONARY VECTOR AUTOREGRESSIVE–MOVING AVERAGE MODELS......Page 587
14.4.1 Calculation of Forecasts from ARMA Difference Equation......Page 590
14.5 STATE-SPACE FORM OF THE VECTOR ARMA MODEL......Page 592
14.6.1 Initial Model Building and Least Squares for Vector AR Models......Page 595
14.6.2 Estimation and Model Checking for Vector ARMA Models......Page 600
14.6.3 Estimation and Inferences for Co-integrated Vector AR Models......Page 602
14.7 EXAMPLE OF VECTOR ARMA MODELING......Page 605
PART FOUR Design of DiscreteControl Schemes\r......Page 613
CH PTER FIFTEEN Aspects of Process Control......Page 614
15.1.1 Process Monitoring......Page 615
15.1.2 Process Adjustment......Page 618
15.2 PROCESS ADJUSTMENT USING FEEDBACK CONTROL......Page 619
15.2.1 Feedback Adjustment Chart......Page 620
15.2.2 Modeling the Feedback Loop......Page 622
15.2.3 Simple Models for Disturbances and Dynamics......Page 623
15.2.4 General Minimum Mean Square Error Feedback Control Schemes......Page 627
15.2.6 Complementary Roles of Monitoring and Adjustment......Page 632
15.3 EXCESSIVE ADJUSTMENT SOMETIMES REQUIRED BY MMSE CONTROL......Page 635
15.3.1 Constrained Control......Page 636
15.4.1 Bounded Adjustment Scheme for Fixed Adjustment Cost......Page 638
15.4.2 Indirect Approach for Obtaining a Bounded Adjustment Scheme......Page 640
15.5 FEEDFORWARD CONTROL......Page 642
15.5.1 Feedforward Control to Minimize Mean Square Error at the Output......Page 644
15.5.2 An Example—Control of the Specific Gravityof an Intermediate Product......Page 647
15.5.3 Feedforward Control with Multiple Inputs......Page 650
15.5.4 Feedforward–Feedback Control......Page 651
15.5.6 Remarks on Fitting Transfer Function–Noise Models Using Operating Data......Page 654
15.6 MONITORING VALUES OF PARAMETERS OF FORECASTING AND FEEDBACK ADJUSTMENT SCHEMES......Page 656
APPENDIX A15.1 FEEDBACK CONTROL SCHEMES WHERE THE ADJUSTMENT VARIANCE IS RESTRICTED......Page 658
A15.1.1 Derivation of Optimal Adjustment......Page 659
Table of Optimal Values for Constrained Schemes......Page 667
APPENDIX A15.2 CHOICE OF THE SAMPLING INTERVAL......Page 668
A15.2.2 Sampling an IMA(0, 1, 1) Process......Page 669
PART FIVE Charts and Tables\r......Page 673
Collection of Tables and Charts......Page 674
Collection of Time Series Used for Examples in the Text and in Exercises......Page 681
References......Page 697
PART SIX Exercises and Problems\r......Page 712
Index......Page 739