Statistical Factor Analysis and Related Methods: Theory and Applications (Wiley Series in Probability and Statistics)

STATISTICAL FACTOR ANALYSIS AND RELATED METHODS: THEORY AND APPLICATIONS......Page 1
Half-title......Page 2
Title Page......Page 3
Copyright Page......Page 4
Dedication......Page 6
Preface......Page 10
Contents......Page 18
1.1 INTRODUCTION......Page 26
1.2.1 The Chi-Squared Distribution......Page 30
1.2.2 The F Distribution......Page 31
1.2.3 The t Distribution......Page 32
1.3 ESTIMATION......Page 33
1.3.1 Point Estimation: Maximum Likelihood......Page 34
1.3.2 The Likelihood Ratio Criterion......Page 37
1.4 NOTIONS OF MULTIVARIATE DISTRIBUTIONS......Page 40
1.5 STATISTICS AND THE THEORY OF MEASUREMENT......Page 44
1.5.1 The Algebraic Theory of Measurement......Page 45
1.5.2 Admissible Transformations and the Classification of Scales......Page 50
1.5.3 Scale Classification and Meaningful Statistics......Page 53
1.5.4 Units of Measure and Dimensional Analysis for Ratio Scales......Page 55
1.6 STATISTICAL ENTROPY......Page 56
1.7 COMPLEX RANDOM VARIABLES......Page 58
EXERCISES......Page 60
2.1 INTRODUCTION......Page 62
2.2 LINEAR, QUADRATIC FORMS......Page 63
2.3.1 Derivative Vectors......Page 67
2.3.2 Derivative Matrices......Page 69
2.4 GRAMMIAN ASSOCIATION MATRICES......Page 72
2.4.1 The Inner Product Matrix......Page 74
2.4.2 The Cosine Matrix......Page 75
2.4.3 The Covariance Matrix......Page 76
2.4.4 The Correlation Matrix......Page 77
2.5 TRANSFORMATION OF COORDINATES......Page 81
2.5.1 Orthogonal Rotation......Page 82
2.5.2 Oblique Rotations......Page 85
2.6 LATENT ROOTS AND VECTORS OF GRAMMIAN MATRICES......Page 87
2.7 ROTATION OF QUADRATIC FORMS......Page 92
2.8 ELEMENTS OF MULTIVARIATE NORMAL THEORY......Page 94
2.8.1 The Multivariate Normal Distribution......Page 95
2.8.2 Sampling From the Multivariate Normal......Page 108
2.9 THE KRONECKER PRODUCT......Page 111
2.10 SIMULTANEOUS DECOMPOSITION OF TWO GRAMMIAN MATRICES......Page 113
2.11.1 Complex Matrices, Hermitian Forms......Page 115
2.11.2 The Complex Multivariate Normal......Page 118
EXERCISES......Page 120
3.1 INTRODUCTION......Page 122
3.2 PRINCIPAL COMPONENTS IN THE POPULATION......Page 126
3.3. ISOTROPIC VARIATION......Page 144
3.4.1 Introduction......Page 152
3.4.2 The General Model......Page 153
3.4.3 The Effect of Means and Variances on PCs......Page 167
3.5 PRINCIPAL COMPONENTS AND PROJECTIONS......Page 171
3.6 PRINCIPAL COMPONENTS BY LEAST SQUARES......Page 185
3.7 NONLINEAR1TY IN THE VARIABLES......Page 187
3.8.1 Introduction......Page 198
3.8.2 Standardized Regression Loadings......Page 199
3.8.3 Ratio Index Loadings......Page 200
3.8.4 Probability Index Loadings......Page 202
EXERCISES......Page 203
4.1 INTRODUCTION......Page 207
4.2 TESTING COVARIANCE AND CORRELATION MATRICES......Page 209
4.2.1 Testing For Complete Independence......Page 210
4.2.2 Testing Sphericity......Page 216
4.2.3 Other Tests for Covariance Matrices......Page 219
4.3.1 Testing Equality of All Latent Roots......Page 227
4.3.2 Testing Subsets of Principal Components......Page 229
4.3.3 Testing Residuals......Page 232
4.3.4 Testing Individual Principal Components......Page 234
4.3.5 Information Criteria of Maximum Livelihood Estimation of the Number of Components......Page 245
4.4.1 Estimates Based on Resampling......Page 248
4.2.2 Residual Correlations Test......Page 253
4.4.3 Informal Rules of Thumb......Page 254
4.5 DISCARDING REDUNDANT VARIABLES......Page 256
4.6.1 Assessing Univariate Normality......Page 259
4.6.2 Testing for Multivariate Normality......Page 260
4.6.3 Retrospective Testing for Multivariate Normality......Page 266
4.7.1 Robustness......Page 267
4.7.2 Sensitivity of Principal Components......Page 268
4.7.3 Missing Data......Page 271
EXERCISES......Page 273
5.2 PRINCIPAL COMPONENTS OF SINGULAR MATRICES......Page 275
5.2.1 Singular Grammian Matrices......Page 276
5.2.2 Rectangular Matrices and Generalized Inverses......Page 277
5.3 PRINCIPAL COMPONENTS AS CLUSTERS: LINEAR TRANSFORMATIONS IN EXPLORATORY RESEARCH......Page 282
5.3.1 Orthogonal Rotations......Page 283
5.3.2 Oblique Rotations......Page 295
5.3.3 Grouping Variables......Page 301
5.4.1 Q-Mode Analysis......Page 303
5.4.2 Multidimensional Scaling and Principal Coordinates......Page 307
5.4.3 Three-Mode Analysis......Page 311
5.4.4 Joint Plotting of Loadings and Scores......Page 322
5.5.1 The Canonical Correlation Model......Page 325
5.5.2 Modification of Canonical Correlation......Page 333
5.5.3 Canonical Correlation for More than Two Sets of Variables......Page 335
5.5.4 Multigroup Principal Components......Page 336
5.6 WEIGHTED PRINCIPAL COMPONENTS......Page 343
5.7 PRINCIPAL COMPONENTS IN THE COMPLEX FIELD......Page 346
5.8.1 Further Optimality Properties......Page 347
5.8.2 Screening Data......Page 349
5.8.3 Principal Components in Discrimination and Classification......Page 351
5.8.4 Mahalanobis Distance and the Multivariate T-Test......Page 353
5.9.1 Proportions and Compositional Data......Page 355
5.9.2 Estimating Components of a Mixture......Page 360
5.9.3 Directional Data......Page 364
EXERCISES......Page 372
6.1 INTRODUCTION......Page 376
6.2 THE UNRESTRICTED RANDOM FACTOR MODEL IN THE POPULATION......Page 378
6.3.1 The Homoscedastic Residuals Model......Page 386
6.3.2 Unweighted Least Squares Models......Page 388
6.3.3 The Image Factor Model......Page 390
6.4.1 The Reciprocal Proportionality Model......Page 392
6.4.2 The Lawley Model......Page 395
6.4.3 The Rao Canonical Correlation Factor Model......Page 404
6.4.4 The Generalized Least Squares Model......Page 406
6.5.1 The Double Heteroscedastic Model......Page 407
6.6 TESTS OF SIGNIFICANCE......Page 409
6.6.1 The Chi-Squared Test......Page 410
6.2.2 Information Criteria......Page 412
6.6.3 Testing Loading Coefficients......Page 417
6.7 THE FIXED FACTOR MODEL......Page 419
6.8 ESTIMATING FACTOR SCORES......Page 420
6.8.1 Random Factors: The Regression Estimator......Page 421
6.8.2 Fixed Factors: The Minimum Distance Estimator......Page 423
6.9 FACTORS REPRESENTING "MISSING DATA:" THE EM ALGORITHM......Page 425
6.10 FACTOR ROTATION AND IDENTIFICATION......Page 427
6.11 CONFIRMATORY FACTOR ANALYSIS......Page 439
6.12 MULTIGROUP FACTOR ANALYSIS......Page 441
6.13 LATENT STRUCTURE ANALYSIS......Page 443
EXERCISES......Page 445
7.1 INTRODUCTION......Page 448
7.2 TIME SERIES AS RANDOM FUNCTIONS......Page 449
7.2.1 Constructing Indices and Indicators......Page 455
7.2.2 Computing Empirical Time Functions......Page 459
7.2.3 Pattern Recognition and Data Compression: Electrocardiograph Data......Page 462
7.3 DEMOGRAPHIC COHORT DATA......Page 464
7.4 SPATIAL CORRELATION: GEOGRAPHIC MAPS......Page 468
7.5. THE KARHUNEN–LOÈVE SPECTRAL DECOMPOSITION IN THE TIME DOMAIN......Page 470
7.5.1 Analysis of the Population: Continuous Space......Page 471
7.5.2 Analysis of a Sample: Discrete Space......Page 479
7.5.3 Order Statistics: Testing Goodness of Fit......Page 486
7.6 ESTIMATING DIMENSIONALITY OF STOCHASTIC PROCESSES......Page 489
7.6.1 Estimating a Stationary ARMA Model......Page 490
7.6.2 Time Invariant State Space Models......Page 492
7.6.3 Autoregression and Principal Components......Page 494
7.6.4 Kalman Filtering Using Factor Scores......Page 502
7.7 MULTIPLE TIME SERIES IN THE FREQUENCY DOMAIN......Page 505
7.7.1. Principal Components in the Frequency Domain......Page 506
7.7.2 Factor Analysis in the Frequency Domain......Page 508
7.8 STOCHASTIC PROCESSES IN THE SPACE DOMAIN: KARHUNEN–LOÈVE DECOMPOSITION......Page 511
7.9 PATTERNED MATRICES......Page 514
7.9.1 Circular Matrices......Page 515
7.9.2 Tridiagonal Matrices......Page 516
7.9.3 Toeplitz Matrices......Page 517
7.9.4 Block-Patterned Matrices......Page 519
EXERCISES......Page 522
8.2 ORDINAL DATA......Page 526
8.2.1 Ordinal Variables as Intrinsically Continuous: Factor Scaling......Page 527
8.2.2 Ranks of Order Statistics......Page 533
8.2.3 Ranks as Qualitative Random Variables......Page 537
8.3 NOMINAL RANDOM VARIABLES: COUNT DATA......Page 543
8.3.1 Symmetric Incidence Matrices......Page 544
8.3.2 Asymmetric Incidence Matrices......Page 547
8.3.3 Multivariate Multinomial Data: Dummy Variables......Page 549
8.4 FURTHER MODELS FOR DISCRETE DATA......Page 558
8.4.1 Guttman Scaling......Page 559
8.4.2 Maximizing Canonical Correlation......Page 563
8.4.3 Two-Way Contingency Tables: Optimal Scoring......Page 565
8.4.4 Extensions and Other Types of Discrete Data......Page 577
8.5 RELATED PROCEDURES: DUAL SCALING AND CORRESPONDENCE ANALYSIS......Page 586
EXERCISES......Page 589
9.2 SERIALLY CORRELATED DISCRETE DATA......Page 595
9.2.1 Seriation......Page 596
9.2.2 Ordination......Page 602
9.2.3 Higher-Dimensional Maps......Page 605
9.3 THE NONLINEAR "HORSESHOE" EFFECT......Page 608
9.4 MEASURES OF PAIRWISE CORRELATION OF DICHOTOMOUS VARIABLES......Page 618
9.4.1 Euclidean Measures of Association......Page 619
9.4.2 Non-Euclidean Measures of Association......Page 621
9.5 MIXED DATA......Page 622
9.5.1 Point Biserial Correlation......Page 623
9.5.2 Biserial Correlation......Page 624
9.6 THRESHOLD MODELS......Page 627
9.7 LATENT CLASS ANALYSIS......Page 632
EXERCISES......Page 646
10.2 LEAST SQUARES CURVE FITTING WITH ERRORS IN VARIABLES......Page 649
10.2.1 Minimizing Sums of Squares of Errors in Arbitrary Direction......Page 652
10.2.2 The Maximum Likelihood Model......Page 660
10.2.3 Goodness of Fit Criteria of Orthogonal-Norm Least Squares......Page 664
10.2.4 Testing Significance of Orthogonal-Norm Least Squares......Page 665
10.2.5 Nonlinear Orthogonal Curve Fitting......Page 669
10.3 LEAST SQUARES REGRESSION WITH MULTICOLLINEARITY......Page 670
10.3.1 Principal Components Regression......Page 672
10.3.2 Comparing Orthogonal-Norm and Y-Norm Least Squares Regression......Page 688
10.3.3 Latent Root Regression......Page 690
10.3.4 Quadratic Principal Components Regression......Page 694
10.4.1 Factor Analysis Regression: Dependent Variable Excluded......Page 696
10.4.2 Factor Analysis Regression: Dependent Variable Included......Page 699
10.5 FACTOR ANALYSIS OF DEPENDENT VARIABLES IN MANOVA......Page 701
10.6 ESTIMATING EMPIRICAL FUNCTIONAL RELATIONSHIPS......Page 703
10.7.1 Capital Stock Market Data: Arbitrage Pricing......Page 707
10.7.2 Estimating Nonlinear Dimensionality: Sliced Inverse Regression......Page 710
EXERCISES......Page 712
References......Page 713
Index......Page 755