Multivariate Statistical Modeling in Engineering and Management

Cover
Half Title
Title Page
Copyright Page
Dedication
Table of Contents
Foreword
Preface
Acknowledgments
Author
Part I Prerequisites
	1 Introduction
		1.1 Data-Driven Decision-Making
		1.2 Variable and Data Types
			1.2.1 Random Variable
			1.2.2 Measurement Scale and Data Types
			1.2.3 Data Sources
		1.3 Models and Modeling
		1.4 Statistical Approaches to Model-Building
			1.4.1 Step 1: Define Problem
			1.4.2 Step 2: Develop Conceptual Model
			1.4.3 Step 3: Design Study
			1.4.4 Step 4: Collect Data
			1.4.5 Step 5: Examine Data
			1.4.6 Step 6: Select a Suitable Model
			1.4.7 Step 7: Estimate Parameters
			1.4.8 Step 8: Verify Model
			1.4.9 Step 9: Validate Model
			1.4.10 Step 10: Interpret Results
		1.5 Multivariate Models
		1.6 Illustrative Problems
		1.7 Case Descriptions
			1.7.1 Case 1: Job Stress Assessment Among Employees in Coke Plant
			1.7.2 Case 2: Job Demand Analysis of Underground Coal Mine Workers
			1.7.3 Case 3: Study of the Process and Quality Characteristics and Their Relationships in Worm Gear Manufacturing
			1.7.4 Case 4: Study of the Process and Quality Characteristics in Cast Iron Melting Process
			1.7.5 Case 5: Employees Safety Practices in Mines
			1.7.6 Case 6: Modeling Causal Relationships of Job Risk Perception of EOT Crane Operators
		1.8 Aims of the Book
		1.9 Organization of the Book
		Exercises
	2 Basic Univariate Statistics
		2.1 Population and Parameter
		2.2 Defining Population: the Probability Distributions
			2.2.1 Univariate Normal Distribution
		2.3 Sample and Statistics
			2.3.1 Measures of Central Tendency
			2.3.2 Measures of Dispersion
		2.4 Sampling Distribution
			2.4.1 Standard Normal Distribution
			2.4.2 Chi-Square Distribution
			2.4.3 T-Distribution
			2.4.4 F-Distribution
		2.5 Central Limit Theorem
		2.6 Estimation
			2.6.1 Confidence Interval for Single Population Mean
			2.6.2 Confidence Interval for Single Population Variance
			2.6.3 Confidence Interval for the Difference Between Two-Population Means
			2.6.4 Confidence Interval for the Ratio Of Two-Population Variances
		2.7 Hypothesis Testing
			2.7.1 Hypothesis Testing Concerning Single Population Mean
			2.7.2 Hypothesis Testing Concerning Single Population Variance
			2.7.3 Hypothesis Testing Concerning Equality Of Two-Population Means
				Scenario I
				Scenario II
				Scenario III
			2.7.4 Hypothesis Testing Concerning Equality of Two-Population Variances
		2.8 Learning Summary
		Exercises
		Notes
	3 Basic Computations
		3.1 Matrix Algebra
			3.1.1 Data as a Matrix
			3.1.2 Row and Column Vectors
			3.1.3 Orthogonal Vectors
			3.1.4 Linear Dependency of a Set of Vectors
			3.1.5 The Gram-Schmidt Orthogonalization Process
			3.1.6 Projection of One Vector On Another
			3.1.7 Basic Matrices
			3.1.8 Basic Matrix Operations
			3.1.9 Determinants
			3.1.10 Rank of a Matrix
			3.1.11 Inverse of a Matrix
			3.1.12 Eigenvalues and Eigenvectors
			3.1.13 Spectral Decomposition
			3.1.14 Singular Value Decomposition (SVD)
			3.1.15 Positive Definite Matrices
		3.2 Methods of Least Squares
			3.2.1 Ordinary Least Squares (OLS)
			3.2.2 Weighted Least Squares (WLS)
			3.2.3 Iteratively Reweighted Least Squares (IRLS)
			3.2.4 Generalized Least Squares (GLS)
		3.3 Maximum Likelihood Method
			3.3.1 Probability Function
			3.3.2 Likelihood Function
			3.3.3 Maximum Likelihood Estimation
		3.4 Generation of Random Variable
			3.4.1 Generation of Univariate Normal Observations
			3.4.2 Generating Multivariate Normal Observations
		3.5 Resampling Methods
			3.5.1 Jackknife
			3.5.2 Bootstrap
		3.6 Learning Summary
		Exercises
Part II Foundations of Multivariate Statistics
	4 Multivariate Descriptive Statistics
		4.1 Multivariate Observations
		4.2 Mean Vectors
		4.3 Covariance Matrix
		4.4 Correlation Matrix
		4.5 Types of Correlation
			4.5.1 Correlation Between Two Ordinal Variables
				Spearman’s Rho
				Gamma Coefficient
			4.5.2 Correlation Between Two Nominal Variables
			4.5.3 Correlation Between One Continuous and One Ordinal Variable
			4.5.4 Correlation Between One Continuous and One Nominal Variable
			4.5.5 Correlation Between One Ordinal and Another Nominal Variable
		4.6 Correlation With Dependence Structure
			4.6.1 Part Correlation
			4.6.2 Partial Correlation
		4.7 Learning Summary
		Exercises
	5 Multivariate Normal Distribution
		5.1 Statistical Distance
		5.2 Bivariate Normal Density Function
		5.3 Multivariate Normal Density Function
		5.4 Constant Density Contours
		5.5 Properties of Multivariate Normal Density Function
		5.6 Assessing Multivariate Normality
			5.6.1 Tests of Univariate Normality
			5.6.2 Tests of Multivariate Normality
			5.6.3 Remedy to Violation of Multivariate Normality
		5.7 Learning Summary
		Exercises
	6 Multivariate Inferential Statistics
		6.1 Estimation of Parameters of Multivariate Normal Distribution
		6.2 Sampling Distribution of X– and S
		6.3 Multivariate Central Limit Theorem
		6.4 Hotelling’s T2 Distribution
		6.5 Inference About Single Population Mean Vector
			6.5.1 Confidence Region
			6.5.2 Simultaneous Confidence Intervals
				Scenario 1: Sampling From When Is Known
				Scenario 2: S Is Unknown and Sample Size Is Large
				Scenario 3: S Is Unknown and N Is Small to Medium
			6.5.3 Hypothesis Testing
				Scenario 1: X ~ Np (., S) and Is Known
				Scenario 2: and Is Unknown But N Is Large
				Scenario 3: , and Is Unknown But N Is Small to Medium
		6.6 Inference About Equality of Two-Population Mean Vectors
			6.6.1 Confidence Region
				Scenario 1: Sampling From Multivariate Normal Populations With Known
				Scenario 2: Sampling From Multivariate Normal Populations With Unknown But Equal
				Scenario 3: Sampling From Multivariate Normal Populations With Unknown and Unequal and Large Samples
			6.6.2 Simultaneous Confidence Intervals
				Scenario 1: Sampling From Multivariate Normal Populations With Known SA and SB
				Scenario 2: Sampling From Multivariate Normal Populations With Unknown But Equal S
				Scenario 3: Sampling From Multivariate Normal Populations With Unknown, Unequal and Large Samples
			6.6.3 Hypothesis Testing
				Scenario 1: Sampling From Multivariate Normal Populations With Known SA and SB
				Scenario 2: Sampling From Multivariate Normal Population With Unknown But Equal S
				Scenario 3: Sampling From Multivariate Normal Population With Unknown, Unequal S and Large Samples
		6.7 Confidence Region and Hypothesis Testing for Covariance Matrix S
			6.7.1 Confidence Region for S
			6.7.2 Hypothesis Testing for S
		6.8 Sampling From Non-Normal Population
		6.9 Learning Summary
		Exercises
Part III Multivariate Models
	7 Multivariate Analysis of Variance
		7.1 Analysis of Variance (ANOVA)
			7.1.1 Conceptual Model
			7.1.2 Assumptions
				Bartlett Test
			7.1.3 Total Sum Squares Decomposition
			7.1.4 Hypothesis Testing
			7.1.5 Estimation of Parameters
				100(1–a)% Confidence Interval (CI) and Simultaneous Confidence Interval (SCI) for
				100(1–a)% CI and Simultaneous Confidence Interval (SCI) for
			7.1.6 Model Adequacy Tests
				Test of Normality
				Test of Independence
				Test of Homogeneity of Variances
				Modified Leven Test
			7.1.7 Interpretation of Results
		7.2 Multivariate Analysis of Variance (MANOVA)
			7.2.1 Conceptual Model
			7.2.2 Assumptions
				Box’s M Test
			7.2.3 Total Sum Squares and Cross Product (SSCP) Decomposition
			7.2.4 Hypothesis Testing
				Multivariate Test Statistics Used in MANOVA
			7.2.5 Estimation of Parameters
				100(1 – .)% Simultaneous CI
			7.2.6 Model Adequacy Tests
			7.2.7 Test of Assumptions
				Test of Normality
				Test of Independence
				Test of Homogeneity of Covariance Matrices
		7.3 Two-Way MANOVA
			7.3.1 Two-Way ANOVA
			7.3.2 Two-Way MANOVA
			7.3.3 Hypothesis Testing
				Factor F1 (Wilks’ Lambda, )
				Factor F2 (Wilks’ Lambda, )
				Interaction F12 (Wilks’ Lambda, )
		7.4 Case Study
		7.5 Learning Summary
		Exercises
	8 Multiple Linear Regression
		8.1 Conceptual Model
		8.2 Assumptions
		8.3 Estimation of Model Parameters
		8.4 Sampling Distribution of
		8.5 Confidence Region (CR) and Simultaneous Confidence Intervals (SCI) for
		8.6 Sampling Distribution of
		8.7 Assessment of Overall Fit of the Model
		8.8 Test of Individual Regression Parameters
		8.9 Interpretation of Regression Parameters
		8.10 Test of Assumptions
			8.10.1 Test of Linearity
			8.10.2 Homoscedasticity Or Constant Error Variance
			8.10.3 Uncorrelated Error Terms
			8.10.4 Normality of Error Terms
		8.11 Remedy Against Violation of Assumptions
			8.11.1 Remedies Against Linearity
			8.11.2 Remedies Against Non-Normality and Heteroscedasticity
		8.12 Diagnostic Issues in MLR
			8.12.1 Outliers
			8.12.2 Leverage Points
			8.12.3 Influential Observations
			8.12.4 Multicollinearity
				Variance Inflation Factor (VIF)
				Tolerance Statistic
				Eigenvalue Structure of the Correlation Matrix, R
				Multicollinearity Condition Number (MCN)
		8.13 Prediction With MLR
		8.14 Model Validation
		8.15 Case Study
		8.16 Learning Summary
		Exercises
		Notes
	9 Multivariate Multiple Linear Regression
		9.1 Conceptual Model
		9.2 Assumptions
		9.3 Estimation of Parameters
		9.4 Sampling Distribution of
		9.5 Assessment of Overall Fit of The Model
		9.6 Test of Subset of Regression Parameters
		9.7 Test of Individual Regression Parameters
		9.8 Interpretation and Diagnostics
		9.9 Case Study
		9.10 Learning Summary
		Exercises
	10 Path Model
		10.1 Conceptual Model
		10.2 Assumptions
		10.3 Estimation of Parameters for Recursive Model
			10.3.1 Normal Equation Approach
			10.3.2 Reduced Form Approach
		10.4 Estimation of Parameters for Non-Recursive Model
			10.4.1 Model Identification
			10.4.2 Estimation of Parameters
				Instrumental Variable (IV) Method
				Two Stage Least Squares (2SLS)
				Three Stage Least Squares (3SLS)
				Maximum Likelihood Estimation
		10.5 Overall Fit AND SPECIFICATION TESTS
			10.5.1 Tests for Endogeneity
			10.5.2 Tests for Over-Identifying Restrictions
				Anderson-Rubin Test
				Sargan Test
			10.5.3 Heteroscedasticity-Robust Tests
			10.5.4 Tests for Identifying Weak Instruments
			10.5.5 Coefficients of Determination (R2)
		10.6 Test of Individual Path Coefficients
		10.7 Case Study
		10.8 Learning Summary
		Exercises
		Notes
	11 Principal Component Analysis
		11.1 Conceptual Model
		11.2 Extracting Principal Components
		11.3 Sampling Distribution of and
		11.4 Adequacy Tests for PCA
			11.4.1 Bartlett’s Sphericity Test
			11.4.2 Number of PCs to Be Extracted
				Cumulative Percentage of Total Variation
				Kaiser’s Rule (Eigenvalue Criteria)
				The Average Root
				The Broken Stick Method
				Scree Plot
			11.4.3 Hypothesis Testing Concerning Insignificant Eigenvalues
		11.5 Principal Component Scores
		11.6 Validation
		11.7 Case Study
		11.8 Learning Summary
		Exercises
	12 Exploratory Factor Analysis
		12.1 Conceptual Model
		12.2 Assumptions
		12.3 Some Useful Results
		12.4 Factor Extraction Methods
			12.4.1 The Principal Component Method
			12.4.2 The Principal Factor Method
			12.4.3 The Maximum Likelihood Method
			12.4.4 Choice of Methods of Estimation
			12.4.5 Degrees of Freedom
		12.5 Model Adequacy Tests
		12.6 Number of Factors
		12.7 Factor Rotation
		12.8 Factor Scores
			12.8.1 Estimation By Principal Component Scores
			12.8.2 Estimation By Least Squares Approach
			12.8.3 Estimation By Regression Method
		12.9 Difference With Principal Component Analysis
		12.10 Validation
		12.11 Case Study
			12.11.1 Background
			12.11.2 Variables And Data
			12.11.3 Data Analysis
			12.11.4 Results And Discussion
			12.11.5 Conclusions
		12.12 Learning Summary
		Exercises
		Notes
1 Factor Analysis Is Not a Causal Model. It Is an Interdependence Model That Is Used to Identify and Quantify Hidden Factors, Usually Lesser in Dimensions. The Use of ‘Cause’ Here Is to Explain the Inner Meaning of a Factor.
	13 Confirmatory Factor Analysis
		13.1 Conceptual Model
		13.2 Estimation of Parameters
			13.2.1 Model Identification
			13.2.2 Model Estimation
		13.3 Model Adequacy Tests
			13.3.1 Absolute Fit Indices
				Goodness of Fit Index (GFI)
				Root Mean Square Residual (RMSR)
				Standardized Root Mean Square Residual (SRMSR)
				Root Mean Square Error of Approximation (RMSEA)
			13.3.2 Relative Fit Indices (RFI)
				Tucker Lewis Index (TLI)
				Normed Fit Index (NFI)
				Comparative Fit Index (CFI)
				General Normed Fit Index (GNFI)
				Relative Non-Centrality Index (RNI)
			13.3.3 Parsimony Fit Indices
				Adjusted Goodness of Fit Index (AGFI)
				Parsimony Normed Fit Index (PNFI)
		13.4 Test of Parameters
		13.5 Fit Indices for Individual Factors
			13.5.1 Construct Validity
			13.5.2 Unidimensionality
			13.5.3 Reliability
		13.6 Model Respecification
		13.7 Case Study
			13.7.1 Background
			13.7.2 Variables and Data
			13.7.3 Analysis and Results
				Overall Fit of The Proposed Model
				Test of Model Parameters
				Test of Fit of Individual Factors
			13.7.4 Model Re-Specifications
			13.7.5 Discussions and Insights
		13.8 Learning Summary
		Exercises
	14 Structural Equation Modeling
		14.1 Prerequisite and Modeling Strategy
		14.2 Variables, Relationships, Terminologies, and Notations
		14.3 Model Equations and Dispersion Matrices
			14.3.1 Structural Model Equations
			14.3.2 Exogenous Factor Model Equations
			14.3.3 Endogenous Factor Model Equations
		14.4 Assumptions
		14.5 Fundamental Relationships
			14.5.1 Fundamental Relationships for the Structural Model
			14.5.2 Fundamental Relationships for the Complete SEM Model
		14.6 Parameter Estimation
			14.6.1 Model Identification
				Measurement Model Identification
				Structural Model Identification
			14.6.2 Model Estimation
			14.6.3 Direct, Indirect, and Total Effects
		14.7 Evaluating Model Fit
		14.8 Test of Model Parameters
		14.9 Other Important Issues in SEM
			14.9.1 Sample Size
			14.9.2 Input Matrix
			14.9.3 Estimation Process
			14.9.4 Model Respecification
		14.10 Case Study
			14.10.1 Background
			14.10.2 Variables and Data
			14.10.3 Conceptual Model
			14.10.4 Measurement Model
				Overall Fit of the Measurement Model
				Parameter Estimation
				Test of Model Parameters
			14.10.5 Structural Model
				Model Identification
				Model Respecification
				Overall Fit of the Structural Model
				Test of Model Parameters
				Direct, Indirect, and Total Effects
			14.10.6 Discussions and Insights
		14.11 Learning Summary
		Exercises
Bibliography
Appendix A1: Cumulative Standard Normal Distribution
Appendix A2: Percentage Points of Student’s T-Distribution
Appendix A3: Percentage Points of the Chi-Square Distribution
Appendix A4: Percentage Points of the F-Distribution (α = 0.10)
Appendix A5: Percentage Points of the F-Distribution (α = 0.05)
Appendix A6: Percentage Points of the F-Distribution (α = 0.01)
Index