ANOVA and ANCOVA: A GLM Approach

ANOVA and ANCOVA A GLM Approach ... 5
Contents ... 7
Acknowledgments ... 15
1 An Introduction to General Linear Models: Regression, Analysis of Variance, and Analysis of Covariance ... 17
	1.1 Regression, Analysis of Variance, and Analysis of Covariance ... 17
	1.2 A Pocket History of Regression, ANOVA, and ANCOVA ... 18
	1.3 An Outline of General Linear Models (GLMs) ... 19
		1.3.1 Regression ... 20
		1.3.2 Analysis of Variance ... 21
		1.3.3 Analysis of Covariance ... 21
	1.4 The \"General\" in GLM ... 22
	1.5 The \"Linear\" in GLM ... 24
	1.6 Least Squares Estimates ... 27
	1.7 Fixed, Random, and Mixed Effects Analyses ... 28
	1.8 The Benefits of a GLM Approach to ANOVA and ANCOVA ... 29
	1.9 The GLM Presentation ... 30
	1.10 Statistical Packages for Computers ... 31
2 Traditional and GLM Approaches to Independent Measures Single Factor ANOVA Designs ... 33
	2.1 Independent Measures Designs ... 33
	2.2 Balanced Data Designs ... 35
	2.3 Factors and Independent Variables ... 36
	2.4 An Outline of Traditional ANOVA for Single Factor Designs ... 37
	2.5 Variance ... 39
	2.6 Traditional ANOVA Calculations for Single Factor Designs ... 41
	2.7 Confidence Intervals ... 46
	2.8 GLM Approaches to Single Factor ANOVA ... 47
		2.8.1 Experimental Design GLMs ... 47
		2.8.2 Estimating Effects by Comparing Full and Reduced Experimental Design GLMs ... 53
		2.8.3 Regression GLMs ... 57
		2.8.4 Schemes for Coding Experimental Conditions ... 57
			2.8.4.1 Dummy Coding ... 57
			2.8.4.2 Why Only (p – 1) Variables Are Used to Represent All Experimental Conditions? ... 60
			2.8.4.3 Effect Coding ... 63
		2.8.5 Coding Scheme Solutions to the Overparameterization Problem ... 66
		2.8.6 Cell Mean GLMs ... 66
		2.8.7 Experimental Design Regression and Cell Mean GLMs ... 67
3 Comparing Experimental Condition Means, Multiple Hypothesis Testing, Type 1 Error, and a Basic Data Analysis Strategy ... 69
	3.1 Introduction ... 69
	3.2 Comparisons Between Experimental Condition Means ... 71
	3.3 Linear Contrasts ... 72
	3.4 Comparison Sum of Squares ... 73
	3.5 Orthogonal Contrasts ... 74
	3.6 Testing Multiple Hypotheses ... 78
		3.6.1 Type 1 and Type 2 Errors ... 79
		3.6.2 Type 1 Error Rate Inflation with Multiple Hypothesis Testing ... 81
		3.6.3 Type 1 Error Rate Control and Analysis Power ... 82
		3.6.4 Different Conceptions of Type 1 Error Rate ... 84
			3.6.4.1 Testwise Type 1 Error Rate ... 84
			3.6.4.2 Family wise Type 1 Error Rate ... 85
			3.6.4.3 Experimentwise Type 1 Error Rate ... 86
			3.6.4.4 False Discovery Rate ... 86
		3.6.5 Identifying the \"Family\" in Familywise Type 1 Error Rate Control ... 87
		3.6.6 Logical and Empirical Relations ... 88
			3.6.6.1 Logical Relations ... 88
			3.6.6.2 Empirical Relations ... 90
	3.7 Planned and Unplanned Comparisons ... 92
		3.7.1 Direct Assessment of Planned Comparisons ... 93
		3.7.2 Contradictory Results with ANOVA Omnibus F-tests and Direct Planned Comparisons ... 94
	3.8 A Basic Data Analysis Strategy ... 95
		3.8.1 ANOVA First? ... 95
		3.8.2 Strong and Weak Type 1 Error Control ... 96
		3.8.3 Stepwise Tests ... 97
		3.8.4 Test Power ... 98
	3.9 The Three Basic Stages of Data Analysis ... 99
		3.9.1 Stage 1 ... 99
		3.9.2 Stage 2 ... 99
			3.9.2.1 Rom\'s Test ... 99
			3.9.2.2 Shaffer\'s R Test ... 100
			3.9.2.3 Applying Shaffer\'s R Test After a Significant F-test ... 102
		3.9.3 Stage 3 ... 105
	3.10 The Role of the Omnibus F-Test ... 107
4 Measures of Effect Size and Strength of Association, Power, and Sample Size ... 109
	4.1 Introduction ... 109
	4.2 Effect Size as a Standardized Mean Difference ... 110
	4.3 Effect Size as Strength of Association (SOA) ... 112
		4.3.1 SOA for Specific Comparisons ... 114
	4.4 Small, Medium, and Large Effect Sizes ... 115
	4.5 Effect Size in Related Measures Designs ... 115
	4.6 Overview of Standardized Mean Difference and SOA Measures of Effect Size ... 116
	4.7 Power ... 117
		4.7.1 Influences on Power ... 117
		4.7.2 Uses of Power Analysis ... 119
		4.7.3 Determining the Sample Size Needed to Detect the Omnibus Effect ... 120
		4.7.4 Determining the Sample Size Needed to Detect Specific Effects ... 123
		4.7.5 Determining the Power Level of a Planned or Completed Study ... 125
		4.7.6 The Fallacy of Observed Power ... 126
5 GLM Approaches to Independent Measures Factorial Designs ... 127
	5.1 Factorial Designs ... 127
	5.2 Factor Main Effects and Factor Interactions ... 128
		5.2.1 Estimating Effects by Comparing Full and Reduced Experimental Design GLMs ... 133
	5.3 Regression GLMs for Factorial ANOVA ... 137
	5.4 Estimating Effects with Incremental Analysis ... 139
		5.4.1 Incremental Regression Analysis ... 140
			5.4.1.1 Step 1 ... 140
			5.4.1.2 Step 2 ... 140
			5.4.1.3 Step 3 ... 141
	5.5 Effect Size Estimation ... 142
		5.5.1 SOA for Omnibus Main and Interaction Effects ... 142
			5.5.1.1 Complete ?2 for Main and Interaction Effects ... 142
			5.5.1.2 Partial ?2 for Main and Interaction Effects ... 143
		5.5.2 Partial ?2 for Specific Comparisons ... 143
	5.6 Further Analyses ... 144
		5.6.1 Main Effects: Encoding Instructions and Study Time ... 144
		5.6.2 Interaction Effect: Encoding Instructions × Study Time ... 147
			5.6.2.1 Simple Effects: Comparing the Three Levels of Factor B at a1, and at a2 ... 148
			5.6.2.2 Simple Effects: Comparing the Two Levels of Factor A at b1, at b2, and at b3 ... 151
	5.7 Power ... 152
		5.7.1 Determining the Sample Size Needed to Detect Omnibus Main Effects and Interactions ... 152
		5.7.2 Determining the Sample Size Needed to Detect Specific Effects ... 154
6 GLM Approaches to Related Measures Designs ... 155
	6.1 Introduction ... 155
		6.1.1 Randomized Block Designs ... 156
		6.1.2 Matched Sample Designs ... 157
		6.1.3 Repeated Measures Designs ... 157
	6.2 Order Effect Controls in Repeated Measures Designs ... 160
		6.2.1 Randomization ... 160
		6.2.2 Counterbalancing ... 160
			6.2.2.1 Crossover Designs ... 160
			6.2.2.2 Latin Square Designs ... 161
	6.3 The GLM Approach to Single Factor Repeated Measures Designs ... 162
	6.4 Estimating Effects by Comparing Full and Reduced Repeated Measures Design GLMs ... 169
	6.5 Regression GLMs for Single Factor Repeated Measures Designs ... 172
	6.6 Effect Size Estimation ... 176
		6.6.1 A Complete ?2 SOA for the Omnibus Effect Comparable Across Repeated and Independent Measures Designs ... 176
		6.6.2 A Partial ?2 SOA for the Omnibus Effect Appropriate for Repeated Measures Designs ... 177
		6.6.3 A Partial ?2 SOA for Specific Comparisons Appropriate for Repeated Measures Designs ... 178
	6.7 Further Analyses ... 178
	6.8 Power ... 184
		6.8.1 Determining the Sample Size Needed to Detect the Omnibus Effect ... 184
		6.8.2 Determining the Sample Size Needed to Detect Specific Effects ... 185
7 The GLM Approach to Factorial Repeated Measures Designs ... 187
	7.1 Factorial Related and Repeated Measures Designs ... 187
	7.2 Fully Repeated Measures Factorial Designs ... 188
	7.3 Estimating Effects by Comparing Full and Reduced Experimental Design GLMs ... 195
	7.4 Regression GLMs for the Fully Repeated Measures Factorial ANOVA ... 196
	7.5 Effect Size Estimation ... 202
		7.5.1 A Complete ?2 SOA for Main and Interaction Omnibus Effects Comparable Across Repeated Measures and Independent Designs ... 202
		7.5.2 A Partial ?2 SOA for the Main and Interaction Omnibus Effects Appropriate for Repeated Measures Designs ... 203
		7.5.3 A Partial ?2 SOA for Specific Comparisons Appropriate for Repeated Measures Designs ... 204
	7.6 Further Analyses ... 204
		7.6.1 Main Effects: Encoding Instructions and Study Time ... 204
		7.6.2 Interaction Effect: Encoding Instructions × Study Time ... 207
			7.6.2.1 Simple Effects: Comparison of Differences Between the Three Levels of Factor B (Study Time) at Each Level of Factor A (Encoding Instructions) ... 207
			7.6.2.2 Simple Effects: Comparison of Differences Between the Two Levels of Factor A (Encoding Instructions) at Each Level of Factor B (Study Time) ... 209
	7.7 Power ... 213
8 GLM Approaches to Factorial Mixed Measures Designs ... 215
	8.1 Mixed Measures and Split-Plot Designs ... 215
	8.2 Factorial Mixed Measures Designs ... 216
	8.3 Estimating Effects by Comparing Full and Reduced Experimental Design GLMs ... 221
	8.4 Regression GLM for the Two-Factor Mixed Measures ANOVA ... 222
	8.5 Effect Size Estimation ... 227
	8.6 Further Analyses ... 227
		8.6.1 Main Effects: Independent Factor—Encoding Instructions ... 227
		8.6.2 Main Effects: Related Factor—Study Time ... 228
		8.6.3 Interaction Effect: Encoding Instructions × Study Time ... 228
			8.6.3.1 Simple Effects: Comparing Differences Between the Three Levels of Factor B (Study Time) at Each Level of Factor A (Encoding Instructions) ... 228
			8.6.3.2 Simple Effects: Comparing Differences Between the Two Levels of Factor A (Encoding Instructions) at Each Level of Factor B (Study Time) ... 228
	8.7 Power ... 230
9 The GLM Approach to ANCOVA ... 231
	9.1 The Nature of ANCOVA ... 231
	9.2 Single Factor Independent Measures ANCOVA Designs ... 232
	9.3 Estimating Effects by Comparing Full and Reduced ANCOVA GLMs ... 237
	9.4 Regression GLMs for the Single Factor, Single-Covariate ANCOVA ... 242
	9.5 Further Analyses ... 245
	9.6 Effect Size Estimation ... 247
		9.6.1 A Partial ?2 SOA for the Omnibus Effect ... 247
		9.6.2 A Partial ?2 SOA for Specific Comparisons ... 248
	9.7 Power ... 248
	9.8 Other ANCOVA Designs ... 249
		9.8.1 Single Factor and Fully Repeated Measures Factorial ANCOVA Designs ... 249
		9.8.2 Mixed Measures Factorial ANCOVA ... 249
10 Assumptions Underlying ANOVA, Traditional ANCOVA, and GLMs ... 251
	10.1 Introduction ... 251
	10.2 ANOVA and GLM Assumptions ... 251
		10.2.1 Independent Measures Designs ... 252
		10.2.2 Related Measures ... 254
			10.2.2.1 Assessing and Dealing with Sphericity Violations ... 254
		10.2.3 Traditional ANCOVA ... 256
	10.3 A Strategy for Checking GLM and Traditional ANCOVA Assumptions ... 257
	10.4 Assumption Checks and Some Assumption Violation Consequences ... 258
		10.4.1 Independent Measures ANOVA and ANCOVA Designs ... 259
			10.4.1.1 Random Sampling ... 259
			10.4.1.2 Independence ... 260
			10.4.1.3 Normality ... 261
			10.4.1.4 Homoscedasticity: Homogeneity of Variance ... 264
		10.4.2 Traditional ANCOVA Designs ... 266
			10.4.2.1 Covariate Independent of Experimental Conditions ... 266
			10.4.2.2 Linear Regression ... 268
			10.4.2.3 Homogeneous Regression ... 272
	10.5 Should Assumptions be Checked? ... 275
11 Some Alternatives to Traditional ANCOVA ... 279
	11.1 Alternatives to Traditional ANCOVA ... 279
	11.2 The Heterogeneous Regression Problem ... 280
	11.3 The Heterogeneous Regression ANCOVA GLM ... 281
	11.4 Single Factor Independent Measures Heterogeneous Regression ANCOVA ... 282
	11.5 Estimating Heterogeneous Regression ANCOVA Effects ... 284
	11.6 Regression GLMs for Heterogeneous Regression ANCOVA ... 289
	11.7 Covariate–Experimental Condition Relations ... 292
		11.7.1 Adjustments Based on the General Covariate Mean ... 292
		11.7.2 Multicolinearity ... 293
	11.8 Other Alternatives ... 294
		11.8.1 Stratification (Blocking) ... 294
		11.8.2 Replacing the Experimental Conditions with the Covariate ... 295
	11.9 The Role of Heterogeneous Regression ANCOVA ... 296
12 Multilevel Analysis for the Single Factor Repeated Measures Design ... 297
	12.1 Introduction ... 297
	12.2 Review of the Single Factor Repeated Measures Experimental Design GLM and ANOVA ... 298
	12.3 The Multilevel Approach to the Single Factor Repeated Measures Experimental Design ... 299
	12.4 Parameter Estimation in Multilevel Analysis ... 304
	12.5 Applying Multilevel Models with Different Covariance Structures ... 305
		12.5.1 Using SYSTAT to Apply the Multilevel GLM of the Repeated Measures Experimental Design GLM ... 305
			12.5.1.1 The Linear Mixed Model ... 307
			12.5.1.2 The Hierarchical Linear Mixed Model ... 311
		12.5.2 Applying Alternative Multilevel GLMs to the Repeated Measures Data ... 314
	12.6 Empirically Assessing Different Multilevel Models ... 319
Appendix A ... 321
Appendix B ... 323
Appendix C ... 331
References ... 341
Index ... 355