Methodology for Multilevel Modeling in Educational Research: Concepts and Applications

Contents
Editor and Contributors
Part I Introduction
1 Hierarchical Linear Modeling and Multilevel Modeling in Educational Research
	1.1 Introduction
	1.2 Theoretical Foundations and Conceptual Frameworks
	1.3 Methodology for Multilevel Modeling
	1.4 Multilevel Analysis of PISA and TIMSS Data
	1.5 Multilevel Modeling in Educational Research
	1.6 Conclusion
Part II Theoretical Foundations and Conceptual Frameworks
2 A Primer for Using Multilevel Confirmatory Factor Analysis Models in Educational Research
	2.1 Introduction
		2.1.1 Conceptual Principles
		2.1.2 Benefits of Using the MCFA Framework
	2.2 Analysis Choices for MCFA
		2.2.1 Ignoring Nesting
		2.2.2 Accounting for Nested Data
		2.2.3 Evaluating Model Fit in MCFA
	2.3 Applied Example
		2.3.1 Areas for Future Research
	References
3 Multilevel Model Selection: Balancing Model Fit and Adequacy
	3.1 Introduction
	3.2 Conceptual Overview of Estimation in MLM
	3.3 Reliability of Cluster J
	3.4 Maximum Likelihood Estimation
		3.4.1 FIML
		3.4.2 REML
		3.4.3 FIML vs. REML
	3.5 Model Selection
	3.6 Criteria for Evaluating Model Fit
		3.6.1 Likelihood Ratio Test (LRT)
		3.6.2 Recommendations for Testing Random Effects
		3.6.3 Other Issues with Modeling Random Slopes
		3.6.4 Information Criteria (ICs)
	3.7 Model Adequacy
		3.7.1 Proportional Reduction in Variance Statistics
		3.7.2 Variance Decomposition Framework for MLM
	3.8 Conclusion
	References
4 Concepts and Applications of Multivariate Multilevel (MVML) Analysis and Multilevel Structural Equation Modeling (MLSEM)
	4.1 Introduction of Multilevel Modeling
	4.2 Multilevel Structural Equation Modeling (MLSEM)
	4.3 Examples of MLSEM Application
	4.4 Issues That Should Be Considered When Reporting MLSEM Applications
	4.5 Multivariate Multilevel Model (MVML)
	4.6 Basic Concepts and Assumptions of the Multivariate Multilevel Model
	4.7 Multivariate Random Intercept Model and Examples
	4.8 Examples of MVML Applications
	4.9 Summary
	References
5 Data Visualization for Pattern Seeking in Multilevel Modeling
	5.1 Introduction
	5.2 Data Source
	5.3 Preliminary Data Visualization
		5.3.1 Profile Analysis by Linking and Brushing
		5.3.2 Binning and Median Smoothing for Examining the Relationship between SES and Math Achievement
		5.3.3 Data Reduction for Examining the Relationship between SES and Math Achievement in Bubble Plot
		5.3.4 Examining Variation between Schools by ANOM Plot
	5.4 Multilevel Modeling
		5.4.1 Computing ICC to Decompose Variance Components by Random Effects Modeling
	5.5 Running a Mixed Model for Fixed and Random Effects using SES
		5.5.1 Using School Mean and Locally Centered SES to Disentangle Within and between Groups
	5.6 Conclusion
	References
6 Doubly Latent Multilevel Structural Equation Modeling: An Overview of Main Concepts and Empirical Illustration
	6.1 Introduction
	6.2 Doubly Latent Multilevel Models
		6.2.1 Contextual vs. Climate Effects in Doubly Latent Multilevel Models
		6.2.2 Measurement and Sampling Errors in Doubly Latent Multilevel Models
		6.2.3 Centering in Doubly Latent Multilevel Models
	6.3 Testing Mediation in Multilevel Structural Equation Modeling Framework
	6.4 An Empirical Illustration Using Doubly Latent Multilevel Structural Equation Modeling
		6.4.1 Theoretical Background and Description of the Study
		6.4.2 Estimating Measurement Model and Reliability
		6.4.3 Estimating Structural Model
	6.5 Conclusions
	References
Part III Methodology for Multilevel Modeling
7 Analyzing Large-Scale Assessment Data with Multilevel Analyses: Demonstration Using the Programme for International Student Assessment (PISA) 2018 Data
	7.1 Introduction
	7.2 PISA 2018 Data
	7.3 Complex Survey Design
		7.3.1 Sampling Weights
		7.3.2 Plausible Values
		7.3.3 Replicate Weights
	7.4 Specifying a Multilevel Model
	7.5 Example Analysis
	7.6 Obtaining the Data
	7.7 Model Specification
	7.8 Analysis
	7.9 Results
		7.9.1 Descriptive Statistics
		7.9.2 Math Achievement Outcome
		7.9.3 MASTGOAL Outcome
	7.10 Conclusion
	Appendix: R syntax
	References
8 Multilevel Modelling of International Large-Scale Assessment Data
	8.1 Introduction
	8.2 Sampling in ILSAs and the Use of Multilevel Models
	8.3 Outcome Variables in ILSAs
		8.3.1 Plausible Values
		8.3.2 Continuous and Non-continuous Performance Outcomes
	8.4 Configuration of Multilevel Models
	8.5 Sampling Weights in ILSAs
	8.6 Software
	8.7 Summary
	References
9 Transparency and Replicability of Multilevel Modeling Applications: A Guideline for Improved Reporting Practices
	9.1 Introduction
	9.2 Statement of Research Questions and Hypotheses
	9.3 Description of the Sampling Procedures
	9.4 Sample Size, Power, and Precision
	9.5 Psychometrics
	9.6 Missing data Treatment
	9.7 Model Specifications
	9.8 Estimation Methods and Software
	9.9 Statistical Inference
	9.10 Interpretation of Regression Coefficients
	9.11 Effect Sizes
	9.12 Assumption Checking
	9.13 Discussion
	9.14 Summary and Checklist
	References
10 Application of Multilevel Models to International Large-Scale Student Assessment Data
	10.1 Introduction
	10.2 Example of Typical Use—Modeling Relationship Between Socioeconomic Background and Student Achievement in PISA 2018
	10.3 Applications
	10.4 Plausible Values and Multilevel Models
	10.5 Survey Weights Adjustments for Multilevel Models
	10.6 Estimation of Standard Errors
	10.7 Multilevel Models with Additional Layers
	10.8 Summary
	References
Part IV Multilevel Analysis of PISA and TIMSS Data
11 Changing Trends in the Role of South African Math Teachers’ Qualification for Student Achievement: Findings from TIMSS 2003, 2011, 2015
	11.1 Introduction
	11.2 Teacher Education of Mathematics Teachers in Post-apartheid South Africa
	11.3 Teacher Allocation and Placement in Schools
	11.4 Research on Teacher Participation in Professional Development Programs
	11.5 Research on the Relationship Between Teacher Qualification and Learning Outcomes
	11.6 Research the Relationship Between Teacher Qualification and Learning Outcomes in Mathematics in South Africa
	11.7 Research Questions
	11.8 Data, Population, and Sample
	11.9 Measures
		11.9.1 Teacher Qualification
		11.9.2 Teacher Covariates
		11.9.3 Student Level Control Variables
		11.9.4 Classroom Context Control Variables
		11.9.5 Student Outcomes
		11.9.6 Analytic Strategy
		11.9.7 Limitations
	11.10 Results
		11.10.1 Percentage of Learners by Qualification Levels
		11.10.2 Teacher Allocations by Qualification Profile and School Student Composition
		11.10.3 Intensity and Focus of Teacher Participation in Formal Professional Development Activities
		11.10.4 Relationship Between Teacher Qualification and Student Outcomes
	11.11 Summary of Findings
	References
12 Revisiting the Relationship Between Science Teaching Practice and Scientific Literacy: Multi-level Analysis Using PISA
	12.1 Introduction
	12.2 Literature Review
		12.2.1 Benefits and Challenges of DI and IBT
		12.2.2 Review of Impacts of Teaching Practices (Inquiry Versus Direct) on Science Achievement
		12.2.3 Relationship Between Inquiry Teaching and Science Achievement in PISA Studies
	12.3 Methods
		12.3.1 Data and Sample
		12.3.2 Measures
		12.3.3 Multivariate Multilevel Model
	12.4 Results
	12.5 Discussion
		12.5.1 Implications for Teacher Education and Professional Development
		12.5.2 Limitations and Future Directions
	12.6 Conclusion
	References
13 Family Meals and Academic Performance: A Multilevel Analysis for Spain
	13.1 Introduction
	13.2 Literature Review
	13.3 Methodological Approach
		13.3.1 Variables
		13.3.2 Multilevel Modeling
	13.4 Results
	13.5 Conclusions
	References
14 Multilevel Modeling of Nordic Students’ Mathematics Achievements in TIMSS 2019
	14.1 Introduction
	14.2 Method
		14.2.1 Participants
		14.2.2 Statistical analyses
	14.3 Results
	14.4 Discussion
	References
15 Teachers’ Perceptions of School Ethical Culture: The Implicit Meaning of TIMSS
	15.1 Introduction
	15.2 Theoretical Background
		15.2.1 Ethics in the Context of National and Universal Culture
		15.2.2 Confusion Around the Definitions of Culture and Climate in the Context of Ethics
		15.2.3 School Ethical Culture
		15.2.4 The Ethical Aspects of TIMSS
	15.3 Method
		15.3.1 Context
		15.3.2 Sample
		15.3.3 Overview of Procedures and Analyses
	15.4 Results
	15.5 Discussion
	15.6 Conclusions
	Appendix: Selection rule for relevant items (Expert judgment)
	References
Part V Multilevel Modeling in Educational Research
16 Why They Want to Leave? A Three-Level Hierarchical Linear Modeling Analysis of Teacher Turnover Intention
	16.1 Introduction
	16.2 Multilevel Modeling and Teacher Turnover Research
	16.3 Teacher Turnover Intention
	16.4 Teacher- and School-Level Characteristics and Teacher Turnover Intention
	16.5 Country-Level Variables and Teacher Turnover Intentions
		16.5.1 Cross-Level Interactions (Moderation Effects)
	16.6 Method
	16.7 Findings
		16.7.1 The Effect of Individual and School Characteristics (Compositional Effects)
		16.7.2 The Effects of Country Variables
		16.7.3 The Cross-Level Interaction (The Moderation Effect of Country Variables)
	16.8 Conclusion and Implications
		16.8.1 Limitations
	References
17 Daycare Centers’ Composition and Non-native Children’s Language Skills at School Entry: Exploring the Nature of Context Effects Using Multilevel Modeling
	17.1 Introduction
	17.2 Purpose of the Present Study
	17.3 Background
		17.3.1 Theoretical Framework
		17.3.2 Summary of Previous Research on Composition Effects
		17.3.3 Research Question
	17.4 Multilevel Modeling as an Analytical Strategy for Context Effects
		17.4.1 Five-Step Analytical Process
		17.4.2 Model Fits and Indices
		17.4.3 Assumptions and Data Requirements
	17.5 Applying MLM: Composition Effects and Non-native Children’s Language Skills
		17.5.1 Sample
		17.5.2 Variables
		17.5.3 Modeling Approach
		17.5.4 Results
	17.6 Discussion
	References
18 Gender Effect at the Beginning of Higher Education Careers in STEM Studies: Does Female Recover Better Than Male?
	18.1 Introduction
	18.2 Empirical Framework
		18.2.1 Internal Student Mobility in HE in Italy
		18.2.2 A Brief Overview on Students’ Performance, Transfer Shock, and Gender
		18.2.3 Using Multilevel Modelling in HE in Italy
		18.2.4 Measuring Academic Performance in This Study
	18.3 Data, Variables, and Method
		18.3.1 Data
		18.3.2 Variables
		18.3.3 Method
	18.4 Results
	18.5 Conclusion
	References
19 Service Satisfaction and Service Quality: A Longitudinal and Multilevel Study of User Satisfaction with Kindergartens in Norway
	19.1 Introduction
		19.1.1 Norwegian Kindergartens: Institutional Setting
	19.2 Service Satisfaction and Quality: Expectations
	19.3 Data and Variables
	19.4 Statistical Approach
	19.5 Results
	19.6 Conclusion
	Appendix
	References
20 Multilevel Modeling and Assessment of the Study-Relevant Knowledge of First-Year Students in a Master's Program in Business and Economics
	20.1 Introduction and Research Focus
	20.2 Theoretical Foundation and Hypotheses
	20.3 The Assessment of Economic Knowledge: Study Design, Instruments, and Sampling
	20.4 Methods and Results of Structural Equation Models in the Multilevel Approach
		20.4.1 The Multilevel Approach in a Structural Equation Model
		20.4.2 Confirmatory Factor Analysis (CFA)
		20.4.3 Two-Dimensional Multilevel Model with Covariates (MMIMIC Models)
	20.5 Discussion and Conclusion
	References
Index