Margins of Error: A Study of Reliability in Survey Measurement (Wiley Series in Survey Methodology)

Cover Page......Page 1
Title: Margins of Error - A Study of Reliability in Survey Measurement......Page 4
ISBN 0470081481......Page 5
Contents (with page links)......Page 7
Preface......Page 11
Acknowledgments......Page 13
Foreword......Page 15
1 Measurement Errors in Surveys......Page 17
1.1 WHY STUDY SURVEY MEASUREMENT ERROR?......Page 18
1.2 SURVEY ERRORS......Page 19
1.2.1 Classifying Types of Survey Error......Page 21
1.3 SURVEY MEASUREMENT ERRORS......Page 22
1.4 STANDARDS OF MEASUREMENT......Page 24
1.5 RELIABILITY OF MEASUREMENT......Page 25
1.6 THE NEED FOR FURTHER RESEARCH......Page 26
1.7 THE PLAN OF THIS BOOK......Page 27
2 Sources of Survey Measurement Error......Page 31
2.1 THE UBIQUITY OF MEASUREMENT ERRORS......Page 32
2.2 SOURCES OF MEASUREMENT ERROR IN SURVEY REPORTS......Page 36
2.2.1 Validity of the Question......Page 38
2.2.2 Comprehensionof the Question......Page 39
2.2.3 Information Accessibility......Page 41
2.2.4 Retrieval of Information......Page 42
2.2.5 Respondent Motivation......Page 45
2.2.6 Communication......Page 46
2.3 CONSEQUENCES OF MEASUREMENT ERROR......Page 48
3 Reliability Theory for Survey Measures......Page 51
3.2 BASIC CONCEPTS OF CLASSICAL RELIABILITY THEORY......Page 52
3.3 NONRANDOM MEASUREMENT ERROR......Page 57
3.4 THE COMMON-FACTORMODEL REPRESENTATION OF CTST......Page 58
3.5 SCALING OF VARIABLES......Page 59
3.6 DESIGNS FOR RELIABILITY ESTIMATION......Page 61
3.7 VALIDITY AND MEASUREMENT......Page 62
3.7.1 Record-Check Studies......Page 64
3.7.2 Split-Ballot Experiments......Page 66
3.8 RELIABILITY MODELS FOR COMPOSITE SCORES......Page 67
3.9 DEALING WITH NONRANDOM OR SYSTEMATIC ERROR......Page 69
3.10 SAMPLING CONSIDERATIONS......Page 71
3.11 CONCLUSIONS......Page 73
4 Reliability Methods for Multiple Measures......Page 75
4.1 MULTIPLE MEASURES VERSUS MULTIPLE INDICATORS......Page 77
4.1.1 Multiple Measures......Page 78
4.1.2 Multiple-Indicators Models......Page 81
4.2 MULTITRAIT-MULTIMETHOD APPROACHES......Page 83
4.2.1 The MTMM Matrix......Page 84
4.3 COMMON-FACTORMODELS OF THE MTMM DESIGN......Page 87
4.3.1 Decomposition of Variance/Covariance......Page 90
4.4 CLASSICAL TRUE-SCORE REPRESENTATION OF THE MTMM MODEL......Page 93
4.5 THE GROWING BODY OF MTMM STUDIES......Page 95
4.5.1 Clarification of Terminology......Page 96
4.6 AN EXAMPLE......Page 99
4.6.1 Interpretation......Page 106
4.7 CRITIQUE OF THE MTMM APPROACH......Page 107
4.8 WHERE ARE WE?......Page 109
5 Longitudinal Methods for Reliability Estimation......Page 111
5.1 THE TEST-RETEST METHOD......Page 112
5.2 SOLUTIONS TO THE PROBLEM......Page 117
5.2.1 The Quasi-Markov Simplex Model......Page 118
5.3 ESTIMATING RELIABILITY USING THE QUASI-MARKOVSIMPLEX MODEL......Page 120
5.4 CONTRIBUTIONS OF THE LONGITUDINAL APPROACH......Page 126
5.5 COMPONENTS OF THE SURVEY RESPONSE......Page 130
5.6 WHERE TO FROM HERE?......Page 132
6 Using Longitudinal Data to Estimate Reliability Parameters......Page 133
6.1 RATIONALE FOR THE PRESENT STUDY......Page 134
6.2.1 National Election Panel Studies......Page 135
6.2.3 The Study of American Families......Page 137
6.3 DOMAINS OF MEASUREMENT......Page 138
6.4 STATISTICAL ESTIMATION STRATEGIES......Page 143
6.5 COMPARISON OF METHODS OF RELIABILITY ESTIMATION......Page 146
6.6 THE PROBLEM OF ATTRITION......Page 151
6.6.1 Estimation of Reliability in the Presence of Incomplete Data......Page 153
6.7 WHICH RELIABILITY ESTIMATES?......Page 162
6.8 CONCLUSIONS......Page 163
7 The Source and Content of Survey Questions......Page 165
7.1 SOURCE OF INFORMATION......Page 166
7.2 PROXY REPORTS......Page 168
7.3 CONTENT OF QUESTIONS......Page 169
7.3.1 Reliability in Measuring Self-Reported Facts......Page 172
7.3.2 The Reliability of Measuring Self-Reported Nonfacts......Page 174
7.4 SUMMARY AND CONCLUSIONS......Page 178
8 Survey Question Context......Page 181
8.1 THE ARCHITECTURE OF SURVEY QUESTIONNAIRES......Page 183
8.2 QUESTIONS IN SERIES VERSUS QUESTIONS IN BATTERIES......Page 187
8.3 LOCATION IN THE QUESTIONNAIRE......Page 188
8.4 UNIT LENGTH AND POSITION IN SERIES AND BATTERIES......Page 191
8.5 LENGTH OF INTRODUCTIONS TO SERIES AND BATTERIES......Page 193
8.6 CONCLUSIONS......Page 195
9 Formal Properties of Survey Questions......Page 197
9.1 QUESTION FORM......Page 199
9.2 TYPES OF CLOSED-FORM QUESTIONS......Page 201
9.2.1 Agree-Disagree Questions......Page 202
9.2.3 Feeling Thermometers......Page 204
9.3 NUMBER OF RESPONSE CATEGORIES......Page 207
9.4 UNIPOLAR VERSUS BIPOLAR SCALES......Page 211
9.5 DON’T KNOW OPTIONS......Page 212
9.6 VERBAL LABELING OF RESPONSE CATEGORIES......Page 216
9.7 SURVEY QUESTION LENGTH......Page 218
9.8 CONCLUSIONS......Page 226
10 Attributes of Respondents......Page 229
10.1 RELIABILITY AS A POPULATION PARAMETER......Page 230
10.2.1 Strategies of Analysis......Page 231
10.3 AGE AND RELIABILITY OF MEASUREMENT......Page 234
10.4 SCHOOLING AND RELIABILITY OF MEASUREMENT......Page 237
10.5.1 Age Differences in Schooling......Page 239
10.5.2 Schooling-AdjustedAge Differences in Reliability......Page 241
10.6 GENERATIONAL DIFFERENCES IN RELIABILITY......Page 243
10.7 MTMM RESULTS BY AGE AND EDUCATION......Page 244
10.8 STATISTICAL ESTIMATION OF COMPONENTS VARIATION......Page 247
10.8.2 Educationand Reliability of Measurementin the HRS......Page 253
10.8.3 Age and Components of Variance......Page 258
10.8.4 Education and Components of Variance......Page 263
10.9 TESTS OF HYPOTHESES ABOUT GROUP DIFFERENCES......Page 270
10.10 CONCLUSIONS......Page 277
11 Reliability Estimation for Categorical Latent Variables......Page 279
11.1 BACKGROUND AND RATIONALE......Page 280
11.2 THE LATENT CLASS MODEL FOR MULTIPLE INDICATORS......Page 281
11.2.1 The Model......Page 282
11.2.2 An Example......Page 284
11.3 THE LATENT CLASS MODEL FOR MULTIPLE MEASURES......Page 288
11.3.1 An Example......Page 289
11.4 THE LATENT MARKOV MODEL......Page 293
11.4.1 The Model......Page 295
11.4.2 An Example......Page 296
11.5 CONCLUSIONS......Page 302
12 Final Thoughts and Future Directions......Page 305
12.1 RELIABILITY AS AN OBJECT OF STUDY......Page 306
12.2 WHY STUDY THE RELIABILITY OF SURVEY MEASUREMENT?......Page 307
12.2.1 Effects of Unreliability on Statistical Inference......Page 308
12.2.3 Effects of Unreliability on Estimates of Statistical Association......Page 314
12.3 THE LONGITUDINAL APPROACH......Page 315
12.4 ASSEMBLING KNOWLEDGE OF SURVEY MEASUREMENT RELIABILITY......Page 317
12.4.1 Some Comparisonsof Reliability Estimates......Page 318
12.4.2 Considerations in Archiving Reliability Estimates......Page 320
12.5 COMPENSATING FOR MEASUREMENT ERROR USING COMPOSITE VARIABLES......Page 324
12.6 CONCLUSIONS......Page 331
12.6.1 Future Agenda......Page 332
12.6.2 Closing......Page 334
APPENDIX Reliability of Survey Measures Used in the Present Study......Page 343
References......Page 383
Index (with page links)......Page 399
WILEY SERIES IN SURVEY METHODOLOGY......Page 407
Back Page......Page 409