Public Program Evaluation: A Statistical Guide

Contents......Page 5
Preface......Page 9
What Is Program Evaluation?......Page 13
Types of Program Evaluations......Page 18
Basic Characteristics of Program Evaluation......Page 23
Assessing Government Performance: Program Evaluation and GPRA......Page 25
A Brief History of Program Evaluation......Page 26
What Comes Next......Page 28
Key Concepts......Page 29
Do It Yourself......Page 30
Program Evaluation and Performance Measurement: What Is the Difference?......Page 31
Benchmarking......Page 34
Reporting Performance Results......Page 35
Conclusion......Page 37
Do It Yourself......Page 39
Defining Defensibility......Page 43
Types of Validity: Definitions......Page 44
Types of Validity: Threats and Simple Remedies......Page 45
Basic Concepts......Page 64
Do It Yourself......Page 65
The Logic of Internal Validity......Page 66
Making Comparisons: Cross Sections and Time Series......Page 69
Threats to Internal Validity......Page 70
Summary......Page 78
Three Basic Research Designs......Page 79
Rethinking Validity: The Causal Model Workhorse......Page 81
Basic Concepts......Page 83
A Summary of Threats to Internal Validity......Page 84
Basic Characteristics......Page 86
Brief History......Page 87
Caveats and Cautions About Randomized Experiments......Page 88
Types of RFEs......Page 92
Issues in Implementing RFEs......Page 104
Threats to the Validity of RFEs: Internal Validity......Page 108
Threats to the Validity of RFEs: External Validity......Page 111
Conclusion......Page 113
Basic Concepts......Page 114
Do It Yourself: Design a Randomized Field Experiment......Page 115
Defining Quasi-Experimental Designs......Page 116
The One-Shot Case Study......Page 117
The Posttest-Only Comparison-Group (PTCG) Design......Page 119
The Pretest Posttest Comparison-Group (PTPTCG) (The Nonequivalent Control-Group) Design......Page 125
The Pretest Posttest (Single-Group) Design......Page 128
The Single Interrupted Time-Series Design......Page 130
The Interrupted Time-Series Comparison-Group (ITSCG) Design......Page 137
The Multiple Comparison-Group Time-Series Design......Page 140
Summary of Quasi-Experimental Design......Page 141
Basic Concepts......Page 142
Do It Yourself......Page 143
What Is a Nonexperimental Design?......Page 144
Back to the Basics: The Workhorse Diagram......Page 145
The Nonexperimental Workhorse Regression Equation......Page 146
Data for the Workhorse Regression Equation......Page 148
Interpreting Multiple Regression Output......Page 151
Assumptions Needed to Believe That b Is a Valid Estimate of B [E(b) = B]......Page 165
Assumptions Needed to Believe the Significance Test for b......Page 183
What Happened to the R 2 ?......Page 189
Basic Concepts......Page 190
Introduction to Stata......Page 193
Do It Yourself: Interpreting Nonexperimental Results......Page 197
8 Designing Useful Surveys for Evaluation......Page 202
The Response Rate......Page 203
How to Write Questions to Get Unbiased, Accurate, Informative Responses......Page 210
Turning Responses into Useful Information......Page 217
For Further Reading......Page 225
Basic Concepts......Page 226
Do It Yourself......Page 227
9 Summing It Up Meta-Analysis......Page 230
Example of a Meta-Analyis: Data......Page 231
Example of a Meta-Analysis: Variables......Page 232
Example of a Meta-Analysis: Data Analysis......Page 233
The Role of Meta-Analysis in Program Evaluation and Causal Conclusions......Page 234
For Further Reading......Page 235
Notes......Page 237
Index......Page 259
About the Authors......Page 271