Causality in Policy Studies: a Pluralist Toolbox

Preface
Acknowledgments
Contents
Chapter 1: Introduction: The Elephant of Causation and the Blind Sages
	1.1 Policy Decisions and Causal Theories
	1.2 The Elephant of Causation
		1.2.1 Elephants by the Principle
		1.2.2 Elephants by the Rules
			1.2.2.1 Regularity
			1.2.2.2 Counterfactual
	1.3 The Blind Sages’ Portrayals as the Book’s Blueprint
		1.3.1 Can this Single Factor Make Any Difference?
		1.3.2 Through Which Structures?
		1.3.3 Through Which Process?
		1.3.4 Considerations and Extensions
	References
Untitled
Chapter 2: Causation in the Social Realm
	2.1 Why Discuss the Ontology of Causation?
	2.2 Scientific Realism About the Social World and Social Causation
		2.2.1 Critical Realism
	2.3 What Is Causation?
		2.3.1 Causal Mechanisms
		2.3.2 Causal Powers
		2.3.3 Manipulability and Invariance
	2.4 Pluralism About Causal Inquiry
		2.4.1 Case Studies and Process Tracing
		2.4.2 Quantitative Research Based on Observational Data
		2.4.3 Randomized Controlled Trials and Quasi-experimental Research
		2.4.4 Generative Models and Simulation Methods
	2.5 Realism and Methodological Pluralism
	References
		Suggested Readings
Chapter 3: Counterfactuals with Experimental and Quasi-Experimental Variation
	3.1 Introduction
	3.2 Causation and Counterfactual Impact Evaluation: The Jargon
		3.2.1 Causes as Manipulable Treatments
		3.2.2 Effects as Differences Between Factual and Counterfactual Outcomes
		3.2.3 What the Data Tell (And When)
	3.3 Shades of Validity
		3.3.1 Internal Validity: The Ability to Make a Causal Claim from a Pattern Documented in the Data
		3.3.2 Statistical Validity: Measuring Precisely the Relationship Between Causes and Outcomes in the Data
		3.3.3 External Validity: The Ability to Extend Conclusions to a Larger Population, over Time and Across Contexts
	3.4 Random Assignment Strengthens Internal Validity
	3.5 Internally Valid Reasoning Without RCTs: Instrumental Variation
		3.5.1 A Tale of Pervasive Manipulation
		3.5.2 General Formulation of the Problem
		3.5.3 Assumptions
			3.5.3.1 The “Monotonicity” Assumption
			3.5.3.2 The “As Good as Random” Assumption
			3.5.3.3 The “Exclusion Restriction”
			3.5.3.4 The “First-Stage” Requirement
		3.5.4 Better LATE than Never
		3.5.5 External Validity of Causal Conclusions
	3.6 Causal Reasoning with Administrative Rules: The Case of Regression Discontinuity Designs
		3.6.1 Larger Classes, Worse Outcomes?
		3.6.2 Visual Interpretation
		3.6.3 General Formulation of the Problem
			3.6.3.1 The Sharp RD Design
			3.6.3.2 The Fuzzy RD Design
		3.6.4 Validating the Internal Validity of the Design
	3.7 Conclusion
	References
		Suggested Readings
Chapter 4: Correlation Is Not Causation, Yet… Matching and Weighting for Better Counterfactuals
	4.1 Introduction
	4.2 Not Just a Mantra: Correlation Is Not Causation Because…
		4.2.1 Causal Inference Entails an Identification Problem
		4.2.2 Each Identification Strategy Entails a Set of Assumptions
		4.2.3 Last but not Least: Model Dependence
	4.3 Preprocessing Data with Matching to Improve the Credibility of the Estimates
		4.3.1 No Magic: What Matching Can and Cannot Do
		4.3.2 Useful Starting Point: Exact Matching
		4.3.3 Propensity Score Tautology
		4.3.4 How to Choose Among Matching Procedures?
		4.3.5 The End: The Parametric Outcome Analysis
	4.4 Empirical Illustration
		4.4.1 Entropy Balancing
		4.4.2 Coarsened Exact Matching
	4.5 Conclusion
	References
		Suggested Readings
Chapter 5: Getting the Most Out of Surveys: Multilevel Regression and Poststratification
	5.1 Introduction
	5.2 How It Works
	5.3 Running Example
		5.3.1 Draw a Sample
			5.3.1.1 Step 1: Fit a Model
			5.3.1.2 Step 2: Construct the Poststratification Frame
			5.3.1.3 Step 3: Predict and Poststratify
		5.3.2 Beware Overfitting
		5.3.3 Partial Pooling
		5.3.4 Sample Size Is Critical
		5.3.5 Stacked Regression and Poststratification (SRP)
		5.3.6 Synthetic Poststratification
		5.3.7 Best Performing
	5.4 Conclusion
	References
Chapter 6: Pathway Analysis, Causal Mediation, and the Identification of Causal Mechanisms
	6.1 Introduction
	6.2 Can Pathways Be Mechanisms?
	6.3 Identifying Causal Mechanisms with Graphs
		6.3.1 Closing the Backdoor
		6.3.2 Closing the Front Door
	6.4 Identifying Indirect Effects
		6.4.1 Indirect Effect in Non-linear Systems
		6.4.2 Indirect Effect When the Cause and the Mediator Interact
		6.4.3 Wrapping Up
	6.5 Applications
		6.5.1 A Mechanistic View on the Worm Wars
		6.5.2 A Mechanistic View on a Chicago School Reform
	6.6 Thou Shall Not Raise Causal Illusions
	References
Chapter 7: Testing Joint Sufficiency Twice: Explanatory Qualitative Comparative Analysis
	7.1 Introduction
	7.2 Interpretability
		7.2.1 Mechanisms and Machines
		7.2.2 Operationalizing Typological Theories
		7.2.3 Assembling Configurational Hypotheses
	7.3 Validity
		7.3.1 QCA’s Algebra
			7.3.1.1 Literals
			7.3.1.2 Operators
				Negation
				Joint Occurrence
				Alternatives
				Necessity and Sufficiency
			7.3.1.3 Truth Tables
		7.3.2 Identifying Valid Inus Hypotheses
			7.3.2.1 Rendering Hypotheses
			7.3.2.2 Tackling Underspecification
				Decision 1: Frequency Cut-Off
				Decision 2: The Consistency Threshold
				Decision 3: The Coverage Cut-Off
			7.3.2.3 Tackling Overspecification
				Irrelevant Components
				A Note on Ambiguity in Solutions
				Dealing with Trivial Factors
	7.4 Soundness
		7.4.1 Gauging for QCA: The Theoretical Side
			7.4.1.1 The Starting Point
			7.4.1.2 Ragin’s Reinvention
			7.4.1.3 Fuzzy Sufficiency and Necessity
		7.4.2 Gauging for QCA: The Empirical Side
			7.4.2.1 Establishing the Universe of Reference
			7.4.2.2 Operationalizing Intension
				Hyper-Specificity
				Hyper-Generality
				The Problem of Missing Values
			7.4.2.3 Identifying Membership Thresholds
	7.5 Summing Up
	References
		Suggested Readings
Chapter 8: Causal Inference and Policy Evaluation from Case Studies Using Bayesian Process Tracing
	8.1 Introduction
	8.2 The Epistemic Foundations of Process Tracing
	8.3 Process Tracing Best Practices and Examples from COVID Research
		8.3.1 Definition of Process Tracing
		8.3.2 How to Do Process Tracing
		8.3.3 Best Practices in Process Tracing
			8.3.3.1 Cast the Net Widely for Alternative Explanations
			8.3.3.2 Be Equally Tough on the Alternative Explanations
			8.3.3.3 Consider the Potential Biases of Evidentiary Sources
			8.3.3.4 Consider Whether the Case Is Most or Least Likely for Alternative Explanations
			8.3.3.5 Make a Justifiable Decision on When to Start
			8.3.3.6 Be Relentless in Getting Diverse Evidence, but Make a Justifiable Decision on When to Stop
			8.3.3.7 Combine PT with Case Comparisons if Relevant
			8.3.3.8 Be Open to Inductive Insights
			8.3.3.9 Use Deduction to Infer What Must Be True if a Hypothesis Is True
			8.3.3.10 Remember Not All PT Is Conclusive
		8.3.4 Examples from COVID Case Studies
	8.4 The “Replication Crisis” and the Comparative Advantages of Process Tracing Case Studies
		8.4.1 The Replication Crisis
		8.4.2 Process Tracing on Complex Phenomena
		8.4.3 Process Tracing in Multimethod Research
		8.4.4 Process Tracing and Generalizing from Case Studies
		8.4.5 Limitations of Process Tracing
	8.5 New Developments in Process Tracing
		8.5.1 Formal Bayesian Process Tracing
		8.5.2 New Modes of Multimethod Research
	8.6 Conclusions
	References
		Suggested Reading
Chapter 9: Exploring Interventions on Social Outcomes with In Silico, Agent-Based Experiments
	9.1 Introduction
	9.2 Agent-Based Modeling
	9.3 Exploring Artificial Policy Scenarios
		9.3.1 Interventions to Increase Competition or Collaboration in Science
			9.3.1.1 Example 1
			9.3.1.2 Example 2
	9.4 Conclusions
	References
Chapter 10: The Many Threats from Mechanistic Heterogeneity That Can Spoil Multimethod Research
	10.1 Introduction
	10.2 Basic Ideas Behind MMR
	10.3 The Problem of Mechanistic Heterogeneity for External Validity in MMR
	10.4 Sources of Mechanistic Heterogeneity in MMR
		10.4.1 Complex Concepts or Measures
		10.4.2 Known and Unknown Omitted Conditions
		10.4.3 Causal and Temporal Dynamics
	10.5 Taking Mechanistic Heterogeneity in MMR More Seriously
	10.6 Concluding Remarks
	References
Chapter 11: Conclusions. Causality Between Plurality and Unity
	11.1 Introduction
	11.2 Two Tales About the Making of Science
		11.2.1 The Viewpoint of the History of Science
		11.2.2 The Perspective of the Philosophy of Science
	11.3 Can We Learn from One Another?
		11.3.1 Ontological Incommensurability?
		11.3.2 Epistemic Incommensurability?
		11.3.3 Methodological Incommensurability?
			11.3.3.1 Design-Based Solutions
			11.3.3.2 Model-Based Solutions
	11.4 Wrapping Up and Looking Ahead
	References