DESIGN OF OBSERVATIONAL STUDIES.

Preface to the First Edition
Preface to the Second Edition
	New Topics in the Second Edition
	R in the Second Edition
Acknowledgments
Contents
Part I Beginnings
	1 Dilemmas and Craftsmanship
		1.1 Those Confounded Vitamins
		1.2 Cochran's Basic Advice
			1.2.1 Treatments, Covariates, Outcomes
			1.2.2 How were Treatments Assigned?
			1.2.3 Were Treated and Control Groups Comparable?
			1.2.4 Eliminating Plausible Alternatives to Treatment Effects
			1.2.5 Exclusion Criteria
			1.2.6 Exiting a Treatment Group After Treatment Assignment
			1.2.7 Study Protocol
		1.3 Maimonides' Rule
		1.4 Seat Belts in Car Crashes
		1.5 Money for College
		1.6 Nature's ``Natural Experiment''
		1.7 What This Book Is About
			1.7.1 Basic Structure
			1.7.2 Structure of Part I: Beginnings
			1.7.3 Structure of Part II: Matching
			1.7.4 Structure of Part III: Design Sensitivity
			1.7.5 Structure of Part IV: Enhanced Design
			1.7.6 Structure of Part V: Planning Analysis
			1.7.7 A Less Technical Introduction to Observational Studies
			1.7.8 Dependence Among Chapters
		1.8 Further Reading
		Bibliography
	2 Causal Inference in Randomized Experiments
		2.1 Two Versions of the National Supported Work Experiment
			2.1.1 A Version with 185 Pairs and a Version with 5 Pairs
			2.1.2 Basic Notation
		2.2 Treatment Effects in Randomized Experiments
			2.2.1 Potential Responses Under Alternative Treatments
			2.2.2 Covariates and Outcomes
			2.2.3 Possible Treatment Assignments and Randomization
			2.2.4 Interference Between Units
		2.3 Testing the Null Hypothesis of No Treatment Effect
			2.3.1 Treated–Control Differences When the Null Hypothesis Is True
				What Would Happen If the Null Hypothesis of No Effect Were True?
				What Experimental Results Might Occur If the Null Hypothesis of No Effect Were True?
			2.3.2 The Randomization Distribution of the Mean Difference
				Randomization Test of No Effect Using the Mean as the Test Statistic
				The Reasoned Basis for Inference in Experiments
				Some History: Random Assignment and Normal Errors
			2.3.3 The Randomization Distribution of Wilcoxon's Statistic
				Computing the Randomization Distribution Under the Null Hypothesis of No Effect
				Testing No Effect with All I=185 Matched Pairs
				Ties and Distribution-Free Statistics
		2.4 Testing Other Hypotheses; Confidence Intervals; Point Estimates
			2.4.1 Testing a Constant, Additive Treatment Effect
			2.4.2 Confidence Intervals for a Constant, Additive Effect
			2.4.3 Hodges–Lehmann Point Estimates of Effect
			2.4.4 The Average Treatment Effect
			2.4.5 Testing General Hypotheses About Treatment Effects
			2.4.6 Multiplicative Effects; Tobit Effects
				A Constant Multiplicative Effect β
				Tobit Effects
				Comparing the Three Effects: Additive, Multiplicative, Tobit
		2.5 Attributable Effects
			2.5.1 Why Use Attributable Effects?
			2.5.2 Aligned Responses: Shifting Attention from Pairs to Individuals
			2.5.3 Thinking About Heterogeneous Effects by Focusing on Small Groups of Pairs
			2.5.4 Computations; Null Distribution
				Ties
				Results for the NSW Experiment: Big Effects, Now and Then
				Uncommon But Dramatic Responses to Treatment
		2.6 Internal and External Validity
		2.7 Summary
		2.8 Appendix: Randomization Distribution of m-Statistics
			2.8.1 Giving Observations Controlled Weight Using a ψ Function
			2.8.2 Scaling
			2.8.3 Randomization Test of a Hypothesized Additive Effect, H0:rTij=rCij+τ0
			2.8.4 m-Tests in the NSW Experiment
		2.9 Further Reading
		2.10 Software
		2.11 Data
		Bibliography
	3 Two Simple Models for Observational Studies
		3.1 The Population Before Matching
		3.2 The Ideal Matching
		3.3 A Naïve Model: People Who Look Comparable Are Comparable
			3.3.1 Assigning Treatments by Flipping Biased Coins Whose Unknown Biases Are Determined by Observed Covariates
				The Ideal Match and the Naïve Model
				What Is the Propensity Score?
			3.3.2 Balancing Property of the Propensity Score
			3.3.3 Propensity Scores and Ignorable Treatment Assignment
			3.3.4 Summary: Separating Two Tasks, One Mechanical, the Other Scientific
		3.4 Sensitivity Analysis: People Who Look Comparable May Differ
			3.4.1 What Is Sensitivity Analysis?
			3.4.2 The Sensitivity Analysis Model: Quantitative Deviation from Random Assignment
			3.4.3 Sensitivity Analysis Model When Pairs Are Matched for Observed Covariates
		3.5 Welding Fumes and DNA Damage
			3.5.1 Sensitivity Analysis When Testing the Hypothesis of No Treatment Effect
			3.5.2 Computations
			3.5.3 Sensitivity Analysis for a Confidence Interval
			3.5.4 Sensitivity Analysis for Point Estimates
		3.6 Amplification of a Sensitivity Analysis
			3.6.1 What Is an Amplification?
			3.6.2 Sensitivity Analysis in Terms of ( Λ, Δ)
			3.6.3 Precise Meaning of ( Λ, Δ)
		3.7 Bias Due to Incomplete Matching
		3.8 Summary
		3.9 Further Reading
		3.10 Software
		3.11 Data
		Appendix: Exact Computations for Sensitivity Analysis
		Bibliography
	4 Competing Theories Structure Design
		4.1 How Stones Fall
		4.2 The Permanent-Debt Hypothesis
		4.3 Guns and Misdemeanors
		4.4 The Dutch Famine of 1944–1945
		4.5 Replicating Effects and Biases
		4.6 Reasons for Effects
		4.7 The Drive for System
		4.8 Further Reading
		Bibliography
	5 Opportunities, Devices, and Instruments
		5.1 Opportunities
			5.1.1 The Well-Ordered World
			5.1.2 Questions
			5.1.3 Solutions
		5.2 Devices
			5.2.1 Disambiguation
			5.2.2 Multiple Control Groups
			5.2.3 Coherence Among Several Outcomes
			5.2.4 Known Effects
				Unaffected Outcomes or Control Outcomes
				Bias of Known Direction
			5.2.5 Doses of Treatment
				Does a Dose–Response Relationship Strengthen Causal Inference?
				Genetic Damage from Paint and Paint Thinners
			5.2.6 Differential Effects and Generic Biases
				What Are Differential Effects?
				Examples of Differential Effects in Observational Studies
				What Are Generic Biases?
		5.3 Instruments
			5.3.1 What Is an Instrument?
			5.3.2 Example: Noncompliance in a Double-Blind Randomized Trial
			5.3.3 Example: Maimonides' Rule
			5.3.4 Notation for an Instrument in a Paired Encouragement Design
			5.3.5 The Hypothesis that Effect Is Proportional to Dose
			5.3.6 Inference About β
			5.3.7 Example: IV Analysis for Maimonides' Rule
			5.3.8 Effect Ratios
			5.3.9 Is the Validity of an Instrument Testable?
			5.3.10 When and Why Are Instruments Valuable?
		5.4 Strengthening Weak Instruments
			5.4.1 Why Strengthen an Instrument?
			5.4.2 A Popular Instrument in Health Outcomes Research: Distance to a Hospital
			5.4.3 How Is an Instrument Strengthened?
			5.4.4 Use of Matching in Strengthening an Instrument
		5.5 Summary
		5.6 Further Reading
		5.7 Software
		5.8 Data
		Bibliography
	6 Transparency
		References
	7 Some Counterclaims Undermine Themselves
		7.1 Appraising Counterclaims
			7.1.1 Types of Counterclaims
			7.1.2 The Logic of Counterclaims That Undermine Themselves
		7.2 An Example: Safety Belts, Injury, and Ejection
			7.2.1 A First Look at an Example: A Claim Prior to a Counterclaim
			7.2.2 The Counterclaim of Selection Bias and Secondary Outcomes
			7.2.3 A Second Look at an Example in Light of a Counterclaim
		7.3 Discussion
			7.3.1 Anticipating Counterclaims
			7.3.2 Some Theory
		7.4 Further Reading
		7.5 Data
		Bibliography
Part II Matching
	8 A Matched Observational Study
		8.1 Is More Chemotherapy More Effective?
		8.2 Matching for Observed Covariates
		8.3 Outcomes in Matched Pairs
		8.4 Summary
		8.5 Further Reading
		References
	9 Basic Tools of Multivariate Matching
		9.1 A Small Example
		9.2 Propensity Score
		9.3 Distance Matrices
		9.4 Optimal Pair Matching
		9.5 Optimal Matching with Multiple Controls
		9.6 Optimal Full Matching
		9.7 Efficiency
		9.8 Summary
		9.9 Further Reading
		9.10 Software
		9.11 Data
		Bibliography
	10 Various Practical Issues in Matching
		10.1 Checking Covariate Balance
		10.2 Simulated Randomized Experiments as Diagnostics
		10.3 Near-Exact Matching
		10.4 Exact Matching
		10.5 Directional Penalties
		10.6 Missing Covariate Values
		10.7 Networks and Sparse Representations of Matching
		10.8 Constraints on Individual Distances
		10.9 Clustered Treatment Assignment
		10.10 Further Reading
		10.11 Software
		Bibliography
	11 Fine Balance
		11.1 What Is Fine Balance?
		11.2 Constructing a Finely Balanced Control Group
		11.3 Controlling Imbalance When Fine Balance Is Not Feasible
		11.4 Fine Balance, Exact Matching and Near-Exact Matching
		11.5 Near-Fine Balance
		11.6 Refined Balance
		11.7 Strength K Balance
		11.8 Cardinality Matching
			11.8.1 What Is Cardinality Matching?
			11.8.2 Cardinality Matching and Outcome Heterogeneity
			11.8.3 Cardinality Matching and Effect Modification
		11.9 Further Reading
		11.10 Software
		11.11 Data
		Bibliography
	12 Matching Without Groups
		12.1 Matching Without Groups: Nonbipartite Matching
			12.1.1 What Is Nonbipartite Matching?
			12.1.2 Treatment-Control Matching Using an Algorithm for Nonbipartite Matching
			12.1.3 Matching with Doses
			12.1.4 Matching with Several Groups
		12.2 Some Practical Aspects of Matching Without Groups
			12.2.1 An Odd Number of Subjects
			12.2.2 Discarding Some Subjects
			12.2.3 Balanced Incomplete Block Designs with Three Groups
			12.2.4 Propensity Scores for Several Groups
		12.3 Matching with Doses and Two Control Groups
			12.3.1 Does the Minimum Wage Reduce Employment?
			12.3.2 Optimal Matching to Form Two Independent Comparisons
			12.3.3 Difference in Change in Employment with Two Control Groups
		12.4 Further Reading
		12.5 Software
		12.6 Data
		Bibliography
	13 Risk-Set Matching
		13.1 Does Cardiac Transplantation Prolong Life?
		13.2 Risk-Set Matching in a Study of Surgery for Interstitial Cystitis
		13.3 Maturity at Discharge from a Neonatal Intensive Care Unit
		13.4 Joining a Gang at Age 14
		13.5 Some Theory
		13.6 Isolation in Natural Experiments
			13.6.1 Brief Review of Differential Effects and Generic Biases
			13.6.2 What Is Isolation?
			13.6.3 Twins Versus a Single Birth, and Their Impact on Labor Supply
			13.6.4 Mass and Safety in Vehicle Crashes Resulting in a Fatality
		13.7 Further Reading
		13.8 Software
		Bibliography
	14 Matching in R
		14.1 Optimal Matching Using R
		14.2 Data
		14.3 Propensity Score
		14.4 Covariates with Missing Values
		14.5 Distance Matrix
		14.6 Constructing the Match
		14.7 Checking Covariate Balance
		14.8 College Outcomes
		14.9 Further Reading
		14.10 Software
		14.11 Data
		Bibliography
Part III Design Sensitivity
	15 The Power of a Sensitivity Analysis and Its Limit
		15.1 The Power of a Test in a Randomized Experiment
			15.1.1 What Is the Power of a Test?
			15.1.2 A Pep Talk About Statistical Power
			15.1.3 Computing Power in a Randomized Experiment: The Two Steps
			15.1.4 Step 1: Determining the Critical Value Assuming the Null Hypothesis Is True
			15.1.5 Step 2: Determining the Power Assuming the Null Hypothesis Is False
			15.1.6 A Simple Case: Constant Effect with Random Errors
		15.2 Power of a Sensitivity Analysis in an Observational Study
			15.2.1 What Is the Power of a Sensitivity Analysis?
			15.2.2 Computing the Power of a Sensitivity Analysis: The Two Steps
			15.2.3 Step 2: Determining the Power When the Null Hypothesis Is False and There Is No Unobserved Bias
			15.2.4 A First Look At the Power of a Sensitivity Analysis
		15.3 Design Sensitivity
			15.3.1 A First Look at Design Sensitivity
			15.3.2 A Formula for Design Sensitivity
			15.3.3 Computing Design Sensitivity with Additive Effects and iid Errors
		15.4 Summary
		15.5 Further Reading
		Appendix: Technical Remarks and Proof of Proposition 15.1
		Bibliography
	16 Heterogeneity and Causality
		16.1 J.S. Mill and R.A. Fisher: Reducing Heterogeneity or Introducing Random Assignment
		16.2 A Larger, More Heterogeneous Study Versus a Smaller, Less Heterogeneous Study
			16.2.1 Large I or Small σ: Which is Better?
			16.2.2 A Simulated Example
			16.2.3 Power Comparisons with Normal, Logistic, and Cauchy Errors
			16.2.4 Design Sensitivity
		16.3 Heterogeneity and the Sensitivity of Point Estimates
		16.4 Examples of Efforts to Reduce Heterogeneity
			16.4.1 Twins
			16.4.2 Road Hazards
			16.4.3 The Genetically Engineered Mice of Microeconomics
			16.4.4 Motorcycle Helmets
		16.5 Summary
		16.6 Further Reading
		Bibliography
	17 Uncommon but Dramatic Responses to Treatment
		17.1 Large Effects, Now and Then
			17.1.1 Are Large but Rare Effects Insensitive to Unmeasured Biases?
			17.1.2 Review of Sect.2.5: Measuring Large but Uncommon Effects
		17.2 Two Examples
			17.2.1 Chemotherapy Intensity and Toxicity in Ovarian Cancer
			17.2.2 DNA Adducts Among Aluminum Production Workers
		17.3 Properties of a Paired Version of Salsburg's Model
		17.4 Design Sensitivity for Uncommon but Dramatic Effects
			17.4.1 Design Sensitivity of Stephenson's Test
			17.4.2 Design Sensitivity of Stephenson's Test under Salsburg's Model
		17.5 Summary
		17.6 Further Reading
		17.7 Software
		17.8 Data
		Appendix: Sketch of the Proof of Proposition 17.1
		Bibliography
	18 Anticipated and Discovered Patterns of Response
		18.1 Using Design Sensitivity to Evaluate Devices
		18.2 Coherence
			18.2.1 Notation with Several Responses
			18.2.2 Responses with a Multivariate Normal Distribution
			18.2.3 Numerical Results for Bivariate Normal Responses
			18.2.4 Practical Implementation for General λ
		18.3 Can Coherence Be Discovered?
			18.3.1 Using Split Samples to Plan for Coherence
			18.3.2 Considering Every Possible λ
			18.3.3 Hedged Bets About λ
			18.3.4 Summary
		18.4 Doses
			18.4.1 Another Way to Write a Signed Rank Statistic
			18.4.2 The Favorable Situation with Doses
			18.4.3 A Formula for the Design Sensitivity with Doses
			18.4.4 Numerical Evaluation of the Design Sensitivity
		18.5 Example: Maimonides' Rule
		18.6 Reactive Doses
		18.7 Further Reading
		18.8 Software
		18.9 Data
		Appendix: Proof of Proposition 18.1
		Bibliography
	19 Choice of Test Statistic
		19.1 Choice of Test Statistic Affects Design Sensitivity
			19.1.1 Design Anticipates Analysis
			19.1.2 A Simple Case: Step-Rank Statistics, Additive Effects, Normal Errors
		19.2 Statistics Built for Superior Design Sensitivity
			19.2.1 Building New Statistics for Use in Sensitivity Analyses
			19.2.2 A New U-Statistic with Superior Design Sensitivity
			19.2.3 Example: Chemotherapy Associated Toxicity
			19.2.4 Example: Lead in the Blood of Smokers
			19.2.5 m-Statistics
			19.2.6 Summary
		19.3 Adaptive Inference
			19.3.1 Using the Data to Choose a Test Statistic
			19.3.2 Example: Adaptive Inference for Chemotherapy Associated Toxicity
			19.3.3 Example: Adaptive Inference for Lead in the Blood of Smokers
		19.4 Design Sensitivity and Dose–Response
			19.4.1 Dose–Response and Evidence of Causality
			19.4.2 Example: Smoking and Periodontal Disease
			19.4.3 Ignoring Doses: Pair Differences in Periodontal Disease
			19.4.4 The Crosscut Test
			19.4.5 Design Sensitivity of the Crosscut Test
			19.4.6 The Stratified Crosscut Test
			19.4.7 An Adaptive Crosscut Test
			19.4.8 The Crosscut Test and Evidence Factors
		19.5 Bahadur Efficiency of Sensitivity Analyses
			19.5.1 Where Do Things Stand?
			19.5.2 Several Types of Efficiency
			19.5.3 Using Bahadur Efficiency in Sensitivity Analyses
		19.6 Further Reading
		19.7 Software
		19.8 Data
		Bibliography
Part IV Enhanced Design
	20 Evidence Factors
		20.1 What Are Evidence Factors?
			20.1.1 A Replication Should Disrupt a Plausible Bias
			20.1.2 Replication and Evidence Factors
			20.1.3 Example: Smoking and Periodontal Disease
			20.1.4 Issues Developed in This Chapter
		20.2 The Simplest Nontrivial Case: Renyi's Partial Ranks
			20.2.1 DNA Damage Among Tannery Workers
			20.2.2 Renyi's Partial Ranks in a Randomized Experiment Under the Null Hypothesis
			20.2.3 Wilcoxon's Stratified Rank Sum Test
			20.2.4 Two Evidence Factors Using Wilcoxon's Rank Sum Test
			20.2.5 Limitations of Partial Ranks
			20.2.6 Sensitivity Analyses for Evidence Factors
		20.3 A Second Example: Smoking and Periodontal Disease
		20.4 Appendix: Some Theory
			20.4.1 A Tiny Example
			20.4.2 Elementary Theory of Groups of Permutation Matrices
			20.4.3 Probability Distributions on a Group of Permutation Matrices
			20.4.4 Invariant Test Statistics
			20.4.5 Ignoring One Evidence Factor
			20.4.6 Symmetric Sensitivity Analyses for Biases That May Be Asymmetric
			20.4.7 Fixing One Evidence Factor
			20.4.8 Joint Behavior of Two Bounds on P-Values
			20.4.9 Joint Distribution of P-Value Bounds for Two Evidence Factors
			20.4.10 Another Example: Blocks of Size Three
			20.4.11 Untidy Blocks
		20.5 Further Reading
		20.6 Software
		20.7 Data
		Bibliography
	21 Constructing Several Comparison Groups
		21.1 Why Compare Several Groups?
			21.1.1 Uses of Several Comparison Groups
			21.1.2 Issues When Constructing More Than One Comparison Group
		21.2 Overlapping Comparison Groups and the Exterior Match
			21.2.1 Two Entwined Comparison Groups
			21.2.2 The Exterior Match
		21.3 Optimal Tapered Matching
		21.4 Building Matched Sets Using an Approximation Algorithm
			21.4.1 A Near Optimal Solution to a Difficult Problem
			21.4.2 What Is Near-Fine Balance with Three Groups?
			21.4.3 An Approximation Algorithm
		21.5 Is It Possible to Attenuate Unmeasured Bias?
			21.5.1 Attenuation: Logically Possible But Small in Magnitude
			21.5.2 Look More, Speculate Less
			21.5.3 Smoking and Elevated Homocysteine Levels
		21.6 Further Reading
		21.7 Software
		Bibliography
Part V Planning Analysis
	22 After Matching, Before Analysis
		22.1 Split Samples and Design Sensitivity
		22.2 Are Analytic Adjustments Feasible?
			22.2.1 Tapered Matching and the Exterior Match
		22.3 Matching and Thick Description
			22.3.1 Thick Description
			22.3.2 What Is Thick Description?
			22.3.3 Example: Mortality After Surgery
		22.4 Further Reading
		22.5 Software
		References
	23 Planning the Analysis
		23.1 Plans Enable
		23.2 Elaborate Theories
			23.2.1 R.A. Fisher's Advice
			23.2.2 What Should a Planned Analysis Accomplish?
		23.3 Three Simple Plans with Two Control Groups
			23.3.1 The Simplest Plan for Analysis with Two Control Groups
			23.3.2 A Symmetric Plan for Analysis with Two Control Groups
			23.3.3 Are the Two Control Groups Nearly Equivalent?
			23.3.4 An Elementary Plan for Analysis with Two Control Groups
			23.3.5 An Alternative Plan for Analysis with Two Control Groups
			23.3.6 Summary
		23.4 Sensitivity Analysis for Two Outcomes and Coherence
		23.5 Sensitivity Analysis for Tests of Equivalence
		23.6 Sensitivity Analysis for Equivalence and Difference
		23.7 Summary
		23.8 Further Reading
		Appendix: Testing Hypotheses in Order
		Appendix: Testing Hypotheses in Order
			What Is a Sequentially Exclusive Partition of a Sequence of Hypotheses?
			Testing Hypotheses in Order
			Sensitivity Analysis for Testing in Order
		Bibliography
Summary: Key Elements of Design
Solutions to Common Problems
	References
Symbols
Acronyms
Glossary of Statistical Terms
Further Reading
	References
Suggested Readings for a Course
	References
Index