Transformational Change for People and the Planet: Evaluating Environment and Development

Preface
Contents
Abbreviations and Acronyms
About the Authors
Transformational Change for People and the Planet: Evaluating Environment and Development – Introduction
	Background
	State of Development Evaluation
	The Sustainability Context
	What It Means for Evaluation
	About This Book
	References
Part I: Transformational Change
	Evaluation for Transformational Change: Learning from Practice
		Evaluation Must Respond to Global Signals to Be Relevant
		Redefinition in the COVID-19 Crisis: Evaluators Are Not Isolated from Changes
		Challenges to Evaluation as a Practice and Form of Transformation
		The Exploratory Nature of This Chapter
		Changes to the Evaluation–Transformation Relationships over Time
			The Crisis Context and Potential Loss of Judgement Proprietorship
			Judging Transformation, the Challenge of Relativism
		Context Ascribes Value and Meaning to the Concepts of Transformation
		Changes in Evaluation Production and Emphasis
		Case Studies on the Evaluation–Transformation Nexus
			South Africa National Department of Land Affairs and Public Service Commission
			The Independent Evaluation Office of the United Nations Development Program: Some Strategic Choices
			Learning from Both Managerial Roles
		Some Conclusions
			Challenge on the Exclusivity of Judgment
			Reflecting on Transformation Drivers
			The Enabling Environment for Transformation
			Political Will and Leadership Support
			The Post-Normal or COVID-19 Era
		References
	Transformational Change for Achieving Scale: Lessons for a Greener Recovery
		Introduction
		A Framework for Transformational Change and Achieving Scale
			Drivers of Change
				Clear Ambition in Design
				Addressing Market and System Reforms Through Policies
				Quality of Project Design and Implementation
				Mechanisms for Financial Sustainability
			Scaling-Up
			Factors Influencing Transformative Change and Scaling-Up
			Example 1: Transformative and Effectively Scaled Up: Lighting Africa – Market-Based Solutions for Energy Access
			Example 2: Review, Ownership, and Partnering: Payments for Ecosystem Services in the Danube Basin
			Adoption of the Intervention
			Sustained Support for Scaling-Up Processes
			Learning for Adaptability and Cost-Effectiveness
		Conclusions
		Appendix
			Projects Discussed in This Chapter
		References
Part II: Drivers of Sustainability
	Introduction
	Reference
	Sustainability After Project Completion: Evidence from the GEF
		Introduction
		Understanding Sustainability
		Analytical Framework
		Data
		Methodology
			Screening and Review
			Assessment Approach
			Limitations
		Findings
			Sustainability During Postcompletion Period
			Broader Adoption and Sustainability
			Factors that Facilitate Sustainability
				Financial Support for Follow-Up
				Political Support
				Follow-Up by, and Capacities of, Executing Partner
				Stakeholder Buy-In
				Project Design
		Conclusion
		Appendix: Completed GEF-funded Projects with Postcompletion Evaluation
		References
	From the Big Picture to Detailed Observation: The Case of the GEF IEO’s Strategic Country Cluster Evaluations
		Introduction
		Challenges and Opportunities in IEO Complex Evaluations
		The Strategic Country Cluster Evaluation Concept
			Applications of the SCCE Approach
			Methodological Considerations
				Geospatial Analysis Following Project Field Visits
		Lessons from the SCCE Experience
		References
	Staying Small and Beautiful: Enhancing Sustainability in the Small Island Developing States
		Introduction
		Environmental Challenges in SIDS
		GEF Interventions in SIDS
			Climate Resilience
			Integrated Resource Management Through Ridge to Reef
			Blue Economy
			Protected Areas
			Land Use Management
			Invasive Alien Species
			Chemicals and Waste
			Renewable Energy and Energy Efficiency
		Performance and Sustainability of GEF Projects in SIDS
		Sustainability
			Factors Affecting the Sustainability of Outcomes in SIDS
				Institutional Capacity, Environmental Awareness, and Economic Pressure
		GEF’s Overall Additionality in SIDS
		Conclusions
		Appendix: Projects Discussed in Chap. 7
		References
			Resources
	Assessing Sustainability in Development Interventions
		The Problem
		The Environment as a Closed System
		Catalyzing Capabilities to Ensure Sustainable Outcomes and Impacts
		Postprogram Evaluation
		Dairy Development Asset Transfer—Malawi
		Environmental Effects
		Discussion
		References
	Can We Assume Sustained Impact? Verifying the Sustainability of Climate Change Mitigation Results
		Measuring Impact and Sustainability
		How Are Sustainability and Impact Defined?
		The Limits of Terminal Evaluations
		Methodology
		Findings
			How Is Sustainability Being Captured?
			How Effectively Is Sustainability Being Captured?
			Project Evaluability
				Resources
				Local Ownership and Partnerships
				Capacity Building
				Emerging Sustainability
				Benchmarks, Risks, and Resilience
				Greenhouse Gas Emissions (Impacts)
			Uncertainty and Likelihood Estimates
		Conclusions and Recommendations
		Appendix
			Projects Discussed in Chapter
		References
Part III: Evaluating Climate Change Mitigation and Adaptation
	Introduction
	Using a Realist Framework to Overcome Evaluation Challenges in the Uncertain Landscape of Carbon Finance
		The Evaluation of the Carbon Market Finance Programme
		Overview of Methodology
			Realist Evaluation as an Approach
				Hypothesis Development
				Coding System
				Evidence Saturation
				Coding Results
			Realist Evaluation as a Framework
		Benefits of the Applied Approach
		Challenges of the Applied Approach
		Improving the Methodology
			Bottom-Up Formulation of ICMO Statements
			Increasing Traceability of Causality by Tailoring the Coding to the Mechanism
			Increasing Variability of Contextual Factors
			Summary of the Modified Methodology
		Conclusion
		References
	Evaluation’s Role in Development Projects: Boosting Energy Efficiency in a Traditional Industry in Chad
		Introduction
		Project Background
			Evaluation Findings
		Project Performance
			Relevance
			Effectiveness
			Efficiency
			Sustainability
		Project Coordination and Management
			Gender Mainstreaming
			Cross-Cutting Issues
		Conclusions
		Value of Evaluation in Development Projects
		Appendix: Methodology
			Sampling Methods and Data Collection
				Desk Review
				Qualitative Methods
				Quantitative Methods
				Site Visits
			Data Analysis Methods and Reporting
		Appendix References
		References
	Enabling Systems Innovation in Climate Change Adaptation: Exploring the Role for MEL
		Introduction
			Study Purpose and Approach
		Complexity, Systems Innovation, and CCA
			CCA, Complex Systems, and Innovation: Evolution to the Present Day
			Systems Innovation—The CCA Future
		MEL’s Role in Enabling Systems Innovation for CCA
			Seven Directions of Change for the CCA MEL Community
		Conclusion
		References
	Assessing the Evaluability of Adaptation-Focused Interventions: Lessons from the Adaptation Fund
		Introduction
			Background to Adaptation Fund
			History and Purpose of Evaluability Assessment
			Evaluation of Climate Change Adaptation
			Study Objectives
		Assessment Approach
			Framework Development
			Process for Implementation
		Analysis
			Logic and Additionality of Adaptation Projects
				Relevance to Evaluability
				Adaptation-Specific Evaluability Considerations
				Findings
			Evidence Base and Baselines: Natural vs. Human Systems
				Relevance to Evaluability
				Adaptation-Specific Evaluability Considerations
				Findings
			Resources Allocated to MEL: Direct vs. Indirect
				Relevance to Evaluability
				Adaptation-Specific Evaluability Considerations
				Findings
			Potential for Postcompletion Evaluation
				Relevance to Evaluability
				Adaptation-Specific Evaluability Considerations
				Findings
		Reflections on the EA Tool Development and Implementation
		Conclusions
		References
	Evaluating Transformational Adaptation in Smallholder Farming: Insights from an Evidence Review
		Introduction
		Methodology
			Key Messages Emerging from the Evidence Synthesis
				Scaling Up Transformative Adaptation Pathways
				Transformative Knowledge Management
				Ecosystem-Based Adaptation and Landscape Approaches in Smallholder Farming
				Policy Shortcomings
		Implications for Monitoring, Evaluation, and Learning
		Future Role of Evidence Reviews in Programmatic Evaluation
		References
Part IV: Evaluation Approaches
	Introduction
	Evaluation at the Endgame: Evaluating Sustainability and the SDGs by Moving Past Dominion and Institutional Capture
		Introduction
		Taking Stock on Evaluation Practice and Resources on Sustainability
		Dominion, Accountability, and Institutional Capture
			Dominion
			Accountability
			Institutional Capture
		Sustainability-Ready Evaluation
		How Can Evaluation Contribute to Checkmating Extinction?
			Recognizing Natural Systems as the Foundation for the Human System Means Adding the Natural System Perspective to All Evaluation Criteria
			Evaluation Standards Will Emphasize Achieving the Larger Goals Identified as Central to Checkmating Extinction
			Standards Need to Shift to Evaluating Against Collective Achievement of Sustainability Goals, and Away from Likely Contributions by Partitioned Organizations and Interventions
			Sustainability Is Imbedded in All Evaluation Criteria Reflecting Nexus, Not Isolated as a Free-Standing Criterion
			Evaluation Standards at the Endgame: Evaluating with Rapid Change and Uncertainty
			Evaluation for the Endgame Relentlessly Pursues Use
		Conclusion: Nexus Requires New Rootstock to Grow Relevant Evaluation Functions
		References
	Importance and Utilization of Theory-Based Evaluations in the Context of Sustainable Development and Social- Ecological Systems
		Challenges in Evaluating Sustainable Development
		Theory-Based Evaluation
		Critical Realism
		Importance of Theory-Based Evaluation Approaches
		Coupled Human and Natural Systems (CHANS)
		Why We Need a Framework Like CHANS
		Appropriate Methodologies
		Conclusion
		References
	Pathway to the Transformative Policy of Agenda 2030: Evaluation of Finland’s Sustainable Development Policy
		Introduction to the Evaluation
			Preparations
			Dissemination
		Implementation of the Evaluation
			Focus of the Evaluation: Sustainable Development Policy and the Coordination Model
			Evaluation Questions
			Carrying Out the Evaluation: Approach, Methods, and Process
		Key Messages from the Evaluation
		The Use and Usefulness of the Evaluation
		Evaluative Lessons for the Future
			Inclusiveness and Participation
			Learning Throughout the Evaluation Process
		References
	Evaluating for Resilient and Sustainable Livelihoods: Applying a Normative Framework to Emerging Realities
		Winds of Change in Development and Response
		Evaluation Methodology of IOE
			Sustainable Livelihood Approach and Evaluation Methodology
				Conceptual Linkage Among Livelihoods, Environment, Resilience, and Agriculture
			Evolution of the IOE Evaluation Methodology
			Accumulated Methodological Experience Through Various Products
		Methods for Assessment Using Normative Frameworks
			Sustainable Livelihood Approach and Qualitative Methods
				Physical Capital
				Financial Capital
				Social Capital
				Human Capital
				Natural Capital
			Methods for an Evolving Methodology
				Constraints
				GIS for the Future
			Future Perspectives on Methodology for Evaluating Climate Adaptation and Natural Resource Management
		References
	Measuring the Impact of Monitoring: How We Know Transparent Near-Real-Time Data Can Help Save the Forests
		Background: What Is GFW?
		GFW Monitors Progress
			Analytics
			Users and Stories
		Quantitative Evaluation
		Qualitative Evaluation
		Discussion and Conclusion
		References
	Application of Geospatial Methods in Evaluating Environmental Interventions and Related Socioeconomic Benefits
		Introduction
		Geospatial Approaches and Methods
			Drivers of Increased Use
			Use of Geospatial Data and Analysis in Monitoring and Evaluation
		Application of Geospatial Approaches by the GEF IEO
			Assessing the Relevance of GEF-Supported Interventions to Combat Land Degradation and Desertification
			Assessing Impacts in GEF-Supported Protected Areas
			Assessing Socioeconomic Co-Benefits
			Assessing Health Co-Benefits
			Assessing Outcome Sustainability in Fragile and Conflict Situations
		Conclusions
		References