Section 1: Introduction; overview of the teaching case; the evaluation approach; scoping and framing. 

What is retrospective impact evaluation (a.k.a. ex-post evaluation or post-project evaluation)? 

Introduction to the teaching case: Save the Children’s first ever retrospective impact evaluation of a Child Sponsorship-funded community development project in Ethiopia.  

Explaining your evaluation approach and guiding principles. 

Grounding and framing a retrospective impact evaluation: Stakeholder consultation first! 

The place of “values” in evaluation; how to determine which values should apply; how to use values to identify criteria and evidence and to draw evaluative conclusions.  

Crafting explicitly evaluative Key Evaluation Questions (KEQs) to guide your whole evaluation. 

Highlighting the OECD DAC Evaluation Criteria within the KEQs. 

Section 2: Evaluating Design, Implementation, and Durability 

Key elements to cover when answering this KEQ. 

Documenting what was implemented and installed in the distant past. 

Dealing with historical data challenges – patchy records, non-existent maps. 

What does “good” design and implementation look like? 

Assessment of strengths, challenges, and aspirations. 

The main “values” relevant to human and natural systems. 

What do “equitable design” and “equitable implementation” look like?  

How to assess coherence and alignment. 

How to assess the durability of installations and programming. 

How to assess “environmentally sustainable” design and implementation. 

Section 3: Evaluating outcomes, impacts, and sustainment. 

Key elements to cover when answering this KEQ. 

Identifying which outcomes and impacts to track. 

What do “equitable outcomes and impacts” look like?  

Mixed methods and triangulation for “robust” conclusions. 

What to do when the evidence you need is unavailable. 

Using rubrics to assess performance. 

Causal inference: What contributed to the observed changes? 

Producing credible answers to KEQs despite severe data limitations. 

Tracking down ripple effects. 

Section 4: Evaluating Value for Investment 

Key elements to cover in answering this KEQ. 

Multiple options for answering the VfI KEQ - and why we chose a cost-benefit analysis here.

Cost-Benefit Analysis, step by step:

• Finding the big-ticket items to include in the analyses. 
• Figuring out the overall cost component. 
• Estimating the magnitude of outcomes and their monetary value.
• Estimating a cost-benefit ratio. 

Using existing research to fill gaps. 

Step-by-step demonstrations using three examples: 

• Increased years of schooling 
• Facilitators trained as teachers 
• Access to clean drinking water 

Section 5: Lessons learned for lasting change; dataviz & reporting; top tips. 

Key elements to cover in answering the “lessons learned” KEQ. 

Identifying what helped and hindered solutions, outcomes, and impacts to last. 

Evaluating transition and exit processes for a completed initiative. 

Dataviz for Retrospective Impact Evaluation.

Graphing rubric-based ratings to show impact and sustainment.

Visualizing responses to direct questions about sustainment. 

Reporting for Retrospective Impact Evaluation.

Common problems and characteristics of great reporting. 

How to structure reports around the KEQs for better clarity. 

Using sensemaking sessions to strengthen preliminary reporting. 

Creating a skeleton report before collecting data. 

Weaving the evidence and values. 

10 tips for doing a great Retrospective Impact Evaluation.