An impact evaluation is measuring how the lives of others change and to what extent that change is due to the program. It serves to make decisions, learn and communicate to the different actors. It includes defining indicators, collecting data and analyzing whether the observed effect can be attributed to the intervention.
What is it and why does it matter?
The impact evaluation answers two questions. Were there changes in people's lives? To what extent do these changes occur due to an intervention? Its value is in providing evidence to decide: adjust the design, prioritize resources, scale what works and close what doesn't.
To avoid confusion: outputs are what happens immediately (for example, workshops given), results are what takes time to happen (skills, income, well-being) and impacts are what changed in the lives of others in a more profound and sustainable way.
Levels of rigor: from observed change to attribution
Not all evaluations look for the same thing. Depending on the objective and resources, you can choose different levels of evidence:
- Potential for change: measuring only at one moment of an intervention, it is useful to have a quick snapshot of how people are at a given moment, although it does not allow comparison with the beginning or inferring causality.
- Evaluate over time: measuring the beginning and end of an intervention, rrequires baseline and closure with consistent instruments to estimate the net change of participants. If the program has already started and it could not be collected ex-ante, an alternative is to use retrospective baseline: at the end you ask how they were before and after and how much they attribute the change to the program. It has limitations that depend on people's memory, but it can be triangulated with qualitative evidence to make the information more solid.
- Evaluate with control: allows you to compare what would have happened if my project had not existed. To do this, it is necessary to measure the group (people or organizations) with which you worked and with a similar group with which you did not work, collecting at the beginning and end of an intervention. This evaluation allows attributing the effect more precisely.
Choose the rigor that your decision requires: learn quickly, improve in progress or demonstrate attribution to scale.
How is it done in practice?
The process combines technical design (“desktop”) and field work. This can be seen like this:
First, define who the program affects and what change is reasonable to expect, aligned with the theory of change (the roadmap to changing the lives of others). Then, land metrics, select the design (simple, over time, or controlled, depending on your needs and capabilities), and agree on an ethical data plan.
Then, implement collection. For time or controlled evaluations, you need a clear baseline. If it does not exist, consider the retrospective version with questions anchored in facts to reduce bias. Finally, analyze and translate findings into decisions: what to scale, what to adjust, what to discard.
- Typical instruments (choose few and well): brief surveys, interviews, focus groups, review of administrative records and systems data.
- Useful metrics: effective coverage, changes in key results and, when applicable, estimates of attributable effect.
- Good practices: informed consent, data protection, and return of results to the people involved.
How to start? And if the train has already passed, what do I do?
The ideal is to plan from the beginning to ensure a baseline, but it is never too late to learn. If the project is already running, you can rebuild a well-designed retrospective baseline. If you decide with control, make sure you have clear eligibility criteria and operational feasibility. The most important thing is to never run out of information for decision making.
What is not an impact evaluation?
It is not a check list of activities or a delivery report. It is insufficient to count workshops or publications. What matters is whether something that matters changed and to what extent the program produced that change. Therefore, differentiating outputs, results and impacts avoids inflating achievements and aligns expectations with real evidence.
Deepen planning with a solid theory of change: What is theory of change?
Frequently asked questions
Conclusion
Impact evaluation is not an end in itself: it is a tool to decide better and be accountable honestly. Define the change that matters, choose the right level of rigor, and turn data into concrete improvements. Do you want to land a useful and realistic evaluation plan for your program? Let's talk.