Using rigorous and innovative methods to measure impact

UNDP makes use of different scientific methods to measure impact. Experimental and quasi-experimental are the most prominent designs, relying on control and treatment groups. The control group (not exposed to the intervention) acts as a counterfactual¹, which allows for a valid comparison of similar groups at the same point in time. This approach enables attribution of impact to the intervention being tested and ensures that confounding factors, such as age, gender and educational level of beneficiaries or shorter-term climate variability, are not responsible for the result.

In an experimental design, the selection of control and treatment groups is random² and integrated at the inception of an intervention. When this random selection is not possible, programme managers and policymakers can turn to quasi-experimental methods – such as regression discontinuity design, instrumental variables, difference-in-difference and matching.

To backstop these designs, the availability of quality, accurate and frequent data is critical. While new technologies (such as remote-sensing and web scraping) coupled with increased computing power create new opportunities for gathering and analysing data, traditional household surveys and administrative data still play a crucial role in measuring policy and intervention outcomes.

Results from causal assessments enable programme managers and policymakers to plan interventions in a rational and evidence-based manner, making choices that are oriented to improving resilience to a changing climate.