IMPACT Study Report #5 - Using Natural Experiments in Crises: Lessons for Evaluation

In 2020, IOM commissioned a three-year impact evaluation (referred to as IMPACT) of the EU-IOM Joint Initiative programme in the Horn of Africa (referred to as JI-HoA), focusing on Ethiopia, Somalia and Sudan. The study had its own innovative elements, among which the use of a natural experiment (NE) that capitalized on the opportunity presented by the coronavirus disease 2019 (COVID-19) lockdown restrictions to further understand the resilience of returnees when faced with extreme shocks.

The pandemic created the conditions that could stress-test the JI-HoA programme and understand better how beneficiaries were able to respond in a crisis. From a methodological perspective, implementing the IMPACT NE-based evaluation generated insights and allowed IOM, Itad and its evaluation partners to complement the other evaluation approaches pursued in the IMPACT study.2 This report highlights some of these complementarities and shares lessons for others who might consider integrating NEs into their programme evaluations.

The IMPACT Study is the impact evaluation of the EU-IOM Joint Initiative programme in the Horn of Africa. Launched in March 2020 and concluded in March 2023, the study focuses on Ethiopia, Somalia and Sudan: the three countries in the region where the programme has the largest reintegration caseload. All the IMPACT Study reports, as well as additional resources such as technical annexes, datasets, data analysis scripts and dissemination material are accessible from the IMPACT Study webpage: https://eastandhornofafrica.iom.int/impact-study.