Analysing causal relationships in a participatory evaluation process

A group of five brazilian NGOs publish collective case study on participatory evaluation

A group of five Brazilian organisations working for equity in education (CEERT, Cidade Escola Aprendiz, Ensina Brasil, IBEAC and Elos Institute) took part in a participatory evaluation process on the contributions of specific programmes to systemic changes in formal and informal education. The process, carried out within the Community of Practice on participatory approaches to Monitoring, Evaluation and Learning hosted by the philanthropic organisation Imaginable Futures, took place over the course of a year, and its reflections and cross-cutting results were recently published as a case study on the Causal Pathways Initiative platform. 

The invitation to publish this paper stems from the nature of the approach to understanding evaluating the causal relationships between the selected programmes, their immediate results, and the systemic changes to which the organisations wish to contribute. Taking the Outcome Harvesting approach as a macro process, the teams collected stories of significant change from the central actors of the programmes under analysis and then constructed the causal pathways of the changes reported in the stories. This is to say that, rather than starting from results predefined by logical models or similar tools, the teams went into the field to investigate, based on the experiences of the people involved in the programmes themselves, what had really changed, how that change had happened, whether the actors identified contributions of the programme to this change and, if so, how. 

Thus, rather than simply proving whether or not the desired results were achieved, the participatory evaluation process sought to provide learning opportunities for all parties involved:

• for the implementer organisations' teams, who were able to broaden their understanding of how their actions contribute to change and make adjustments to programmes when necessary;

• for the funding organisation, which was able to gain a cross-sectional view of how a group of its grantees contribute to broader changes aligned with its investment strategy, as well as what challenges they faced along the way;

• for the programme participants involved in the evaluation, who had the opportunity to tell their stories and, in doing so, also reflect on how the reported changes had taken place and what these changes meant to them. By involving these actors in the evaluation process, the evaluation takes on the character of an intervention, i.e., participation in the evaluation process itself can contribute to the desired changes or bring about new ones. 

The journey involved a team of facilitators supporting each organisation in the stages of participatory evaluation according to their needs, while also sustaining the collective nature of the inquiry. At different times, the five organisations came together for learning sessions, reflection on the process and results achieved, and exchange of experiences. One of these moments took place in June 2024, when representatives of the teams presented the ongoing process at the 11th Seminar of the Brazilian Monitoring and Evaluation Network (RBMA) in Belém. 

Bruna Viana, Denise dos Santos, Vinícius Lima, Natacha Costa, Natasha Costa, Flávia Kolchraiber, Marina Apgar and Lara Santos presenting at the XI Seminar of the Brazilian Monitoring and Evaluation Network (Rede Brasileira de Monitoramento e Avaliação - RBMA), in Belém, June 2024.

With the conclusion of this participatory evaluation process, the group inaugurated the third phase of the Community of Practice, which featured an immersive week-long course on participatory approaches to Monitoring, Evaluation and Learning from the Institute of Development Studies, offered by Marina Apgar and Angela María-Baez in February in São Paulo. The third phase, which will continue until the end of 2025 with facilitation by Marina Apgar, Bruna Viana and Aline Rocha, focuses on supporting teams in structuring institutional participatory evaluation systems and processes, with new rounds of participatory evaluation to be carried out as part of this process.

Members of the Community of Practice during the Participatory Monitoring and Evaluation for Learning course, offered by Marina Apgar and Ángela-María Baez in São Paulo, February 2025.

To read the case study in English, click here. If you prefer to access the Portuguese version, click here.

Read other case studies available on the Causal Pathways Initiative platform by clicking here.

Technical details:

Funding organisation: Imaginable Futures

Monitoring and evaluation process coordination: Marina Apgar 

Facilitators: Bruna Viana and Irenildes Silva

Participating organisations: Ashoka, CEERT, Cidade Escola Aprendiz, Ensina Brasil, IBEAC and Instituto Elos.