Evaluating Stepping Stones: A Review of Existing Evaluations and Ideas for Future M&E Work

Commissioned by ActionAid International (AAI), this 39-page report reviews monitoring and evaluation (M&E) data on "Stepping Stones" (SS) - a participatory training package developed by Strategies for Hope, with the support of ActionAid, in the mid-1990s to address the prevention and spread of HIV, and to increase the care of people living with HIV and AIDS (PLWHA), by promoting communication and relationship skills within households and communities. AAI's intention here is not to summarise or present all SS evaluations to date, but to highlight the key issues emerging from them - including how systematic and comprehensive the existing documents have been, the key processes and methodologies used, and the most significant overall findings - and to identify the gaps which need filling. In brief, this report reveals that there was strong support for the programmes among those who had used it or seen it being used, and good evidence linking the programmes to improved communication and positive behaviour change.

As detailed here, the SS training programme is lengthy and spread over many sessions, each one building on the one before enabling real behaviour change to happen and be supported during the process. It involves people working in separate age and sex groups, to encourage openness and discussion; it is designed to enable women and men and the wider community to decide how to promote respect, listening, and communication between sexual partners and within families. The goal is to enable individuals and communities to find their own solutions to the threat of HIV/AIDS, with a focus on filling gaps and addressing the shortcomings of the most prevalent HIV/AIDS messages (e.g., those focused on the dangers of AIDS and/or promoting the ABC (abstain, be faithful, and use condoms) approach to prevention). While people work essentially within their peer groups, there are periodic community meetings held to share issues and to ask others to change their attitudes and behaviour on specific, locally identified issues. There is an accompanying video, for use during the training process, presenting illustrative material in short clips, which was developed in a village in Uganda where the work was piloted. (Editor's Note: Please visit the Stepping Stones Feedback website to learn more about this manual and how to access it.)

AAI found, in looking at the existing reviews of SS, that only a few go deeply into the SS process and outcomes, explore the methodological issues, and analyse in some detail the achievements and challenges from using the SS approach. (Appendix 2 of the report details the evaluations consulted; Appendix 5 features a proposed list of process indicators for SS.) Thus, many of the findings from existing SS evaluations are often rather generalised, based on self-reporting or observation soon after trainings end, with changes not explored in any detail. However, AAI has identified several findings, which are detailed in this report. A few highlights include:

• There is almost universal support for, and appreciation of, SS as a change process from those with first-hand experience of using it or seeing it used.
• Only a few reviews refer to the issue of spread/coverage, but the figures given in those cases are significant. A survey carried out in Uganda in 1997 indicated that 22,400 adults had been through SS training, the majority of them women. The World Bank estimated that AAI in Mozambique had reached 500,000 from 1999-2003, while in Zambia the coverage was 3,500 young people and over 100 adults. In 2004, the Malawi training was found to have reached 1,150 teachers and SS was to be rolled out across the entire primary school sector, reaching 55,000 teachers in 2006.
• Almost every review reported an improvement in communication, usually between spouses or children and parents, as a result of the SS training.
• Most reviews report positive increases in the knowledge and understanding of HIV/AIDS, its causes, and how to prevent its spread.
• Many of the findings of the different reviews supported the view that SS contributes to changes in knowledge and attitudes around sexual behaviour, gender relations, and those affected by HIV/AIDS.
• Most reviews referred to positive changes in behaviour such as increased use of condoms, more respect for women, less domestic violence, respect for women to refuse sex within marriage, better communication between couples and parents-children, and more cooperation around household chores and income.

Based on these findings, and this review process, AAI offers several recommendations, such as the following:

1. When SS is introduced, there needs to be clarity about factors including: its purpose, the problems it is designed to address, and the people for whom it is designed. Those implementing SS should consider which elements in the particular context will support positive change, and what are the barriers, external to the SS process, which need to be taken into account. A baseline needs to be collected so that changes can later be compared with this.
2. For M&E, it is important to be clear who needs what information, how it can best be collected, who can fund it, who is the best team/individuals to do it, and what measurement tools are most appropriate.
3. More resources need to go into M&E to ensure a build-up of learning and knowledge from the grassroots experience to feed into modification, adaptations, and policy work around HIV/AIDS prevention and mitigation. The nature and quality of the SS training needs to form part of any review.
4. Monitoring skills need to be prioritised and developed, with trainers, facilitators and staff implementing SS, and the results of monitoring need to be collected and analysed by HIV/AIDS teams, both for using in evaluation and for sharing with others working in this area. They need to be accessible to others working in this field wherever possible.
5. All evaluation reports need to meet some minimum standards of describing the programme, presenting the key aims, the budget and basic costs, the implementation process, the methods used and why they have been chosen and the key findings - with overall trends and exceptions, coverage and reach, and, where possible, who benefited most and how, and who benefited least and why.
6. A range of methods should be used, including revisits to old sites 3-5 years after SS has ended. More use of participant observation, triangulation of reported change, social surveys, and group reflection can be used to deepen and widen the evaluation data available.
7. There needs to be an open, accessible means for the storing of M&E on SS, well publicised and available to all.

In conclusion, the author notes that "It is hoped that this review of the existing publicly available M&E data on SS will contribute to the on-going debates around participatory approaches to HIV&AIDS in a number of ways....It has raised questions about how much seems to be expected of this training and how often the purposes it is used for remain rather vague and wide-ranging....It is hoped that the issue of what assessment/performance methodologies to use to evaluate participatory methods has been opened out and that in future these discussions can take place within a clear understanding of the need to link methodology to the overall purposes for, and the contexts in, which SS is being used. Above all the need for independent and critical review alongside community and agency monitoring should now be clear; this needs to be widely disseminated to promote shared learning and deepen understanding of the role and potential of participatory approaches to behaviour change."

Editor's note: Since this report was published, SS has made many of the reports reviewed within it available for download in PDF format on the references page of the SS website. In addition, the South African Medical Research Council has published a policy brief summarising the findings from an independent randomised clinical trial (RCT) of SS (which echo many of the findings identified by the AAI report); click here to access these reports.

SS is now part of the Strategies for Hope series; click here to access the Strategies for Hope website, where the manual and video may be ordered.