President's Malaria Initiative (PMI) External Evaluation Report

This report shares findings of an external evaluation of the first five years (FY 2006-FY 2010) of the President’s Malaria Initiative (PMI). PMI, led by the United States Agency for International development (USAID) and implemented with the U.S. Centers for Disease Control and Prevention, works to reduce malaria-related deaths by 50% in 15 countries that have a high burden of malaria through proven curative and preventive interventions, including insecticide-treated nets (ITNs), indoor residual spraying (IRS), intermittent preventive treatment for pregnant women (IPTp), and artemisinin-based combination therapies (ACTs). Overall, the evaluation found that PMI is, by and large, a very successful, well-led component of the United States Government’s Global Health Initiative. Through its major contributions to the global malaria response via its collaborations with multilateral and bilateral partners, effective relationship with the Global Fund, and contributions to reinvigorating national malaria control programmes, PMI has made substantial progress toward meeting its goal of reducing under-5 child mortality in most of the 15 focus countries.

Based on extensive document reviews, key informant interviews of U.S.-based and country-based personnel, field visits to five of the PMI focus countries, and extensive debate within the team, the evaluation report offers five policy recommendations and five technical recommendations based on the outcome of the research.

This summary looks specifically at the evaluation from a communication point of view and seeks to pull out some of the key points and issues raised around communication. These relate particularly to the Principles of Operation which include a community-based approach and capacity building. It also looks at one of the key features of the PMI, which is around working in partnerships at international, national, and local level.

Community based approaches (under Principle 1: Use of Comprehensive, Integrated Package of Interventions)
In an effort to achieve the desired results, PMI has collaborated with the National Malaria Control Program (NMCP) and other Ministry of Health (MOH) units to employ community-based approaches to deliver interventions, especially in rural areas and hard-to-reach villages. The main approaches are community case management (CCM) of malaria, community distribution of LLINs, and community-based behavior change communication (BCC) and information, education, and communication (IEC) activities.

Working through sub-grants to CBOs, FBOs, and NGOs, PMI supports community distribution of LLINs in all 15 countries. These organisations engage community volunteers to distribute nets during mass campaigns or through door-to-door distribution. In some countries, the volunteers not only distribute nets but are also responsible for hanging them. In one of the countries implementing CCM, LLINs are distributed to children under 5 who are seen at clinics. According to the report, PMI has made a significant investment in community-based BCC/IEC activities across the 15 focus countries to increase understanding and demand for malaria prevention and treatment interventions. AS stated in the report, PMI estimates that these activities have reached millions of people in urban, peripheral, rural, and isolated villages across the focus countries since PMI's inception, although it is difficult to precisely quantify utilisation.

Capacity building and participation (under Principle 3: NMCP Strengthening and Country Ownership)
From the outset, PMI made an important strategic decision to make the NMCP its central partner, recognise the NMCP's leadership role, and work within that framework. PMI carried out important activities to strengthen the NMCP’s capacity to lead, advance the sense of country ownership, and encourage countries to assume the responsibility of partner and stakeholder coordination.

The evaluation team found the NMCP managers consider themselves to be the key actors in the overall coordination of PMI programming. For the most part, they value PMI's respect for their important national role. The team agrees with the majority of NMCP managers and partners who reported that PMI "performed well" in putting the principles of NMCP strengthening and country ownership into practice. The report however explains that the performance of individual NMCPs was highly variable. The stronger national programmes, though aided by PMI personnel, did not report that their capacity had been strengthened markedly. Some NMCPs may have been strengthened due to their interaction with PMI and became more independent, while a third group are still deeply dependent on external support and have made less progress in capacity strengthening.

Partnerships

Realising the need for achieving and documenting impact, PMI from its inception placed emphasis on the development of effective partnerships in the beneficiary countries. The results of the evaluation showed that:

• PMI developed strong partnership with almost all the NMCPs in the 15 focus countries and recognised the NMCP's lead role in malaria control.
• PMI is recognised as a key partner at country level and its contribution is well appreciated by most of the partners. Some described PMI as "flexible", "more transparent", "inclusive in designing its approaches" and "receptive to ideas and suggestions."
• Most partners believe that PMI has invigorated and strengthened malaria partnerships at the country level. The partners note the role PMI plays in creating efficient communication within the partnership. In conversations with the evaluation team, they acknowledged the strategic and technical contributions of some of the Residential advisors, along with their understanding of the specific roles of other partners and their sensitivity to the requirements of international work.
• PMI is generally viewed as an exemplary partner by partners because it does not use its large, broad presence and substantial financial support to gain undue influence within the partnership.
• PMI has had a worthwhile role in the global partnership, especially with the Global Fund and the RBM Partnership. Active and collegial engagement within the partnership appears to have contributed significantly to the attainment of PMI’s goal and objectives. It is unusual that a bilateral donor initiative with so much financial and technical resource invests so much effort in collaborating with a range of partners and interested parties. Not only do partners appreciate this, but there are indications that the NMCPs see this as helpful for their engagement with partners. PMI clearly sees partner engagement as well worth the effort.
• PMI was however not seen as a good partner in one specific area. PMI leadership and selected technical personnel may have their reservations about Affordable Medicines Facility—malaria, but the evaluation team heard from multiple sources that the perception of an openly antagonistic manner in which PMI voices its concerns is unjustified and potentially inappropriate. As the AMFm is being evaluated by others, the evaluation team suggests all parties “let the data speak” to resolve questions over the benefits and impacts of the AMFm program in an empirical fashion.