Core Group Polio Project (CGPP) Final Evaluation Report 2017

"Across the different countries, both government and WHO attribute the increased coverage in the CGPP target areas to the work of the project community mobilizers. They acknowledge that getting families to agree or even to seek out the polio vaccine was a major barrier. Project community mobilizers were able to overcome this challenge through their knowledge of the local culture and efforts to educate the community."

This is the final evaluation of the 2012-2017 project phase of the CORE Group Polio Project (CGPP), a multi-country, multi-partner initiative providing financial support and on-the-ground technical guidance to strengthen host country efforts to eradicate polio. It is intended to document achievements, actions, and lessons. Findings will inform plans for the next phase.

CGPP began in 1999, with the CORE Group channeling United States Agency for International Development (USAID)-funded grants to international and national non-governmental organisations (NGOs) to support polio eradication by mobilising communities to take part in immunisation campaigns, routine immunisation (RI), and acute flaccid paralysis (AFP) surveillance. Since then, the CGPP has worked in a total of 9 countries. The first phase of the CGPP included India, Ethiopia, Angola, Bangladesh, Nepal, and Uganda. Once the World Health Organization (WHO) declared Bangladesh, Nepal, and Uganda free of polio, they graduated from the programme; Angola joined the others in 2016. The CGPP later added Nigeria, South Sudan, and the Horn of Africa, or HOA (Kenya and Somalia). Currently, the CGPP is working through 9 international and 30 local NGO partners to reach conflict-affected zones and under-served populations such as refugees, internally displaced persons (IDPs), and nomads in very remote areas.

Each CGPP country has its own Secretariat, a small team of neutral technical advisors who facilitate communication, coordination, and transparent decision making among all partners. Through this model, the countries coordinate and promote civil society engagement in polio eradication, while simultaneously infusing a community-level component through the focused activities of thousands of community health workers (CHWs). The Secretariat in each country handles coordination with the host government and others engaged in the Global Polio Eradication Initiative (GPEI), such as the United Nations Children's Fund (UNICEF), WHO, Rotary International, and the US Centers for Disease Control and Prevention (CDC).

To assess the 2012-2017 project phase, an independent consultant conducted a quantitative household survey in the country target areas, apart from South Sudan, which is only working on surveillance and campaign monitoring; the Angola programme closed in 2016. Each CGPP Secretariat contracted a local firm to collect and analyse the qualitative and quantitative data and to generate a report. The reports were reviewed by the evaluator and/or the CGPP Technical Advisor for M&E [monitoring and evaluation]. The quantitative surveys were conducted between July and September 2017, with reports finalised by November 2017.

The qualitative data collection consisted of focus groups with female caregivers and with fathers separately along with individual interviews of community mobilisers, government health workers, and Secretariat staff. Religious leaders were interviewed in Somalia, Kenya, and Nigeria. The qualitative data collection occurred during the same period as the quantitative survey, but guidance called for selecting different communities than those involved in the quantitative survey to prevent bias and interview fatigue.

The Indicator Tracking Table (see Annex A) lists the objectives, intermediate results, and indicators; these were measured either by routine monitoring or by survey comparisons from the baseline and endline surveys. In general, progress was made against all objectives, with variation between countries due to contextual challenges, such as civil unrest and militant group ideology. (As noted here, vaccine hesitance or resistance and porous borders through which nomads and IDPs transit from the under-vaccinated regions continue to add to the complexity of the challenge.) Besides fully engaging the partners in the national collaborative efforts against polio and now participating in documenting results and planning the post-polio transition, the country programmes reportedly made their greatest progress in contributing to improved RI coverage and coverage of oral polio vaccine type 3 (OPV3) and to AFP surveillance, including adequate stool samples.

Major findings from the endline evaluation, as excerpted from the report, include:

• "Community mobilizers in each of the countries are well-respected, knowledgeable and influential in convincing parents to seek immunization and are the top source of health information in hard-to-reach communities. They are the primary source of knowledge on polio and polio campaigns in CGPP focal areas. Their influence and credibility has grown over the life of the program.
• Coverage of routine immunization, including OPV3, in the CGPP target areas has increased and is now higher than national coverage data in most countries.
• The increases in percent of children 12-23 months fully immunized are particularly notable in Nigeria (from 33% at baseline to 57%) and in Ethiopia (from 24.7% at baseline to 43.6%).
• The use of newborn tracking and education from community mobilizers has increased OPV birth dose, with Ethiopia reporting increases from 52.0% to 54%; India from 64.2% to 78.9%; and Nigeria, from 54.9% to 98.6%.
• The CGPP has supported SIAs [supplementary immunisation activities] through monitoring, planning, social mobilization and logistical support with activities varying by country. Children under 5 missed in each SIA in Nigeria has decreased from 4.5% to 1.5%. In India, houses missed has decreased from 5.9% to 4.4%.
• Non-polio AFP surveillance (NPAFP) has been a success as measured by the indicator of identifying >2 per 100,000 children under age 15. Additionally, the average percentage of adequate stool samples is about 90% across five countries...
• South Sudan has established a timely, accurate, and robust community based surveillance system....Post campaign monitoring has evolved to become a great strength of CGPP South Sudan.
• In 2014, polio was eradicated in India, despite deeply entrenched social resistance to immunization campaigns. The Secretariat developed innovative Behavior Change Communication strategies for social mobilization efforts. The use of child registries provided solid, up-to-date data to improve and track routine coverage rates, and allowed for the improvement of birth dose timing and tracking. These strategies have been employed elsewhere with success."

Furthermore, the report asserts that each of the countries has come up with new methods or strategies during the current phase. (The specific progress and challenges for achieving results are elaborated in the country-specific sections of the report.) Briefly:

• Kenya and Somalia led the Cross-Border Health Initiative, which involves the convening of inter-country border stakeholders to synchronise campaigns and other efforts to reach every child. The HOA Secretariat supported outreach to nomadic populations by providing social mobilisation and transportation for health facility staff to support integrated RI outreach.
• Ethiopia established a mobile device and web-based disease surveillance system, developed a system for newborn tracking of polio birth dose, and strengthened cross-border collaboration with CGPP HOA.
• India has developed strategies to motivate fathers to increase participation in seeking immunisations for their children. Under-served communities were reached mainly through interpersonal communication (IPC) sessions, group meetings, and the creative use of mixed media. The Secretariat has also developed special certificates for parents who completed timely immunisation of their children.
• Nigeria established a network of community volunteers and expanded their capacity to deliver healthcare interventions beyond polio eradication, such as water, sanitation, and hygiene (WASH), nutrition, and malaria prevention, and it leveraged community structures to boost surveillance at all levels.
• South Sudan (see also Related Summaries, below): instituted Independent Campaign Monitoring across the entire country to measure the reliability and accuracy of campaigns; recruited and trained nearly 2,000 "key informants", such as traditional healers and birth attendants, religious leaders, and community leaders, to improve community-based surveillance (CBS); and facilitated cross-border meetings to synchronise campaigns and build relationships with neighbouring country health systems.
• Angola utilised a network of 2,710 CHWs to promote CBS, increased RI using community registers, and leveraged skills of CHWs to tackle other health initiatives such as the use of treated bed nets and malaria drugs.

The report offers a series of recommendations, organised around these topics:

• Data-driven programme recommendations - For example: "The CGPP must continue to find innovative and inclusive strategies to combat complacency, especially in countries that have been polio-free for a number of years. As campaign frequency decreases, it is important that communication messages remain strong and continue to inform parents of the importance of vaccination. Transition strategies should also include strong messaging."
• M&E recommendation: "The project's Technical Advisor for Monitoring and Evaluation has begun to organize the project data and develop electronic systems for continual organization and updating. This process should continue to ensure that all data and information collected in the new phase can be easily utilized and lessons learned can be effectively shared."
• Transition recommendations - For example: "Careful documentation of the process of preparing and mentoring/supporting social mobilizers would be useful for partners and national stakeholders who may wish to use the CGPP strategy of social mobilization in the future."

Some concluding thoughts related to methodology: "Measuring impact of the project outcomes and outputs at the project-wide level is impeded by lack of indicators on which every country consistently reports and/or includes in the survey reports....In summary, the project strategy and activities are effective in achieving the results for which indicator data is available. Anecdotal evidence and the qualitative data indicate that the project is most likely achieving increased OPV coverage and AFP surveillance, however quantitative data is not available across all countries to assess the level of achievement."