Salvation Army World Service Organization

This report shares the results of a midterm evaluation (MTE) of the Chikankata Child Survival Project (CCSP), which was launched October 1 2005 by The Salvation Army (TSA) World Service Office (SAWSO) and TSA Chikankata Health Services (CHS). It is a 5-year project, funded by the United States Agency for International Development (USAID), which aims to reduce maternal and under-five child mortality among 53,521 direct beneficiaries in the Mazabuka and Siavonga Districts of Zambia's Southern Province. CCSP collaborates with the Zambian Ministry of Health (MOH) by serving rural health centres (RHCs), and has 4 intervention areas: malaria prevention and treatment, immunisation coverage for children, childhood and pregnant women's nutrition, and maternal and newborn care.

The central strategy being used is the Care Group model, according to which every household with women of reproductive age is cared for and visited every month by community health volunteers, called Care Group Volunteers. Each Care Group Volunteer is assigned 10 households within walking distance. The Care Group Volunteers are organised into care groups of 10 members. The Care Groups meet twice a month for training, reporting, and discussion of their home visits. The Care Groups are trained and supervised by Field Facilitators, who visit each group twice a month. Each Field Facilitator has 8-12 care groups under his or her supervision. The Field Facilitators in turn are supervised by Field Supervisors; each Supervisor works with 4-5 Field Facilitators.

Specifically, "Care Group mothers act as early adopters of new health behaviors and model these to their neighbors....It is a labor-intensive model because of the large number of volunteers who have to be trained and supervised. The benefit is the intensive peer-to-peer interaction regarding behavior change. It relies on personal interaction to nurture behavior change rather than printed health education and communication tools. It is very appropriate for the culture and the socioeconomic conditions of this project. Paper is expensive and scarce so it cannot be widely used for health communications. The literacy rate is low so printed text would not be very useful anyway. Also the local culture is highly relationship-oriented so that the one-on-one interaction among peers is a very appropriate method for communicating behavior change."

Care Groups do, as explained here, use some printed materials in their communication for behaviour change work; the MTE team recommends improving the quality of these materials, noting that each message should be expressed in picture form, with images that predominate on the page. "An additional communication channel would be to make posters with pictures and place these in the RHCs. They can add color to what are often bare walls and can be used as teaching tools at the center. Another way that the pictures can be used is to print them in the form of small posters that can be given to the Care Group Volunteers to be displayed in their homes. People value this kind of decoration, and thus can be used as incentives. For example, they can be handed out for years of service as a Care Group Volunteer."

In addition, a key communication method that this project used was to put each of the messages to song. "Care Group Volunteers are taught the songs as part of their training and are sung at almost every training session. The songs are used during home visits and community events. Singing is an integral part of the culture and consequently is a most appropriate method for behavior change communication."

Another community mobilisation strategy was the development of 20 pilot men's groups - whose members hail from fellowship groups from TSA churches - in the Mazabuka catchment area. In their groups the men learn about child survival issues relating to malaria, immunisation, and nutrition. The men's groups participated in disseminating health messages to other men within and outside the church. They were instrumental in distributing ITNs in their areas. The MTE team interviewed 4 men's groups: "All of them stated that men are now getting more involved in children's health than in the past. They have come to realize that the whole community should be involved, not just the women."

In addition to these interviews, other monitoring and evaluation (M&E) assessment strategies used as part of the MTE include: knowledge, practices, and coverage (KPC) survey, health facility assessment, organisational capacity assessment with local partners, participatory learning in action (PLA), lot quality assurance sampling, community-based monitoring techniques, and participatory evaluation techniques.

The main accomplishments of the project, according to the MTE, are as follows:

• CCSP contributed to an increase in childhood immunisation coverage from 33% to 84%: "A critical factor was the work of the Care Group Volunteers. They mobilized the community to attend growth monitoring sessions and they followed up with mothers who fall behind in their children's vaccination schedule. The relatively low amount of effort in this intervention (10%) has yielded a very high return, largely due to the Care Group Volunteers along with the training and supervision by the Field Facilitators. Another critical factor is the project's collaboration with the TSA Chikankata hospital in immunization coverage. The hospital sends out monthly medical caravans in the Mazabuka District. Project staff members inform communities about the caravan schedule and work with the Care Group Volunteers to mobilize mothers to bring their children for scheduled vaccinations. The hospital administrator stated that attendance at the caravans has more than doubled because of this collaboration."
• 85% of mothers interviewed identified fever as a sign of malaria and 91% of these women stated that they would seek care for an episode of child fever that same day. This is a substantial increase from the baseline, where 11% stated they would seek care within 24 hours.
• 86% of mothers of children under 5 years reported that their children slept under an insecticide-treated net (ITN). This is an increase from 22% at the baseline.
• 90% of the mother of children under 5 reported that they had their child's growth monitoring card on hand, and over 80% of the cards were up-to-date in recording immunisations, growth monitoring attendance, and weight measurement.
• 98% of the pregnant women had their antenatal cards in their possession and 100% of them had at least one prenatal visit recorded on the card.
• 58% of home births in the month of May (2008) were done by Trained Traditional Birth Attendants (TTBA). This is an increase from 13% at the baseline.
• 81% of pregnant women ate iron-rich food the day before the evaluation.
• All of the 28 Care and Prevention Teams (CPTs) that were interviewed were active in problem-solving.

Overall, at the MTE CCSP was found to have surpassed the end of project (EOP) targets for the proportion of children 0-23 months who sleep under an ITN and pregnant women who sleep under an ITN, the proportion of children 12-23 months who are fully vaccinated, and the percent of mothers of children 0-23 whose birth was attended by trained personnel and who had at least one postpartum check-up. In each of these indicators the MTE percentages have increased significantly and in some cases even doubled.

Following is a summary of the main constraints, problems, and areas that, per the MTE, need further attention:

• The distribution of long-lasting ITNs is insufficient in the project area, and record keeping of coverage is mostly non-existent at the RHCs.
• The availability of clean delivery kits (CDKs) is insufficient. Only 35% of the pregnant women had a CDK at home. Price was a factor in limiting access to CDKs.
• 31% of the TTBAs did not know the sequence of postnatal checkups for mothers and newborns.
• Only 44% of the Care Group Volunteers stated that they visited all of their households in the month of July (2008). The foundation of the Care Group model is that each household is visited once a month, thus it is crucial that this percentage be increased substantially.
• The evaluation team found that there was almost no relationship between the Care Groups and the CPTs. An organisational relationship is essential for the sustainability of the Care Groups.

Main MTE conclusions:

• CCSP is on track in implementing the interventions in the Detailed Implementation Plan (DIP).
• The Care Group Volunteers, TTBAs, and CPT/Neighbourhood Health Committees (NHCs) are effectively mobilising the community for immunisations, growth monitoring, and ITN distribution.
• CCSP made good progress in educating women about the benefits of immunisations, the use of ITNs, consumption of iron-rich foods, and the importance of delivering babies with trained personnel.
• CCSP is a well managed project that has a good system for data management and is using data from field reports for monthly planning.

Key recommendations:

• CCSP managers, the District Health Management Team (DHMT), and the USAID/Zambia Mission should collaborate to identify the bottlenecks in the distribution of long-lasting ITNs.
• Community health workers (CHWs) at each RHC should set a goal of 90% of underweight children recuperating a normal weight by one month's time.
• CCSP staff should investigate the barriers to the availability of CDKs for pregnant women.
• CCSP staff should conduct a doer, non-doer assessment of the reasons why some women still chose to deliver without trained assistance.
• TTBAs should be trained as soon as possible after the MTE on postnatal care, and support of postnatal care should be part of their responsibilities as TTBAs.
• Field Facilitators should work with their Care Groups to set goals for membership, attendance, and home visitation, and then facilitate a process whereby the groups make their own plans for meeting the goals. (This is crucial for sustainability).
• CCSP staff should be involved in building linkages between CPTs and the Care Groups in their area. Each of these entities should get to the point where they are mutually supportive, independent of CCSP staff.

Amongst the additional impacts described in the report is the following: "The large network of women who have leadership in the community as Care Group Volunteers has contributed to the improving the status of women. Over 2,000 women have been trained and are active in their community. They have an organizational structure, the Care Groups, which they manage on their own. CCSP staff members are working to build a system of accountability between the Care Groups and the community leaders in the CPTs and the NHCs. Over time, the Care Group leaders will become members of the CPTs and be respected for having an important role in the development of the community."

Development Experience Clearinghouse (DEC) Express, August 7 2009.