Post on 16-Apr-2018
Malawi Primary Health Care (PHC) Project Evaluation
Clinton Health Access Initiative (CHAI) - Malawi July 2014
An operational and health impact evaluation of the multi-partner
implemented emergency essential medicines
parallel supply chain in Malawi
Government of Malawi
Lilongwe
2
Acknowledgements
The CHAI evaluation team would like to thank the support of the Malawi MOH,
UNICEF, CHAM, and the other organizations and partners who provided feedback on
this report. The authors of this report would like to specifically acknowledge the
support and time of the staff of the 59 health facilities interviewed for this report.
3
Table of Contents Acronyms ................................................................................................................................................ 4
Executive Summary ................................................................................................................................. 5
Purpose, Objectives and Scope ............................................................................................................... 9
Evaluation Methodology ....................................................................................................................... 10
Evaluation Findings ............................................................................................................................... 13
A. Relevance .............................................................................................................................. 13
B. Effectiveness .......................................................................................................................... 16
C. Efficiency ............................................................................................................................... 29
D. Impact ................................................................................................................................... 32
E. Sustainability ......................................................................................................................... 36
Lessons Learned .................................................................................................................................... 40
Conclusion and Recommendations ...................................................................................................... 42
Annexes ................................................................................................................................................. 43
Annex 1: Evaluation Terms of Reference .......................................................................................... 43
Annex 2: Evaluation Team ................................................................................................................ 49
Annex 3: Evaluation Questions ......................................................................................................... 50
Annex 4: Phase I Kit Contents ........................................................................................................... 52
Annex 5: Phase II Kit Contents .......................................................................................................... 53
Annex 6: Comparison of Phase I and Phase II Items ......................................................................... 56
Annex 7: List of Interviewed Partners and Sites Visited ................................................................... 58
Annex 8: Reported List of “Most” and “Least” Important PHC kit, Over and Undersupplied Items by
Facility Level ...................................................................................................................................... 62
Annex 9: Analysis of Most Important, Least Important, Stocked Out, Under and Oversupplied Kit
Item Lists ........................................................................................................................................... 66
Annex 10: Tracer Item Selection Matrix ........................................................................................... 68
Annex 11: Under and Overstocked Items and Expiries..................................................................... 69
Annex 12: Evaluation Surveys ........................................................................................................... 71
Annex 13: Bibliography ..................................................................................................................... 85
4
Acronyms
ARIs Acute Respiratory Infection
CCTV Closed Circuit Televison
CHAI Clinton Health Access Initiative
CHAM Christian Health Association of Malawi
CMST Central Medical Stores Trust
DAC Development Assistance Committee
DFID Department for International Development (UK)
DHO District Health Office (or Officer)
FEFO First Expiry First Out
HMIS Health Management Information System
HSA Health Service Assistant
HTSS Health Technical Support Services - MOH
JSI John Snow Inc
KfW Kreditanstalt für Wiederaufbau (Reconstruction Credit Institute)
LMIS Logistics Management Information System
MDG Millennium Development Goals
MMR Maternal Mortality Rate
MOH Ministry of Health
O&M IT/S O&M Information Technology Services
OECD Organization for Economic Cooperation and Development
OPD Out Patient Department
PHC Primary Health Care
PMPB Pharmacy Medicines and Poisons Board
PODs Proof of Delivery
RTT RTT Group (Now Imperial Health Sciences)
UNICEF United Nations Children’s Fund
WHO World Health Organization
Executive Summary
PHC Project Overview
The Primary Health Care (PHC) Essential Medicines project was launched in January 2012 to address
the essential medicine and health commodity stock out and under supply emergency in Malawi. A
project of national reach, the project supplied standardized kits of essential medicines and supplies
on a monthly basis to over 620 Ministry of Health (MOH) and Christian Health Association of Malawi
(CHAM) facilities in all of Malawi’s 28 districts through a parallel supply chain. The project received a
total of USD39M1 in funding from the governments of the United Kingdom, through DFID, Norway,
through the Royal Norwegian Embassy in Malawi, and Germany, through KfW.
The project was coordinated by UNICEF under a multi-partner project approach. PHC Project
partners included the MOH, the Pharmacy, Medicines and Poisons Board (PMPB), the Central
Medical Stores Trust (CMST), international donors, and John Snow Inc. (JSI), who oversaw the
warehousing and distribution planning for the kits. The project was designed to be implemented
from January 2012 to June 2013. It was then extended to the fourth quarter of 2013 in order to
distribute the remaining kits and supplies.2 The last full distribution during this phase of the project
took place in September 2013, with partial deliveries continuing into October 2013.
PHC Project Goals and Objectives
The overall goal of the PHC project was to contribute towards keeping mortality and morbidity
among children under 5 and pregnant women on a constant declining rate in Malawi. It was
intended to meet the 2015 Millennium Development Goals (MDG) 4 &53 by averting drug shortages
over the implementation of the project. Operational objectives included: the implementation of an
efficient and accountable supply chain system; national level project oversight; a functioning
independent monitoring mechanism; health workers sensitized and knowledgeable about the kit
items; and an effective communication campaign that sensitized communities about the kit project.
In addition, in order to address longer term national supply system concerns, the Central Medical
Stores Trust (CMST) was to receive 55% of the kit costs from recipient MOH and CHAM facilities to
support their recapitalization initiative.
1
Using end of project September 2013 USD / GBP, NOR and EURO conversion rates 2
During the extension period ( June to October 2013) the warehousing and distribution of the kits was transitioned to UNICEF 3
MDG4: Reduce child mortality by two-thirds, between 1990 and 2015, the under-five mortality rate; MDG5: Reduce by three quarters, between 1990 and 2015, the maternal mortality ratio
6
Project Accomplishments (as of September 2013)
15,844 kits in total were distributed to over 620 MOH and CHAM facilities in 28 districts
The total number of medicines and supplies distributed by the PHC project potentially
served up to 9,487,697 patients.
The kits program reduced product stock outs from 31% to as low as 7%, with 81% of
facilities reporting that the project reduced the incidence and duration of product stock
outs.
An estimated MKW 4,202,126,365 (US$8,670,690) in total went to the recapitalization of
the CMST from the PHC project.
Under its multi-partner model, the project was rapidly set up and exhibited a high degree of
supply chain visibility and controls and all PHC kit contents passed PMPB quality testing.
Project Implementation Challenges
Communication and coordination challenges were experienced with CHAM facilities as many
facilities believed that the kits were free.
As a result of the standardization of the kit contents and their quantities, over and under
stocks were experienced with 75% of recipient facilities reporting some level of product
redistribution to mitigate expiries. However, there were relatively few expiries experienced
during its implementation.
Two major in-country distribution delays were experienced. The first in April 2012, due to
vendor delays, and the second in June 2012, due to the delays in the Beira corridor and the
relocation of the UNICEF warehouse in Copenhagen.
Overall Project Findings
The PHC project was a timely and relevant project and while it did not eliminate essential medicine
and supply stock outs in Malawi, it had a significant impact in reducing their occurrence and
duration. Eighty-one percent (81%) of facilities interviewed agreed or strongly agreed that the PHC
project prevented stock outs at their facility. The kits program reduced product stock outs nationally
from 31% in 2011 to as low as 7% in 2013 for selected items.4 In addition, a total of 89% facilities
either strongly agreed or agreed that the kits increased patient access to essential medicines. In total
15,844 kits were distributed with no quality issues regarding their contents reported. These needed
medicines and supplies potentially served up to 9,487,697 patients.
The overarching objective of the project was to contribute towards keeping mortality and morbidity
among children under 5 years and pregnant women on a constant declining rate to meet the 2015
MDG 4 & 5. While there is no way to conclusively link the contribution of the kits program to keeping
mortality and morbidity among children under 5 and pregnant women on a constant declining rate,
it is clear that the PHC’s improvement in the availability of specific commodities contributed to this
overall decreasing trend.
Overall, the PHC project was also found to be effectively and efficiently implemented. The rapid
planning and launch of the project was particularly notable, given the national scope and complexity
4 See Page 10 for selected tracer items
7
of the project. The project demonstrated a high degree of project visibility as demonstrated by
routine coordination meetings, regular PMPB product quality reporting, as well as the continuous
independent monitoring of the project’s implementation. The project achieved a high degree of
inventory and supply chain visibility in particular, as demonstrated by monthly distribution
summaries, the complete accountability of PODs, as well as the immediate identification of an
incidence of theft in 2012. A total of 88% of the health facilities said that the PHC kits met their
overall delivery expectations. In general about 69% of the sites said the deliveries occurred within
one week of their expectations.
The PHC project contributed an estimated MKW 4,202,126,365 (US$8,670,690) to the
recapitalization of the CMST. While this is a significant contribution, as a result of invoicing and
payment challenges, this potentially only represents roughly 65% of the revenue that CMST should
have received from the kits program. CHAM facilities, in particular, were noted for their delayed or
lack of payment for the PHC kits, mostly due to budget constraints. In addition there were
communication and project coordination issues with CHAM sites, who were initially under the
impression that the PHC kits were free.
Despite the prioritization of communication outreach and mass media sensitization efforts, 48% of
facility staff felt that the mass communication campaign did not reach patients and communities,
with 28% of clinical staff reporting that they had first learned about the kit program when the kits
arrived at their facility. As result of the standardized nature of the kits, 75% of facilities reported
redistributing oversupplied or unneeded PHC items to other facilities or to their district hospitals. In
addition, despite the overall improvement of LMIS rates in Malawi, the recording of PHC items on
stock cards was inconsistent among facilities, with about 24% of monthly stock entries
undocumented or incorrect, potentially introducing those products to a risk of leakage at the health
facility level.
Conclusion
The PHC was a timely and effective medicine and health commodity supply delivery intervention. It
reduced the occurrence and duration of health commodities stock outs, particularly at the rural
health facility levels, and its contribution to primary health care in Malawi is a considerable
achievement.
The PHC project’s contribution to the recapitalization of CMST was meant to address concerns about
the sustainability of the project. However, as evidenced by the eventual revival of the kits project in
2014, the recapitalization of CMST did not yield capacity and performance outputs by the end of the
PHC project as intended. While the PHC project was largely a project success, the MOH, CMST and
other partners, ultimately need to address the broader supply system issues that prompted the need
for the PHC project intervention in the first place. By implementing the 2012 Joint Strategy for
Supply Chain, and strengthening the public supply system, Malawi’s public health system will be able
to better plan against and respond to emergency stock out situations and begin the process of
building a more sustainable national supply delivery system.
8
Evaluation Recommendations
Design kits specific to health centers with maternity wards, facilities with surgical wards, and
central hospitals for future kit project implementation.
Revive and implement the 2012 Joint Strategy on Supply Chain Integration Plan.
Strengthen CMST’s invoicing and account reconciliation capacity.
Standardize and streamline product redistribution and relocation policies and procedures.
Implement more targeted supply management improvements at the DHO level.
Strengthen the inventory management capacity and performance of CHAM facilities.
9
Purpose, Objectives and Scope
Evaluation Purpose
The purpose of the evaluation is to conduct a comprehensive and systematic evaluation and review
of the impact of the PHC Essential Medicines project during its implementation. As Malawi’s public
health system maintains multiple, parallel supply and distribution systems, which pose longer-term
coordination, financing and sustainability challenges, this evaluation is intended to provide more
evidence on the effectiveness, implementation, and health impact of such a parallel system.
In 2012, the MOH, CMST and partners developed and agreed to the implementation of the
Integrated Supply Chain Strategy. As such, this evaluation also aims to inform the implementation of
that strategy and to draw operational lessons from the successes and challenges of the PHC project.
Evaluation Objectives
The objective of this evaluation is to assess the implementation of the PHC project and its health
impact in Malawi, particularly its impact ensuring patient access to Primary Health Care products.
The PHC project was assessed on its relevance, effectiveness, efficiency, impact, sustainability, and
equity.
Scope of Evaluation
This evaluation assesses the planning, establishment, implementation, and to a lesser extent, post-
project planning phases. It was guided by evaluation questions based on the original UNICEF RFP
(see Annex 1). Among the areas investigated in this evaluation include project coordination, supply
chain efficiency and effectiveness, the communication components of the project, as well as health
system capacity building. The evaluation had a special focus on understanding how patients
benefitted from the PHC project, especially the project’s contribution to reducing the burden of
maternal and under-five mortality rates in Malawi.
10
Evaluation Methodology
Evaluation Framework
Based on the original submitted RFP required evaluation questions, the CHAI evaluation team
evaluated the following project areas (Figure 1):
Figure 1 Evaluation Framework
Evaluation Methods
Data Sources
The evaluation was implemented from February to May 2014. The evaluation included surveys that
were implemented at 59 health facilities. In addition, partner and donor key informant interviews
were conducted, along with a comprehensive review of PHC project documentation. Subject-
relevant research on available data was also conducted. Reviewed documented sources included:
• Health center and hospital inventory management stock cards • UNICEF project management documentation • Logistics Management Information System (LMIS) • Health Management Information System (HMIS) • O&M IT/S and Management Services PHC Reports • Various relevant kit system program evaluations and literature on essential medicine
distribution programs The evaluation conformed to the OECD Quality Standards for Development Evaluation and
implemented a rigorous internal review process that included feedback from MOH and development
partner organizations on the draft report.
PHC Evaluation Guiding Questions
Based on the evaluation framework, the team developed a series of high-level evaluation questions,
which guided the development of the research agenda for the evaluation, partner interview tools, as
well as health facility surveys [See annex for the evaluation questions as well as the surveys].
11
Site Selection and Informational Interviews
The evaluation site selection included all MOH
central hospitals, 10 district hospitals, and
randomly selected health centers from the
northern (~20%), central (~30%), and southern
regions (~50%). A total of 59 sites were visited (44
MOH sites and 15 CHAM sites). Changes were
made to the initial list of randomly selected health
centers due to the identified overrepresentation
of sites in one district, or one particular CHAM
denomination. Health facility interviews were conducted both in Chichewa and English depending on
the comfort level of the interviewed staff. During the visits, interviews were also conducted with
village chiefs, head men and members of the community drug committees when possible. For a
complete list of health facilities visited and partners interviewed, see Annex 7.
Tracer Item Selection
In order to understand recordkeeping practices for kit items and assess potential leakage levels, 10
kit items were selected as tracer items. They were designated to be reflective of priority primary
health disease burdens as well as representative of each kit box (1-7, and briefly 1-8), as well as the
loose items that arrived under a different schedule than the regular boxes. Additional criteria were
also selected to better understand the feasibility of data collection [See Annex 10].
Amoxycillin susp bot
Cotrimoxazole, 480mg tabs
Erythromycin tabs 100s
Glucose 5% 500ml iv 20s
Oxytocin Inj
Paracetamol, 500mg tabs
Quinine inj 10s
Sodium chloride 500ml iv 20s
Sulphadoxine/Pyrimeth 1000s
Syringe+needle, 2ml unit
Evaluation Limitations
Sample Size: A total of 59 health facilities were visited during this evaluation due to time and team capacity
constraints. Due to the absence of staff the day site visit occurred, the clinical survey for Nthisi District
Hospital and the inventory manage survey Zomba could not be completed. As a result, the findings and
recommendations for both the clinical and inventory management areas were each based on data from 58
facilities.
Stock Card Data Quality: In about 31% of the cases, inventory management stock cards were incomplete,
not able to be located, or otherwise inaccessible. This evaluation team has attempted to mitigate this
concern by quantifying these gaps with the goal of identifying the levels of confidence in the data collected.
Health Center Institutional Memory: The length of time staff has worked at a health facility will likely impact
their ability to recall PHC project implementation details. Informants who have been at their facility for a
Total Sites Visited by Facility Type
Facility Type MOH CHAM Grand Total
CENTRAL HOSPITAL 4 - 4
DISTRICT HOSPITAL 10 - 10
HEALTH CENTRE 30 10 40
MISSION HOSPITAL - 4 4
RURAL HOSPITAL - 1 1
Grand Total 44 15 59
12
shorter period might be less knowledgeable about the implementation of the project at their facility, which
could have also introduced interviewee recall bias.
The following table reflects the amount of time interviewed staff have worked at their respective facilities:
Staff posted at health facility5
Clinical Management
Survey (n = 58)
Inventory Management
Survey (n = 58)
A. % of Staff who have been at their facility since 2010
41% 38%
B. Average # months worked 58 months (4.8 Years)
52 months (4.3 Years)
C. Median # months worked 36 month (3 years)
42 months (3.5 years)
D. % of staff started before the project [ Before January 2012]
60% 72%
E. % of staff started within the implementation of the PHC project [Between January 2012 – June 2013]
28% 24%
F. % of staff started during the PHC project extension period
[Between July - Sept 2013] 2% 2%
G. % of staff started in 2013 [After October 2013]
10% 2%
5 Benchmarked against the staff who participated in the survey site visits in March 2014
13
Evaluation Findings
The findings of this evaluation have been organized into six categories in compliance with the Development
Assistance Committee (DAC) Quality Standards for Development Evaluation. A qualitative ratings system has
been established using the following categories: Poor, Adequate, Good and Excellent.
The following table summarizes the broad findings in each category.
Evaluation Category Evaluation Score
Relevance Excellent
Effectiveness Good / Excellent
Efficiency Good / Excellent
Impact Good / Excellent
Sustainability Poor
Equity Excellent
In the following subsections, each of these evaluation categories is examined in further detail. First, the
framing questions of the evaluation category are presented, then the findings are presented, with the key
points bolded at the beginning of the section.
A. Relevance
Evaluation Category: Relevance Evaluation Score: Excellent
Category Framing Questions
To what extent are the objectives of the program still valid?
Are the activities and outputs of the program consistent with the overall goal and the attainment of
its objectives?
Are the activities and outputs of the program consistent with the intended impacts and effects?
Evaluation Findings Summary
By meeting the critical supply gaps in the Malawi public health system that occurred in 2011, the PHC
Project was a timely and relevant project. The project’s utilization of a kit system, along with its partner
structure and project oversight mechanisms, was consistent with its objectives of rapidly establish a supply
chain that achieved timely and routine medicine and health supply deliveries of a national scope.
Detailed Findings
Project context
The Primary Health Care (PHC) Essential Medicines Project was launched in January 2012 in response to the
persistent medicine and health commodity under stocks and stock outs experienced by Malawi in 2011. It
14
was estimated that as much as 60%6 of all health centers and hospitals in Malawi experienced chronic stock
outs or understocks in 2011.
Figure 2 Stock outs or under supply by % in 2011 (by quarter)
These health commodity supply challenges occurred in a context of the national financial and political
challenges that occurred in 2011. These challenges included civil unrest, chronic fuel shortages, and high
inflation. In addition, the suspension of donor general and sector budget support, as well as reduced tobacco
exports in 2011 resulted in a national FOREX crisis.7 This lack of FOREX, cuts to government budgets, as well
as the inflation-induced reduction of purchasing power presented severe national health commodity
procurement challenges for public and private organizations alike. In 2011, the Central Medical Store was
reestablished and registered as the Central Medical Store Trust (CMST). In 2011, it had no CEO or governing
board and had only recently began to establish its revolving capital fund. As a result, it did not have the
inventory and operational capacity to address the health commodity emergency that emerged in 2011.
The MOH executed an emergency 6 month procurement, but also made an appeal to development partners
to support a broader emergency response to these stock outs. Discussions began in July 2011 to design a
comprehensive essential medicines procurement, storage and distribution project. This project, approved in
November 2011, received a total of USD37 million8 in funding from the governments of the United Kingdom,
through DFID, Norway, through the Royal Norwegian Embassy in Malawi, and Germany, through KfW. This
funding was intended to fund the project for 18 months.
The overall goal of the PHC project was to contribute towards keeping mortality and morbidity among
children under 5 and pregnant women on a constant declining rate in Malawi to meet the 2015 Millennium
Development Goals (MDG) 4 &5, by averting drug shortages over the implementation of the project. The
actual attainment of the MDG goals can be assessed in second half of 2014, when the results of the Malawi
End-line MDG survey become available.
6 This is based on an analysis of 10 tracer items that were a part of the PHC project evaluation and is a rough proxy for
national rates. The actual national stock out rate at that time may be higher or lower. 7 http://www.imf.org/external/pubs/ft/survey/so/2012/car080912a.htm
8 The exchange rate at the time of project approval in November 2011
15
Project Objectives
The operational objectives of the project were the:
1. Timely delivery of the pre-packed medicine and health commodity kits to the PHC facilities through a
functional, efficient and accountable supply chain system
2. Functioning project coordination mechanism established which will provide project management and
oversight at the national level
3. Functional Independent monitoring mechanism of essential commodities at facility level in place
4. Health facility workers sensitized and knowledgeable on appropriate application of the kit system
and rational use of medicines
5. Effective Communication Campaign that informs communities of availability of drugs at PHC sites
A sixth objective was not documented, but understood by the partners, was to leverage the PHC kits project
to support to the CMST recapitalization initiative and strengthen the CMST’s procurement, warehousing and
service delivery capacity.
Alignment to objectives
The project design was well aligned to the project objectives. The overall project was carefully designed to
achieve Objective 1. A prepackaged “kit” procurement and distribution model was selected by the partners
to expedite the launch of the project and as a way to contend with national distribution capacity limitations.
The project was implemented using a “Phase I” and a “Phase II” kit. The Phase I kits were distributed from
January to March/April 2012 and were based off a Zimbabwe essential medicines kit design. These kits were
made up of 37 items and its use of preexisting supplies in the UNICEF pipeline enabled the rapid start-up of
the project. The Phase II kits were designed by representatives from UNICEF, CMST, HTSS, the World Health
Organization (WHO), and to a limited extent, also DHOs. Their design, assembly and procurement occurred
in parallel to the distribution of the Phase I kits. The Phase II kits were made up of 61 items and mostly
shipped by sea. The PHC Project also consisted of loose items bulk-purchased by UNICEF. These items were
not in the kits because they were either too bulky (like the umbilical clamps) or were controlled items (such
as diazepam injectable or phenobarbitone) and so required special importation waivers and so were on a
separate distribution schedule from the rest of the kits.
The selection of partners and assignment of activities was also well-aligned to the context. As a result of its
national presence, experience with kit production, and current supplier contracts, UNICEF was selected as
the procurement partner to oversee the implementation of the entire project. JSI, due to its logistics
expertise, and existing subcontract arrangements with Cargo Management Logistics (CML) and RTT, was
selected to manage the planning, warehousing and distribution for these kits. In order to address the
substantial warehousing capacity required to store and distribute the kits, RTT (now Imperial Health
Sciences) was also subcontracted by JSI to provide expanded warehousing services for the kits. Investments
were made at the RTT warehouse in less than 3 months in late 2011. These included floor reinforcement, the
installation of racking and CCTV to upgrade their facilities to a pharmaceutical grade warehouse in
preparation for the arrival of the first consignment in January 2012.
16
The project was also able to develop an approach relevant to the goal of ensuring adequate oversight, as
described in Objectives 2 and 3. Under its partnership model, MOH, donor and partner coordination was a
central component of the PHC project. Partner meetings were first implemented monthly and then
bimonthly. Independent monitoring of the project was implemented by O&M Information
Technology/Systems and consisted of eight rounds of health facility spot checks designed to continuously
identify urgent project issues during its implementation.9 For additional information please refer to “Project
Coordination in the Effectiveness section.
In addition, an important element of this project was its public awareness campaign, tailored to achieve
Objectives 4 and 5. The campaign consisted of a mass media campaign, which included radio spots,
newspapers adverts, and even a television documentary. The goal of this mass media outreach was to
sensitize the public on the utilization of the drugs and product availability. In addition, there was also a
health facility outreach component to the project. The outreach component targeted all 28 districts, and
focused on DHOs, District Information Officers, and health facility drugstore staff, who were in turn
responsible for disseminating information to the health center level.
As an emergency response, the project was intended to relieve pressure off CMST while it established its
revolving capital fund and strengthened its operational capacity, towards achievement of the sixth objective.
The Central Medical Stores Trust (CMST) was to receive 55% of the kit costs from recipient MOH and CHAM
facilities in support of their recapitalization initiative.
B. Effectiveness
Evaluation Category: Effectiveness Evaluation Score: Good /Excellent
To what extent were the objectives achieved / are likely to be achieved?
What were the major factors influencing the achievement or non-achievement of the objectives?
Evaluation Findings Summary
The PHC project established and implemented an effective supply chain for essential medicines. In its
national distribution of a total of 15,844 kits, with only two major monthly delays, the project
demonstrated its supply chain distribution effectiveness. Routine partner meetings, independent
monitoring and a high degree of supply chain accountability, demonstrated the project’s effective project
coordination and controls.
The communication aspect of the project was found to be the least effective in terms of meeting its goals.
In interviews, 48% facility staff felt that the mass communication campaign had been ineffective and did
not reach patients and communities. In addition, project coordination with CHAM facilities was ineffective,
as evidenced by the perception that the kits were free, and the high percentage of the CHAM facilities
eventually opting out of the program. The amendment of kit deliveries was also a noted challenge. CMST
received an estimated US$8,670,690 from the PHC kit project, however, this represents only 65% of what
they should have received, mostly due to non-payment of invoicing issues.
9 June, August, and October 2012; January, April, July, and October 2013; and January 2014
17
The project achieved 5 of its 7 stated health service outputs targets, and by improving product availability,
it contributed to overall disease management capacity, particularly at the health center level.
Detailed Findings
Effectiveness was assessed in terms of supply chain performance, coordination, communication, and health
service outputs.
Supply Chain Performance
The first and primary objective of the PHC kits program was “Timely delivery of the pre-packed medicine kits
to the PHC facilities through a functional, efficient and accountable supply chain system.” The evaluation
examined the effectiveness of the PHC parallel supply chain across various aspects.
Stock Out Analysis
Eighty-one percent (81%) of facilities
interviewed agreed or strongly
agreed that the PHC project
prevented stock outs at their facility.
This parallels their view on increased
patient access to medicines. Many
facilities highlighted the dependability
of the kits supply relative to the more
inconsistent CMST supply at that
time. The 2012 and 2013 LMIS data
for the selected 10 kit tracer
items for this evaluation (See
Annex 10 for the complete list of items) shows that the consistent delivery of the kits drove down the rate of
facility stock outs from 25% to 7% during the implementation period of the PHC project (See Figure 3
above).10 There is an observable increase in stock outs in Q4 2013 after the distribution of the remaining kits
and kit items ended in September/October 2013. The MOH attributed the increase of stock outs in Q4 2013
due to the end of the kits program.11
However, the kits did not completely eliminate stock outs, and occasionally items within the kits themselves
stocked out. Thirty-three percent (33%) of the facilities said they frequently experienced stock outs even
with the kit program, with 55% saying they only sometimes experienced stock outs, and only 12% of
facilities saying they never experienced commodity stock outs. Seventy-five percent (75%) of the facilities
attributed stock outs to the short supply of health commodities, both in the kits and from CMST system.
Even with the kits, Malawi’s general environment of undersupply continued to be a challenge.
10
The 3% increase in Q3 2012 is largely attributed to increased facility stock out of Sodium chloride 500ml iv 20s and Oxytocin. 11
Monthly Inventory & Reporting Performance, HTSS, MOH, December 2013
Figure 3 PHC Tracer Item Facility Stock Out Rate: 2012 - 2014
18
Across the health facility service levels, the following items were reported as the most frequently stocked
out items from the PHC kit (alphabetically):
Amoxycillin tabs 1000
Bandage gauze 8x4m
Benzylpenicillin 5miu 50s
Calamine lot 500ml
Cotton wool 500g
Gentamycin inj 50s
Gloves exam latex M 100s
Ibuprofen 200mg tabs 100s
Paracetamol syrup bot
Paracetamol tabs, 500mg 1000s
Quinine 300mg tabs 100
Quinine inj 10s
Order Responsiveness
Kit contents, by definition, are standardized and cannot be amended. The total number of kits, however,
should have been tailored to need. However, 69% facilities strongly disagreed or disagreed that it was easy
to make changes to the number if kits. Only 14% of facilities interviewed strongly agreed or agreed that it
was easy to amend the number of kits received at their facility.
Commodity Management Capacity
The PHC project was not specifically designed to develop
in-country capacity for inventory control, quantification,
ordering and distribution planning. While the routine
and dependable supply of medicines and supplies can
indirectly reinforce stock keeping practices and reporting
practices, the results appeared mixed for the PHC
project. Fifty-nine (59%) of the sites interviewed agreed
or strongly agreed that the kits had improved the
inventory management skills of their facility. Eight-six
percent (86%) of those interviewed said they received no
specific training related to the kits, and so the skills-
building was most often understood in terms of the day-
to-day fact that commodities were available in their
stock rooms to manage. As an example, one MOH facility
mentioned that the start of the kits program was
timely as it helped their recently arrived HSA to
apply their MOH training at their facility.
Twenty-two (22%) disagreed and 2% strongly
disagreed that the kits had strengthened their
inventory management capacity. A large
number of interviewed health facility staff
reported that they had already received
drugstore and inventory reporting training from
the MOH, and since the kits used the same stock
cards, there was nothing particular to the PHC
Figure 4 Interviewee Response to inventory management skills building
Figure 5 LMIS Reporting Rates, 2012 - 2014
19
project that improved their skills.
While LMIS reporting rates did improve during the implementation of PHC project (see Figure 5), more
notable increases occurred during the end of the project, with the kit only contributing to the larger MOH
efforts to improve LMIS reporting. This includes training, mentoring, and supportive supervision efforts that
were rolled out by the MOH in late 2012 and throughout 2013. In the course of two years, national
reporting rates more than doubled from 39% to 86%, which is a significant achievement and should be
recognized. While this reporting improvement is indeed notable, as noted earlier in this evaluation,
inventory data quality remains an issue (see sections below).
Over and Understocked Items
While the benefits of using a kit distribution system include simplified procurement, storage, distribution,
and well as a reduced risk of leakage, the quantity of the items in these kits are not based on health facility
consumption data. As a result there is an inherent risk of over and understocks in this model. Annex 11
includes a reported list of the most over and understocked commodities.
Facility level stock management
The cycle check reports highlight poor recordkeeping at CHAM facilities in particular during the
implementation of the kits project. As an example, the August 2012 report from Independent Monitors
indicated that 13% of the CHAM facilities visited had no functioning stock card system.
Items from the PHC kits were recorded on the same stock card as MOH CMST items and commodities from
other supply systems. However, initially there appeared to be some divergent guidance of this practice, with
34% of CHAM facilities being advised to create separate stock cards for kit items. Only 10% of those facilities
actually ended up implementing such a parallel system.12 This rationale for these separate systems was to
better segregate free MOH supplies from the fee-based CHAM procured stock. The integration of free and
non-free supplies was a challenge CHAM experienced during the PHC project.
During the site visits for this evaluation inventory data was collected on 10 tracer items. Data was collected
from drugstore stock cards for a total of 12 months for each product (May 2012 to April 2013) and compared
with delivered quantities. Across all the products about 31% of monthly stock entries were excluded from
this analysis due the inability to locate or access facility stock cards. The lack of stock cards varied from
product to product. In general, it was observed that slower moving items, such as sodium chloride, glucose
5%, and erythromycin, tended to have the least complete information. While the stock entry for faster
moving items, such as paracetomol and cortimoxazole, tended to be more complete. This is presumably due
to the fact that they were more frequently prescribed and so their stock cards required more frequent
updating and were better maintained.
12
Conducting Spot Checks in Health Facilities – First Cycle. O&M IT/S and Management Services Ltd, p. 7 & 24
20
The tracer item unrecorded percentages are as follows:
“Unrecorded” in the table above is defined as products that were delivered and not recorded on drugstore
stock cards. Unrecorded product is at greater risk of leakage, as well as expiry, since it not documented in
facility records. Amoxicillin was eventually excluded from this analysis due to the inconsistency of the unit of
measure recorded, indicating that at many facilities amoxicillin in both syrup and tablet form were
frequently entered on the same stock card. It is important to recognize the limitations of the above
estimates due to missing or inaccessible health facility stock cards.
Health Facility Storage Practices
Fifty-three percent (53%) of the sites visited indicated that they had inadequate storage space when
managing the kit products. Some facilities indicated that due to the lack of space or shelving they kept kit
items in the boxes they were delivered in. This sometimes resulted in the stacking of boxes on top of each
other, which made the implementation of FEFO inventory management and the identification of expired or
expiring products difficult.
Product Expiry
As a result of the long expiry dates of the kit contents and the fast moving nature of most of the kit items,
product expiries were found to be a rare occasion during the implemented evaluation site visits. In the
monitoring cycle reports, almost zero wastage was reported until the end of 2013.
Expiries were also largely mitigated through the relocation of slow moving or near expiring items to higher
volume facilities, such as a district or central hospitals for use or further redistribution. The eighth (and final)
cycle report issued in January 2014 identified a list of items that had expired in 2013 and were expected to
expire in the first half of 2014. They included:
Products Expired in 2013
• Albdendazole
• Benzyl Benzoate lotion
• Cortimoxazole
• Doxycycline
• Erythromycin
• Metronidazole
• Promethazine
Tracer Item
Units
Delivered
Units
Recorded on
Stock Card
Total
Unrecorded
Units % Unrecorded
Amoxycillin susp bot 141,840 Undetermined Undetermined Undetermined
Sodium chloride 500ml iv 20s 14,100 8,864 5,236 37%
Erythromycin tabs 100s 7,800,000 5,017,096 2,782,904 36%
Glucose 5% 500ml iv 20s 15,320 10,340 4,980 33%
Sulphadoxine/Pyrimeth 1000s 2,684,000 1,834,018 849,982 32%
Cotrimoxazole 480mg tabs 500s 23,166,000 17,659,733 5,506,267 24%
Paracetamol tabs, 500mg 1000s 20,096,000 16,075,192 4,020,808 20%
Oxytocin Inj 32,426 25,980 6,446 20%
Quinine inj 10s 102,480 95,426 7,054 7%
Syringe 2ml w needle 100s 105,400 102,200 3,200 3%
21
Products to expire by mid-2014
• 50% Dextrose
• Albendazole
• Benzyl Benzoate lotion
• Cotrimoxazole
• Doxycycline
• Hydrochlorhiazide
See Annex 11 for additional details on expired items.
Product Quality
In general, nearly all facilities spoke highly of the quality of the items contained in the kits and voiced no
complaints or concerns. The PMPB conducted random sampling throughout the implementation of the PHC
project and all products passed post-marking testing. Only one site visit mentioned a quality issue when it
was reported that IV fluid arrived with some stains on it, they were reported to the PMPB and then
subsequently returned. No additional complaints were submitted to the PMPB.
As it was an emergency situation at the time, UNICEF received a blanket waiver from the PMPB to import the
kits into Malawi. As controlled substances, phenobarbital tabs and injection, as well as diazepam, required
more specialized approvals and so were among the items distributed on separate schedule as loose items.13
This blanket waiver prompted media reporting alleging the kits project hurt local suppliers. In response, two
press releases, “Donors, Government have not ignored local pharmaceutical suppliers” (22 February 2012)
and “Drug certification and quality control in Malawi” (15 March 2012), were released by the government
and the PMPB. The latter was to address reports that the medicines being imported by UNICEF were not
certified by the PMPB.
Project Coordination
PHC Project Structure
The PHC project was implemented under an effective partnership model. As a comprehensive supply
delivery project of national scope, various health sector organizations and partners contributed to the
execution of the PHC project under UNICEF project coordination. The clarity of partner roles (see below) has
been highlighted among the strengths of the project.
Partner Project Role & Responsibilities
UNICEF
Overall Project Coordination and Oversight
Procurement and international freight
Project Communications
Independent project monitoring through O&M
IT/S
DFID, Germany (KfW),
and Royal Norwegian Embassy
Project financing
Project monitoring
MOH – HTSS Project monitoring
National inventory monitoring
13
Primary Health Care Essential Medicines Support for Malawi – Consolidated May Report, UNICEF Malawi, June 2013. P.24
22
MOH - District Health Office District level inventory monitoring and reporting
Commodity redistribution
JSI [RTT & CML]
Distribution planning and oversight
Warehousing and Distribution
Technical advisor
CMST Kit recipient invoicing
PMPB
Product waivers
Certfication of RTT warehouse
Post-marketing surveillance
Health Centers, Community Hospitals,
District Hospitals and Central Hospitals
(MOH & CHAM)
Kit receipt and storage
Patient treatment
Health facility inventory record-keeping and
LMIS reporting
Project Start-Up
The ability of the project to take advantage of existing contractual agreements enabled the rapid roll-out of
the PHC project and was a key strength of the PHC project. As the procurement partner, UNICEF was able to
supply from existing supplies from the UNICEF pipeline. While DFID, under a “conditional gift” mechanism,
was able to transfer pooled donor funding for the UK, Norway and Germany, to USAID and plug into an
existing contract with JSI, and in turn, their subcontractors, CML and RTT.
Coordination Meetings
Routine partner project meetings organized by UNICEF were a critical element of the management of the
project during the start-up and planning phase of the project and subsequently the monitoring of the
project. Initially these meetings were monthly and then they were bimonthly, with one or two delays in the
latter due to scheduling issues. The agenda of these meetings included updates on kit procurement and
distribution, presentations on the findings by the independent monitors, updates from the UNICEF
communication team, as well as reports on post-marketing surveillance by the PMPB. As the recapitalization
of CMST was also one of the objectives of the PHC project, it was also a regular agenda item. In fact, the
initial coordination meetings were held at CMST offices. In general, these coordination meetings were
mostly attended by central level partners, with only periodic attendance by CHAM or MOH DHOs.
Independent Project Monitoring
O&M IT/S and Management Services acted as an independent monitor of the PHC project. It implemented 8
quarterly health facility “spot-checks.” The first report was submitted in June 2012 and the last was
completed in January 2014. In each cycle they visited between 60 - 70 facilities and developed findings and
recommendations. They assessed various aspects of the PHC project, including kit delivery totals, record
keeping quality, product under and over supply situations, storage quality, as well as progress on kit
invoicing by CMST and payments made by districts, central hospitals and CHAM facilities.
All cycle reports included system improvement recommendations and provided rich information on the
performance of the kit program during its implementation. As noted above, findings and recommendations
23
were shared during the project coordination meetings. In one instance, over the course of the first two
cycle visits, particularly poor record keeping and drugstore infrastructure was identified at a site in Thyolo
at a site in Mchinji.14 These sites were discussed at a coordination meeting and it was decided that kits
would no longer be sent to these sites until improved record keeping at those facilities was observed.15
These spot-checks provided the project coordination team a high degree of visibility into the project and
the information to quickly response to identified issues as they arose.
Partner Coordination Challenges
Project coordination with CHAM appeared to cause the most challenges at the beginning of the project. This
is evidenced by the initial misperception by CHAM facilities that the kits were free during the initial months
of the project. The fact that 55% of the cost of the kits were intended to contribute to the recapitalization of
CMST appeared to be poorly communicated. It was only when CHAM facilities received invoices from CMST
did this become clear to many CHAM facilities.16 This misperception resulted in some CHAM sites returning
kits, stock-piling kits for their eventual return, or ultimately opting out of the project entirely. While 172
CHAM facilities were a part of the PHC project at the planning stage, only 58 CHAM facilities maintained
their participation in the project by mid-2012.
Inconsistent CHAM representation at the bimonthly coordination meetings is one likely factor for this poor
coordination. In addition, as an umbrella organization for multiple faith-based health service delivery
organizations, internal communication has been noted as being an organizational challenge. In addition,
CHAM underwent restructuring from 2010 to 2012 which resulted in the turnover of organizational
leadership, and may have limited their ability to consistently engage on PHC project-related issues.
Project Visibility and Security Responsiveness
Another key strength of the PHC project coordination was performance and implementation visibility. In
addition to the bimonthly coordination meetings and the O&M cycle reports, the coordinating partners
actively collected data against performance targets and UNICEF conducted 2 internal project progress
reports. Visibility into the distribution process, in particular, was an operational strength. Partners and
donors received distribution updates almost on a daily basis. At the end of each distribution, a summary
would be published in the local papers to inform the public that supplies were being delivered to health
centers. In addition, all proofs of deliveries (PODs) were provided to all PHC partners, including the
programs.
Product security was also a part of this project visibility. The theft of some kits was identified at the RTT
warehouse via CCTV in May 2012. Donors noted that UNICEF and JSI were quick to respond and report out
on the issue. A total of $46,000 was recouped in insurance money which was put back into operational
expenses. Another allegation of theft and resale of kits were raised at one point, was quickly investigated
but then proved to be unsubstantiated. The response to these incidences reflect the project visibility the
PHC project was able to achieve.
14
These facilities were reintroduced to the kit distribution system after introducing the use of stock cards, training staff in drug store management, repairing a drugstore wall, as well as installing a burglar bar in late 2012. 15
PHC Essential Medicines Kits Project Coordination Meeting Notes, 21 August 2012 16
Delayed invoicing by CMST may also have exacerbated this misperception
24
Health Service Outputs
Target achievement
At the start of the project, the PHC kits project management team identified 7 indicators related to
appropriate treatment of common health conditions among children. Data was collected from the FY
2009/2010 HMIS report to provide baseline rates and targets were set to maintain these baseline levels over
time. Setting these targets as a flatline indicator was deemed appropriate as the kits program intended to
respond to the national health commodity crisis and ensure continued operation of the health system at the
same level. The following table summarizes the 7 indicators, the baseline levels, and the final levels
reported in the HMIS 2012/13 report. It also lists if the targets set at the outset of the project were achieved:
Outcome Outcome Indicators 2009/2010
baseline (HMIS report)
2013 Target as stated in
logframe
2012/2013 result
(HMIS report)
Target achieved?
Availability of basic sets of essential medicines in assisted government and CHAM primary health care facilities in all 28 districts. This will enable appropriate treatment of common health conditions especially women and children to pre-crisis levels.
1.1) Proportion of children under-five seen at facility level for non-bloody diarrhea (per 1,000 children)
135
135 or above
155 Yes
1.2) In patient diarrhea death rate among under five (per 1000 cases)
3.0 3.0 or below 1.1 Yes
1.3) Acute Respiratory Infection (ARI) inpatient death rate among under five (per 1000 cases)
1.5 1.5 or below
0.9 Yes
1.4) Out Patient Department (OPD) visits (per 1,000 population)
1,316 1,400 or above
1,108 No
1.5) Under Five ARI new cases in under five population (per 1000 children)
500
500 or better 212 Yes17
1.6) Number of children under-five new cases seen at health facility level for ARI (per year)
1,100,100 1,100,100 or above
564,600 No
1.7) Number of children under-five seen at health facility level for non-bloody diarrhoea (per year)
324,000 324,000 or above
413,000 Yes
Overall, five out of seven of the outcome indicators were achieved. Over the course of the project, the
proportion (1.1) and number (1.7) of children seen at the facility level for non-bloody diarrhea increased.
This indicates increased care seeking by caregivers and increased provision of services for non-bloody
diarrhea, which can be lethal if not managed correctly with ORS, Zinc, and antibiotics. The rate of new cases
of ARI in the under-5 population (1.5) declined and in-patient death rates for diarrhea (1.2) and for ARI (1.3)
both had noticeable declines in the period under review. Because these indicators reflect in-patient
outcomes, the changes in this indicator can be reasonably presumed to be a factor of improved services
17
Note that the target for indicator 1.5 was unclear as worded: “Targets to be the same as or better than baseline”. In
this case, we are assuming that “better” means a lower number of new ARI cases in the under five population
25
within the health facility, as well as improved care seeking behaviors such that caregivers bring children for
care earlier in their illness.
Outpatient department visits (1.4) declined
in the period under review, which was
contrary to the target of maintaining levels
of patient visits. This decline reflects a year-
on-year decline dating to the 2009/2010
fiscal year. Declining OPD utilization appears
to be driven in large part by the dramatic
reduction in visits related to malaria (See
Figure 6). Between 2009/2010 and
2012/2013, the share of all OPD cases
attributable to malaria dropped from 48% of
all cases to 21%. This reflects a marked
decline in malaria cases in the under-5
population, which has only recently been reflected in the overall population. In the same time period, the
death rates from malaria have also declined (data not shown) indicating that the malaria program has
achieved successes in both preventing malaria cases and effectively treating them. Because the malaria
burden remains so high, and because malaria-related visits contribute so heavily to OPD utilization, it is
reasonable to link the decline in utilization to the successes of malaria control.
The number of ARI cases in children under-5 seen at the health facility each year (1.6) also declined instead
of going up as indicated in the target. The number of cases seen at the facility declined by nearly half
between 2009/2010 and 2012/2013. This decline in visits should be driven primarily by the declining case
rate in the population, which declined by 58%. This decline may be due in part to other public health
successes beyond the kits program. The pneumococcal vaccine was introduced nationally for all children
under the age of 1 in November 2011. Uptake was immediately high in the target age group, and at least
some of the decline may be attributable to reduced incidence of pneumococcal illnesses. Another possible
explanation may have to do with improved community case management of ARI, where children treated in
their communities may not have their data captured in the HMIS, however it is notable that no similar
decline was seen in the diarrhea cases, which may also be addressed through community case management.
In the case of both indicators that did not achieve the targets set out at the beginning (1.4 & 1.6), the flaw
may be with the indicators and non-achievement may actually reflect public health successes.
Health Service Utilization
In order to complement health service utilization data obtained from the HMIS, health workers at the
sampled facilities were asked for their perceptions of the impact of the kits program on health service
utilization. Clinicians (n=58) were asked whether the kits program encouraged more patients to attend the
facility. Overall, 86% of respondents responded yes, with an additional 3% stating that this was somewhat
true. The results were similar for the subset of clinicians who had been at the facility since January 2010 or
earlier (data not shown). Seventy-five (75%) of clinicians who had been at their present facility since January
2010 or earlier (n=24) felt that there were more patients attending the facility than before the kits program,
0
500
1000
1500
Axi
s Ti
tle
Year of HMIS report
Declining malaria cases in Malawi
Malaria cases<5per 1000 <5population
Malaria casesper 1000population
Figure 6 Malawi Malaria Cases
26
and 78% of those who thought there were more
patients attending (n=18) felt that this was
attributable to the kits program.
This is an interesting finding to note in light of the
overall decline in OPD utilization registered in the
HMIS and it may reflect a perception of general
community openness to service utilization, rather
than an actual increase in numbers of patient visits.
In fact, by using HMIS data to examine the OPD
utilization just in the facilities sampled for the
assessment (excluding 6 facilities with incomplete
data), there is also a marked decrease in OPD
utilization after 2010, with a significant drop from
2010 to 2011 and then relatively steady utilization from 2011 - 2013.
Qualitative responses volunteered by health
workers at 3 facilities shed further light on the
matter. These clinicians felt that the number of
onward referrals from the facilities had reduced
as a result of the kits program, because when
patients found the medicines they needed,
there was less need to refer them to a higher
level. It could also be hypothesized that with
good commodity availability, patients may be
less likely to make repeat visits to the health
facility for the same condition. Reducing
unnecessary referrals and unnecessary repeat
patient visits would be a positive outcome of
the kits program, although it would lead to a
reduction in OPD utilization in the national data. While the available data is not strong enough to
conclusively determine the contribution of the kits program to reduced OPD utilization, there is further
indication that the decline is actually an indication of health system success rather than failure in the context
of the kits program.
Even more importantly than patient visits was the impact that the program had on patient access to needed
medicines. Overall, 90% of respondents agreed or strongly agreed that the kits program increased patient
access to needed essential medicines. A similar question was asked to clinicians who had been at their
current facilities since at least January 2010, asking them to compare access to essential medicines before
the kits program in 2010 and in 2013, and 75% of these respondents (n=24) observed increased access with
most of the remaining observing no change.
Figure 7 Patient Facility Attendance Responses
Figure 8 Annual OPD visits
27
Disease management
In order to understand how the kits program contributed to the ability of health workers to manage
diseases, health workers were asked about their ability to manage individual target diseases and conditions
with the resources available to them. Overall health workers indicated a high degree of comfort with disease
management, particularly for commonly managed conditions including diarrhea, ARI, and fever, with 84% -
88% able to manage all or most cases. Comfort levels were lower with management of syphilis, with only
76% of health workers able to manage all or most cases, though this may reflect the syndromic management
approach where syphilis is not treated as a disease entity on its own. Syphilis management may also have
been complicated by the unavailability of diagnostic tests, which were not included in the kits. Comfort
levels were least with severe or complicated malaria and postpartum care with 71% and 66% respectively.
Further analysis shows that the majority of those facilities where the health worker was not able to manage
the majority of cases in these last two instances were low level health centers where referrals for severe
malaria or postpartum complications would be appropriate.
Encouragingly, the numbers of health workers who reported not being able to manage any cases were
negligible for all conditions except for post-partum care. Again, those not able to provide post-partum care
were located in lower level facilities with no delivery wards. The most common cause of not being able to
manage cases was the need for referral to a higher level (60%), which is appropriate in many cases. The
second most common cause for not being able to manage cases was lack of medicines (14%). Although this
is a minority of facilities, this is concerning and reflects continued issues with stock outs of medicines
required to manage common health conditions.
Communication
The communication component of the PHC project was a key element of its design. The three objectives of
the communication initiative were to A) ensure the utilization of the medicines B) sensitize communities on
their availability and C) sensitize health facilities, DHOs and the DHMT on recapitalization. This initiative
included a mass media component (radio, newspaper and television documentaries) as well as management
outreach meetings. This evaluation only focused on objectives A & B.
At the health facility level, twenty-eight (28%) clinical staff first learned about the kit program when the
kits first arrived at their facility. Ten percent (10%) learned about the kits from the radio. The rest (48%)
first learned about the PHC project through other various management channels. This includes directly
from the DHO (7%), a CHAM management memo (7%), and MOH Meeting (5%). Three percent (3%) heard
about the kits directly from a visiting UNICEF representative. Much of the sensitization and management
communication efforts were focused on the district level and it appears that this messaging did not always
trickle down to the health center level given the varied responses as to how facility staff first learned about
the kits.
Diarrhea ARI Syphilis Deworming Fever
Severe/
complicated
malaria
Post-partum
care
Able to manage all or most cases 84% 86% 76% 86% 88% 71% 66%
Able to manage some cases 14% 12% 19% 12% 10% 26% 24%
Not able to manage any cases 0% 0% 3% 0% 0% 2% 9%
28
In facility interviews 48% facility staff felt that the mass communication campaign was ineffective and did
not reach patients and communities. Many clinicians said that their communities did not receive
newspapers and that they also did not recall community members mentioning the PHC radio and newspaper
advertisements and ultimately doubted that they were reached by them. Facilities often mentioned that,
like with some facility staff, community members often first find out about the kits when they saw the
delivery trucks arriving in their community.
Strengthening CMST
The recapitalization of CMST was among the main objectives of the PHC project. CMST was to receive 55% of
the kit revenues from both MOH and CHAM facilities. As far as the distribution of complete kits, a total of
3,703 Phase I and 12,141 Phase II kits were distributed under the PHC kits program. As indicated in the table
below, the amount that CMST is expected to be capitalized is roughly USD$13,229,360.
The invoicing from CMST and payment by the MOH and CHAM as closely monitored during the partner
coordination meetings as well as documented in the O&M reports. In general, it was observed by the
independent monitors that the invoicing from CMST was regular and timely, with some challenges related to
delayed payments from the Central Hospitals (at least initially) and more notably from CHAM facilities.
The table below reflects the total financial amount invoiced and received by CMST as of April 2014:
As reflected in the above table, overall, 76% of invoiced totals have been paid to CMST. Eighty-three (83%)
of the amount invoiced to MOH was paid, while only 24% of the amount invoiced to CHAM hospitals having
been paid. CMST was not able to provide information related to the amount invoiced to CHAM community
health and CHAM community hospitals. According to the cycle reports, it was unclear if all the CHAM health
facilities received invoices from their umbrella organizations. In terms of their lack of payment, many
facilities cited budgeted constraints and other financial commitments.18 Payments from CHAM to CMST may
18
6th
and 8th
cycle reports
Total Kits
Distributed
Cost / Kit
(USD) Total Expensed
55% of Total
Expensed
Total Phase I 3,703 $594 $2,199,582 $1,209,770
Total Phase II 12,141 $1,800 $21,853,800 $12,019,590
Grand Total $13,229,360
Health FacilityTotal Kits
Received (#)
Total Amount
Due for Kits
Received (MKW)
Total Amount
Invoiced
(MKW)
Total Amount
Paid (MKW)
Total
Outstanding
Balance
(MKW)
% of Revenue
Received
1. MOH - Health Centres 10,744 2,712,554,307 2,712,554,307 2,352,943,987 359,610,320 87%
2. MOH - District Hospitals 2,200 718,480,827 718,480,827 590,541,642 127,939,185 82%
3. MOH - Central Hospitals 792 245,326,853 245,326,853 125,016,244 120,310,610 51%
4. CHAM - Health Centres Data Unavailable Data Unavailable Data Unavailable Data Unavailable Data Unavailable Data Unavailable
5. CHAM - Community Hospitals Data Unavailable Data Unavailable Data Unavailable Data Unavailable Data Unavailable Data Unavailable
6. CHAM - Hospitals 2,201 525,764,377 525,764,377 126,474,044 399,290,333 24%
Grand Total 15,937 4,202,126,365 4,202,126,365 3,194,975,917 1,007,150,447 76%
29
have also been complicated by some CHAM facilities requesting quarterly distributions, the return of unused
kits,19 and the fact that many CHAM sites eventually opted out of the project. While 172 CHAM facilities may
have been a part of the project initially, as of mid-2012, kits were only being delivered to 58 CHAM sites. In
addition, the inclusion of the loose items, and challenges related to fluctuating Malawian kwacha, may have
also further complicated the tracking of invoices and payments related to the PHC project.
While CMST may have not yet received all the revenue that was owed to them yet, it was clear that revenue
from the kits was significant. The MKW 3,194,975,917 in revenue CMST received from the kits converts to
US$8,670,690 in September 2013 dollars.20 However, this also represents a deficit of US$4,558,670 in
expected revenue, with CMST only receiving roughly 65% of what they should have from the kits program21.
C. Efficiency
Evaluation Category: Efficiency Evaluation Score: Good / Excellent
Category Framing Questions
Were objectives achieved on time?
Were activities cost-efficient?
Was the program or project implemented in the most efficient way compared to alternatives?
Evaluation Findings Summary
The project was implemented on time and largely met its targets for supply chain efficiency. The cost per
kit was reasonable and the majority of funds were expensed on direct commodity-related costs. However,
due to the emergency nature of the project and the limited number of logistics providers of sufficient
capacity, opportunities to explore additional cost efficiencies were limited. Due to the accepted risk of over
and under stocks integral to kit systems, 75% of PHC kit recipient facilities reported redistributing some kit
items to some extent.
Detailed Findings
Timeliness
Project efficiency was demonstrated by the rapid 5 month planning and start-up of the project as well as the
project’s performance in distributing the kits.
The PHC project was designed from July to October 2011, with the first kit order submitted in November and
delivered to Malawi in January 2012. This is a notable achievement as in addition to the actual design of the
project, project start-up work included: donor funding discussions; JSI, RTT and CML contract coordination;
the refurbishment of the RTT warehouse (which increased volume by roughly 50%);22 the development of a
19
35 kits were returned by CHAM from August to October 2012, 4th
Cycle Report, January 2103, p.12 20
1USD = MKW368, 30 September 2013 (End of kits program) rate, http://www.oanda.com/currency/converter/ 21
As CMST uses Malawian Kwacha, due to fluctuating exchange rates, this should be considered a rough estimate. 22
USAID | DELIVER PROJECT Performance Evaluation Malawi 2007 – 2013, December, 2013. p. vii
30
performance management framework; as well as the execution of an international procurement and kit
packing operations.
The distribution of the PHC kits was also very efficient as only two gaps in its monthly distribution were
experienced during the entire implementation of the project. These delays were mainly due to upstream
procurement and unexpected port issues, rather than the distribution providers. The PHC distribution
system regarded as an efficient and reliable supply source for essential medicines and health supplies by the
receiving facilities. Eighty-eight (88%) of health facilities that received the PHC kits said that their delivery
expectations were met.
In the transition to the full, Malawi-specific kit design under Phase
II supplier delays were encountered. For a short period certain
items were sourced from an approved supplier in Kenya, and due
to issues with the Beira port in Mozambique, the kits were also
temporarily run through South Africa. Due to these operational
adjustments, variations in the Phase II kit configurations were
introduced with the box configuration going from 7 boxes to 8,
and then eventually back to 7 again.
The fuel shortages in 2012 were also a major challenge for the
national kit distribution. The MOH established mechanisms with
the Malawi Energy Regulatory Authority (MERA) to ensure access
to fuel for deliveries through both government and parallel supply
systems.23
The original end date of the project was June 2013 and was
extended to September / October 2013 in order to distribute the
excess stock that emerged from the distribution gaps in April and
June 2012, as well as other remaining stock that had accumulated
in the country. This extension period also included the distribution
of the product already in the UNICEF procurement pipeline.
Initially, an option was proposed during the project coordination meetings in early 2013 that CMST would
distribute the kits post-June 2013, however, it subsequently determined that further operational reforms
and improvements still needed to occur at CMST.
Cost Efficiency
In support of the PHC project, donors provided a total of USD39,230,794.24 Eighty percent (80%) went to
procurement and related management costs, and 20% went to warehousing, distribution and related
management costs. Of the 80% in UNICEF procurement and management costs, 95% of that amount went
to the procurement of drugs and only 5% went to direct and indirect project implementation costs.25 This
indicates that in-country project implementation funding was used efficiently.
23
Primary Health Care Essential Medicines Support for Malawi – Consolidated May Report, UNICEF Malawi, June 2013. 24
In September 2013 USD 25
UNICEF Funds Utilization Report, Primary Health Care Essential Medicines Support for Malawi – Consolidated May Report, June 2013
Figure 9 Facility Perceptions on PHC Kit Delivery
31
Even though they were not free, even with the reduced cost, both DHOs and CHAM facilities reported that
the kit project was able to reduce pressure on their budgets and which made them go further. One CHAM
facility mentioned that even with the 45% discount, many of the kit items were much less expensive than
the same products sold by private suppliers for the PHC project. This savings enabled this facility to procure
more medicines and then pass along the savings to their patients.
JSI distributed the kits from January 2012 to June 2013 and from July to September/October 2013 they were
distributed by UNICEF. On average, it cost $506 to distribute a kit under the JSI-managed distribution
compared with $354 to manage under the UNICEF-managed system. This higher cost is perhaps most likely
explained by the added costs of the sub-contracting of warehousing and distribution services to RTT and
CML. Due to the emergency nature of the project, the PHC project tagged onto an existing warehousing and
distribution arrangement under USAID. If contract negotiations for warehousing and distribution services
had been more dedicated and the project planning horizon was longer, more opportunities for identifying
cost saving in this area might have been further explored.
Efficiency consideration of the kits approach
While kitting costs add additional costs to the procurement process, in the terms of performance, they also
introduce efficiency gains, demonstrated by the rapid start-up of the project, as well as a more streamlined
distribution planning and implementation process.
However, by their nature, kit programs are also limited in the efficiencies they can achieve at the facility
level due to the standardization of their contents and their quantities. As a result of the risk of product over
and under supply, redistribution is an inherent part of a kit system and was a regular occurrence under the
PHC project. Forty-four (75%) of the 58 facilities visited reported the relocation of PHC items to manage
oversupplied stocks. Most of the relocation was from Health Centers to District Hospitals, and from low to
high patient volume sites. These were mostly slow moving items or items that were not appropriate for
health centers, particularly if they lacked a maternity ward. It was noted by a CHAM representative, that
since the kit contents were standardized, it was also difficult to relocate items because after a certain point,
Donor
Procurement,
Shipping & Mgmt -
UNICEF
Logistics,
Distribution & Mgmt -
JSI /RTT
Logistics, Distribution
& Mgmt - UNICEF
(Extension)
Total ContributionUSD
Exchange Rate
30 September 2013* Total USD
DFID - GBP 9,132,541 4,492,262 30,950 £13,655,753 1.614 $22,034,377
Norway - NOK 59,600,000 - 2,950,000 kr 62,550,000 0.167 $10,436,468
Germany (KfW) - EUR 5,000,000 - - € 5,000,000 1.352 $6,759,950
Total (USD) $31,440,113 $7,248,534 $523,158 $39,230,794
* End of Project Extension Date, Exchange rates from Oanda.com
Logistics,
Distribution & Mgmt
Total Kits
Distributed
Donor
Contribution
(GBP)
Donor
Contribution
(NOK)
Donor
Contribution
(USD)
Average
Distribution
Cost / Kit
(USD)
JSI, RTT & CML 14,313 4,492,262 0 7,248,534.27$ 506.43$
UNICEF 1,531 30,950 2,950,000 542,147.18$ 354.11$
GBP 1.614
NOK 0.167
USD
Exchange Rate
30 September 2013
32
facilities were all oversupplied with the same items. Albendazole, Amoxycillian, Erythromycin and Benzyl
Benzoate were specifically highlighted as having this problem. As the kits were not distributed for free, the
lack of shared financial management systems between the MOH and CHAM (as well between individual
CHAM sites themselves), further added a layer of complexity to redistribution of oversupplied items.
D. Impact
Evaluation Category: Impact Evaluation Score: Good / Excellent
Category Framing Questions
What has happened as a result of the program or project?
What real difference has the activity made to the beneficiaries?
How many people have been affected?
Evaluation Findings Summary
The PHC demonstrated project impact by increasing product availability at health facilities, and in turn,
contributed to the decreasing mortality and morbidly trends among pregnant women and children under
5. The overarching objective of the project was to contribute towards keeping mortality and morbidity
among children under 5 years and pregnant women on a constant declining rate to meet the 2015 MDG 4.
While there is no way to conclusively link the contribution of the kits program to keeping mortality and
morbidity among children under 5 on a constant declining rate, it is clear that the PHC’s improvement in
the availability of specific commodities contributed to this overall decreasing trend. Maternal mortality
has also declined, though at less marked rates. Specific kit contents relevant to the key direct causes of
maternal mortality may have contributed to its decline during the implementation of the project.
Detailed Findings
Under-5 child mortality
General Trends in Under-5 child mortality
Malawi has made steady progress in reducing Under-5 mortality over the last several decades. The
Demographic and Health surveys have recorded a steady and sustained decline.
In 2014, in the report titled Committing to
Child Survival: A Promise Renewed: Progress
Report 2013, UNICEF declared that Malawi,
along with 6 other countries, had achieved
Millennium Development Goal 4, related to
child survival in 2014, a year early. The
country had gone from 244 under-5 deaths for
1,000 live births in 1991 down to 71, a 71%
33
reduction, with an annual decline in that period of 5.6%.
The exact causes of these overall declines are not entirely clear. It is likewise not possible to estimate the
exact contribution made by the kits program to the continuous decline in child mortality in Malawi.
Population-level mortality assessments
require vast amounts of rigorously collected
data, which is beyond the scope of this assessment. Additionally, due to the wide reach of the kits program,
there is no effective “control” population that could provide a comparator to assess how the program
contributed. However, by assessing the overall contributors to child mortality in Malawi, and determining
how the kit contents addressed these, we can determine if the kits were able to contribute to the sustained
declines.
First, we should assess whether the overall decline in child mortality is primarily caused be environmental
factors beyond the health system that could underlie improved survival. One common underlying factor
that leads to overall reduced mortality is improving levels of nutrition. Poor nutrition has been shown to
underlie as many as 50% of all under-5 deaths in countries like Malawi. While Malawi has registered some
gains in declining rates of severe malnutrition in children under 5, from 6% prevalence of wasting (low
weight for height) in the 2004 DHS to 4% prevalence in 2010. Chronic malnutrition in the form of stunting
(low height for age) in children under 5 also declined from 53% in 2004 to 47% in 2010. While these declines
are welcome, these malnutrition levels remain extremely high, and continue to contribute to continued
mortality of children under five.
A second potentially underlying cause of
improving health might be improving access to
water and sanitation facilities. This data is
collected in both the 2008 national census and the
2010 DHS. In the DHS, overall 79.3% of the
population obtains drinking water from an
improved source, most commonly a tube well or
borehole. However only 32.6% of the population
reports treating water before drinking, increasing
the likelihood those disease-causing contaminants
may be in the water at the time of drinking. Only
8.8% of all households use an improved toilet
facility, with the majority (75.8%) of people using a
pit latrine without a slab or an open pit. This data
indicates that water and sanitation continue to
pose a health challenge to children under 5.
In light of the continuing environmental and nutritional challenges that children under-5 face, the major
gains in mortality may be presumed to be related in large part to factors within the health system.
Causes of under-5 deaths in Malawi
Neonatal deaths are the most common cause of under-5 deaths. Between 1990 and 2012, neonatal deaths
rose from 21% of all under-5 deaths to 31%. Because neonatal deaths have declined at a slower rate than
other categories of under-5 deaths, the major improvements in mortality have come in the other age
Figure 11 Source: World Health Organization. Malawi: Health Profile. 2010.
Figure 10 Trends in Childhood Mortality, UNICEF 2013
34
categories. Infectious diseases continue to be the primary cause of deaths after the neonatal period. Both
neonatal deaths and deaths from infectious diseases are primarily addressed within the health system.
One part of the reduction in mortality should be attributable to prevention of diseases through vaccines and
other interventions. Malawi has maintained reasonably high coverage of vaccines through the Expanded
Programme on Immunisation (EPI), and has also been an early adopter of new vaccines including
pentavalent vaccine in 2002, pneumococcal vaccine in 2011, and rotavirus vaccine in 2012.26 The overall
decline in malaria cases registered in the HMIS system (as noted earlier in this evaluation) also indicates
improving malaria primary prevention. But there remains an important component of appropriate treatment
of illnesses, especially acute respiratory infection (ARI), fever, and diarrhea.
However, a strong case can be made that effective case management of common conditions contributing to
under-5 mortality has also assisted to reduce mortality. The following table summarizes the major causes of
under-5 mortality in Malawi, and the kit contents that addressed them. Some of the kit contents were
geared towards prevention of common conditions, some directly treated the disease causes, and some were
used to manage potential complications arising from the disease, for example, morbidity and mortality
associated with high fever caused by malaria.
Contribution of the kits program to declining under-5 child mortality
The contents of the kits were well-targeted towards treating the major causes of under-5 deaths in
Malawi. The most common category of deaths in children under-5 is neonatal deaths. Eighty-nine (89%) of
neonatal deaths in Malawi are caused by 3 contributing conditions: prematurity, infections, and birth
asphyxia, all three of which are addressed in some part by the kit contents. So even though the kit contents
were not targeted towards the immediate lifesaving of premature infants, the selected contents could
contribute to reduction in neonatal deaths.
Infants born prematurely are more than 10 times more likely to die in the perinatal or neonatal period than
infants who reach full term, and prematurity is responsible for 37% of all neonatal deaths. The kits included
essential commodities to ensure delivery of quality antenatal care services to mothers, thereby preventing
prematurity and neonatal deaths. Sulphadoxine/Pyrimethine prevents malaria in pregnancy and iron folate
prevents anemia in pregnancy; both malaria and anemia are known contributors to premature birth.
Infections, and particularly ascending genital tract infections, are also important contributors to prematurity
in Malawi, and antibiotics provided in the kits could be used to treat STIs identified during antenatal care.
Under-5 mortality cause
Relevant kit contents
Neonatal
Antenatal care and prevention of prematurity
SP - Prevention of malaria in pregnant women
Iron folate – Prevention of anemia in pregnant women
Benzyl Penicillin – Treatment of syphilis and sexually transmitted infections Treatment for neonates
Phenobarbitone, Diazepam – Treatment and prevention of seizures
Epinephrine – Cardiac resuscitation
Ampicillin, Cotrimoxazole, Metronidazole, Chloramphenicol – Treatment of
26
Pentavalent vaccine protects against 5 infectious agents: diphtheria, pertussis, tetanus, Hepatitis B, and Hib. Pneumococcal vaccine protects against severe pneumonia and meningitis. Rotavirus vaccine protects against severe diarrhea caused by rotavirus.
35
bacterial infections
Tetracycline eye ointment – eye infection and general hygiene
Glucose 5% - Hypoglycemia
Acute Respiratory Infection/Pneumonia
Amoxicillin, Erythromycin—Treatment of ARI
Fever/Malaria
Treatment of malaria
Quinine – Treatment of severe malaria and malaria not responding to first line Treatment of malaria complications and fever
Paracetemol syrup – Treatment of fever
Iron Folate – Treatment of anemia caused by malaria
HIV/AIDS Cotrimoxazole – Prevention and treatment of opportunistic infections.27
Diarrhea
ORS – Prevention and treatment of dehydration
Zinc – Reducing duration of diarrheal episode, prevent dehydration
Doxycycline, Metronidazole—Treatment of bacterial diarrhea not responding to other treatments
Albendazole – Prevention/treatment of diarrhea caused by parasites
The kits also included a suite of medications that can be used for treatment of neonatal infections, another
primary cause of neonatal deaths. The third primary cause of neonatal death, birth asphyxia, requires skilled
practitioners and is less responsive to specific commodities. Nonetheless, the medications provided assisted
to prevent neonatal deaths.
The drugs also provided the backbone for the
country’s successful program for the Integrated
Management of Childhood Illnesses (IMCI). This
approach allows lower cadre health workers to
use clinical algorithms based on symptoms to
presumptively diagnose and treat several of the
major and commonly recurring illnesses for
children including ARI, fever, and diarrhea.
Conclusive demonstrations of correlation
between the kits program and declines in child
mortality are beyond the scope of this evaluation.
However, by providing essential commodities, the
kits program allowed for effective case
management of common conditions that too often become deadly in the Malawian context. Therefore it is
reasonable to conclude that the kits program was able to contribute to the sustained declines in under-5
child mortality in Malawi.
Maternal mortality
General trends in maternal mortality
27
While cotrimoxazole for prevention of opportunistic infections is provided through the HIV supply chain, the cotrimoxazole contained in the PHC kits could be used to treat opportunistic infections in children who had not yet been diagnosed with HIV.
0
200
400
600
800
1000
1200
19
90
19
92
19
94
19
96
19
98
20
00
20
02
20
04
20
06
20
08
20
10
20
12
MM
R
Maternal Mortality Ratio in Malawi (# of deaths per 100,000 births)
Figure 12 Maternal Mortality Estimation Interagency Working Group, 2014
36
Although it is unlikely that Malawi will reach its MDG target for maternal mortality, there is evidence that
the rates of maternal deaths have declined over time. Much of this may be attributable to an increasing
percentage of facility births, as well as improving maternal health practices by health workers. Skilled
attendance at birth rose from 51.8% in 1990 to 65.8% in 2013. In 2010, the MDHS indicates that 73% of all
women delivered in a health facility. While the overall trend in maternal mortality is positive, the most
recent estimates indicate that progress has slowed somewhat since 2005.
Causes of maternal mortality in Malawi
The exact causes of maternal mortality in Malawi are
not well understood. However, the recent report of
Confidential Inquiry into Maternal Death (MoH 2013)
showed that 56.2% of maternal deaths are due to direct causes like bleeding, eclampsia, sepsis,
complications of abortions, obstructed labour and ruptured uterus. These findings are consistent with the
causes of maternal mortality in the region.
Contribution of the kits program to declining maternal mortality
As in the case of child mortality, the kits program provided essential products for management of some of
these direct causes of maternal mortality. Key among these is oxytocin, which is used for labor induction
and to control hemorrhage. Chlorhexidine is also an essential commodity to prevent infection in the birth
canal, and magnesium sulfate to manage pre-eclampsia. In addition, there are several commodities that
address other possible factors in maternal deaths including HIV, malaria and anemia (See table below).
Maternal mortality cause Relevant kit contents Hemorrhage Oxytocin – increase uterine contractions and control hemorrhage
Sepsis / Infections Chlorhexidine – Cleansing of birth canal
Benzyl Penicillin, Amoxicillin, Metronidazole, genatamycin—treatment of Sepsis and infections
Hypertensive disorders
Magnesium sulfate – management of pre-eclampsia
Oxytocin – labor induction to prevent obstructed labor
Other possible factors in maternal deaths
Maternal mortality cause Relevant kit contents HIV/AIDS Co-trimoxazole—management of opportunistic infections
Malaria Doxycycline – Malaria prophylaxis
Sulfadoxine Pyrimethamine - Malaria prophylaxis
Quinine – Treatment of severe malaria/ second line
Anemia Fefol – Prevention of anemia
Others Paracetamol, Cannula, Sterile gloves
Promethazine – prevent/control nausea/vomiting during and after surgery/delivery
E. Sustainability
Evaluation Category: Sustainability Evaluation Score: Poor
Category Framing Questions
Figure 13 : MOH 2013. Confidential Inquiry into causes of maternal deaths, Malawi.
37
To what extent did the benefits of a program or project continue after donor funding ceased?
What were the major factors which influenced the achievement or non-achievement of sustainability
of the program or project?
Evaluation Findings Summary
While the PHC project contributed $8.6M to the recapitalization of CMST, this support did not translate
into the expected operational and financial capacity at CMST by the end of the project as originally
intended. This was due to factors external to the implementation of the PHC project, including progress on
CMST internal reforms and operational improvements.
Detailed Findings
Benefit continuation after project end
The PHC project was designed as an emergency measure to meet the national supply gap in essential
medicines that Malawi struggled with for the last several years, which reached a critical point in 2011. During
the implementation of the PHC project, essential medicine stock outs at health facilities were reduced to as
low as 7%. When the project ended in September/October 2013, there was a clear increase in product stock
outs to 12% in Q4 2013. As a result, this evaluation concludes that the PHC project did not achieve its system
sustainability goals by the end of its implementation. This conclusion is further supported by the stock outs
that occurred immediately after the end of the PHC project and the eventual revival of the kits project again
in 2014.
Factors contributing to non-achievement
A rationale for the PHC project was to create a “breathing space” for CMST to institute procurement and
internal operational reform, funded from a revolving capital fund. However, despite the PHC’s contribution
to the CMST’s recapitalization, it did not translate into expected operational capacity on the part of CMST by
the time the project ended. While CMST only received roughly 65% of what they should have from the kits
program,28 as noted earlier in this report, the US$8.6M recouped by CMST is still significant amount of
funding by any standard. Expectations on the pace of reform and capacity building at CMST between the
MOH, partners and donors were clearly not aligned. The Joint Strategy for Supply Chain Integration Strategy,
was in part, developed to add structure to this reform, however, it remains unclear to what extent its
milestones have been achieved by MoH. Additional developments, like the delayed submission of DFID
“Component II” essential medicines order in 2013, due to CMST-DFID outsourcing negotiations, added an
additional layer of complexity to CMST reform planning. In light of the significant contribution of the PHC
project to CMST recapitalization, barriers to operational reform at this point, are perhaps less a case of
financial constraints, but more a case of internal operational reform planning and its execution.
Equity
Category: Equity Evaluation Score: Excellent
Category Framing Questions
28
As CMST uses Malawian Kwacha, due to fluctuating exchange rates, this should be considered a rough estimate.
38
How broadly were the benefits of the project shared among various populations and regions?
Where the benefits of the PHC project equally shared among the different levels of the health
system?
Evaluation Finding Summary
The PHC project achieved equity in its benefits due to its national reach, inclusion of all service levels in the
Malawi health system, as well as in its selection of kit items that targeted a broad spectrum of diseases
and conditions. By increasing product availability at public facilities in all levels of the health system, the
project increased equitable access to health care in Malawi.
Detailed Findings
Benefits across populations and regions
The PHC project provided essential medicines and supplies to over 620 health facilities in 28 districts. A total
of 15,844 kits were distributed during the implementation of the PHC project, which potentially reached an
estimated 9,487,697 patients. As a result of its national scope, this evaluation concludes that the project
achieved a high level of equity in terms the beneficiaries of the PHC project.
A total of 89% facilities either strongly agreed or
agreed that the kits increased patient access to
essential medicines. One facility said, "People were
dying here and were saved because of those drugs.”
Another clinician reported overhearing patients
talking about improved availability of drugs at the
health center due to the kits program. The only
facilities that disagreed with this increase in
availability were CHAM sites, which were able to
procure directly from private suppliers and so less
reliant on the kits program. One central hospital felt
that while the kits mitigated product stock outs, the
quantities were inadequate for their routine patient
load and that the kit items did not dramatically
change overall patient access to medicines.
Fifty-nine percent (59%) of facilities said that regardless of quantity, they felt that the items selected met
patient needs, with 34% saying the items somewhat met their needs, and 7% saying the items selected did
not meet the needs of their health facility. In terms of quantity, 60% of facilities said that the quantities in
the kits were met patient needs, while 40% said they did not. Isolating for just the health centers that were
interviewed, 65% agreed that the quantities meet the needs of their facilities, while 35% said they did not.
While the overwhelming facilities believed the PHC increased patient access to medicines, health facility
observations on the standardization of both the content of the kits and their quantities were not entirely
consistent – even among the health centers for which they were designed. Fifty-nine percent (59%) of
facilities said that regardless of quantity, they felt that the items selected met patient needs. While 60% of
Figure 14 Patient Access Survey Question Responses
39
the facilities felt that the quantities in the kit were adequate for patient needs. [See Analysis of Most
Important, Least Important, Stocked Out, Under and Oversupplied Kit Items in the Annex section]
As a part of this evaluation, health facilities were asked which of the PHC items they thought were the most
and the least “important.” Importance was defined in terms of both having the broadest use and its
criticality in terms of meeting the needs of that facility. Items listed as the “least” important should be
understood as items perceived as having least relevance for their patient needs.
Across all health service levels, inclusive of MOH and CHAM sites, the following items were considered the
most “important” items in the PHC kits (alphabetized):
Amoxycillin susp bot
Amoxycillin tabs 1000
Benzylpenicillin 5miu 50s
Cotrimoxazole 480mg tabs 500s
Doxycycline tabs 1000
Gentamycin inj 50s
Gloves exam latex M 100s
Metronidazole tabs 1000s
Paracetamol tabs, 500mg 1000s
Quinine 300mg tabs 100
Quinine inj 10
Across all health service levels, inclusive of MOH and CHAM sites, the following items were considered the
least “important” items in the PHC kits for their respective facility (alphabetized):
Albendazole 400mg tabs 100s
Benzoic/salicylic acid ont 40g
Benzyl Benzoate lotion 1000ml
Epinephrine (or Adrenaline) inj 10s
Promethazine tabs 100s
Sulphadoxine/Pyrimeth 1000s
Suture non abs DEC3 36s
Suture abs DEC4 36s
Water for Inj 50s
Benefits across service delivery levels
The PHC project distributed medicines and supplies to all service levels in the MOH service delivery system;
this includes health centers, district hospitals and central hospitals. While most CHAM facilities eventually
opted-out of the PHC project, mostly due to financial reasons, their health centers and community and
mission hospitals also benefited from the PHC project through reported cost savings. As such, this evaluation
finds that equity across health service delivery levels and systems was also achieved.
The items in the kits included products that address various conditions and diseases. They included broad-
spectrum use antibiotics, such as cotrimoxazole and amoxycillian. Maternal and neonatal products, such as
oxytocin and and 2ml syringes, were also included. Antimalarial products such as, Sulphadoxine/Pyrimeth
and injectable quinine were also included in the kits. This evaluation concludes that equity in the
implementation of the PHC project as also achieved due to the selection of PHC kit contents.
40
Lessons Learned
The higher level lessons learned from this evaluation are:
Standardized kit distributions are effective in emergency situations and in environments characterized by
chronic undersupply and limited inventory management capacity. In the Malawi context, they are most
appropriate for rural, public health facilities.
By simplifying procurement, warehousing, and distribution planning, the utilization of the kit system under
the PHC project enabled it to quickly establish a medicine and health commodity distribution system of
national reach. As expected, the use of a kit system produced inventory over and under stocks - however it
did not result in a substantial incidence of product expiry in 2012 and 2013. This is attributed to the
appropriate selection of kit contents and their quantities, the active relocation of products as well as the
long shelf life of the PHC commodities.
MOH health centers were clearly the facilities that most benefited from the PHC project. While the hospitals
found the kits very useful, kit products were often quickly consumed in a few weeks, or often in a few days,
and so had less of an impact on their high patient volumes. Similarly, as evidenced by the number of CHAM
facilities that opted-out of the project, the PHC project was also less of an effective solution for CHAM
facilities.29 As a private organization, with facility based procurement budgets, they are able to directly
procure from private suppliers and so less dependent on the government supply in times of crisis.
Unless kits of multiple variations are available for the various health facility level and their needs, in order to
maximize the benefits of a kit distribution model in Malawi, any future kit distribution project should focus
on only supplying MOH rural health centers.
Increased product availability at the PHC level reduces patient burdens at higher levels in the health
system and leads to overall improved health service delivery.
The routine and dependable monthly delivery was cited the most often as the main strength of the PHC
project. Facilities often expressed that due to the routine supply made available by the kits, patients had
more confidence in their health center. Eighty-six (86%) of health facilities reported that they observed an
increase of patients at their respective facilities due to the PHC project. In the Malawi environment, patient
demand appears to rises to meet the available medicine and health commodity supply.
Several health centers observed that the PHC project was able to reduce the number of patient referrals to
district and central hospitals. While this evaluation was not able to confirm this observation from other data
sources, it concludes that investment in PHC services (supported by a routine and reliable supply system)
likely also has broader health system benefits in Malawi.
29
CHAM facilities were not recipients of the streamlined kits that were distributed in 2014
41
Due to the over and under supplies that result from the implementation of a kit distribution model,
product redistribution is a critical mechanism for averting product stock outs, overstocks and expiry.
Product redistribution is a frequently used mechanism by health facilities in Malawi to manage oversupplied,
expiring or unneeded commodities. Seventy-five percent (75%) of health facilities interviewed for this
evaluation reported relocating products from the PHC kits to other health facilities, usually their district or
referral hospital. Health centers without maternal or surgical wards, for example, would often relocate
either slow moving or unused products to facilities that did have these wards.
The redistribution of health center oversupplies is managed at the DHO level and as a process, is not
standardized, effectively planned or documented. It is also often limited by transportation capacity or the
proximity between overstocked and understocked facilities. This lack of standardization limits opportunities
to more optimally address undersupply and stock out issues and also introduces a risk of product leakage.
Under the PHC project, CHAM experienced redistribution challenges due to difficulties related to transferring
product ownership and costs. While due to the standardization of the kit items, both MOH and CHAM also
found it a challenge to redistribute certain products to other facilities as they were also overstocked with the
same items.
Ad-hoc product relocations add a hidden operational, financial and potentially also a health impact cost
when patients do not receive the medicines they need. Since the centralization of drug budgets occurred in
mid-2013, districts have less capacity to procure from private suppliers in the event of a stock out, which has
further increased the dependence on redistribution as a supply strategy. Until Malawi is able to implement
an effective and fully functioning health commodity order-based (“pull”) system at the facility level, kit based
or not, effective product redistribution will need to be a part of the national logistics system in Malawi.
The DHOs play a critical role in the effective planning, oversight and management of medicine and health
commodity stocks at the health center level.
The DHOs play a key role in the performance outcomes of health centers, in particular, effective inventory
and supply management. They aggregate and report on stock levels, submit resupply orders to CMST, and as
noted above, also manage product redistribution. Similarly, they also need to be able to effectively voice and
communicate the needs of their health centers to decision-makers at the central level, and so must also
serve as advocates for their district health facilities. In addition, DHOs are the mechanism in which
information is shared and improvement initiatives are rolled out to lower level facilities. The fact that
roughly a third of health facilities first heard about the kit project when the kits arrived is indicative of the
inconsistent effectiveness of communications from the DHO level to their respective health facilities.
DHOs are an overburdened and critical pivot point in the management of the Malawi health system and are
on the forefront of its clinical and operational performance and success. Due to their significant
responsibilities, they have to be experts in many technical and functional areas. They need to be effective
managers, planners, problem-solvers, and in general, also competent and effective leaders. Under Malawi’s
decentralized system DHOs have direct oversight and leadership over the health facilities in their district, as
a result, supply chain management trouble shooting and improvement initiatives should be more effectively
owned and led at this level in the health system.
42
Conclusion and Recommendations
Conclusion
The PHC was a timely and effective medicine and health commodity supply delivery intervention. It reduced
the occurrence and duration of health commodity stockouts, particularly at the rural health facilities levels,
and this is a notable achievement. However, the PHC project was intended to be an intervention of a limited
duration.
The PHC project’s contribution to the recapitalization of CMST was meant to address concerns about the
sustainability of the project. However, as evidenced by the increased stock outs that occurred immediately
after the end of the kits project, as well as the revival of the kits project in 2014, the recapitalization of CMST
did not yield capacity and performance outputs by the end of the PHC project as intended. While the PHC
project was a largely successful project, and its support to the provision of primary health care in Malawi
considerable, the MOH, CMST and other partners, need to return to the implementation of the targets and
milestones of the 2012 Joint Strategy for Supply Chain Integration in order ensure the longer-term effective
operations of CMST and decrease the kind of stock out situations that put Malawi’s health system at risk and
require interventions like the PHC project in the first place.
Evaluation Recommendations
If standardized kit distributions are planned in the future, multiple kits configurations should be
developed for health centers with maternity wards, facilities with surgical wards, and central
hospitals to better mitigate the risk of medicine and supply over and understocks.
The MOH, CMST and partners should revive and implement the priorities set out in the 2012 Joint
Strategy on Supply Chain Integration in order to strengthen the capacity of CMST to better mitigate
emergency stock out scenarios in the future. As a part of this, there should be knowledge and skills
transfer between the parallel supply chains and CMST.
The invoicing and account reconciliation functions of CMST needs to strengthened in order for
ensure its capacity to capture revenue and in turn sustainably fund its operations.
Under Malawi’s “push” system, a significant amount of product redistribution occurs which
introduces a hidden cost and burden to the national supply chain. In its current implementation, it’s
a fairly ad-hoc and undocumented process, and so needs to be better standardized and streamlined.
As the District Health Office (DHO) plays a critical role in monitoring inventories and the
management oversight of the health center level, more targeted supply management improvements
and capacity building need to be targeted at, and led from, the DHO level.
As an large private sector health service provider, the inventory management capacity and
performance of CHAM facilities needs to also be strengthened and better integrated into the MOH
system.
49
Annex 2: Evaluation Team
The CHAI evaluation team was made up of a cross-disciplinary professionals from medical,
pharmacy, supply chain, health research and management consulting backgrounds. The team is
made up of CHAI Malawi staff who have strong Malawi health context experience in health systems
strengthening, health research and supply chain management, as well as exceptional relationships
with local Ministry of Health and implementing partner organizations.
The following team structure was implemented:
Andrew MvulaProject Coordinator
Dr. Bridget MaleweziMedical Advisor
Emily Churchman Kobayashi
MNCH Advisor
Sean Donato Project Lead
Newton ChagomaEvaluation Analyst
Serena ParcellEvaluation Analyst
Chris ConnellyEvaluation Analyst
Diallo AfadhaliPharmacist Advisor
Maeve MagnerCHAI Global Supply
Chain Advisor
Survey and Data Collection Team Evaluation Technical Review Team
Clement BandaM&E Advisor
Godwin NyirendaEvaluation Analyst
Experience and Qualifications
Sean D
onato
Res
earc
h T
eam
Lea
der
Dr.
Brid
get M
ale
wezi
Me
dic
al A
dvis
or
Em
ily C
hurc
hm
an K
obayashi
MN
CH
Advis
or
Adnre
w M
vula
Stu
dy C
oord
inato
Chris C
onnolly
E
valu
atio
n A
naly
st
New
ton C
hagom
a
Evalu
atio
n A
naly
st
Godiw
n N
yirenda
Evalu
atio
n A
b
Dia
llo A
fadhall
Pharm
acis
t A
dvis
or
Sere
na P
arc
ell
Evalu
atio
n A
naly
st
Cle
me
nt B
anda
M&
E O
ffic
er
Ma
eve M
agner
Supply
Chain
Advis
or
Health Evaluation and Research
Medicine
Clinical Pharmacy
Public Health
Supply Chain and Logistics
Supply Chain Impact Evaluation
Procurement
Health Systems Strengthening
Maternal and Child Health
Mixed Method Research
Statistical Analysis
Financial Analysis
Process Improvement
Malawi Health Context
50
Annex 3: Evaluation Questions
I. Operational Evaluation Questions
1. Project Management
A. Project Coordination and Planning
How did the partners work together to implement the project and meet stated PHC goals?
How effective were the PHC coordination meetings?
B. Supply Chain Efficiency and Effectiveness
How did you store the kits and what were the storage conditions?
How were the kit contents managed with the other stock? Did it change your inventory practices or recordkeeping?
Number of kits procured and delivered to each of the Center?
What was the cost of the kit medicines, relative to the total kit costs, vs the total cost of the program?
What was the performance against agreed-upon delivery schedules?
What was the product leakage rate?
What were the over and understock totals and rates?[Risk of loss due to oversupply]
What were the product delivery lead-times?
C. Product Selection
Did the contents meet disease priorities? Were there any disease priorities which were not covered by the kit program?[High level]
What was the suitability of the drugs in the kits? [Kit content appropriateness]
What were the contents of the kits for Phase I vs. Phase II?
D. Project Communication
Was the PHC Essential Medicines Project Communication strategy effective?
2. Product Availability
Availability of Quality Medicines and Health Supplies
Did any of the kit contents arrived expired or expire during its storage?
Did the kits arrive late?
Did the PMPB list of PSM items overlap with the kit items? Did any of the products show up on the PMPB report? Were there any product recalls?
To what extent were kit contents available to patients?
Was the product availability adequate for the patient population? [Quantity question]
Was the product of a good quality when delivered?
What did you do when a kit product stocked out?
What was the amount delivered?
Where there stock outs of essential medicines in your facility?
II. Health Impact Evaluation Questions
3. Health Service Outputs
A. Disease Management
How many women have benefited from prenatal iron supplementation that reduces the incidence of iron deficiency anemia?
To what extent has the Project eased the management of Diarrhea, ARI, Postpartum care, syphilis, de-worming, fever, severe and complicated malaria and reduced malaria prevalence in pregnant women?
B. Health Service Utilization
What impact did the kits have on patient visits to Center?
51
What impact did the PHC program have on health service utilization?
C. Commodity Management Capacity
The recapitalization of CMST was a stated objective of the PHC project, to what extent was this achieved?
What was the impact of the PHC program on health facility worker inventory management skills?
What was the impact of the PHC program on LMIS reporting at Center?
What was the impact of the PHC project on the CMST reform process?
4. Public Health Impact
Impact on mortality and morbidity among children under 5 and pregnant women
Did the products provided reinforce or help to maintain current declining disease trends?
How has the Project contributed towards keeping mortality and morbidity among children under 5 years and pregnant women on a constant declining rate to meet the 2015 MDG 4 (Under Five Mortality rate of 78 / 1000 LB)?
Is there a correlation between product availability and the uptake of services for pregnant women?
Is there a correlation between product availability and the uptake of under-5 health services?
52
Annex 4: Phase I Kit Contents
Phase I Kit: Distributed January to April/May 2012 (37 Items in Total)
# Item description Qty per kit # Item description Qty per kit
1 Amoxicillin 250mg tabs/PAC-1000 3 21 Water for inj 10ml amp/BOX-50 1
2 Amoxici.pdr/oral sus 125mg/5ml/BOT-100ml 15 22 Zinc 20mg tablets/PAC-100 2
3 Benzylpenicillin pdr/inj 3g vial/BOX-50 1 23 Albendazole 400mg chewable tabs/PAC-100 2
4 Ciprofloxacin 500mg tabs/PAC-10 15 24 Fe(as fum.)+folic 60+0.4mg tab/PAC-1000 2
5 Diazepam inj 5mg/ml 2ml amp/BOX-10 pt 1 25 Miconazole nitrate cream 2%/TBE-30g 10
6 Doxycycline 100mg tabs/PAC-1000 3 26 Bandage,gauze,8cmx4m,roll 10
7 Epinephrine inj 1mg/ml 1ml amp/BOX-10 1 27 Cotton wool,500g,roll,non-ster 5
8 Erythromycin 250mg tabs/PAC-100 14 28 Compress,gauze,10x10cm,n/ster/PAC-100 3
9 Glucose hyperton.inj 50% 50mL vl/BOX-20 1 29 Tape,adhesive,Z.O.,2.5cmx5m 5
10 Hydrochlorothiazide 25mg tabs/PAC-100 10 30 Syringe,disp,2ml,ster/BOX-100 1
11 Lidocain inj 2% 50ml vial/BOX-5 2 31 Syringe,disp,5ml,ster/BOX-100 1
12 Metronidazole 250mg tabs/PAC-1000 2 32 Needle,disp,19G,ster/BOX-100 2
13 New formula Oral reh.salts,1L s/BOX-100 1 33 Needle,disp,23G,ster/BOX-100 1
14 Paracetamol 500mg tabs/PAC-100 40 34 Sut,abs,DEC2,need 3/8 26mm,tri/BOX-36 1
15 Paracetamol 120mg/5ml or.sol/BOT-60ml 25 35 Sut,nonabs,DEC3,need 3/8 30mm,tri/BOX-36 1
16 Povidone iodine sol 10%/BOT-500ml 4 36 Needle,disp,21G,ster/BOX-100 1
17 Salbutamol 4mg tabs/PAC-1000 1 37 Gloves,exam,latex,medium,disp/BOX-100 5
18 Sulf.400mg+Trimet.80mg tabs/PAC-500 20
19 Sulf.100mg+Trimet.20mg disp.tab/PAC-100 50
20 Tetracycline eye ointment 1%/TBE-5g 10
53
Annex 5: Phase II Kit Contents
Phase II Kit: Distributed January to June 2012 – September /October 2013 (61 Items in Total)
S0305020 1 Apron,protection,plastic,disp/PAC-100 2 BOX
S0512111 2 Bandage,gauze,8cmx4m,roll 10 EA
S1559005 3 Benz.benzylpeni.pdr/inj 1.44g vl/BOX-50 1 BOX
S1520011 4 Benzylpenicillin pdr/inj 3g vial/BOX-50 1 BOX
S0709210 5 Cannula,IV short,18G,ster,disp 25 EA
S0709225 6 Cannula,IV short,22G,ster,disp 25 EA
S0709230 7 Cannula,IV short,24G,ster,disp 25 EA
S0323302 8 Catheter,Foley,CH14,ster,disp 2 EA
S0323303 9 Catheter,Foley,CH18,ster,disp 2 EA
S0523050 10 Compress,paraffin,10x10cm,ster/BOX-10 4 EA
S1501000 11 Epinephrine inj 1mg/ml 1ml amp/BOX-10 1 BOX
S1550025 12 Fe(as fum.)+folic 60+0.4mg tab/PAC-1000 14 PAC
S1551960 13 Gentamicin inj 40mg/ml 2ml amp/BOX-50 1 BOX
S0330030 14 Gloves,exam,latex,pwdfree,large/BOX-100 5 EA
S0330025 15 Gloves,exam,latex,pwdfree,medium/BOX-100 10 EA
S1555205 16 Lidocaine inj 1% 50ml vial/BOX-5 1 BOX
S0305030 17 Mask,surgical,disp/BOX-100 2 BOX
S1560600 18 Quinine inj 300mg/ml 2ml inject/BOX-10 8 BOX
S1560610 19 Quinine sulfate 300mg tabs/PAC-100 1 PAC
S0782300 20 Syringe,RUP,5ml,w/fixed ndl/BOX-100 2 BOX
S0504000 21 Tape,adhesive,Z.O,perforated,10cmx5m 3 EA
S1543805 22 Water for inj 10ml amp/BOX-50 4 BOX
S1580020 23 Zinc 20mg tablets/PAC-100 10 PAC
S1555370 24 Albendazole 400mg chewable tabs/PAC-100 10 PAC
S1515020 25 Benz.acid 6%+salic.acid 3% oint./TBE-40g 13 TBE
S8914100 26 Carton f/dividers 249 x 140 x 163 mm 1 Ea
S1555945 27 Clotrimazole 500mg vag.tabs/applicator 20 EA
S0523005 28 Compress,gauze,10x10cm,n/ster/PAC-100 10 EA
S1552300 29 Hydrochlorothiazide 25mg tabs/PAC-100 20 PAC
S1553000 30 Ibuprofen 200mg tabs/PAC-100 10 PAC
S1504200 31 Magn.trisilicate compound tabs/PAC-1000 6 PAC
S1555930 32 Nystatin oral sus 100,000IU/ml/BOT-30ml 25 BOT
S1555990 33 Paracetamol 125mg/5ml or.sol/BOT-60ml 100 BOT
S1555965 34 Paracetamol 500mg tabs/PAC-1000 16 PAC
S1559200 35 Promethazine 25mg tabs/PAC-100 18 PAC
S1562015 36 Salbutamol 4mg tabs/PAC-1000 4 PAC
# Material Desription Qty Unit
Box: 2/7 (13)UNICEF
Material
code
UNICEF
Material
code
Material Desription Qty Unit#
Box: 1/7 (23)
54
S1519500 37 Benzyl benzoate 25% lotion /BOT-1000ml 1 BOT
S0519600 38 Cotton wool,500g,roll,non-ster 10 EA
S1544400 39 Doxycycline 100mg tabs/PAC-1000 7 PAC
S0327520 40 Gloves,surg,pwdfree,7.5,ster,s.u.,pair 100 EA
S1555650 41 Metronidazole 250mg tabs/PAC-1000 7 PAC
S1537100 42 Sulf.400mg+Trimet.80mg tabs/PAC-500 36 PAC
S0782303 43 Syringe,RUP,2ml,w/fixed ndl/BOX-100 1 BOX
S1505060 44 Amoxicillin 250mg tabs/PAC-1000 10 PAC
S1523000 45 Calamine lotion/BOT-500ml 2 BOT
S1546315 46 Erythromycin 250mg tabs/PAC-100 60 PAC
S1552115 47 Glucose hyperton.inj 50% 50mL vl/BOX-20 1 BOX
S1561130 48 Oral reh.salts,orange Flavour1Ls/CAR-100 3 CAR
S1568045 49 Sulfadox+Pyrimeth 500+25mg tabs/PAC-1000 2 PAC
S0564050 50 Sut,abs,DEC4,need 1/2 30mm,round/BOX-36 1 BOX
S0565011 51 Sut,nonabs,DEC3,need 3/8 30mm,tri/BOX-36 1 BOX
S0782302 52 Syringe,RUPw/SIP,10ml,w/fixed nd/BOX-100 1 BOX
S1510000 53 Tetracycline eye ointment 1%/TBE-5g 150 TBE
S1505046 54 Amoxici.pdr/oral sus 125mg/5ml/BOT-100ml 120 BOT
S1564326 55 Sodium chl.inj 0.9% 500ml w/giv.s/BOX-20 1 BOX
S1552106 56 Glucose inj 5% 500ml w/giv.set/BOX-20 1 BOX
Box: 4/7 (10)UNICEF
Material
code
# Material Desription Qty Unit
Material Desription Qty Unit
#
Box: 3/7 (7)
Box: 7/7 (1)
UNICEF
Material
codeMaterial Desription Qty Unit
UNICEF
Material
Qty Unit
Box: 6/7 (1)UNICEF
Material
code
Material Desription Qty Unit#
UNICEF
Material
codeMaterial Desription#
#
Box: 5/7 (1)
55
Material number # Material Description
S1545300 57 Oxytocin inj. 10 IU, 1ml amp/BOX-10
S0566010 58 Clamp,umbilical cord,ster,s.u.
S1543625 59 Diazepam inj 5mg/ml 2ml amp/BOX-10 pt
S6780202 60 Phenobarb.200mg/ml inj 1ml amp/BOX-10 pt
S1559312 61 Phenobarbital 30mg tabs/PAC-100 pt
Loose Items - Outside Kits (5)
56
Annex 6: Comparison of Phase I and Phase II Items
Comparison Question Answer
What was the difference between contents of phase I and phase II kits?
Phase II Kits are richer in items and tackle a broader range of diseases
What was the added advantage of the contents of kit II?
It addresses maternal, child health, surgery, severe malaria, mental health and includes injectable antibiotics and more ways of administration
What were the disadvantages of the items in the phase II kits?
No anti-diabetic, no anti-ulcer, no antidote, no anti-emetic, only one type of anesthetic which is again local
Did the kit contents include items appropriate for Primary Health Care?
Yes, but we still need to assess at what level (from field information).
Kit I Kit II
1 type of non-steroidal anti-inflammatory drug,
1 type of anesthetic which is again only local
No mental health, at the exception of Diazepam which can have multiple uses,
No anti-diabetic,
No anti-ulcer,
No anti-emetic drugs,
No anti-allergic,
No antacid,
No antidote,
No transfusion kit,
No normal saline
No anti-diabetic,
Only one type of anesthetic which is again local,
No anti-ulcer,
No antidote,
No anti-emetic.
Phase I Items Not Included in Phase II Kits
Ciprofloxacin 500mg tabs/PAC-10
Miconazole nitrate cream 2%/TBE-30g
Needle,disp,19G,ster/BOX-100
Needle,disp,21G,ster/BOX-100
Needle,disp,23G,ster/BOX-100
Povidone iodine sol 10%/BOT-500ml
Sulf.100mg+Trimet.20mg disp.tab/PAC-100
Sut,abs,DEC2,need 3/8 26mm,tri/BOX-36
Tape,adhesive,Z.O.,2.5cmx5m
57
Phase II Kit Items Not Included in Phase I Kits
Apron plastic 100s
Benzathine Penicillin 1.44g 50s
Benzoic/salicylic acid oint 40g
Benzyl Benzoate lotion 1000ml
Calamine lot 500ml
Cannula iv 18g
Cannula iv 22g
Cannula iv 24g
Catheter foley CH14
Catheter foley CH18
Clotrimazole vag pess box
Compress paraffin 10x10cm
Gentamycin inj 50s
Gloves exam latex L 100s
Gloves surgical 7.5 pair
Glucose 5% 500ml iv 20s
Ibuprofen 200mg tabs 100s
Magnesium Sulphate inj
Magnesium trisilicate tabs 1000s
Mask surgical 100s
Nystatin oral susp
Oxytocin Inj
Phenobarbitone inj
Phenobarbitone tabs
Promethazine tabs 100s
Quinine 300mg tabs 100
Quinine inj 10s
Sodium chloride 500ml iv 20s
Sulphadoxine/Pyrimeth 1000s
Suture abs DEC4 36s
Syringe 10ml w needle 100s
Umbilical cord clamps
Zinc Oxide tape 10x5m
58
Annex 7: List of Interviewed Partners and Sites Visited
MOH, Donors and Implementing Partners (15)
Organization Interviewee Title Interview Date
CHAM Lumbani Makwakwa Pharmacy Manager 28-Feb-14
CMST Saidi Sangala Financial Accountant 7-Apr-14
DFID Susan Clapham Health and HIV Adviser 26-Mar-14
JSI Phillip Kamutenga Country Director 20-Feb
KfW Lamulo Nsanja Country Economist 4-Apr-14
MOH-HTSS Brian Chimwenje Senior Pharmacist 14-Apr-14
MOH-HTSS David Bagonza Supply Chain Advisor 14-Apr-14
Norwegian Embassy Hildegunn Tobiassen First Secretary 6-Mar-14
PMPB Godfrey Kadewele Chief Executive Officer 4-Apr-14
UNICEF Dennis Chimenya Communication for Development Officer 27-Mar-14
UNICEF John Blasco Supply and Logistics Specialist 27-Mar-14
UNICEF Nyson Chizani Monitoring and Evaluation Specialist 3-Mar-14
UNICEF Sam Chriwa PHC Specialist 3-Mar-14
USAID Ugbede-Ojo Abu Supply Chain Advisor 26-Mar-14
MOH Central Hospitals (4)
Region Facility Name District
Interview Date Interviewee Title
Central Kamuzu Lilongwe 4/8/2014 Dr. Munthali Medical Doctor
4/7/2014 Tadala Hamisi Pharmacist in-Charge
North
Mzuzu Mzimba North 3/14/2014 Brave Nyirenda Radiology Technologist/Drug committee member
Cosum Moyo Pharmacy Technician
South Queens Blantyre 3/21/2014
Dr. Dube Paedtrician
Sylvester Gausi Pharmacy Technician
Zomba Zomba 3/27/2014 Mr. Chaheka Principle Clinician
District Hospitals (10)
Region Facility Name District Interview Date Interviewee Title
South Chikwawa Chikwawa 17-Mar-14 Charles Kwenje Clinical Officer
Labson Tembo Pharmacy Technician
North Chitipa Chitipa 11-Mar-14 Given Chilimiro Clinical Officer
59
Jeremiah Chiumia Pharmacy Technician
Central Dowa Dowa 21-Mar-14 George Mhasa Clinical Technician
Mercy Matapila Pharmacy Technician
North Karonga Karonga 12-Mar-14 Alberto Balasiya Clinical Officer
Voster Msutu Pharmacy Technician
Central Kasungu Kasungu 20-Mar-14 Sosten Lankhulani District Medical Officer
Kondwani Kautsa Pharmacy Technician
South Machinga Machinga 27-Mar-14 Charles Mtibo District Medical Officer
Kingston Kapida Pharmacy Technician
South Mangochi Mangochi 26-Mar-14 Dr. william Peno District Medical Officer
Andy Kishyombe Pharmacy Technician
South Mwanza Mwanza 18-Mar-14 Dr. Godwin Ulaya District Medical Officer
Mcjones Nyirongo Pharmacy Technician
Central Ntchisi Ntchisi 20-Mar-14 Robert Kombe Pharmacy Technician
South Thyolo Thyolo 24-Mar-14 Botha Matron
Nelson Nachinga Pharmacy Technician
MOH Health Centres (30)
Region Facility Name District
Interview Date Interviewee Title
Central Area 18 Lilongwe 31-Mar-14 Grace Chikhwaya Nurse/Midwife(Drugstore Clerk)
South Beleu Chikwawa 17-Mar-14 Hannah Ng'oma Nurse/Midwife
Edward Awali Drugstore clerk
Central Bwanje Ntcheu 12-Mar-14 Stephen Mpinganjira Medical Assistant
Dennis Chikakuda Drugstore clerk
Central Chadza Lilongwe 13-Mar-14 Chifundo Nkhoma Medical Assistant
Charles Chikawale Drugstore clerk
North Chikwina Nkhata-Bay
17-Mar-14 Aidan Chinsula Senior Medical Assistant
17-Mar-14 Aidana Chinselu Senior Medical Assistant
South Chimembe Blantyre 19-Mar-14 Paul Salijeni Nurse /Midwife Technician/ Drugstore Clerk
South Chiumbangame
Mangochi 24-Mar-14 Hastings Chunga Medical Assistant
South Chonde Mulanje 26-Mar-14 Mr. Munthali Medical Assistant
Juma Mizati Pharmacy Technician
Central Diamphwe Lilongwe 14-Mar-14 Clemecia Nyoni Nurse/Midwife
Laston Owen Drugstore clerk
South Domasi Zomba 28-Mar-14 Leonard Magombo Medical Assistant
North Kachere Nkhata-Bay
18-Mar-14
Roosevelt Njerenje/Janet Nkusang'ombe
Clinical Officer/Nurse
Idah Manda Drugstore clerk
South Kakoma Chikwawa 17-Mar-14 Williams Duba Nurse/Midwife
60
Central Kawale Lilongwe 3-Apr-14 Paliwin Malikebu
Senior Medical Assistant
4-Apr-14 Tamanda Kazaku nurse/Midwife
Central Lobi Dedza 11-Mar-14 Justin Mwalabu Medical Assistant
Ella Nambote Drugstore clerk
South Lundu Blantyre 19-Mar-14 Micheal Matewere Medical Assistant
South Makhanga Nsanje 20-Mar-14 Lucius Gama Medical Assistant
North Malidade Mzimba North
13-Mar-14
MacFarland Phiri/Ken Chirwa
Nurse/Clerk
Ken Chirwa Drugstore clerk
Central Maluwa Lilongwe 10-Mar-14 Bartholomeu Andrea Medical Assistant
Doreen Mphika Drugstore clerk
North Mkoma Mzimba South
19-Mar-14 Precious Chibweya Senior HAS
Micheal Chirwa Medical Assistant
North Mpamba Nkhata-Bay
17-Mar-14 Chawanangwa Luwe Medical Assistant
Central Nalunga Lilongwe 1-Apr-14
Joseph Dickson Nthanda
Medical Assistant
Zuze Ndalamira Drugstore clerk
South Namwera Mangochi 24-Mar-14
Coby Maunda Medical Assistant
Steria Shauli Drugstore clerk
South Nankumba Mangochi 25-Mar-14 Clement Masoambeta Medical Assistant
Ingris Maunde Drugstore clerk
South Naphimba Mulanje 26-Mar-14 Francis Sibande Medical Assistant
South Ndamera Nsanje 28-Mar-14 Macford Chikopa Medical Assistant
Asautsaine Nkhumani Drugstore clerk
Central Ngoni Lilongwe 31-Mar-14 D.R.C Gondwe Medical Assistant
Dyson Njale Drugstore clerk
South Ngwelelo Zomba 27-Mar-14 Golden kachimanga Medical Assistant
South Nkalo Chiradzulu 24-Mar-14 Mr. Khulumura Medical Assitant
James Kaphwiti Drustore Clerk
South Ntaja Machinga 26-Mar-14 Akuzike Nkali Medical Assistant
Aaron Mchere Drugstore clerk
Central Tembwe Mchinji 2-Apr-14 Odala Mbewe Medical Assistant
Chrispin Kachipande Drugstore clerk
CHAM Mission / Rural Hospitals (6)
Region Facility Name District
Interview Date Interviewee Title
North Ekwendeni Mzimba North 13-Mar-14 Albert Mirongo Principle Clinician
Agness Hara Nurse
South Malamulo Thyolo 25-Mar-14 Dr. Shank Medical Doctor
Harry Muyaya Stores Supervisor
South Mlambe Blantyre 20-Mar-14 Mary Chikapa Matron
John Sailes Drugstore Assistant
South Mposa Machinga 26-Mar-14 Precious Chambakata Drugstore clerk
South Mulanje Mulanje 26-Mar-14 Dickson Chisale Pharmacy Technician
Dr. Shakespeare Medical Supplies
61
Central Ntonda [Rural Hospital]
Ntcheu 13-Mar-14
Stanley Namajagali & Elizabeth Gondwe
Medical assistant & Administrator
Elizabeth Bulaimu Pharmacy Attendant
CHAM Health Centres (10)
Region Facility Name District Interview
Date Interviewee Title
North Chambo Chitipa 11-Mar-14 Siesta Ngwira Clinical Officer
Central Gowa Ntcheu 13-Mar-14
Marlyne Ziwoya
Nurse/Midwife
Grace Zinkanda
Drugstore clerk
South H Parker Zomba 27-Mar-14 Stanley Mweiwa
Medical Assistant/ Drug store Clerk
Central Kasina Dedza 11-Mar-14
Sister Victoria Ogu
Nurse in-charge
Gyliselia Hola Pharmacy Attendant
North Lusubilo Rehabilitation Center
Karonga 12-Mar-14 Christina Kasilika
Nurse
Central M'bwatalika Lilongwe 2-Apr-14 Chimwemwe Ndoliro
Nurse/Midwife In-charge/Drug store Manager
South Mposa Machinga 26-Mar-14 Frank Nyozani Nurse
South Nsambe Neno 18-Mar-14
Mr. Nkhoma Clinical Officer
Stanley Nkhoma
Pharmacy Clerk
South PIM Chiradzulu 24-Mar-14 Damson Erick Nurse/Midwife
Chance Uladi Nurse/Midwife
Central St Mary's Rehabilitation
Dowa 1-Apr-14 Sister Jolly
Nurse/Midwife
In-charge/Drug store Manager
62
Annex 8: Reported List of “Most” and “Least” Important PHC kit, Over
and Undersupplied Items by Facility Level
Below is a list of PHC kit items that were considered by facilities as the “most” important for their facility,
based on service level:
Health Center
1) Paracetamol tabs, 500mg 1000s 2) Amoxycillin tabs 1000 3) Cotrimoxazole 480mg tabs 500s 4) Metronidazole tabs 1000s 5) Quinine inj 10s 6) Quinine 300mg tabs 100 7) Gloves exam latex M 100s
8) Benzylpenicillin 5miu 50s 9) Doxycycline tabs 1000 10) Gentamycin inj 50s 11) Amoxycillin susp bot 12) Ibuprofen 200mg tabs 100s 13) Erythromycin tabs 100s
DHO / District Hospital
1. Quinine inj 10s 2. Benzylpenicillin 5miu 50s 3. Gentamycin inj 50s 4. Gloves surgical 7.5 pair
5. Cotrimoxazole 480mg tabs 500s 6. Oxytocin Inj 7. Gloves exam latex M 100s 8. Paracetamol tabs, 500mg 1000s
Central Hospitals
1) Gloves exam latex M 100s 2) Benzathine Penicillin 1.44g 50s 3) Cannula iv 18g 4) Gloves exam latex L 100s 5) Mask surgical 100s
6) Metronidazole tabs 1000s 7) Suture abs DEC4 36s 8) Syringe 10ml w needle 100s 9) Quinine inj 10s
Mission Hospital
1) Paracetamol tabs, 500mg 1000s 2) Gentamycin inj 50s 3) Quinine 300mg tabs 100 4) Amoxycillin tabs 1000
5) Benzathine Penicillin 1.44g 50s 6) Benzylpenicillin 5miu 50s 7) Gloves exam latex M 100s 8) Gloves surgical 7.5 pair
Rural / Community Hospitals
1) Benzylpenicillin 5miu 50s 2) Doxycycline tabs 1000 3) Gentamycin inj 50s
4) Amoxycillin susp bot 5) Ibuprofen 200mg tabs 100s 6) Erythromycin tabs 100s
Below is a list of PHC kit items that were considered by facilities as the “least” important for their facility,
based on service level:
Health Center
1) Benzyl Benzoate lotion 1000ml 2) Epinephrine (or Adrenaline) inj 10s 3) Magnesium Sulphate inj 4) Mask surgical 100s 5) Catheter foley CH18 6) Glucose 50% 20s 7) Promethazine tabs 100s
8) Benzoic/salicylic acid ont 40g 9) Catheter foley CH14 10) Zinc tabs 100s 11) Doxycycline tabs 1000 12) Apron plastic 100s 13) Clotrimazole vag pess box 14) Sulphadoxine/Pyrimeth 1000s 15) Water for Inj 50s
63
DHO / District Hospital
1) Benzyl Benzoate lotion 1000ml 2) Promethazine tabs 100s 3) Sulphadoxine/Pyrimeth 1000s 4) Albendazole 400mg tabs 100s 5) Benzoic/salicylic acid ont 40g
6) Epinephrine (or Adrenaline) inj 10s 7) Sut non abs DEC3 36s 8) Suture abs DEC4 36s 9) Water for Inj 50s
Central Hospitals
1) Albendazole 400mg tabs 100s 2) Benzyl Benzoate lotion 1000ml 3) Doxycycline tabs 1000 4) ORS 100s
5) Sulphadoxine/Pyrimeth 1000s 6) Zinc Oxide tape 10x5m 7) Zinc tabs 100s
Mission Hospital
1) Promethazine tabs 100s 2) Benzoic/salicylic acid ont 40g 3) Catheter foley CH14
4) Magnesium trisilicate tabs 1000s 5) Sut non abs DEC3 36s 6) Tetracycline eye oint 5g
Rural / Community Hospitals
1) Compress paraffin 10x10cm 2) Epinephrine (or Adrenaline) inj 10s
3) Gentamycin inj 50s 4) Magnesium Sulphate inj
By health facility level, the following items were reported by the sites as the most frequently stocked out
items provided in the PHC kits:
Health Centers – Most Frequently Stocked Out
1) Ibuprofen 200mg tabs 100s 2) Paracetamol tabs, 500mg 1000s 3) Amoxycillin tabs 1000 4) Quinine 300mg tabs 100 5) Calamine lot 500ml 6) Bandage gauze 8x4m
7) Cotton wool 500g 8) Paracetamol syrup bot 9) Gentamycin inj 50s 10) Metronidazole tabs 1000s 11) Erythromycin tabs 100s
DHO / District Hospital – Most Frequently Stocked Out
1) Bandage gauze 8x4m 2) Benzylpenicillin 5miu 50s 3) Ibuprofen 200mg tabs 100s
4) Gentamycin inj 50s 5) Zinc Oxide tape 10x5m
Central Hospitals30 – Most Frequently Stocked Out
Benzylpenicillin 5miu 50s
Cannula iv 24g
Cotton wool 500g
Epinephrine (or Adrenaline) inj 10s
Gentamycin inj 50s
Gloves exam latex M 100s
Mask surgical 100s
Oxytocin Inj
Mission Hospitals – Most Frequently Stocked Out
1) Paracetamol tabs, 500mg 1000s 2) Cannula iv 24g 3) Gloves exam latex M 100s
4) Glucose 50% 20s 5) Quinine inj 10s
Rural / Community Hospitals – Most Frequently Stocked Out
Gloves exam latex M 100s
30
These items are not ranked by frequency of stock outs as these items received the same scoring by facilities interviewed
64
By health facility level, the following items were reported by the sites as the most frequently undersupplied
items provided in the PHC kits:
Health Center – Most Undersupplied
1) Ibuprofen 200mg tabs 100s 2) Quinine 300mg tabs 100 3) Paracetamol tabs, 500mg 1000s 4) Amoxycillin tabs 1000 5) Bandage gauze 8x4m 6) Calamine lot 500ml
7) Syringe 2ml w needle 100s 8) Benzylpenicillin 5miu 50s 9) Fe+Fol tabs 1000s 10) Metronidazole tabs 1000s 11) Nystatin oral susp 12) Amoxycillin susp bot
DHO / District Hospital – Most Undersupplied
1) Benzylpenicillin 5miu 50s 2) Gentamycin inj 50s 3) Amoxycillin tabs 1000 4) Cannula iv 24g 5) Gloves surgical 7.5 pair 6) Metronidazole tabs 1000s 7) Benzathine Penicillin 1.44g 50s
8) Compress paraffin 10x10cm 9) Cotton wool 500g 10) Quinine 300mg tabs 100 11) Syringe 10ml w needle 100s 12) Paracetamol tabs, 500mg 1000s 13) Sodium chloride 500ml iv 20s 14) Zinc Oxide tape 10x5m
Central Hospitals – Most Undersupplied
1) Gloves exam latex M 100s 2) Cannula iv 24g 3) Calamine lot 500ml
4) Epinephrine (or Adrenaline) inj 10s 5) Gentamycin inj 50s 6) Gloves exam latex L 100s
Mission Hospital – Most Undersupplied
1) Benzathine Penicillin 1.44g 50s 2) Benzylpenicillin 5miu 50s 3) Cannula iv 24g 4) Gentamycin inj 50s
5) Gloves exam latex M 100s 6) Ibuprofen 200mg tabs 100s 7) Paracetamol syrup bot 8) Paracetamol tabs, 500mg 1000s
Rural / Community Hospitals – Most Undersupplied
1) Ibuprofen 200mg tabs 100s 2) Quinine inj 10s
By health facility level, the following items were reported by the sites as the most frequently oversupplied
items provided in the PHC kits:
Health Center – Most Oversupplied
1) Albendazole 400mg tabs 100s 2) Doxycycline tabs 1000 3) Promethazine tabs 100s 4) Glucose 50% 20s 5) Mask surgical 100s 6) Sulphadoxine/Pyrimeth 1000s 7) Water for Inj 50s 8) Fe+Fol tabs 1000s
9) Hydrochlorothiazide 25mg tabs 100s 10) Erythromycin tabs 100s 11) Apron plastic 100s 12) Epinephrine (or Adrenaline) inj 10s 13) Magnesium trisilicate tabs 1000s 14) Tetracycline eye oint 5g 15) Cotrimoxazole 480mg tabs 500s
DHO / District Hospital – Most Oversupplied
1) Promethazine tabs 100s 2) Hydrochlorothiazide 25mg tabs 100s
7) Clotrimazole vag pess box
8) Epinephrine (or Adrenaline) inj 10s
65
3) Cotrimoxazole 480mg tabs 500s 4) Doxycycline tabs 1000 5) Sulphadoxine/Pyrimeth 1000s 6) Albendazole 400mg tabs 100s
9) Glucose 50% 20s 10) Magnesium trisilicate tabs 1000s 11) Sut non abs DEC3 36s 12) Suture abs DEC4 36s
Central Hospitals – Most Oversupplied
1) Albendazole 400mg tabs 100s 2) Sulphadoxine/Pyrimeth 1000s 3) Benzathine Penicillin 1.44g 50s
4) Fe+Fol tabs 1000s 5) Hydrochlorothiazide 25mg tabs 100s 6) Zinc tabs 100s
Mission Hospital – Most Oversupplied
1) Cotrimoxazole 480mg tabs 500s 2) Promethazine tabs 100s
3) Tetracycline eye oint 5g
Rural / Community Hospitals – Most Oversupplied
1) Albendazole 400mg tabs 100s 2) Cotton wool 500g 3) Cotrimoxazole 480mg tabs 500s
66
Annex 9: Analysis of Most Important, Least Important, Stocked Out,
Under and Oversupplied Kit Item Lists
The table organizes the various item lists reported by the facilities. The table below includes the most
referenced items during the survey interviews. The comments column attempts to provide and explanation
for the respective categorizations of the products discussed.
# Item M
ost
Im
po
rtan
t
Un
der
sup
pli
ed
Lea
st im
po
rtan
t
Ov
ersu
pp
lied
Fre
qu
entl
y S
tock
ed
Ou
t
Comments
1 Albendazole 400mg tabs 100s
Multiple deworming interventions likely resulted in an overstock of supplies
2 Amoxycillin tabs 1000
As this is a multiple use item, it’s a fast moving and widely used across all health service levels
3 Bandage gauze 8x4m The was most likely provided in low quantities
4 Benzoic/salicylic acid not 40g
This item is used for uncommon skill conditions (rashes,etc.). The expertise to treat the skin conditions requiring this product is often minimal
5 Benzyl Benzoate lotion 1000ml
This item is used mostly to treat scabies in Malawi, which is a rare condition. The expertise to treat the skin conditions requiring this product is often also minimal.
6 Benzylpenicillin 5miu 50s
As this is a multiple use item, it’s a fast moving and widely used across all health service levels
7 Calamine lot 500ml
This is an antihistamine and broadly used for various skin conditions and is a fast-moving item.
8 Cannula iv 24g
This item is used for collecting blood samples and IV transfusions and is a multiple use item. This was likely supplied in low quantities.
9 Catheter foley CH14
This item is used mostly for complicated procedures (urine retention) which the admission of patients and mostly used at a secondary or tertiary level.
10 Catheter foley CH18
This item is used mostly for complicated procedures (urine retention) which the admission of patients and mostly used at a secondary or tertiary level. This is just a higher caliber that CH14.
11 Cotrimoxazole 480mg tabs 500s
It is the drug used for treatment of most opportunistic infections in HIV and TB, and is supplied by multiple programs.
12 Cotton wool 500g
Multipurpose (cleaning wounds, maternity during delivery, etc.). Only 10 rolls provided in the kits so at likely risk of frequent stock outs.
13 Doxycycline tabs 1000
Routine use for STIs and urinary tract infections is broadly used. As with cotrimoxazole, multiple programs supply this product.
14 Epinephrine (or Adrenaline) inj 10s
Used for resuscitation and used mostly at the secondary and tertiary level. Its administration requires advanced training only found at those levels.
67
15 Fe+Fol tabs 1000s Used for the treatment of anemia caused by multiple conditions. As with cotrimoxazole, and doxycycline, this product is often supplied by multiple programs.
16 Gentamycin inj 50s As this used for treating severe bacterial infections, it’s fast moving and widely used across all health service levels
17 Gloves exam latex M
100s
Used for infction prevention, this is a multiple use item, it’s a fast moving and widely used across all health service levels
18 Glucose 50% 20s Routine IV fluid, mostly used at a secondary and tertiary level
19 Ibuprofen 200mg tabs
100s
Broad pain-reduction use and often overprescribed at all levels
20 Magnesium Sulphate inj Used for the treatment of severe high blood pressure in pregnant women with complications most often utilized at the secondary and tertiary levels
21 Mask surgical 100s The items are for surgical/theatre use which is not available at the primary level
22 Metronidazole tabs
1000s
Commonly used for treatment of multiple conditions (diarrhea, etc.) at a primary level and so is a fast-moving item.
23 Paracetamol syrup bot Used in the reduction of pain and fever caused by multiple conditions in children. This is also a fast moving item.
24 Paracetamol tabs, 500mg
1000s
Used in the reduction of pain and fever caused by multiple conditions. This is also a fast moving item.
25 Promethazine tabs 100s Used for the treatment of vomiting in adults and not widely used at the health center level.
26 Quinine 300mg tabs 100
The drug is for treatment of Malaria which is endemic in Malawi. It is meant for treatment of severe malaria, but some facilities also use it for simple malaria which is out of the treatment guidelines. The low quantity might have been put deliberately to prevent over prescription which can lead to drug resistance.
27 Quinine inj 10s
The drug is for treatment of Malaria which is endemic in Malawi. It is meant for treatment of severe malaria, but some facilities also use it for simple malaria which is out of the treatment guidelines. The is likely undersupplied because severe malaria is very common.
28 Sulphadoxine/Pyrimeth
1000s
The drug is no longer used at a first line treatment of Malaria. It is only used as a prophylaxis for Malaria in pregnancy and is provided by multiple programs like cotrimoxazole.
29 Tetracycline eye oint 5g Routine eye antibiotic, no longer commonly used.
30 Water for Inj 50s
It is used for dilution of intramuscular or intravenous drugs (injections) which is not common at a primary level. Some injectables come already packed with water. It's possible that the amount of water provided exceed the injectibles they were meant to compliment.
31 Zinc tabs 100s
While reported as "least" important, but it is a life-saving drug for children with diarrhea. Many interventions use this product, but clinicians and health workers may need more training and education on this item as it appears underutilized.
Annex 10: Tracer Item Selection Matrix
# PHC Project Item Product Description Tracked on Stock Card
Primary Care Use
Maternal and Neonatal
Commodity
Antimalarial Commodity
Multipurpose Commodities
O&M Report Tracer Item
Additional Rationale for Inclusion
1 Sulphadoxine/ Pyrimeth 1000s
Preventive treatment for malaria in pregnancy and for complicated malaria.
It represents women. Represents Box 1
2 Paracetamol, 500mg tabs
Anti-pain medication
Widely used commodity;
3 Erythromycin tabs 100s
broad-spectrum antibiotic
Represents Box 7
4 Cotrimoxazole, 480mg tabs
bacteriostatic antibiotic
480mg: Children represented
5 Syringe+needle, 2ml unit
injecting drug, for low vein calibre patients
Neonatal commodity
6 Quinine inj 10s injectable antimalarial
Only treatment for severe and complicated malaria
7 Oxytocin Inj increase uterine contraception during labor
Maternal and neonatal commodity; should be available at facilities with maternity wing. From the list of loose items
8 Amoxycillin susp bot multipurpose antibiotic, against body infection
Represents Box 4
9 Sodium chloride 500ml iv 20s
I.V infusion, increase body concentration in salt and rehydration
Represents Box 5
10 Glucose 5% 500ml iv 20s
I.V infusion, increase body concentration in sugar
Represents Box 6
Annex 11: Under and Overstocked Items and Expiries Across all health service levels, inclusive of MOH and CHAM sites, the following items were considered the
most undersupplied item in the PHC kits (alphabetized):
Amoxycillin susp bot
Amoxycillin tabs 1000
Bandage gauze 8x4m
Benzylpenicillin 5miu 50s
Calamine lot 500ml
Fe+Fol tabs 1000s
Ibuprofen 200mg tabs 100s
Metronidazole tabs 1000s
Nystatin oral susp
Paracetamol tabs, 500mg 1000s
Quinine 300mg tabs 100
Syringe 2ml w needle 100s
Across all health service levels, inclusive of both MOH and CHAM sites, the following items were considered
the most oversupplied item in the PHC kits (alphabetized):
Albendazole 400mg tabs 100s
Cotrimoxazole 480mg tabs 500s
Doxycycline tabs 1000
Erythromycin tabs 100s
Fe+Fol tabs 1000s
Glucose 50% 20s
Hydrochlorothiazide 25mg tabs 100s
Mask surgical 100s
Promethazine tabs 100s
Sulphadoxine/Pyrimeth 1000s
Tetracycline eye oint 5g
Water for Inj 50
During the evaluation site visits, when asked which kit items expired the most at their facilities the following
products, organized by service level, were mentioned:
Health Center
• Albendazole 400mg tabs 100s
• Amoxycillin tabs 1000
• Benzylpenicillin 5miu 50s
• Epinephrine (or Adrenaline) inj 10s
DHO/District Hospital
• Benzyl Benzoate lotion 1000ml
• Epinephrine (or Adrenaline) inj 10s
• Phenobarbitone inj
• Phenobarbitone tab
Mission Hospital
• Benzathine Penicillin 1.44g 50s
• Diazepam Inj
• Promethazine tabs 100s
Central Hospital
• Zinc tabs 100
When reviewing the above lists, it’s important to consider that most of the items came up only once among
the 58 facilities as having expired. Epinephrine was the product most mentioned as having expired.
While relocation was a critical mechanism of mitigating product expiry, (as noted earlier in this report) due to
the standardization of the kits the facilities) would often have an oversupply of the same item making
relocation a challenge. Amoxycillin, Benzyl Benzoate, Glucose 5% and Erythromycin were mentioned as the
products that expired because other facilities were also oversupplied with them and so they could not be
relocated.
71
Annex 12: Evaluation Surveys
Health Impact Evaluation - Health Center Survey Clinician Survey
Facility Name: Region ☐ North ☐Central
☐ South
Sector
☐MOH ☐CHAM District:
Interviewee: Date:
Title: Data Collector:
Has the clinician worked at this facility since
January 2010? ☐ A. Yes ☐ B. No
# of years the clinician worked at this facility
Evaluation Category: 1D. Project Communication
Evaluation Question: 15. Was the PHC Essential Medicines Project Communication strategy effective?
FIRST WE ARE GOING TO SPEAK BREIFLY ABOUT THE COMMUNICATION STRATEGY FROM UNICEF. UNICEF TRIED TO REACH FACILITIES AND PATIENTS TO INFORM THEM OF THE KITS PROGRAM, SO WE’D LIKE TO KNOW IF AND HOW THAT COMMUNICATION REACHED YOU AND YOUR FACILITY 1) How did you learn about the kit program? 2) What communications did your facility receive regarding the PHC kit project? 3) In general, did you feel that PHC communications reached patients and communities?
A. ☐Yes B. ☐No C. ☐Don't Know
If so, by which method? How? 4) If so, what do you think the impact this communication had was on patient behavior?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: 2A. Availability of Quality Medicines and Health Supplies
Evaluation Question: 16. To what extent were kit contents available to patients?
1) In general, did the PHC kit program increase patient access to needed essential medicines?
A. ☐Strongly Agree B. ☐Agree C. ☐Disagree
72
D. ☐Strongly Disagree E. ☐Don't Know
2) Why?
QUESTION BELOW ONLY FOR HEALTH WORKERS WHO HAVE BEEN AT THE FACILITY SINCE 2010 OR EARLIER:
3) When you compare 2010 (before the kits program) to 2013 (with the kits program), do you see a change in the level of access patients had to essential medicines?
A. ☐Yes - More Access / B. ☐Yes - Less Access / C. ☐No Change /
D. ☐NA / E.☐ Don't Know
4) If you observed a change, what do you think are the causes of this difference? 5) Since the end of the PHC Kit project what changes to patient access have you observed?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: 1C. Product Selection
Evaluation Question: 12. What was the suitability of the drugs in the kits? [Kit content appropriateness]
FOR THESE QUESTIONS
1) In general, regardless of quantity supplied, where the contents of kits appropriate for the needs of your facility? If everything came in the right quantities, was the content of the kits appropriate for your facility?
A. ☐Yes B. ☐No C. ☐Somewhat D. ☐Don't Know
2) Regardless of quantity supplied, which five kit items were the most important for assisting patients? 3) Regardless of quantity supplied, which five kit items were the least important for assisting patients? 4) Which products, which were not included, did you think should have been included in the kits?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
73
Evaluation Category: 2A. Availability of Quality Medicines and Health Supplies
Evaluation Question: 23. Was the product of a good quality when delivered?
1) Was the product of a good quality?
A. ☐Yes B. ☐No C. ☐Don't Know
2) If not, what were specific issues that occurred?
3) Have you reported any quality concerns to the DHO or PMPB? 4) Did the patients provide any feedback on the quality of the products? 5) Were patients overall happy with the quality of the products? 6) If patients had negative feedback, what were specific issues that occurred?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: 3A. Disease Management
Evaluation Question: 1. To what extent has the Project eased the management of Diarrhea, ARI, Postpartum care, syphilis, de-worming, fever, severe and complicated malaria and reduced malaria prevalence in pregnant women?
Now I am going to ask you some questions about management of common conditions at your facility. For each condition I'm interested in knowing how much you are able to manage the cases, and the primary reason you may not be able to manage some or all of the cases; 1. Diarrhea
A. ☐Able to manage all cases B. ☐Able to manage most cases
C. ☐Able to manage some cases D. ☐Not able to manage any cases
2. ARI
A. ☐Able to manage all cases B. ☐Able to manage most cases
C. ☐Able to manage some cases D. ☐Not able to manage any cases
3. Postpartum care
A. ☐Able to manage all cases B. ☐Able to manage most cases
C. ☐Able to manage some cases D. ☐Not able to manage any cases
74
4. Syphilis
A. ☐Able to manage all cases B. ☐Able to manage most cases
C. ☐Able to manage some cases D. ☐Not able to manage any cases
5. De-worming
A. ☐Able to manage all cases B. ☐Able to manage most cases
C. ☐Able to manage some cases D. ☐Not able to manage any cases
6. Fever
A. ☐Able to manage all cases B. ☐Able to manage most cases
C. ☐Able to manage some cases D. ☐Not able to manage any cases
7. Severe and complicated malaria
A. ☐Able to manage all cases B. ☐Able to manage most cases
C. ☐Able to manage some cases D. ☐Not able to manage any cases
8. For cases that you are not able to manage (listed above) , what is the primary reason:
A. ☐Lack of medicines B. ☐Lack of diagnostics
C. ☐Lack of training D. ☐Cases requiring management at a higher level (referral)
E. ☐Other, specify:________________
9. Reduced malaria prevalence in pregnant women 1) What percentage of pregnant women attending this facility receive malaria prophylaxis (estimate is fine)?
A. ☐100% B. ☐80-99% C. ☐50-79% D. ☐20-49% E. ☐1-20% F.☐ 0%
QUESTION BELOW ONLY FOR HEALTH WORKERS WHO HAVE BEEN AT THE FACILITY SINCE 2010 OR EARLIER:
3) Have you observed any reduction in prevalence of malaria among pregnant women when you
compare before the kits and after the kits?
A. ☐Yes B. ☐No C. ☐Don't Know
75
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: 3B. Health Service Utilization
Evaluation Question: 3. What impact did the kits have on patient visits to Center?
1) Do you feel that the kits program encouraged more patients to attend this facility?
A. ☐Yes B. ☐No C. ☐Somewhat D. ☐Don't Know
2) Why or why not?
Only for health workers who have been at the facility since 2010 or earlier: 3) When you compare 2010 (before the kits program) to 2013 (with the kits program), do you see a
change in the number of patients attending your facility?
A. ☐Yes -Increase B. ☐Yes-Decrease C. ☐No Change D. ☐NA E. ☐Don't Know
4) If you observed a change, what do you think are the causes of this difference? Do you think this change was related to the kits?
A. ☐Yes B. ☐No C. ☐Somewhat D. ☐Don't Know
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Overall High Level Site Findings & Additional Interviewer Notes – DO NOT LEAVE THIS SPACE BLANK
Additional Questions
a) What was the best thing (strength) about the PHC Project? B) What was the worst
(weakness)?
76
Operations Evaluation - Health Center Survey
Inventory Management Survey
Evaluation Category: Availability of Quality Medicines and Health Supplies
Evaluation Question: 17. What was the amount delivered?
1) How many kits did you receive per month?
2) Did this amount change at any point? Why?
A. ☐Yes / ☐ B. No / ☐C. Don't Know
3) Was it easy to request more/less kits?
A. ☐Strongly Agree / ☐ B. Agree / ☐C. Disagree / ☐D. Strongly Disagree /
☐E. Neither Agree nor Disagree / F. ☐Don't Know
4) Why?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Facility Name: Region ☐ North ☐ Central
☐South
Sector: ☐MOH ☐CHAM District:
Interviewee: Date:
Title: Data Collector:
Has the interviewee worked at this facility since January 2010?
☐ A. Yes ☐ B. No
# of years/months the interviewee worked at this
facility
77
Evaluation Category: Supply Chain Efficiency and Effectiveness
Evaluation Question: 6. What was the performance against agreed-upon delivery schedules?
1) When did you expect the kits to arrive?
2) Do you agree with this statement?: “The kits arrived according to my delivery expectations”
A. ☐Strongly Agree / ☐ B. Agree / ☐C. Disagree / ☐D. Strongly Disagree /
☐E. Neither Agree nor Disagree / F. ☐Don't Know
3) Do you agree with this statement?: “ I was informed of expected delivery dates in advance.”
A. ☐Strongly Agree / ☐ B. Agree / ☐C. Disagree / ☐D. Strongly Disagree /
☐E. Neither Agree nor Disagree / F. ☐Don't Know
4) Do you agree with this statement?: “ I received the expected quantities [of kits] with each delivery.”
A. ☐Strongly Agree / ☐ B. Agree / ☐C. Disagree / ☐D. Strongly Disagree /
☐E. Neither Agree nor Disagree / F. ☐Don't Know
5) If they were late, in general, by how many days/weeks?
A. ☐1-2 days/ B. ☐3-4 days / C. ☐5-6 days / D. ☐7-8 days /
E. ☐More than a week / F. ☐ 10 days / G. ☐Over 2 weeks /
H. ☐Over a month / ☐I. N/A
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: Supply Chain Efficiency and Effectiveness
Evaluation Question: 10. How did you store the kits and what were the storage conditions?
1) Did you have any storage challenges?
A. ☐Yes / ☐ B. No / ☐C. Don't Know
2) If so, what were they?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
78
Evaluation Category: Supply Chain Efficiency and Effectiveness
Evaluation Question: 11. How were the kit contents managed with the other stock? Did it change your inventory practices or recordkeeping?
1) How were the kit contents managed with the other stock? 2) What integration issues emerged as a result of these two essential medicines delivery systems?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: Commodity Management Capacity
Evaluation Question: 5. What was the impact of the PHC program on health facility worker inventory management or recordkeeping skills?
1) Were there any training or mentorship provided specifically related to the inventory management of the kits?
A. ☐Yes / ☐ B. No / ☐C. Don't Know
2) Did the kits program impact your inventory management system?
A. ☐Yes / ☐ B. No / ☐C. Don't Know
If so, how?
3) Do you agree with this statement?: “The kit program improved the inventory management skills of health facility staff.” [Record keeping, organized stock rooms, LMIS reporting, etc.]
A. ☐Strongly Agree / ☐ B. Agree / ☐C. Disagree / ☐D. Strongly Disagree /
☐E. Neither Agree nor Disagree / F. ☐Don't Know
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: Availability of Quality Medicines and Health Supplies
Evaluation Question: 21. Was the product availability adequate for the patient population? [Quantity question]
1) Did the quantities in the kit, in general, meet patient needs?
A. ☐Yes / ☐ B. No / ☐C. Don't Know
2) Which 3 items in the kit were undersupplied in terms of quantity? 3) Which 3 items in the kit were oversupplied in terms of quantity?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK: 3 Undersupplied:
79
3 Oversupplied:
Evaluation Category: Availability of Quality Medicines and Health Supplies
Evaluation Question: 19. Where there stockouts of essential medicines in your facility?
1) In general, did stockouts of essential medicines and supplies occur?
A. ☐Frequently / B. ☐Sometimes / C. ☐Never / D. ☐Don't Know
2) 2) Which five commodities were the most often stocked out? [List below] 3) 3) Of those five, how often did they stockout?
3A) Item 1: _________________________________________
A. ☐Frequently / B. ☐Sometimes / C. ☐Never / D. ☐Don't Know
3A) Item 2: _________________________________________
A. ☐Frequently / B. ☐Sometimes / C. ☐Never / D. ☐Don't Know
3A) Item 3: _________________________________________
A. ☐Frequently / B. ☐Sometimes / C. ☐Never / D. ☐Don't Know
3A) Item 4: _________________________________________
A. ☐Frequently / B. ☐Sometimes / C. ☐Never / D. ☐Don't Know
3A) Item 5: _________________________________________
A. ☐Frequently / B. ☐Sometimes / C. ☐Never / D. ☐Don't Know
4) Why do you think these stockouts occurred? 5) Do you agree with this statement?: The kits program prevented product stockouts at your facility?
A. ☐Strongly Agree / ☐ B. Agree / ☐C. Disagree / ☐D. Strongly Disagree /
☐E. Neither Agree nor Disagree / F. ☐Don't Know
6) What impact did the kits program have on stockouts? [Definition of stockout: When a product is not available in facility inventory, regardless of duration of absence]
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Evaluation Category: Availability of Quality Medicines and Health Supplies
Evaluation Question: 20. What did you do when a kit product stocked out?
80
1) What did you do when a kit product stocked out?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK
Evaluation Category: Availability of Quality Medicines and Health Supplies
Evaluation Question: 22. Did any of the kit contents arrived expired or expire during its storage?
1) Did any of the kit contents arrive expired?
A. ☐Yes / ☐ B. No / ☐C. Don't Know
2) If so, what did you do with them? 3) What 3 kit items expired [during storage at the facility] the most? Why? What did you do with them? What did they do with stock that was nearing expiry?
Response / Interviewer Notes - NOTES SHOULD ALIGN WITH THE QUESTION NUMBERS LISTED ABOVE – DO NOT LEAVE THIS SPACE BLANK:
Overall High Level Site Findings & Additional Interviewer Notes – DO NOT LEAVE THIS SPACE BLANK
Additional Questions
Aside from UNICEF and CMST, did this facility receive or procure essential medicines and
supplies from other sources and supplies?
What was the process for transferring stock that was near expiry?
Stock Card Data Collection Section
Box 1 - Quinine inj 10s
M
ay
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
Box 2 - Sulphadoxine/Pyrimeth 1000s
May
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
Box 2 - Paracetamol, 500mg tabs
M
ay
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
2013
Ap
r
201
3
Total Amount Received
Notes
82
Box 2 - Syringe+needle, 2ml unit
May
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
2012
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
2013
Ap
r
201
3
Total Amount Received
Notes
Box 3 - Cotrimoxazole, 480mg tabs
M
ay
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
Box 4 - Amoxycillin susp bot
M
ay
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
83
Box 5 - Sodium chloride 500ml iv 20s bottles
May
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
Box 6 - Glucose 5% 500ml iv 20s bottles
M
ay
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
Box 7 - Erythromycin tabs 100s
M
ay
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
84
Box 8 / Loose - Oxytocin Inj
M
ay
201
2
Ju
n
201
2
Ju
l
201
2
Au
g
20
12
Sep
2012
Oct
2012
No
v
2012
Dec
2012
Jan
2013
Feb
2013
Mar
20
13
Ap
r
201
3
Total Amount Received
Notes
Additional Notes
Annex 13: Bibliography
Bornbusch, Alan and James Bates. “Multiplicity in Public Health Supply Systems: A Learning
Agenda.” Global Health: Science and Practice 2013, Volume 1, Number 2
Kassebaum et al. “Global, regional, and national levels and causes of maternal mortality during 1990–2013: a systematic analysis for the Global Burden of Disease Study 2013” Lancet. Published Online May 2, 2014. Management Sciences for Health. “Managing Access to Medicines and Health Technologies.”
Chapter 26: Kit System Management, 2012.
Maternal Mortality Estimation Inter-Agency Group, 2014. Maternal mortality in 1990-2013,
Malawi. Downloaded on May 13, 2014.
Ministry of Health – Government of Malawi, December 2013. Health Management Information
Bulletin: Annual Report July 2012 – June 2013.
Ministry of Health – Government of Uganda, 2011. Assessment of the medicines kit-based supply
system in Uganda.
“Conducting Spot Checks in Health Facilities.” First, Second, Third, Fourth, Fifth, Sixth, Seventh &
Eight Cycle Reports. Prepared by O&M IT/S and Management Services Limited. June 2012 – January
2014
ORC Macro. Malawi Demographic and Health Survey. 2010.
UNICEF, September 2013. Committing to Child Survival: A Promise Renewed: Progress Report 2013
UNICEF, November 2012. Primary Health Care Essential Medicines Support for Malawi –
Consolidated Progress Report.
UNICEF, June 2013. Primary Health Care Essential Medicines Support for Malawi – Consolidated
Progress Report.