Post on 15-Jun-2018
A quasi-experimental study
Impact of a Health Governance
Intervention on Provincial Health
System Performance in Afghanistan
Author
Mahesh Shukla, MD, DrPH, MPA,
Senior Technical Advisor, Management Sciences for Health
Photo: Jawad Jalali/Afghan Eyes
RESEARCH
REPORT
Funding was provided by the United States Agency for International Development (USAID) under Cooperative
Agreement AID-OAA-A-11-00015. The contents are the responsibility of the Leadership, Management, and
Governance Project and do not necessarily reflect the views of USAID or the United States Government.
About the LMG Project
Funded by the US Agency for International Development (USAID), the Leadership, Management, and
Governance (LMG) Project (2011–2017) is collaborating with health leaders, managers, and
policymakers at all levels to show that investments in leadership, management, and governance lead
to stronger health systems and improved health. The LMG Project embraces the principles of
country ownership, gender equity, and evidence-driven approaches. Emphasis is also placed on good
governance in the health sector—the ultimate commitment to improving service delivery and
fostering sustainability through accountability, engagement, transparency, and stewardship. Led by
Management Sciences for Health (MSH), the LMG consortium includes Amref Health Africa,
International Planned Parenthood Federation (IPPF), Johns Hopkins University Bloomberg School of
Public Health (JHSPH), Medic Mobile, and Yale University Global Health Leadership Institute (GHLI).
Table of Contents
List of Abbreviations and Acronyms ............................................................................................................................. 4
1. Abstract ........................................................................................................................................................................... 5
Background ..................................................................................................................................................................... 5
Methods and Findings ................................................................................................................................................... 5
Conclusion ...................................................................................................................................................................... 5
Keywords ........................................................................................................................................................................ 6
2. Problem Statement ....................................................................................................................................................... 6
3. Institutional Context .................................................................................................................................................... 6
4. Methodology .................................................................................................................................................................. 8
Intervention summary .................................................................................................................................................. 8
Measures of health system performance ................................................................................................................. 9
Governance measurement ........................................................................................................................................ 10
5. Sample Size ................................................................................................................................................................... 11
6. Model ............................................................................................................................................................................. 12
7. Results ............................................................................................................................................................................ 12
Governance measurement results .......................................................................................................................... 14
Health system performance ..................................................................................................................................... 15
Difference-in-differences regression analysis ....................................................................................................... 24
Focus group discussions ............................................................................................................................................ 31
8. Discussion ..................................................................................................................................................................... 33
Limitations..................................................................................................................................................................... 35
Strengths ....................................................................................................................................................................... 36
9. Conclusion .................................................................................................................................................................... 36
Areas for future research.......................................................................................................................................... 37
10. Implication for Practice ........................................................................................................................................... 37
References ......................................................................................................................................................................... 38
Appendix 1 ........................................................................................................................................................................ 40
Overall Health Governance Self-Assessment Tool for the PPHCC ............................................................... 40
Health Governance Standards-based Self-Assessment Tool for the PPHCC .............................................. 42
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
4
List of Abbreviations and Acronyms
ANC Antenatal care
CDR Case detection rate
CHW Community health worker
DD Difference-in-differences
HMIS Health management information system
LMG Leadership, Management, and Governance Project
MDG Millennium development goals
MOPH Ministry of public health
NATO North Atlantic Treaty Organization
NGO Nongovernment organization
OPD Outpatients department
PNC Postnatal care
PPHCC Provincial public health coordination committee
SDG Sustainable development goals
SY Solar year
TB Tuberculosis
TT2+ Two doses of Tetanus Toxoid
UNICEF United Nations Children’s Fund
USAID United States Agency for International Development
WHO World Health Organization
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
5
1. Abstract
Background
Poor health governance contributes to poor health outcomes and may constrain a country’s progress in
attaining the ambitious health targets of the Sustainable Development Goals. Yet, governance has rarely
been used as a lever to improve the performance of the health sector and health systems. Lack of a
clear body of evidence linking governance interventions to better health system performance is one
likely reason. Evidence of this type is not only limited, but is often anecdotal and lacks a control or
comparison group.
This dearth of evidence means that donors are reluctant to invest in governance interventions, and
governments are at a loss as to where to start. The USAID-funded Leadership, Management, and
Governance (LMG) Project conducted the present study in Afghanistan to examine the causal impact of
a provincial-level health governance intervention on the performance of provincial health systems. The
intervention consisted of governance action planning, implementation of the action plan, and self-
assessment of governance performance before and after the intervention.
Methods and Findings
This quasi-experimental study compares nine health system performance indicators between 16
intervention provinces and 18 non-intervention provinces using a difference-in-differences analysis to
draw inference.
The authors found a statistically and practically significant impact of the intervention on six indicators.
Specifically, the intervention increased a province’s rate of outpatient department visits per person by an
average of 18 percentage points (p<0.01), and achievements in Penta 3 immunization by 17 percentage
points (p<0.01), antenatal visits by 14 percentage points (p<0.01), postnatal visits by 12 percentage
points (p<0.01), tuberculosis case detection by 11 percentage points (p<0.01), and facility delivery by 5
percentage points (p<0.01). No impact was detected on the achievements in tetanus toxoid
administration to pregnant mothers and tuberculosis cure. The secular trend was by far a stronger and
more significant predictor of increasing rates of tetanus toxoid administration to pregnant mothers.
Achievements in community health worker home visits and new family planning users decreased
(p<0.01) by 2 and 1 percentage points respectively. The size of the positive effects is large while that of
the negative effects is small.
Conclusion
Governing bodies at different levels provide an opportunity for governance reform. In this study, a
governance intervention implemented by provincial health coordination committees had a beneficial
impact on provincial health system performance. This indicates the value of increasing the number of
health governance interventions in different health system settings and ensuring that such interventions
are subjected to rigorous impact evaluations.
These study results are relevant to the prevailing situation in low-income countries, where weak health
systems are a significant barrier to addressing outbreaks and epidemics and to providing high quality
health services. Improving the governance of health systems is one way to better enable them to meet
their objectives.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
6
Keywords
Governance and health, health governance intervention, health system governance, health system
performance, governance impact
2. Problem Statement Governance has infrequently been used as a lever to improve health sector and health system
performance, arguably because of its perceived sensitivity and complexity [1, 2, 3, 4, 5]. Sustainable
Development Goal (SDG) 16 emphasizes improving governance [6]. Poor governance has contributed
to poor health outcomes [7, 8, 9, 10, 11] and may constrain progress in reaching the SDG 3 health
targets. Good governance could be instrumental in achieving these ambitious health targets [12, 13].
One of the reasons that donors have lacked interest in investing in governance interventions as a way to
improve health system performance is that no clear body of evidence exists on the causal relationship
between the two, especially in low- and middle-income country settings [14]. Governments are at a loss
about where to start. Evidence is sparse and oftentimes anecdotal, or is limited to before and after
studies, usually with no control or comparison group. Our review of studies included in a recent
systematic review [14] showed that most stop at demonstrating association and do not examine
causation. Donors are eager for the evidence demonstrating the link of better governance to
improvements in health system performance; governments continue to remain largely indifferent
because of the politically sensitive nature of governance reforms, or because they do not know where
to begin such reforms.
This quasi-experimental study examines the causal relationship between a provincial-level health
governance intervention and provincial health system performance by comparing performance in
intervention and non-intervention provinces using a difference-in-differences methodology to draw
inference. The study contributes critical empirical evidence to a technical area in need of such. Our
findings indicate the potential of improved governance to improve health system performance. We
conclude that policymakers whose goal is to improve health system performance or health service
delivery should consider implementing interventions to improve governance of the health sector and
health systems.
3. Institutional Context In Afghanistan, shuras, or informal consultative assemblies of elders, have a long and well-established
tradition of resolving disputes and solving contentious issues in communities. By contrast, the Provincial
Public Health Coordination Committee (PPHCC) is a formal multi-stakeholder committee that was
established in the early 2000s by the Ministry of Public Health (MOPH) to carry out a set of distinct
responsibilities at the provincial level [15]. PPHCCs have been functional since that time in all 34 of
Afghanistan’s provinces. Their level of functionality varies from province to province, with the majority
tending to be weak and ineffective. PPHCCs provide a forum for coordination and information sharing
among various stakeholders in the provincial health system. They discuss community health concerns,
and coordinate and participate in all stages of emergency response. They also oversee health posts and
health facilities by conducting joint monitoring and supervision visits during which a subset of committee
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
7
Box 1: Three-phase governance intervention
First phase: Baseline measurement of governance and participatory development of health systems governance
development action plans
The PPHCC teams participated in 2-day workshops facilitated by public health and governance experts. The
first day was spent working in groups discussing actions to apply the effective governing practices (cultivating
accountability, engaging stakeholders, setting a shared strategic direction, and stewarding resources) in their
work over the next six months to better meet the health needs and expectations of the people. Using the
governance guide and the framework of governing practices, each committee identified actions to be taken in
the next six months to improve their governance and developed a governance action plan. The four practices
provided the organizing framework to structure their committee deliberations and governance action plan. On
the second day, participants carried out a self-assessment of their governance performance at baseline.
Second phase: Implementation and monitoring
In implementing their governance action plans, the PPHCCs worked to improve engagement with the public
and communities, and to become more transparent, accountable, and responsive. No additional resources
were made available to the provinces and districts to carry out their planned activities. Committees monitored
implementation of their action plans so that underperformance could be identified and corrected along the
way. The PPHCCs monitored actions in the plan on a monthly basis using a simple Excel-based monitoring
instrument. Progress reports were sent to the Provincial Liaison Directorate of the MOPH.
The PPHCCs monitored the progress by the extent to which actions were implemented. Progress on an action
or activity was classified in five categories: Not started (0%), early stage (1-25% of an action is completed), two
intermediate stages (26-50% or 51-75% of an action is completed), and advanced stage of completion (76-100%
of an action is completed).
Third phase: Evaluation
The PPHCCs evaluated their performance during the governance action plan implementation period in 2-day
workshops held six months after the implementation began. The PPHCC re-assessed their governance, as a
committee, using the same instruments they had used at baseline.
members joins the provincial health team. They are expected to meet on a monthly basis and coordinate
delivery of the basic package of health services as well as the essential package of hospital services. In
essence, this committee governs the provincial health system. That said, it should be noted that their
governance role is limited—they are an advisory, consultative and coordinating body. They have
oversight authority, but no authority to hire and fire. Setting strategy, making rules and regulations,
establishing policy and procedures, and allocating resources are all tasks predominantly performed by
the Ministry and the government at the national level.
The PPHCC is a multi-sectoral governing body chaired by the Provincial Public Health Director. It has
21 members comprising nine appointed provincial public health officers, the provincial hospital director,
the director of the Institute of Health Sciences, two representatives of NGOs that provide health
services at the health post and health facility levels, two district health officers, and one representative
from each of the following: The Ministry of Women’s Affairs, the Provincial Reconstruction Team, the
private health sector, the elected provincial council, UNICEF, and WHO. Thirteen members have voting
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
8
powers, including six officials from the provincial public health office; the provincial hospital director; and
members representing the private health sector, provincial council, UNICEF, WHO, and NGOs.
Decisions made in the PPHCC are usually consensus-based. When consensus is not reached, a decision
is put to a vote. A decision requires a quorum and a majority of members voting in favor. The members
are not paid a salary and do not receive sitting fees for serving on the PPHCC. The governance
development intervention in this study described in the methods section was primarily implemented by
the PPHCCs in their provinces.
4. Methodology This study examines the causal impact of a provincial health governance intervention on provincial health
system performance in Afghanistan. The study design is quasi-experimental and uses Health Management
Information System (HMIS) data to isolate impact by comparing performance in provinces with the
governance intervention to those without, using a difference-in-differences analysis strategy. We
hypothesized that the governance intervention improved health system performance.
We use difference-in-differences analysis to draw inference. In this methodology, outcomes are
observed for two groups for two time periods: pre- and post-intervention. One group is exposed to an
intervention while the other is not. The difference between post- and pre-intervention in the control
group is subtracted from the difference in the intervention group. This methodology removes biases in
the post-period comparisons between the intervention and control group that could result from
permanent differences between the groups, as well as biases from comparisons over time in the
intervention group that could be the result of a time trend [16].
Intervention summary
In each intervention province, the governance intervention was conducted in three phases over a period
of six months, and targeted four essential governing practices: cultivating accountability, engaging
stakeholders, setting a shared strategic direction, and stewarding resources.
In the first phase, using the provincial health system governance guide and the framework of essential
governing practices (http://www.lmgforhealth.org/content/key-practices-good-governance), the PPHCCs
identified actions to be taken to improve the governance of their provincial health system, and designed
a governance action plan for this purpose. They also measured the committee’s governance
performance at baseline using self-assessment instruments. In the second phase, the committees
implemented and monitored their action plans over a period of six months. In the third and final phase,
the committees evaluated the implementation of their action plan and measured governance
performance post-intervention using the same self-assessment instruments. At this stage, they also
examined each of the action items in the action plan to assess progress, and to assess if they planned on
continuing their implementation beyond the initial six-month period. See Box 1 for the three phases of
the intervention and Table 1 for examples of activities selected by the PPHCCs to improve their
governance. Throughout the intervention period, a donor-funded provincial public health advisor who
was already embedded in the provincial public health office to provide technical assistance in public
health management supported the PPHCC in the implementation of their governance development plan
and its monthly monitoring. We described this intervention and its pilot testing in detail in an earlier
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
9
journal publication [15]. In order to obtain feedback on the intervention at the end of pilot
implementation in the initial three provinces, we held focus group discussions with the PPHCC
members of these three provinces.
Table 1 illustrates actions planned by the PPHCCs.
Table 1: Examples of activities selected by the PPHCCs to implement their governance action plan
Governing practice Examples of activities
Improve stakeholder engagement Interview patients and health service users
Invite religious, youth, and women leaders to meetings
Provide feedback to consultative assemblies at health facility level
Consult community leaders on a regular basis
Cultivate accountability Share information on resources and performance with communities
and stakeholders
Encourage health workers to share their challenges during joint
monitoring visits
Review health workers’ job descriptions and provide clear service
delivery targets
Set a shared strategic direction Constitute a team of representatives from the community, health
service users, other health system stakeholders, and district health
officers from each district to identify the health needs and challenges
faced by the communities, and to communicate these needs to the
PPHCC for consideration while deciding their strategic direction
Invite health facility shura members to the PPHCC meetings to better
understand community health concerns
Steward resources responsibly Train provincial public health office staff and health workers in ethical
conduct
Recognize health workers with outstanding performance
Involve the community in health facility monitoring
Use data, information, evidence, and technology for decision making
Measures of health system performance
The focus of the study is health system performance, since it has a direct influence on population health
outcomes. The governance intervention was not undertaken to improve governance for the sake of
governance. Instead, the intention was to improve organizational or health system performance through
improvements in governance. For this reason, the outcomes of interest we selected were health system
performance indicators in addition to governance self-assessment scores.
We collected HMIS data on percentage achievement against the MOPH-established targets on nine
indicators of health systems performance: 1) proportion of children less than one year of age who
received the Penta3 vaccine; 2) proportion of pregnant women who received at least one antenatal care
visit; 3) proportion of women who received at least one postnatal care visit; 4) proportion of pregnant
women who received two doses of tetanus toxoid; 5) proportion of deliveries conducted at a health
facility; 6) outpatient visit rate per person; 7) community health worker home visit rate; 8) proportion
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
10
of new family planning users in target population; and 9) TB case detection rate. In addition, the TB cure
rate was chosen as a health outcome variable because TB is highly prevalent and is a public health
priority in Afghanistan.
Why did we select these indicators? We expected to see a change in maternal and child health
indicators because maternal and newborn care and child health and immunization are the top two
priority elements among the seven elements of the Basic Package of Health Services [17], which is a
mainstay of primary health care in Afghanistan. Likewise, family planning and tuberculosis control are top
public health priorities of the MOPH and provincial health systems in Afghanistan. Any systemic
improvement in the provincial and district health systems, for example its governance, was therefore
expected to be reflected in the indicators related to maternal and child health, family planning and
tuberculosis.
These indicators were also selected because they are proxies of overall health system performance.
Health system strengthening interventions focusing on improving leadership, management, governance,
or health financing will have a direct bearing on the coverage of health services and on access to the
health services the system delivers. Hence, we expected to see improvement in these indicators.
Governance measurement
We developed two self-assessment instruments to measure governance performance and to be used by
committees as a whole. These instruments were based on the extant role of the committees, and also
the expanded role they aspired to take on to make their governance more effective and people-
centered. The two measurement instruments are summarized in Table 2 and are available in full in the
Appendix 1.
One of the two instruments was for self-assessment of performance on the PPHCC governance
responsibilities. The PPHCCs graded their own performance on a 1-10 scale on each responsibility of
the committee. The other instrument helped the PPHCCs self-assess their governance against
established standards in 11 provincial public health core functions. No progress on a standard was
scored 0, 1-25% accomplishment was scored 1, 26-50% was scored 2, 51-75% was scored 3, and 76-
100% was scored 4.
Table 2: PPHCC governance measurement instruments used in the study
#
Name of the
governance self-
assessment scale
Based on Administered to
whom
Frequency of
administration
What is
measured
1
Overall PPHCC
governance
performance
assessment scale
Roles and
responsibilities of
the PPHCC PPHCCs (PPHCC
as a whole did
collective self-
assessment)
Pre- and Post-
intervention
PPHCC
governance
performance
2
Governance
standards- based
scale
PPHCC governance
standards
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
11
Intervention timeline
Afghanistan has 34 provinces. Of these, 16 provinces implemented the governance intervention.1 The 18
non-intervention provinces served as comparison provinces.2 The MOPH and the U.S. Agency for
International Development’s Leadership, Management, and Governance (LMG) Project, which assists the
MOPH, selected provinces based on where the LMG Project was active. Thus, the selection of
provinces was purposeful instead of random.
Figure 1: Intervention timeline
The provinces were staggered in their implementation of the governance intervention over a period of
27 months—three provinces implemented it from the month of April to October of 2012 [Sawar to
Mizan of Solar Year (SY) 1392], nine provinces did so from the month of July to December of 2013
[Saratan to Qaws of SY 1393], and the remaining four provinces implemented it from the month of
February to July of 2014 [Dalwa of SY 1393 to Saratan of SY 1394]. Figure 1 depicts the time line in
terms of the Gregorian calendar.
5. Sample Size Province-month or the month in which a province was observed is the unit of analysis for analyzing the
impact of intervention on the health system performance indicators. We had 13 months of pre-
intervention data when none of the provinces had the intervention introduced. Similarly, we had eight
months of post-intervention data when all 16 intervention provinces had completed their intervention.
Thus there were a total of 714 province-months in the analysis sample ((16+18) provinces*(13+8)
months).
1 Intervention provinces (16): Badakhshan, Baghlan, Bamyan, Farah, Faryab, Ghazni, Helmand, Hirat, Jawzjan, Kandahar,
Khost, Nimroz, Paktika, Paktya, Takhar, and Wardak 2 Comparison provinces (18): Badghis, Balkh, Dykundi, Ghor, Kabul, Kapisa, Kunar, Kunduz, Laghman, Logar, Nangarhar,
Nooristan, Panjsher, Parwan, Samangan, Sar-e-Pul, Urozgan, and Zabul
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
12
6. Model We used the simplest set up where outcomes are observed for two groups for two time periods. One
of the groups (intervention provinces) is already exposed to the treatment (provincial health governance
intervention) in the post-period but not in the pre-period. The second group (comparison provinces) is
not exposed to the treatment during either period. Since we observe the same provinces within the
group in each time period, the average gain in the comparison provinces is subtracted from the average
gain in the intervention provinces. This removes biases in second period comparisons between the two
groups that could be the result of permanent differences between those groups, as well as biases from
comparisons over time in the treatment group that could be the result of time trends.
Data varies by province (i), and month (t). Outcome is Yit, and there are two periods: pre and post. The
intervention will occur in a group of provinces. There are three key variables: Interventionit, Postit, and
Interventionit*Postit (or intervention*post).
Yit = β0 + β1 Interventionit + β2 Postit + β3 (Interventionit*Post)it + εit .......................................(1)
where
Interventionit =1 if observation i belongs to the province that will eventually be treated
Postit =1 in the period when the intervention had already occurred
Interventionit*Postit is the interaction term i.e., intervention provinces after the intervention
The coefficient of our interest is β3, which multiplies the interaction term, (Interventionit*Post)it. β3
denotes the true effect of treatment or the effect of treatment on the treated.
We also use the following extended form of this model that controls for more variables:
Yit = β0 + β1 Interventionit + β2 Postit + β3 (Interventionit*Post)it + β4 Povertyit + β5 Labor Participation
Rateit + β6 Female Literacy Rateit + β7 Securityit + εit .................................................................(2)
where
Poverty is the poverty headcount rate in the province, and security denotes the security rating
in the province based on a scale of 1 (worst security) to 5 (acceptable security as decided by
NATO during this time for transition purposes). The poverty headcount rate, labor participation
rate, and female literacy rate were added because of their likely influence on the outcome
variables. Data on these rates were taken from provincial briefs published by the World Bank in
June 2011.
7. Results
Governance action plan implementation
The PPHCCs implemented many specific governance actions. For example, they invited religious, youth,
and women leaders to PPHCC meetings; provided feedback to health facility consultative council
members; ensured they better assessed, understood and addressed community health concerns; shared
information on resources and performance with communities and stakeholders; encouraged health
workers to share their challenges during joint monitoring visits; recognized health workers for
outstanding performance; involved the community in health facility monitoring, and used data,
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
13
information, and evidence for PPHCC decision making. At the end of the six-month intervention period,
they assessed their performance in implementing these governance actions.
Figure 2 depicts the progress of the intervention provinces in implementing governance action plans at
the end of the initial six-month implementation phase. As mentioned earlier, the provinces were
staggered in their implementation of the intervention. Three provinces that implemented the
intervention in the first phase, five provinces that implemented the intervention in the second phase, and
four provinces that implemented the intervention in the third phase are shown in the chart through
three bars. Four provinces that implemented the intervention in the fourth and final installment did not
carry out the post-intervention self-assessment, and hence do not appear in the chart.
The figure shows that the intervention provinces reported a high level of completion of their
governance development action plans. On average, the intervention provinces completed 55% of their
action plan activities in six months, and an additional 29% were at an intermediate stage of completion.
The committees planned on continuing more than 90% of these actions in the future beyond the original
six-month implementation period.
Figure 2: Progress in implementing governance action plans at the end of the initial six-month
implementation period
58% 58% 50%
55%
20% 13%
15%
16%
10%
14%
16%
13%
6% 4%
8% 6%
7% 12% 12% 10%
0%
20%
40%
60%
80%
100%
Wardak, Khost
and Herat
(Phase 1
provinces)
Takhar, Baghlan,
Badakshan,
Jawzjan and
Faryab (Phase 2
provinces)
Ghazni, Paktya,
Helmand and
Paktika (Phase 3
provinces)
Overall for 12
provinces
Percentage of activities not started
Percentage of activities that did not
progress much
Percentage of activities half
complete
Percentage of activities at advanced
stage
Percentage of activities near
complete or complete
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
14
Governance measurement results
PPHCCs of the intervention provinces self-assessed their governance performance at baseline and again
after six months of intervention (See Table 3 and Figure 3). PPHCCs of the non-intervention provinces
did not do these self-assessments. Overall, we found significant improvements in self-assessed
governance scores. PPHCCs improved their governance score on average by 23.7% and 22.6% using two
different scales (one based on roles and responsibilities [47 items; = 0.91] and the other based on
governance standards [46 items; = 0.97]). The overall improvement upon adding the scores from both
scales is 23.4%. Improvement was higher in the provinces that began with a lower baseline governance
score.
Table 3: PPHCC governance self-assessment scores in the intervention provinces
# Province PPHCC overall
governance performance
scale based on PPHCC
roles and responsibilities
(Maximum score 450)
Governance standards- based
scale for the PPHCC
(Maximum score 184)
PPHCC overall governance
performance
(Maximum score 634)
Pre Post Percent Point
Change
Pre Post Percent Point
Change
Pre Post Percent Point
Change
1 Badakhshan 265 363 22 90 175 46 355 538 29
2 Baghlan 285 415 29 112 119 4 397 534 22
3 Bamyan 306 * 119 * 425 *
4 Farah 205 * 94 * 299 *
5 Faryab 253 346 21 85 145 33 338 491 24
6 Ghazni 153 375 49 66 146 43 219 521 48
7 Helmand 188 381 43 77 129 28 265 510 39
8 Hirat 300 331 7 109 124 8 409 455 7
9 Jawzjan 253 382 29 95 129 18 348 511 26
10 Kandahar 312 * 148 * 460 *
11 Khost 255 351 21 96.5 143 25 351.5 494 22
12 Nimroz 256 * 101 * 357 *
13 Paktika 250 293 10 95 99 2 345 392 7
14 Paktya 213 416 45 101 150 27 314 566 40
15 Takhar 356 347 -2 133 159 14 489 506 3
16 Wardak 256.5 308 11 91.5 132 22 348 440 15
Overall
percentage point
improvement in
governance
23.7 22.6 23.4
*Four intervention provinces, i.e., Bamyan, Farah, Kandahar, and Nimroz did not do the governance self-assessments.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
15
The intervention provinces scored higher on both the governance measurement scales. Out of a total
score of 450 of the scale based on PPHCC roles and responsibilities, the intervention provinces scored
higher after going through the governance intervention (359 ± 38) as opposed to before the
intervention (252 ± 52); thus producing a statistically significant increase of 107 (95% CI, 61 to 153),
t(11) = 5.11, p = 0.0002, d=2.34. Effect size is large and significant (Cohen's d = 2.34).
Similarly, out of a total score of 184 of the scale based on PPHCC governance standards, the
intervention provinces scored higher after going through the governance intervention (137.5 ± 20) as
opposed to before the intervention (96 ± 17); resulting in a statistically significant increase of 42 (95%
CI, 25 to 58), t(11) = 5.52, p = 0.0001, d=2.23. Effect size is large and significant (Cohen's d = 2.23).
Figure 3: PPHCC governance self-assessment scores, pre- and post-intervention
Total possible governance score = 634
Scales used: PPHCC overall governance performance scale (maximum score 450) and governance standards- based scale (maximum
score 184)
PPHCCs improved their governance score by an average of 23.4%. Improvement was higher in the provinces that started with a lower
baseline score on governance.
In the study, the governance intervention was a means to improve health system performance.
Having seen the results of governance measurement, let us now turn to the results of health
system performance measurement.
Health system performance
Descriptive statistics
As shown in Table 4, over the 48-month period for which HMIS data was collected, the performance of
provinces (both intervention and comparison) was more than 200% of the MOPH-established target for
the outpatient department (OPD) visit rate per person and tetanus toxoid administration. However, this
figure may be the result of the MOPH establishing a very low target for these indicators. In the
particular case of tetanus toxoid administration, it may also be due to the health staff reporting each of
the two doses separately. Performance was also more than 100% of the MOPH-established target for
antenatal care visits and Penta3 immunization. Provinces performed less than 100% of the MOPH-
0
100
200
300
400
500
600
700
Ghaz
ni
Helm
and
Pak
tya
Far
yab
Pak
tika
Jaw
zjan
War
dak
Khost
Bad
akhsh
an
Bag
hla
n
Hir
at
Tak
har
Pre
Post
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
16
established target for postnatal care visits, facility delivery, and TB case detection rate and cure rates;
and less than 50% of the MOPH-established target for community health worker home visits and new
family planning users.
Table 4: Indicator-wise average achievement (%) against MOPH-set targets
# Indicator All provinces Intervention Provinces Comparison Provinces
Obs M (SD) 95% CI Obs M (SD) 95% CI Obs M (SD) 95% CI
1 penta3 1632 135 (42) [133, 137] 768 136 (44) [133, 139] 864 134 (39) [132, 137]
2 anc 1632 121 (33) [120, 123] 768 124 (34) [122, 126] 864 119 (32) [117, 121]
3 pnc 1632 84 (29) [82, 85] 768 85 (26) [84, 87] 864 82 (32) [80, 84]
4 delivery 1632 61 (25) [60, 62] 768 64 (25) [62, 65] 864 59 (25) [57, 60]
5 tt2+ 1632 306 (122) [300, 312] 768 295 (115) [287, 303] 864 316 (127) [307, 324]
6 opd 1632 221 (59) [218, 224] 768 209 (51) [206, 213] 864 231 (65) [227, 235]
7 chw hvr 1632 31 (17) [30, 32] 768 39 (16) [38, 40] 864 24 (15) [23, 25]
8 newfp 1632 20 (8) [19, 20] 768 19 (8) [19, 20] 864 21 (8) [20, 21]
9 tbcdr 1632 75 (38) [73, 77] 768 84 (37) [81, 86] 864 68 (38) [65, 70]
10 tbcure 1632 77 (46) [75, 79] 768 81 (47) [78, 84] 864 74 (45) [71, 77]
Notes:
a) Obs is the number of observations in terms of the number of province-months
b) M is the mean and SD is standard deviation. These numbers are the percentage achievement against MOPH-
established targets. For example, Penta3 immunization = 135 indicates that intervention provinces had achieved 135
percent of the target set by the MOPH for immunization of children less than one year of age with three doses of
pentavalent vaccine.
c) Penta3 (penta3) is the percentage achievement among children less than one year of age who received the third dose
of pentavalent vaccine (combined vaccine with five individual vaccines conjugated into one, intended to protect people
from Haemophilus Influenza type B, whooping cough, tetanus, hepatitis B and diphtheria), calculated against MOPH-
established target
d) TT2+ rate (tt2+) is the percentage achievement among pregnant women who received the TT2+ vaccine, calculated
against the MOPH-established target e) Facility delivery (delivery) is the percentage achievement among women who delivered at a health facility, calculated
against MOPH-established target
f) One ANC visit (anc) is the percentage achievement among pregnant women who received at least one antenatal care
visit, calculated against MOPH-established target
g) One PNC visit (pnc) is the percentage achievement among new mothers who have received at least one postnatal
care visit, calculated against MOPH-established target
h) TB CDR (tbcdr) is the TB case detection rate X 100
i) OPD visit rate per person is the total number of OPD visitors in the quarter / Quarterly Target Population
j) CHW home visit rate (chw hvr) is calculated as the number of home visits X 100 / Number of Target visits in a
month
k) New FP (newfp) is New Family Planning users, calculated as the number of New FP Users in the month X 100 /
Monthly Target Population
l) TB Cure Rate (tbcure) is calculated as the number of cases that completed treatment and were smear-negative / the
number that started treatment X 100
As shown in Table 5, eight of ten indicators increased significantly (p < .01) from the pre-period to the
post-period in the intervention provinces. The new family planning user rate and TB cure rate did not
experience significant rises. Similarly, eight of ten indicators increased significantly (p < .01 for seven
indicators and p < .05 for one indicator) in non-intervention provinces. The TB case detection rate and
TB cure rate did not significantly improve. Difference-in-differences was statistically significant for four
variables: antenatal care visits (p < .05); postnatal care visits (p < .01); Penta3 immunization (p < .01),
and the OPD visit rate per person (p < .05).
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
17
Table 5: Health system performance and health outcomes, pre- and post-period
# Performance
Indicator
Intervention Provinces$
Comparison Provinces$$
Difference-
in-
differences
Confidence
Interval
p-
value
Pre
-
inte
rvention
Post
-
inte
rvention
Diffe
rence
Confiden
ce
Inte
rval
Pre
-
inte
rvention
Post
-
inte
rvention
Diffe
rence
Confiden
ce
Inte
rval
Health System Performance
1 Penta 3 124 (28) 148 (70) 24** [14,35] 129 (31) 136 (37) 7* [1, 15] 17** [4, 29] 0.0087
2 One ANC visit 111 (34) 136 (36) 24** [17,32] 114 (36) 125 (31) 11** [4, 18] 14* [3, 24] 0.0112
3 One PNC visit 69 (23) 98 (27) 29** [24,35] 72 (33) 90 (34) 17** [11, 24] 12** [3, 21] 0.0085
4 Facility delivery 53 (22) 74 (27) 22** [16,27] 50 (23) 66 (27) 16** [11, 21] 5 [-2, 13] 0.1497
5 TT2+ 175 (78) 350 (71) 175** [159, 192]
199 (84) 365 (118) 166** [145, 186]
10 [-17, 37] 0.4684
6 OPD visit 176 (44) 241 (48) 66** [55,76] 208 (62) 256 (70) 48** [34, 61] 18* [0.4,35] 0.0442
7 CHW home visit 36 (14) 42 (17) 6** [2, 9] 21 (14) 29 (16) 8** [5, 11] -2 [-12, 8] 0.6827
8 New FP user 18 (8) 20 (8) 1 [-0.2,3] 19 (8) 22 (7) 3** [1, 5] -1 [-4, 1] 0.2376
9 TB case detection
77 (34) 91 (44) 14** [5, 22] 68 (44) 71 (37) 3 [-12, 6] 11 [-1, 23] 0.0800
Health Outcome
10 TB cure rate 76 (33) 81 ( 43) 4 [-4, 13]
74 (44) 69 (30) -5 [-13, 4] 9 [-3, 21] 0.1297
*p < .05. **p < .01. $208 observations in the pre-period and 128 observations in the post- period (total 336 observations) $$234 observations in the pre-period and 144 observations in the post- period (total 378 observations)
Notes:
a) 124 (28) in the first row should be read as M(SD) (i.e.,, mean=124 and standard deviation=28). These numbers are
the percentage achievement against the MOPH-established target. For example, Penta3 immunization = 124 indicates
that intervention provinces had achieved 124 percent of the target set by the MOPH for immunization of children
below one year of age with pentavalent vaccine during the pre-intervention period. A difference of 24 in the fifth
column of the first row means a difference of 24 percentage points.
b) Indicators 1-8 are calculated as the percentage achievement as against MOPH-established targets, and not counts.
c) p-value was obtained by conducting two-sample equal variance two-tailed t-tests.
We performed a significance test in the pre-intervention period to see if the intervention and
comparison provinces were similar in profile in terms of the ten indicators, i.e., if there was a parallel
trend in the pre-intervention period. Intervention facilities had on average lower achievements in Penta
3 immunization, antenatal visits, postnatal visits, tetanus toxoid administration to pregnant mothers,
outpatient department visits, and new family planning users, while had higher achievements in facility
delivery, community health worker home visits, and tuberculosis case detection and cure. We
performed t-tests to see if these differences were statistically significant. See table 6 for results.
During the pre-intervention period, there is no statistically mean difference, at .05 level of significance, in
the achievements in six areas, i.e., Penta 3 immunization, antenatal visits, postnatal visits, facility delivery,
new family planning users, and tuberculosis cure between intervention and comparison provinces.
There are statistically significant differences, at .05 level of significance, between intervention and
comparison provinces in the achievements in four areas, i.e., tetanus toxoid administration to pregnant
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
18
mothers, outpatient department visits, community health worker home visits, and tuberculosis case
detection.
Table 6: Results of t-tests in the pre-intervention period
# Outcome variable
Pre-intervention period
Intervention provinces Comparison provinces df t p
Obs M (SD) Obs M (SD) 95% CI
for mean diff.
1 Penta 3 208 124.43 (28.14) 234 129.36 (30.81) -0.60, 10.47 440 -1.75 0.080
2 One ANC visit 208 111.30 (33.59) 234 113.90 (35.91) -3.92, 9.12 440 -0.78 0.434
3 One PNC visit 208 68.97 (22.56) 234 72.34 (32.65) -1.93, 8.68 440 -1.24 0.212
4 Institutional delivery 208 52.62 (21.99) 234 50.20 (23.21) -6.65, 1.83 440 1.11 0.264
5 TT2+ 208 174.68 (78.04) 234 199.02 (83.53) 9.17, 39.51 440 -3.15 0.001
6 OPD visits 208 175.49 (44.14) 234 208.44 (61.74) 22.80, 43.09 440 -6.38 0.000
7 CHW home visits 208 35.86 (14.30) 234 20.99 (13.76) -17.49, -12.24 440 11.12 0.000
8 New FP users 208 18.13 (7.93) 234 18.64 (7.88) -0.97, 1.98 440 -0.67 0.502
9 TB case detection rate x
100 208 77.13 (33.78) 234 68.06 (43.98) -16.46, -1.66 440 2.40 0.016
10 TB Cure Rate 208 76.22 (33.24) 234 73.85 (44.34) -9.77, 5.03 440 0.62 0.529
Notes:
Obs = observations in terms of the number of province-months
M=Mean, SD=Standard deviation, and CI=confidence interval.
t is t-statistic and p is p-value.
We plotted mean monthly achievement rates for the indicators over the 48-month period. See Figures
4-13 for these plots. Months 1-13 are the pre-intervention period and months 41-48 are the post-
intervention period. Note that pre-intervention trends are approximately parallel for the first nine of
the ten plots. Figures 4-13 show the monthly average for each of the ten indicators in terms of
percentage achievement against the MOPH-established targets in the pre-intervention and post-
intervention periods.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
19
Figure 4
Figure 5
10
012
014
016
018
0
Perc
ent a
ch
ievem
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison provinces Fitted values
Intervention provinces Fitted values
Penta 3 immunization80
90
10
011
012
013
014
015
0
Perc
ent a
ch
ievem
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
Pregnant women who have received at least one ANC visit
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
20
Figure 6
Figure 7
60
80
10
0
Perc
ent a
ch
ievem
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
Pregnant women who have received at least one PNC visit
15
017
019
021
023
025
027
0
Perc
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
Outpatient visit rate per person per month
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
21
Figure 8
Figure 9
45
50
55
60
65
70
75
80
Perc
ent a
ch
ievem
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
Institutional delivery
10
020
030
040
0
Perc
ent a
ch
ievem
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
Tetanus toxoid administration to pregnant mothers
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
22
Figure 10
Figure 11
20
25
30
35
40
Perc
ent a
ch
ievem
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
Community health worker home visits15
20
25
Perc
ent a
ch
ievem
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
New family planning users
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
23
Figure 12
Figure 13
50
70
90
11
0
Perc
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
TB case detection rate
50
70
90
11
0
Perc
ent
13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48
0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month
Comparison Provinces Fitted values
Intervention provinces Fitted values
TB cure rate
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
24
Difference-in-differences regression analysis
In the full model, we found a statistically and practically significant impact of the intervention on six
indicators. Specifically, the intervention increased a province’s rate of outpatient department visits per
person by an average of 18 percentage points (p<0.01), and achievements in Penta 3 immunization by 17
percentage points (p<0.01), antenatal visits by 14 percentage points (p<0.01), postnatal visits by 12
percentage points (p<0.01), tuberculosis case detection by 11 percentage points (p<0.01), and facility
delivery by 5 percentage points (p<0.01). No impact was detected on the achievements in tetanus
toxoid administration to pregnant mothers and tuberculosis cure. The secular trend was by far a
stronger and more significant predictor of increasing rates of tetanus toxoid administration to pregnant
mothers. Achievements in community health worker home visits and new family planning users
decreased (p<0.01) by 1 and 2 percentage points respectively. The size of the positive effects is large
while that of the negative effects is small.
We conducted difference-in-differences regression analysis. We began with Model 1, which had three
key variables: Interventionit, Postit, and Interventionit*Postit. In Model 2, we added four covariates:
poverty headcount rate of the province, labor force participation rate of the province, and female
literacy rate of the province, and its security categorization. Finally, we added province and time fixed
effects in Model 3.
As mentioned earlier, we had 13 months of pre-intervention data and eight months of post-intervention
data, thus a total of 714 province-months in the analysis sample.
Table 7: Effect of the intervention on health system performance and health outcomes (regression
coefficients)
Variable Model 1
Basic model
Model 2
with covariates
Model 3 with
province and time fixed effects
B SE B β B SE B β B SE B β
Penta3
intervention -4.93 3.93 -0.05 -9.24* 3.96 -0.10* -1.36 6.14 -0.01
post 7.53 4.37 0.08 7.53 4.30 0.08 9.92** 3.29 0.11**
intervention x post 16.81** 6.38 0.15** 16.81** 6.27 0.15** 16.81** 4.53 0.15**
poverty -0.26** 0.08 -0.11** -0.44** 0.16 -0.19**
labor force participation
-0.07 0.12 -0.02 -0.73** 0.24 -0.25**
female literacy -0.68* 0.34 -0.09* -1.35** 0.39 -0.18**
security 4.49** 1.38 0.13** 7.61** 2.56 0.22**
R2 0.0414 0.0796 0.5456
F for change in R2 10.23** 8.72** 17.41**
ANC
intervention -2.59 3.27 -0.03 0.43 3.28 0.00 13.76** 3.28 0.19**
post 10.91** 3.64 0.14** 10.91** 3.56 0.14** 10.68** 3.56 0.14**
intervention x post 13.50* 5.31 0.14* 13.50* 5.19 0.14* 13.50** 5.19 0.14**
poverty 0.20** 0.07 0.10** 0.15 0.07 0.07
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
25
labor force
participation -0.09 0.10 -0.04 -1.11** 0.10 -0.45**
female literacy 0.84** 0.28 0.13** -1.24** 0.28 -0.20**
security -2.37* 1.15 -0.08* 1.33 1.15 0.04
R2 0.0616 0.1105 0.7127 -0.05
F for change in R2 16.61** 12.53** 39.45**
PNC
intervention -3.37 2.80 -0.05 -2.42 2.75 -0.03 7.23* 2.94 0.11*
post 17.44** 3.11 0.26** 17.44** 2.98 0.26** 16.92** 1.57 0.26**
intervention x post 12.03** 4.53 0.14** 12.03** 4.35 0.14** 12.03** 2.17 0.14**
poverty 0.35** 0.05 0.21** 0.04 0.07 0.02
labor force
participation -0.27** 0.08 -0.12** -1.23** 0.11 -0.56**
female literacy 0.39 0.23 0.07 -0.83** 0.18 -0.15**
security 2.35* 0.96 0.09 4.62** 1.22 0.18**
R2 0.1356 0.2080 0.8015
F for change in R2 37.12** 26.49** 63.60**
Institutional Delivery
intervention 2.41 2.32 0.04 3.25 2.32 0.06 7.06** 2.31 0.13**
post 16.14** 2.58 0.30** 16.14** 2.51 0.30** 15.80** 1.24 0.29**
intervention x post 5.46 3.76 0.08 5.46 3.67 0.08 5.46** 1.71 0.08**
poverty 0.14** 0.04 0.10** -0.41** 0.06 -0.29**
labor force participation
-0.21** 0.07 -0.11** -0.67** 0.09 -0.37**
female literacy 0.61** 0.20 0.13** -0.27 0.14 -0.06
security 0.96 0.81 0.04 -3.45** 0.96 -0.16**
R2 0.1313 0.1712 0.8316
F for change in R2 35.77** 22.04** 71.62**
TT2+
intervention -24.34** 8.39 -0.10** -24.41** 8.44 -0.10** -5.52 11.15 -0.02
post 165.61** 9.33 0.66** 165.61** 9.16 0.66** 183.88** 5.98 0.73**
intervention x post 9.84 13.60 0.03 9.84 13.35 0.03 9.84 8.22 0.03
poverty 0.54** 0.18 0.08** 0.85** 0.29 0.13**
labor force participation
-0.63* 0.26 -0.07* -3.90** 0.44 -0.46**
female literacy -2.98** 0.72 -0.14** -8.74** 0.70 -0.41**
security -5.20 2.95 -0.05 13.44** 4.64 0.14**
R2 0.4737 0.4955 0.8192
F for change in R2 213.00** 99.06** 65.71**
OPD
intervention -32.94** 5.41 -0.25** -27.95** 5.11 -0.21** 33.76** 5.48 -0.26**
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
26
post 47.80** 6.01 0.36** 47.80** 5.54 0.36** 50.62** 2.94 0.38**
intervention x post 17.71* 8.77 0.10* 17.71* 8.08 0.10* 17.71** 4.04 0.10**
poverty 0.84** 0.10 0.24** 0.84** 0.14 0.24**
labor force participation
0.08 0.15 0.01 -0.84** 0.21 -0.18**
female literacy -2.69** 0.44 -0.24** -4.04** 0.34 -0.36**
security -10.05** 1.79 -0.19** -4.47 2.28 -0.08
R2 0.2252 0.3451 0.8452
F for change in R2 68.79** 53.15** 79.15**
CHW HVR
intervention 14.86** 1.43 0.43** 15.32** 1.36 0.45** 35.61** 1.06 1.05**
post 7.81** 1.59 0.22** 7.81** 1.47 0.22** 7.37** 0.57 0.21**
intervention x post -2.08 2.32 -0.04 -2.08 2.15 -0.04 -2.08** 0.78 -0.04**
poverty -0.20** 0.02 -0.22** -0.37** 0.02 -0.41**
labor force participation
0.20** 0.04 0.17** 0.65** 0.04 0.55**
female literacy 0.38** 0.11 0.13** 0.01 0.06 0.00
security -1.55** 0.47 -0.11** -3.70** 0.44 -0.27**
R2 0.2121 0.3275 0.9158
F for change in R2 63.72** 49.12** 157.61**
New FP
intervention -.50 0.74 -0.03 0.73 0.70 0.04 4.77** 0.72 0.30**
post 2.91** 0.82 0.18** 2.91** 0.76 0.18** 2.99** 0.39 0.18**
intervention x post -1.46 1.20 -0.07 -1.46 1.11 -0.07 -1.46** 0.53 -0.07**
poverty 0.03* 0.01 0.08* -0.11** 0.01 -0.26**
labor force participation
0.20** 0.02 0.37** 0.06* 0.02 0.11*
female literacy 0.51** 0.06 0.38** -0.03 0.04 -0.02
security -0.09 0.24 -0.01 2.62** 0.30 0.42**
R2 0.0255 0.1712 0.8161
F for change in R2 6.20** 20.84** 64.36**
TB CDR
intervention 9.06* 3.79 0.11* 6.91 3.69 0.08 5.50 4.96 0.06
post 2.93 4.22 0.03 2.93 4.00 0.03 5.61* 2.66 0.06*
intervention x post 10.79 6.15 0.10 10.79 5.84 0.10 10.79** 3.66 0.10**
poverty 0.31** 0.07 0.14** 0.05 0.13 0.02
labor force participation
-0.10 0.11 -0.03 -0.04 0.19 -0.01
female literacy -0.43 0.31 -0.06 -1.47** 0.31 -0.21**
security 7.09** 1.29 0.22** 7.39** 2.06 0.23**
R2 0.0397 0. 1393 0. 6805
F for change in R2 9.78** 16.32** 30.89**
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
27
TB Cure Rate
intervention 2.37 3.66 0.03 0.04 3.64 0.00 9.22 7.01 0.11
post -4.60 4.07 -0.05 -4.60 3.95 -0.05 -8.46* 3.76 -0.10*
intervention x post 9.04 5.93 0.09 9.04 5.77 0.09 9.04 5.17 0.09
poverty -0.34** 0.07 -0.16 -0.33 0.18 -0.16
labor force participation
-0.20 0.11 -0.07 -0.10 0.27 -0.03
female literacy -1.59** 0.31 -0.23** -0.72 0.44 -0.10
security -4.51** 1.27 -0.14** 2.81 2.92 0.09
R2 0.0089 0.0686 0.2916
F for change in R2 2.14 7.43** 5.97**
*p < .05. **p < .01.
Nine indicators increased in the post-intervention period as compared to the pre-intervention period;
the increase was statistically significant (p<0.01 for eight and p<0.05 for one). The TB cure rate
decreased in the post-intervention period (p<0.05).
Model 1 (Basic model)
The coefficient on intervention X post (Intervention*Post) variable is statistically significant for antenatal
and postnatal care visits, Penta3 immunization, and the OPD visit rate per person as dependent
variables. There was no statistically significant difference between the intervention and comparison
provinces in terms of the first three of these outcome variables at baseline. However, the intervention
provinces had a 33% lower OPD visit rate at baseline when compared to non-intervention provinces
(see intervention = -32.94** for OPD in Table 4). There was a statistically significant secular trend of
increase in the antenatal care, postnatal care, and OPD visit rates.
Treatment (i.e., governance interventions) significantly predicted the four outcome variables (see
coefficient on intervention X post variables): Penta3 immunization achievement, R2 = .04, F(3, 710) =
10.23, p < .001; antenatal care visits, R2 = .06, F(3, 710) = 16.61, p < .001; postnatal care visits, R2 = .13,
F(3, 710) = 37.12, p < .001; and the OPD visit rate per person, R2 = .22, F(3, 710) = 68.79, p < .001. It is
important to note that the intervention explained a small proportion of variance in at least two of these
four variables. The intervention explained 23% of the variance in the OPD visit rate, 14% of the variance
of the postnatal care visit achievement, 7% of the variance in antenatal care visit achievement, and 4% of
the variance in Penta3 immunization achievement.
On average, the governance intervention increased a province’s OPD visit rate per person by 18
percentage points, Penta 3 immunization achievement by 17 percentage points, antenatal visit
achievement by 14 percentage points, and postnatal visit achievement by 12 percentage points.
The governance intervention did not have impact on achievements in any of the remaining six variables:
facility delivery, community health worker home visits, tetanus toxoid administration to pregnant
mothers, new family planning users, the TB case detection rate, and the TB cure rate.
There was a strong secular trend of increasing facility delivery, tetanus toxoid administration to
pregnant mothers, and new family planning users. The secular trend was by far a stronger and more
significant predictor of these three outcome variables.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
28
At baseline, the achievement of tetanus toxoid administration to pregnant mothers was on average 24%
lower in intervention provinces than in comparison provinces, whereas the TB case detection rate was
9% higher in the intervention provinces. Community health worker home visit achievement was 15%
higher in the intervention provinces at baseline.
Model 2 (Model with covariates)
The main results of Model 1 were maintained after controlling for poverty headcount ratio, labor force
participation rate, female literacy rate, and security categorization of the province. On average, the
governance intervention increased a province’s OPD visit rate per person by 18 percentage points,
Penta 3 immunization achievement by 17 percentage points, antenatal visit achievement by 14
percentage points, and postnatal visit achievement by 12 percentage points. Model 2’s main results are
very similar to Model 1’s main results.
Female literacy had a positive and statistically significant association with achievements in antenatal care
visits, facility delivery, community health worker home visits, and new family planning users. Contrary to
expectation, female literacy was inversely associated with Penta3 immunization achievement,
achievement in administering tetanus toxoid to pregnant mothers, the OPD visit rate per person, and
the TB cure rate.
As expected, deteriorating security had a statistically significant inverse association with antenatal visit
achievement, the OPD visit rate per person, community health worker home visit achievement, and the
TB cure rate. Contrary to what one would expect, deterioration in security was associated with
increase in Penta3 immunization achievement, postnatal care visit achievement, and TB case detection
rate in a statistically significant manner.
Poverty was associated with a decrease in achievement of Penta3 immunization and community health
worker home visits, as well as the TB cure rate. Surprisingly, poverty was associated with an increase in
achievement of antenatal and postnatal care visits, facility delivery, tetanus toxoid administration to
pregnant mothers, the OPD visit rate per person, new family planning users, and the TB case detection
rate.
Higher labor force participation was associated with a decrease in achievements in postnatal care visits,
facility delivery, and tetanus toxoid administration to pregnant mothers in a statistically significant
manner. It was associated with increases in achievement of community health worker home visits and
new family planning users.
Model 3 (Model with province and time fixed effects)
Province and time fixed effects were added in Model 3. The coefficient on intervention X post
(Intervention*Post) variable is statistically significant for achievements in antenatal and postnatal care
visits, facility delivery, Penta3 immunization, community health worker home visits, new family planning
users, and the OPD visit rate per person and TB case detection rate as dependent variables. At baseline,
there was no statistically significant difference between the intervention and comparison provinces in
terms of achievement in Penta3 immunization, tetanus toxoid administration to pregnant mothers, and
the TB case detection rate, and the TB cure rate. However at baseline, the intervention provinces had
higher achievement rates in antenatal and postnatal care visits, facility delivery, OPD visits, community
health worker home visits, and new family planning users. There was a statistically significant secular
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
29
trend of increase in the achievement in all the indicators except the TB cure rate which showed a
statistically significant secular trend of decrease with time.
Treatment (i.e., the governance intervention) significantly predicted the eight outcome variables (see
coefficient on intervention X post variables): Penta3 immunization achievement, R2 = .54, F(46, 667) =
17.41, p = .0000; antenatal care visits, R2 = .73, F(46, 667) = 39.45, p = .0000; postnatal care visits, R2 =
.81, F(46, 667) = 63.60, p = .0000; facility delivery, R2 = .83, F(46, 667) = 71.62, p = .0000; the OPD visit
rate per person, R2 = .84, F(46, 667) = 79.15, p = .0000; community health worker home visits, R2 = .91,
F(46, 667) = 157.61, p = .0000; new family planning users, R2 = .81, F(46, 667) = 64.36, p = .0000; and
the TB case detection rate, R2 = .68, F(46, 667) = 30.89, p = .0000. It is important to note that the
intervention explained a significantly higher proportion of variance in Model 3 as compared to Model 1.
That is, the intervention explained at least 50% of the variance whereas Model 1 had explained less than
25% variance.
On average, the governance intervention increased a province’s OPD visit rate per person by 18
percentage points, Penta 3 immunization achievement by 17 percentage points, antenatal visit
achievement by 14 percentage points, postnatal visit achievement by 12 percentage points, facility
delivery achievement by 5 percentage points, and the TB case detection rate by 11 percentage points.
On the contrary, the intervention decreased a province’s achievement in new family planning users by 1
percentage point, and community health worker home visits by 2 percentage points.
The governance intervention did not have an impact on the remaining two variables: tetanus toxoid
administration to pregnant mothers and the TB cure rate.
Sensitivity analysis
One crucial assumption for difference-in-differences estimations is the exogeneity of the policy
implementation. A question might be posed whether the governance intervention was exogenous in this
case. The intervention provinces were the ones already being assisted by USAID for a long time. The
comparison provinces were assisted by other donors, the European Union and the World Bank, which
might bias the results. We had no direct way to find out if the better results were driven by the work or
actions implemented by USAID compared with other donors.
To verify whether results were mixing up the effect of the intervention with some other unobservable
trend or confounding factor, we conducted a series of falsification and robustness checks. We found our
results to be robust according to all the checks we carried out.
The pre-intervention period spanned the first 13 months of the data. We conducted a “placebo test" on
this data where the initial six months are treated as the pre-intervention period and months 7-13 are
treated as the post-intervention period. We should not see any impact of the intervention in this
placebo test since, in reality, the intervention had not occurred during this period.
Table 8 shows the results of the placebo test. As expected, we did not find any impact of the placebo
intervention on Penta3 immunization rates. The coefficient on intervention X post (Intervention*Post)
variable was not statistically significant. The result goes against the potential endogeneity of the
governance intervention (getting the causation wrong).
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
30
Table 8: Placebo test conducted in the pre-intervention period data (Difference-in-differences
regression coefficients)
Variable
Model 1
Basic model
Model 2
with covariates
Model 3 with
province and time fixed effects
B SE B β B SE B β B SE B β
Penta3
intervention -4.85 4.15 -0.08 -7.99 4.13 -0.13 -8.63* 4.10 -0.14*
post -2.70 3.88 0.04 -2.70 3.79 -0.04 -5.57 3.85 -0.09
intervention x post -0.14 5.66 -0.00 -0.14 5.53 -0.00 -0.14 2.85 -0.00
poverty -0.10 0.07 -0.06 0.06 0.10 -0.03
labor force participation
-0.13 0.11 -0.06 -0.72** 0.15 -0.34**
female literacy -0.01 0.30 -0.00 -0.69** 0.25 -0.13**
security 5.60** 1.25 0.24** 7.75** 1.65 0.33**
R2 0.2607 0.0483 0.7469
F for change in R2 1.34 4.20** 29.28**
As a next step, we conducted a robustness check by considering several outcomes that had nothing to
do with the health governance intervention. Firstly, we generated a placebo indicator with random
values within a plausible range and tested the results. Secondly, we serially tested poverty headcount
rate, labor participation rate, and female literacy rate as outcome variables in the model. Again, we
should not see any impact of the intervention either on the placebo or the three real world indicators
(poverty, labor participation, and female literacy) that are not likely to be influenced by the health
governance intervention that too in a period of six months. Indeed, we did not find any impact on any of
the four indicators. Table 9 shows these results.
Table 9: Robustness check using several outcomes that had nothing to do with the health
governance intervention (Difference-in-differences regression coefficients)
Variable Model 1
Basic model
Model 2
with covariates
B SE B β B SE B β
Poverty
intervention -5.07** 1.76 -0.13** -6.87** 1.72 -0.18**
post -0.00 1.96 -0.00 -0.00 1.89 -0.00
intervention x post 0.00 2.85 0.00 0.00 2.76 0.00
labor force
participation -0.07 0.05 -0.05
female literacy -0.57** 0.14 -0.17**
security 2.12** 0.60 0.14**
R2 0.0185 0.0851
F for change in R2 4.45** 10.96**
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
31
Labor force
participation
intervention -1.53 1.36 -0.05 -3.05* 1.21 -0.10*
post -0.00 1.51 -0.00 -0.00 1.32 -0.00
intervention x post 0.00 2.21 0.00 0.00 1.92 0.00
poverty -0.03 0.02 -0.04
female literacy -1.34** 0.09** -0.54**
security -1.15** 0.42** -0.10**
R2 0.0029 0.2470
F for change in R2 0.68 38.64**
Female literacy
intervention -1.35* 0.55 -0.11* -1.06* 0.43 -0.09*
post -0.00 0.61 -0.00 -0.00 0.47 -0.00
intervention x post 0.00 0.89 0.00 0.00 0.68 0.00
poverty -0.03** 0.00 -0.11**
labor force participation
-0.17** 0.01 -0.42**
security -1.73** 0.13 -0.37**
R2 0.0136 0.4123
F for change in R2 3.26* 82.68**
Random values
intervention 1.11 1.65 0.03 0.59 1.69 0.01
post -2.27 1.83 -0.06 -2.27 1.83 -0.06
intervention x post 2.38 2.67 0.05 2.38 2.67 0.05
poverty -0.00 0.03 -0.00
labor force
participation -0.04 0.05 -0.04
female literacy 0.07 0.14 0.02
security 1.18* 0.59 0.08*
R2 0.2657 0.2323
F for change in R2 1.32 1.33
Power analysis
We tested the power of our statistical model and statistical tests to detect the change in indicators. Our
model and tests were adequately powered; statistical power was 0.80 or above.
Focus group discussions
As stated earlier, we held focus group discussions with the PPHCC members to obtain feedback on the
intervention at the end of pilot implementation in the initial three provinces. The committees reported
many achievements and successes, as summarized in box 2.
There were also goals that committees wanted to accomplish but could not; for example, better
coordination between shuras or committees at different levels and more effective communication with
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
32
Box 2: Findings of focus group discussion
1. Enhanced transparency and accountability: PPHCC members noted that their meetings had become more
regular; had clear agendas; and minutes of meetings were documented and made public. The members
reported that new sub-committees that oversee accountability, for example financial audit and
transparency subcommittee and governance subcommittee were established at the province level;
information and decisions of the committees were shared through emails, press conferences, magazines,
newsletters and social media websites; private sector pharmacies and food stores that lacked legal
documents or did not follow regulatory standards were closed down; expired medicines were rounded
up through monitoring visits; and judicial proceedings were started against corrupt health facility
personnel.
The members stated that their activities as a committee became more transparent and involved diverse
stakeholders; inter-sector coordination and collaboration improved; there was an improved focus on
patients’ health problems at the community level; community health needs were defined, and discussed;
coordination, communication, and information sharing improved; the frequency of data analysis and
presentation to the committees increased, and more committee decisions were evidence based.
2. Inclusion of stakeholders and stakeholder views: The PPHCC members reported that steps were taken to
recruit more women to community health nursing educator posts; suggestion and complaint boxes were
placed outside health posts and the community complaints were discussed during regular meetings;
attendance of members at the meetings improved; civil society groups, community leaders and
representatives from other sectoral departments were invited to the committee meetings; community
concerns were discussed as a standing agenda item during committee meetings; and vaccination rejection
in some villages was addressed by negotiating with elders and through community mobilization efforts.
3. Setting shared strategic direction: The PPHCC members reported that the MOPH guidelines and policies
were introduced and shared in the PPHCC meetings; service delivery data was analyzed and shared more
frequently with committee members; and annual plans were developed and shared with the stakeholders
in the province and the MOPH.
4. Stewardship of resources: A committee reported success in mobilizing resources to build toilets in the
health facility and completing the overdue construction of a community health center. Health center
buildings and facilities were improved through community support at a few places, and inspections and
enforcement of quality standards also improved.
The members observed that there was a better link between committees at provincial and district levels.
Communities began participating in health facility monitoring, gave feedback on the quality of health
services, and became motivated to contribute to health service delivery improvements.
the public. Some of the actions in the governance action plans needed substantial extra resources that
the committees did not have. Overall, lack of resources emerged as one of the common challenges the
committees faced in completing their action plans. The security situation often prevented engagement of
provincial and district governors, and did not allow for the level of health facility monitoring that the
committees would have liked to achieve. Resources for training and education of committee members
were found to be inadequate. Community expectations on health service delivery rose, and the
committees did their best to meet these expectations by mobilizing community support and resources.
Overall, committees reported notable changes in their knowledge, skills, and behaviors, including feeling
more capable, responsive, and accountable than they were before the intervention. Committee
functioning became more systematic and regular, and members felt more responsible for their decisions.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
33
Committees also noticed improvements in their effectiveness; referral of TB cases for treatment
improved in one district, un-served remote areas were identified, and 90% of them were covered
through establishment of mobile teams in another district. One committee reported that antenatal and
postnatal care visits increased, and other maternal and child health services improved. Because of
increased community engagement, committees felt they could solve problems at the health facility level
in collaboration with the local community. This experience showed them that they could build trust with
the communities by working with them.
Committee members said they would continue applying effective governing practices in the future
mainly because they felt their achievements in the short six month period were encouraging. They
became aware of weaknesses in their governing skills and capacity and resolved to improve. Members
thought they gained many benefits at a personal level because of changes in their attitudes and behavior.
They also became aware of their stewardship role and wanted to do more for the communities they
served. The intervention, PPHCC members told us, renewed their commitment to their governance
responsibilities. They observed that periodic governance assessments (because they became aware of
their governance responsibilities and their governance performance) and the overall experience
developed their capacity in discharging their governance role. The committees recommended that the
MOPH should officially introduce the approach in all the provinces and districts, and expressed interest
in sharing their experience with the uncovered jurisdictions.
8. Discussion We found that health systems’ governance can be improved even in fragile and conflict affected
environments. Focus group discussions showed that when health governing bodies in the communities,
the district, and the province worked in coordination, community health concerns could be effectively
represented and addressed and health systems could become more responsive to the community needs
within the available limited resources.
How an organization or health system is governed is a determinant of its performance. Good
governance enables effective and efficient management of people, money, medicines, and information.
Sound management, in turn, facilitates the work of doctors, nurses, and other health workers, enabling
them to provide safe, timely, effective, and efficient care and services that are respectful of and
responsive to individual patient or health service user preferences, needs, and values. We had
hypothesized that the governance intervention would thus have a positive impact on the health system
performance indicators used in the study.
First of all, through the focus groups, we learned how governance could translate into improved health
system performance. In our intervention, we introduced four practices of good governance – cultivating
accountability, engaging with stakeholders, setting a shared strategic direction, and stewarding resources
– that provided an organizing framework for leaders to develop governance improvement plans. When
the Provincial Health Coordination Committee members, who have a predominant governing role,
designed and implemented their governance action plans based on these practices, their governing
behavior, and consequently the governance of their provincial health systems, improved. Inter-sector
and inter-departmental collaboration received a boost; this is relevant in the context of health as the
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
34
work of many sectors other than health influences the health status of the population.
The PPHCC members designed their action plan in a participatory and consultative manner. This
seemed to foster a sense of responsibility to successfully implement it. The intervention was focused on
people, i.e., health system leaders governing in close partnership with health managers, health providers,
health workers, community leaders, health service users, and governance leaders in other sectors. This
helped ensure the intervention was meaningful for the governance leaders as well as for the community.
As health systems become decentralized, sub-national structures and committees are entrusted with the
responsibility to coordinate, implement, and oversee health services; they are expected to play a
governance role. That they are closer to the people helps. Governing bodies at the community level can
represent the unresolved health needs of their communities to governing bodies at the district and
provincial levels, which may have more resources to address them.
We discovered interventions to improve governance skills are feasible in fragile and conflict-affected
environments, when practices of good governance are consistently applied, periodically assessed and
continuously improved.
Through the quantitative part of the study, we saw a significantly positive impact on six out of nine
indicators of provincial health system performance, a marginally negative impact on two indicators of
provincial health system performance, and no impact on the remaining one indicator of health system
performance and one indicator of health outcomes. We found significant positive impact on achievement
of antenatal and postnatal care visits, facility delivery, Penta3 immunization, outpatient department visits,
and tuberculosis case detection. There was no impact on achievements in tetanus toxoid administration
to pregnant mothers, and the TB cure rate. We also saw marginally negative impact on community
health worker home visits and new family planning users though the size of this negative effect is small;
i.e., less than 2 percentage points.
There could be several reasons for these mixed results – six positive, two null, and two negative.
Firstl, all the ten result areas are important national priorities. However, the PPHCCs in the
intervention provinces might have given the six positive result areas (antenatal and postnatal care, facility
delivery, immunization, OPD visits, and tuberculosis case detection) even higher priority in their
province.
Second, these six positive results might have been “low hanging fruit.” More difficult-to-achieve
indicators (for example, TB cure or new family planning users) could not be accelerated during the six
months of intervention. However, at this time, we don’t know if more time would have made a
difference. It is also possible that the PPHCCs did not sustain their improved governance practices long
enough to have a statistically and also clinically significant impact on the four indicators (tetanus toxoid
administration to pregnant mothers, tuberculosis cure, community health worker home visits, and new
family planning users). Note that the performance in tetanus toxoid administration is already exceedingly
high and there may have been little margin for further improvement.
Third, it might be the case that insecurity posed disproportionate hurdles in the intervention provinces.
Fourth, the governance intervention in and of itself might not be enough to make a difference to the
remaining four intractable indicators. In other words, those indictors may have also needed an
intervention at the health service management level or service delivery level, or perhaps at a
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
35
combination of both levels. Interventions at the health manager, provider or patient level could
potentially have helped. Finally, all of the intervention provinces have been assisted by USAID for more
than a decade, whereas other provinces were assisted by other donors (the European Union and the
World Bank) for a similar length of time. There might have been a qualitative difference in the support
provided by USAID when compared to the other donors. It is possible that the intervention provinces
were comparatively well-prepared to derive disproportionate benefit from the intervention, or at least
in the six areas where we saw improvements.
Limitations
These findings may not be generalizable to all low- and middle-income countries. Note that Afghanistan
is an archetypal case of a fragile and conflict affected environment. It may not be representative of low-
income countries in general. It might be the case that governance interventions bear impact in systems
that are severely underperforming and have modest goals like Afghanistan. We do not know for sure if
they are as effective in systems that are moderately underperforming and need to reach much higher
standards. Recall that governance improvement in this case was higher in the provinces where the
baseline was lower.
However, our study does invite readers to make connections between its elements and their own
experience. Moreover, the evidence generated by our study is significant given that there is a scarcity of
evidence in this area. The difference-in-differences method has several shortcomings, including possible
endogeneity of the interventions themselves, issues relating to the standard error of the estimate, and
requirement of a parallel trend assumption [18].
Moreover, effects identified by difference-in-differences may not have a causal interpretation if the policy
change occurred in a jurisdiction that derives unusual benefits from doing so [19]. All of the intervention
provinces in our study were USAID-assisted provinces, whereas most of the comparison provinces
were assisted by other donors. We do not know if the intervention provinces were able to derive
unusual benefits from the governance intervention because they were comparatively better-prepared in
health management and/or health service delivery. This might be the case because, in all the intervention
provinces, the provincial public health advisor embedded in the provincial public health office had been
providing public health management support for years before the intervention began. The comparison
provinces did not have such advisors for any considerable length of time.
The simplicity of the difference-in-differences method comes at a price in terms of assumptions. To
obtain impact estimates, the crucial identifying assumption one has to make is - the counterfactual trend
is the same for treated and non-treated units [20]. This assumption can only be tested if more data are
available.
The identifying assumption requires the investigators to assume whatever happened to the control
group over time is what would have happened to the treated group in the absence of the program [21,
22]. It is relatively strong assumption and tends to overlook all other possible differences between the
groups that could have led to the observed outcomes. The difference-in-differences method attributes
any differences in trends between the treatment and control groups to the intervention as long as those
differences occur at the same time as the intervention. The estimation becomes biased if there are other
factors that affect the difference in trends between the two groups [23, 24].
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
36
To mitigate this methodological limitation, we demonstrated a broadly parallel trend during the pre-
intervention period by plotting mean monthly rates of all health system performance variables during the
48-month period that includes the 13-month pre-intervention period. We also performed t-tests in the
pre-intervention period to see if the intervention and comparison provinces were similar in profile in
terms of the ten indicators.
Another limitation of the study is its short duration. We do not know if six months were enough for the
governance intervention to have an effect on health system performance.
The Hawthorne effect (i.e., governance improved because governance leaders modified their behavior
for the purpose of the assessment and because their performance was being ‘observed’) could also be a
plausible alternative explanation for the results. Governance self-assessments in comparison jurisdictions
could have either refuted or established the Hawthorne effect as the reason for improvement in
governance scores. We could not carry out these assessments due to a lack of time and resources.
PPHCC members measured governance by conducting the self-assessments collectively as a group.
There is an element of subjectivity in the self-assessments. We reduced this bias by designing and
conducting collective self-assessments. Group self-assessments are less vulnerable to subjectivity than
individual self-assessments because group processes can moderate over-rating—if one member of a
group over-rates performance on an item, another group member can bring this to the attention of the
group, which can in turn affect the group’s final rating of that item.
Strengths
Our study has several strong points. It examines the contribution of governance to health system
performance and generates early evidence in this regard using a rigorous quantitative method. It builds
on the Project’s earlier work in the field [15] and establishes the link between the governance of a
health system and its performance.
Our study results are relevant to the situation obtaining in many low-income countries where weak
health systems have been a significant barrier to effectively addressing outbreaks and epidemics and to
providing good quality services to citizens. The international donors and development partners working
in these countries are looking for ways to strengthen the health systems; improving their governance is
one way to do this.
9. Conclusion We conclude that a provincial health governance intervention has the potential to positively impact
health system performance. However, this potential cannot be taken for granted. We still do not know
if the positive impact is experienced every time such interventions are implemented or when
implemented at all different levels of the health systems. We saw beneficial impact only in 67% (six out
of nine) of the health system performance indicators that were included in the study. This is not to imply
that these interventions are not worthy of consideration. Instead, they need to be studied more and the
results documented more systematically.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
37
Areas for future research
This study yields directions for future research. Our intervention lasted six months; we recommend that
longer duration governance interventions be the subject of future studies. Governance improvements
might need time to translate into improvements in health system performance at health facility level.
They might need to be implemented over sustained period of time to realize gains in the health system
performance.
Governance is an enabler of management and management is an enabler of service provision. Our
intervention focused on governance from the perspective of the people who govern; it did not directly
involve health managers or service providers. All three groups have a role to play in improving health
governance. In addition, the provinces sit between the national ministry at the top, and the districts,
health facilities, and communities below. Governance happens at all of these levels, but our intervention
only focused on the provincial level. We recommend future studies of interventions at the different
levels, as well as complex multi-dimensional and multi-level studies.
There is also a case to be made for conducting similar interventions and research in different types of
organizations delivering health services - public organizations, civil society organizations and private for-
profit organizations - to examine the link between their governance and their organizational or health
system performance. There is a multitude of sophisticated studies examining the link between corporate
governance and profitability of a firm [25, 26, 27, 28], but we hardly find such studies examining the link
between health systems governance and system performance.
Cost effectiveness studies are also lacking in this arena, including comparing the cost-effectiveness of
implementing a governance intervention on the top of a public health intervention to implementing only
a public health intervention. Finally, this is mainly a quantitative study. More qualitative studies should be
done in the future to open “the black box” that sits between governance and organizational
performance [29].
10. Implication for Practice Our study has an implication for practice. Entry points for improving governance in the health sector
may not be readily obvious to policymakers. Governing bodies have the potential to contribute to
improved health services and the health status of populations. Often they do not get necessary support
in terms of clear mandates, authority, and resources. We have designed and successfully tested an
approach to improving governance of a provincial health system that enhances its performance. This
approach could be of interest to governing bodies in the health sector of low- and middle-income
countries.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
38
References
1. Siddiqi S, Masud TI, Nishtar S, Peters DH, Sabri B, Bile KM, Jama MA. Framework for assessing
governance of the health system in developing countries: gateway to good governance. Health Policy.
2009; 90(1):13-25.
2. Veillard JHM, Brown AD, Barış E, Permanand G, Klazinga NS. Health system stewardship of
National Health Ministries in the WHO European region: concepts, functions and assessment
framework. Health Policy. 2011;103(2):191-199.
3. WHO Regional Office for Europe. Governance for health in the 21st century. A study
conducted for the WHO Regional Office for Europe. Smart governance for health and well-being. 2011.
Available at: http://www.euro.who.int/__data/assets/pdf_file/0019/154252/Health-2020-Conf.-
Governance-for-health-....pdf. Accessed 29 Jul 2016.
4. Mikkelsen-Lopez I, Wyss K, de Savigny D. An approach to addressing governance from a health
system framework perspective. BMC international health and human rights. 2011. BMC Int Health Hum
Rights. 2011;11:13.
5. Brinkerhoff D, Bossert T. Health governance: concepts, experience and programming options.
2008.
6. United Nations General Assembly. Resolution adopted by the General Assembly on 25
September 2015 70/1. Transforming our world: the 2030 agenda for sustainable development. 2015.
Available at: http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E. Accessed 29 Jul
2016.
7. Gupta S, Davoodi HR, Tiongson E. Corruption and the provision of health care and education
services. In Political Economy of Corruption. 1st edition. Edited by Jain AK. London and New York:
Routledge. 2002:111–41.
8. Azfar O, Kähkönen S, Meagher P. Conditions for effective decentralized governance: A synthesis
of research findings. College Park: IRIS (Institutional Reform and the Informal Sector) Center, University
of Maryland, Center for Institutional Reform and the Informal Sector. 2001:67-73.
9. Delavallade C. Corruption and distribution of public spending in developing countries. J Econ
Fin. 2006;30(2):222-39.
10. Lindelow M, Serneels P. The performance of health workers in Ethiopia: results from qualitative
research. Soc Sci Med. 2006; 62(9):2225-35.
11. Rajkumar AS, Swaroop V. Public spending and outcomes: Does governance matter? J Dev Econ.
2008; 86(1):96-111.
12. Olafsdottir AE, Reidpath DD, Pokhrel S, Allotey P. Health systems performance in sub-Saharan
Africa: governance, outcome and equity. BMC Public Health. 2011; 11:237-44.
13. Svensson J, Björkman M. Power to the people: evidence from a randomized field experiment of
a community-based monitoring project in Uganda. Q J Econ. 2009;124(2):735-69.
14. Ciccone DK, Vian T, Maurer L, Bradley EH. Linking governance mechanisms to health outcomes:
A review of the literature in low-and middle-income countries. Soc Sci Med. 2014;117(86-95).
15. Anwari Z, Shukla M, Maseed BA, Wardak GFM, Sardar S, Matin J, Rashed GS, Hamedi SA, Sahak
H, Aziz AH. Implementing people-centred health systems governance in 3 provinces and 11 districts of
Afghanistan: a case study. Conflict and health. 2015; 9(1):1-6.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
39
16. Wooldridge J. What’s new in econometrics? Difference-in differences estimation, Lecture notes
10 for NBER. Cambridge: NBER Summer Institute. 2007. Available at:
http://www.nber.org/WNE/Slides7-31-07/slides_10_diffindiffs.pdf. Accessed 27 Jul 2016.
17. Newbrander W, Ickx P, Feroz F, Stanekzai H. Afghanistan's basic package of health services: its
development and effects on rebuilding the health system. Global Public Health. 2014;9(Suppl 1):S6–S28.
http://doi.org/10.1080/17441692.2014.916735
18. Epstein AJ. Effects of report cards on referral patterns to cardiac surgeons. J Health Econ.
2010;29(5):718–31.
19. Besley T, Case A. Unnatural experiments? Estimating the incidence of endogenous policies. The
Economic Journal. 2000; 110: F672-94.
20. Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-
differences approach. JAMA. 2014;312(22):2401.
21. Abadie A. Semiparametric difference-in-differences estimators. The Review of Economic Studies.
2005; doi:10.1111/0034-6527.00321.
22. Angrist JD, Pischke J. Mostly harmless econometrics: An empiricist's companion: Princeton
University Press; 2008.
23. Card D, Krueger AB. Minimum Wages and Employment: A Case Study of the Fast-Food Industry
in New Jersey and Pennsylvania. Am Econ Rev. 1994; 84(4):772-793.
24. Bertrand M., Duflo E. and Mullainathan S. How Much Should We Trust Differences-in-
Differences Estimates? Quarterly Journal of Economics. 2004; 119(1):249-275.
25. Black BS, Jang H, Kim W. Does corporate governance predict firms' market values? Evidence
from Korea. Journal of Law, Economics, and Organization. 2006;22(2):366-413.
26. Gompers PA, Ishii JL, Metrick A. Corporate governance and equity prices. Q J Econ.
2003;118:107–55.
27. Black B. The corporate governance behavior and market value of Russian firms. Emerging
Markets Review. 2001:89–108.
28. Durnev A, Kim E. To steal or not to steal: Firm attributes, legal environment, and valuation.
Journal of Finance. 2005:1461–93.
29. Maxwell JA. Using qualitative methods for causal explanation. Field methods. 2004; 16(3):243-64.
doi: 10.1177/1525822X04266831.
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
40
Appendix 1
Overall Health Governance Self-Assessment Tool for the PPHCC
Directions: For each item, note briefly the internal strengths and weaknesses within the PPHCC as well as
the opportunities and threats that exist in the current environment. You will rate performance on a
scale of from 1 to 10, with 10 being the best possible rating.
# Responsibility Performance (scale 1–10)
Role and responsibilities as per ToR
1 Meeting the MOPH policies, priorities, objectives, strategies, and standards:
a. Overall
b. In relation to the BPHS
c. In relation to the EPHS
2 Coordination of all stakeholders:
a. At the provincial level
b. In the districts and the communities
c. With the MOPH
3 Improvement of services:
a. Overall quality
b. Accessibility
c. Sustainability
4 Input into provincial planning, consolidation of the provincial plan, and monitoring of its implementation
5 Oversight: Development and implementation of a joint supervision plan
6 Coordinate an emergency response and participate in it
7 Coordinate and participate in immunization campaign
8 Provide technical and consultation support to the PPHD
9 Sharing of information related to service delivery, MOPH policies, strategies, standards, new initiatives, and any problems and challenges
10 Identify sites for new health facilities
11 Mediate disputes that arise among stakeholders and ensure shared understanding between governmental organizations and NGOs (cases referred by DPHO)
12 Follow up of the work of committees:
a. HMIS Committee
b. EPI Task Force
c. Emergency Task Force
d. CDC Committee
e. Reproductive Health Committee
f. Maternal and Child Survival Committee
g. Child and Adolescent Health Committee
h. Nutrition Committee
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
41
i. Pharmacy Committee
j. Environmental Health Committee
k. Any other provincial health-related committee
Expanded role and responsibilities
13 Advocate and ensure openness, transparency, accountability, honesty, and inclusiveness:
a. In the health system of the province
b. In the affairs of the provincial health directorate
14 Set strategic direction:
a. For 3–5 years
b. Strategic plan is based on the strategic direction
15 Advocate and support achievement of health outcomes, responsiveness and patient satisfaction, and patient safety
16 Nurture relationships with the communities and the people served
17
Stewardship of scarce resources:
a. Ethical and best use of available resources for achievement of health outcomes for the
people served
b. Exert influence across different sectors for achieving best health outcomes for the population in province
c. Provide vision and direction for health systems
d. Collect and use information and evidence on health system performance to ensure
accountability and transparency
18 Ensure adequate financial resources
19 Provide financial oversight
20 Monitor performance
21 Support high performance
22 Strengthen health services in the province
23 Continuous improvement of the functioning of the PPHCC
24 Build a competent PPHCC
25 Advocate for decentralization
26 Any other responsibility PPHCC fulfills
Scoring criteria
The maximum score that can be earned is 450.
Score of 338 and above: Outstanding governance
Score of 226–337: Meets most requirements
Score of 113–225: Needs improvement
Score below 113: Unsatisfactory governance
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
42
Health Governance Standards-based Self-Assessment Tool for the
PPHCC
This instrument is based on 11 PPHO core functions and 46 PPHC governance standards.
Scoring guide
0% 1%–25% 26%–50% 51%–75% 76%–100%
No activity Minimal activity Moderate
activity
Significant
activity Optimal activity
0%, or
absolutely
no activity
Greater than
zero, but no
more than 25% of
the activity
described within
the standard is
met
Greater than
25%, but no more
than 50% of the
activity described
within the
standard is met
Greater than
50%, but no more
than 75% of the
activity described
within the
standard is met
Greater than 75%
of the activity
described within
the standard is
met
Instrument
# ToR 0% 1%–25% 26%–50% 51%–75% 76%–100%
I Oversight for health situation and trend assessment
1 Facilitates access to appropriate resources for community health status monitoring, and mobilizes resources and support for the surveys that are centrally designed
2 Promotes broad-based participation and coordination among all entities active in collecting, analyzing, and
disseminating community health status data
3 Provides oversight and support for community health status
monitoring efforts
4 Guides improvements in the health status monitoring efforts
II Oversight of monitoring and evaluation of health services
1 Facilitates access to the necessary resources to conduct periodic monitoring and evaluations
2 The PPHCC itself evaluates the health services
3 Makes sure that regular supportive supervision, monitoring and evaluation of health services provided in the public and
the private sectors in the province take place
4 Encourages all relevant stakeholders provide input into
monitoring and evaluation processes
5 Reviews evaluation results and utilizes these results to improve health service performance
III Oversight for data and information management
1 Facilitates access to appropriate resources for data and information management
2 Promotes broad-based participation and coordination among all entities active in data and information management tasks
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
43
# ToR 0% 1%–25% 26%–50% 51%–75% 76%–
100%
3
Reviews data and information management on a quarterly
basis, and provides oversight and support for data and information management efforts
4 Guides improvements in overall data and information management
IV Oversight for health service delivery
1 Oversees public and nongovernmental organizations and the private sector responsible for delivery of health services
2 Allows community monitoring of the delivery of health services
3 Facilitates community input in problem identification and problem solving
4 Conducts periodic reviews of health service delivery with special attention to services for vulnerable populations
V Oversight of coordination, communication, and intersectoral collaboration activities
1 Facilitates access to national, state, and local resources that could be used in support of these activities
2
Establishes and oversees the implementation of policies to
support activities to inform, educate, and empower people about public health issues, and reviews these activities in light of community needs
3 Makes sure that all population subgroups have an opportunity to provide input on health issues and health
services
4 Exerts influence across sectors to protect and promote
health of the community
VI Supporting health service delivery
1
Ensures that the PPHO is supporting the implementing
NGOs through joint visits, assessment of quality of service, and assistance in staff recruitment and training
2 Encourages members from lead and secondary NGOs to surface their issues in a timely fashion
3 Mobilizes community support of the NGOs delivering appropriate services
4 Recognizes NGOs delivering quality services to vulnerable populations
VII Oversight of health resource management
1 Establishes and oversees the implementation of policies designed to assure efficient and effective use of physical,
financial, and human resources and drugs and supplies
2
Establishes and oversees the implementation of policies
designed to assure improvements in workforce, management, and leadership quality
3 Facilitates access to national, state, and local resources available for workforce training, leadership development, and continuing education
4 Provides for the training and continuing education of the
PPHCC
5 Assists in mobilizing resources for the provincial health
system and the public health services
Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)
44
# ToR 0% 1%–25% 26%–50% 51%–75% 76%–
100%
VIII Oversight of preventive and clinical services, environmental health services, and forensic medicine services
1
Oversees and supports the delivery of preventive and
clinical services, environmental health services, and forensic medicine services
2 Facilitates the community monitoring of the delivery of these services
3 Encourages community input into the delivery of these services
4 Ensures transparency, accountability, and ethical and moral integrity in the provision of these services
IX Oversight of strategic and annual planning
1 Maintains and annually reviews documentation of its mission
statement
2 Assesses and advocates for adequate resources and the MOPH’s support
3 Supports planning for improvement in the health of the population in the province and works to strategically align
community resources for this purpose
4 Oversees implementation of the annual plan
X Oversight of management of health emergencies
1 Supports planning for emergency response and works to
strategically align community resources for this purpose
2 Facilitates access to appropriate resources for management of health emergencies
3 Promotes broad-based participation and coordination among all entities active in management of health emergencies
4 Provides oversight and support for management of health emergencies
XI Nurturing community relationship and involvement
1 Ascertains people’s preferences, needs, problems, challenges, and issues in health service delivery
2 Mobilizes community input in the planning and
implementation of the health services
3 Mobilizes community input in monitoring, evaluation and
ensuring accountability in health service delivery
4
Provides relevant feedback to its stakeholders and the
communities in
the province