Impact of a Health Governance Intervention on Provincial ... of a Health... · A quasi-experimental...

A quasi-experimental study

Impact of a Health Governance

Intervention on Provincial Health

System Performance in Afghanistan

Author

Mahesh Shukla, MD, DrPH, MPA,

Senior Technical Advisor, Management Sciences for Health

Photo: Jawad Jalali/Afghan Eyes

RESEARCH

REPORT

http://www.lmgforhealth.org/

http://www.LMGforHealth.org?utm_source=report-template&utm_medium=pdf



Funding was provided by the United States Agency for International Development (USAID) under Cooperative

Agreement AID-OAA-A-11-00015. The contents are the responsibility of the Leadership, Management, and

Governance Project and do not necessarily reflect the views of USAID or the United States Government.

About the LMG Project

Funded by the US Agency for International Development (USAID), the Leadership, Management, and

Governance (LMG) Project (2011–2017) is collaborating with health leaders, managers, and

policymakers at all levels to show that investments in leadership, management, and governance lead

to stronger health systems and improved health. The LMG Project embraces the principles of

country ownership, gender equity, and evidence-driven approaches. Emphasis is also placed on good

governance in the health sector—the ultimate commitment to improving service delivery and

fostering sustainability through accountability, engagement, transparency, and stewardship. Led by

Management Sciences for Health (MSH), the LMG consortium includes Amref Health Africa,

International Planned Parenthood Federation (IPPF), Johns Hopkins University Bloomberg School of

Public Health (JHSPH), Medic Mobile, and Yale University Global Health Leadership Institute (GHLI).

Table of Contents

List of Abbreviations and Acronyms ............................................................................................................................. 4

1. Abstract ........................................................................................................................................................................... 5

Background ..................................................................................................................................................................... 5

Methods and Findings ................................................................................................................................................... 5

Conclusion ...................................................................................................................................................................... 5

Keywords ........................................................................................................................................................................ 6

2. Problem Statement ....................................................................................................................................................... 6

3. Institutional Context .................................................................................................................................................... 6

4. Methodology .................................................................................................................................................................. 8

Intervention summary .................................................................................................................................................. 8

Measures of health system performance ................................................................................................................. 9

Governance measurement ........................................................................................................................................ 10

5. Sample Size ................................................................................................................................................................... 11

6. Model ............................................................................................................................................................................. 12

7. Results ............................................................................................................................................................................ 12

Governance measurement results .......................................................................................................................... 14

Health system performance ..................................................................................................................................... 15

Difference-in-differences regression analysis ....................................................................................................... 24

Focus group discussions ............................................................................................................................................ 31

8. Discussion ..................................................................................................................................................................... 33

Limitations..................................................................................................................................................................... 35

Strengths ....................................................................................................................................................................... 36

9. Conclusion .................................................................................................................................................................... 36

Areas for future research.......................................................................................................................................... 37

10. Implication for Practice ........................................................................................................................................... 37

References ......................................................................................................................................................................... 38

Appendix 1 ........................................................................................................................................................................ 40

Overall Health Governance Self-Assessment Tool for the PPHCC ............................................................... 40

Health Governance Standards-based Self-Assessment Tool for the PPHCC .............................................. 42

Impact of a Health Governance Intervention on Provincial Health System Performance in Afghanistan (April 2017)

4

List of Abbreviations and Acronyms

ANC Antenatal care

CDR Case detection rate

CHW Community health worker

DD Difference-in-differences

HMIS Health management information system

LMG Leadership, Management, and Governance Project

MDG Millennium development goals

MOPH Ministry of public health

NATO North Atlantic Treaty Organization

NGO Nongovernment organization

OPD Outpatients department

PNC Postnatal care

PPHCC Provincial public health coordination committee

SDG Sustainable development goals

SY Solar year

TB Tuberculosis

TT2+ Two doses of Tetanus Toxoid

UNICEF United Nations Children’s Fund

USAID United States Agency for International Development

WHO World Health Organization


5

1. Abstract

Background

Poor health governance contributes to poor health outcomes and may constrain a country’s progress in

attaining the ambitious health targets of the Sustainable Development Goals. Yet, governance has rarely

been used as a lever to improve the performance of the health sector and health systems. Lack of a

clear body of evidence linking governance interventions to better health system performance is one

likely reason. Evidence of this type is not only limited, but is often anecdotal and lacks a control or

comparison group.

This dearth of evidence means that donors are reluctant to invest in governance interventions, and

governments are at a loss as to where to start. The USAID-funded Leadership, Management, and

Governance (LMG) Project conducted the present study in Afghanistan to examine the causal impact of

a provincial-level health governance intervention on the performance of provincial health systems. The

intervention consisted of governance action planning, implementation of the action plan, and self-

assessment of governance performance before and after the intervention.

Methods and Findings

This quasi-experimental study compares nine health system performance indicators between 16

intervention provinces and 18 non-intervention provinces using a difference-in-differences analysis to

draw inference.

The authors found a statistically and practically significant impact of the intervention on six indicators.

Specifically, the intervention increased a province’s rate of outpatient department visits per person by an

average of 18 percentage points (p<0.01), and achievements in Penta 3 immunization by 17 percentage

points (p<0.01), antenatal visits by 14 percentage points (p<0.01), postnatal visits by 12 percentage

points (p<0.01), tuberculosis case detection by 11 percentage points (p<0.01), and facility delivery by 5

percentage points (p<0.01). No impact was detected on the achievements in tetanus toxoid

administration to pregnant mothers and tuberculosis cure. The secular trend was by far a stronger and

more significant predictor of increasing rates of tetanus toxoid administration to pregnant mothers.

Achievements in community health worker home visits and new family planning users decreased

(p<0.01) by 2 and 1 percentage points respectively. The size of the positive effects is large while that of

the negative effects is small.

Conclusion

Governing bodies at different levels provide an opportunity for governance reform. In this study, a

governance intervention implemented by provincial health coordination committees had a beneficial

impact on provincial health system performance. This indicates the value of increasing the number of

health governance interventions in different health system settings and ensuring that such interventions

are subjected to rigorous impact evaluations.

These study results are relevant to the prevailing situation in low-income countries, where weak health

systems are a significant barrier to addressing outbreaks and epidemics and to providing high quality

health services. Improving the governance of health systems is one way to better enable them to meet

their objectives.


6

Keywords

Governance and health, health governance intervention, health system governance, health system

performance, governance impact

2. Problem Statement Governance has infrequently been used as a lever to improve health sector and health system

performance, arguably because of its perceived sensitivity and complexity [1, 2, 3, 4, 5]. Sustainable

Development Goal (SDG) 16 emphasizes improving governance [6]. Poor governance has contributed

to poor health outcomes [7, 8, 9, 10, 11] and may constrain progress in reaching the SDG 3 health

targets. Good governance could be instrumental in achieving these ambitious health targets [12, 13].

One of the reasons that donors have lacked interest in investing in governance interventions as a way to

improve health system performance is that no clear body of evidence exists on the causal relationship

between the two, especially in low- and middle-income country settings [14]. Governments are at a loss

about where to start. Evidence is sparse and oftentimes anecdotal, or is limited to before and after

studies, usually with no control or comparison group. Our review of studies included in a recent

systematic review [14] showed that most stop at demonstrating association and do not examine

causation. Donors are eager for the evidence demonstrating the link of better governance to

improvements in health system performance; governments continue to remain largely indifferent

because of the politically sensitive nature of governance reforms, or because they do not know where

to begin such reforms.

This quasi-experimental study examines the causal relationship between a provincial-level health

governance intervention and provincial health system performance by comparing performance in

intervention and non-intervention provinces using a difference-in-differences methodology to draw

inference. The study contributes critical empirical evidence to a technical area in need of such. Our

findings indicate the potential of improved governance to improve health system performance. We

conclude that policymakers whose goal is to improve health system performance or health service

delivery should consider implementing interventions to improve governance of the health sector and

health systems.

3. Institutional Context In Afghanistan, shuras, or informal consultative assemblies of elders, have a long and well-established

tradition of resolving disputes and solving contentious issues in communities. By contrast, the Provincial

Public Health Coordination Committee (PPHCC) is a formal multi-stakeholder committee that was

established in the early 2000s by the Ministry of Public Health (MOPH) to carry out a set of distinct

responsibilities at the provincial level [15]. PPHCCs have been functional since that time in all 34 of

Afghanistan’s provinces. Their level of functionality varies from province to province, with the majority

tending to be weak and ineffective. PPHCCs provide a forum for coordination and information sharing

among various stakeholders in the provincial health system. They discuss community health concerns,

and coordinate and participate in all stages of emergency response. They also oversee health posts and

health facilities by conducting joint monitoring and supervision visits during which a subset of committee


7

Box 1: Three-phase governance intervention

First phase: Baseline measurement of governance and participatory development of health systems governance

development action plans

The PPHCC teams participated in 2-day workshops facilitated by public health and governance experts. The

first day was spent working in groups discussing actions to apply the effective governing practices (cultivating

accountability, engaging stakeholders, setting a shared strategic direction, and stewarding resources) in their

work over the next six months to better meet the health needs and expectations of the people. Using the

governance guide and the framework of governing practices, each committee identified actions to be taken in

the next six months to improve their governance and developed a governance action plan. The four practices

provided the organizing framework to structure their committee deliberations and governance action plan. On

the second day, participants carried out a self-assessment of their governance performance at baseline.

Second phase: Implementation and monitoring

In implementing their governance action plans, the PPHCCs worked to improve engagement with the public

and communities, and to become more transparent, accountable, and responsive. No additional resources

were made available to the provinces and districts to carry out their planned activities. Committees monitored

implementation of their action plans so that underperformance could be identified and corrected along the

way. The PPHCCs monitored actions in the plan on a monthly basis using a simple Excel-based monitoring

instrument. Progress reports were sent to the Provincial Liaison Directorate of the MOPH.

The PPHCCs monitored the progress by the extent to which actions were implemented. Progress on an action

or activity was classified in five categories: Not started (0%), early stage (1-25% of an action is completed), two

intermediate stages (26-50% or 51-75% of an action is completed), and advanced stage of completion (76-100%

of an action is completed).

Third phase: Evaluation

The PPHCCs evaluated their performance during the governance action plan implementation period in 2-day

workshops held six months after the implementation began. The PPHCC re-assessed their governance, as a

committee, using the same instruments they had used at baseline.

members joins the provincial health team. They are expected to meet on a monthly basis and coordinate

delivery of the basic package of health services as well as the essential package of hospital services. In

essence, this committee governs the provincial health system. That said, it should be noted that their

governance role is limited—they are an advisory, consultative and coordinating body. They have

oversight authority, but no authority to hire and fire. Setting strategy, making rules and regulations,

establishing policy and procedures, and allocating resources are all tasks predominantly performed by

the Ministry and the government at the national level.

The PPHCC is a multi-sectoral governing body chaired by the Provincial Public Health Director. It has

21 members comprising nine appointed provincial public health officers, the provincial hospital director,

the director of the Institute of Health Sciences, two representatives of NGOs that provide health

services at the health post and health facility levels, two district health officers, and one representative

from each of the following: The Ministry of Women’s Affairs, the Provincial Reconstruction Team, the

private health sector, the elected provincial council, UNICEF, and WHO. Thirteen members have voting


8

powers, including six officials from the provincial public health office; the provincial hospital director; and

members representing the private health sector, provincial council, UNICEF, WHO, and NGOs.

Decisions made in the PPHCC are usually consensus-based. When consensus is not reached, a decision

is put to a vote. A decision requires a quorum and a majority of members voting in favor. The members

are not paid a salary and do not receive sitting fees for serving on the PPHCC. The governance

development intervention in this study described in the methods section was primarily implemented by

the PPHCCs in their provinces.

4. Methodology This study examines the causal impact of a provincial health governance intervention on provincial health

system performance in Afghanistan. The study design is quasi-experimental and uses Health Management

Information System (HMIS) data to isolate impact by comparing performance in provinces with the

governance intervention to those without, using a difference-in-differences analysis strategy. We

hypothesized that the governance intervention improved health system performance.

We use difference-in-differences analysis to draw inference. In this methodology, outcomes are

observed for two groups for two time periods: pre- and post-intervention. One group is exposed to an

intervention while the other is not. The difference between post- and pre-intervention in the control

group is subtracted from the difference in the intervention group. This methodology removes biases in

the post-period comparisons between the intervention and control group that could result from

permanent differences between the groups, as well as biases from comparisons over time in the

intervention group that could be the result of a time trend [16].

Intervention summary

In each intervention province, the governance intervention was conducted in three phases over a period

of six months, and targeted four essential governing practices: cultivating accountability, engaging

stakeholders, setting a shared strategic direction, and stewarding resources.

In the first phase, using the provincial health system governance guide and the framework of essential

governing practices (http://www.lmgforhealth.org/content/key-practices-good-governance), the PPHCCs

identified actions to be taken to improve the governance of their provincial health system, and designed

a governance action plan for this purpose. They also measured the committee’s governance

performance at baseline using self-assessment instruments. In the second phase, the committees

implemented and monitored their action plans over a period of six months. In the third and final phase,

the committees evaluated the implementation of their action plan and measured governance

performance post-intervention using the same self-assessment instruments. At this stage, they also

examined each of the action items in the action plan to assess progress, and to assess if they planned on

continuing their implementation beyond the initial six-month period. See Box 1 for the three phases of

the intervention and Table 1 for examples of activities selected by the PPHCCs to improve their

governance. Throughout the intervention period, a donor-funded provincial public health advisor who

was already embedded in the provincial public health office to provide technical assistance in public

health management supported the PPHCC in the implementation of their governance development plan

and its monthly monitoring. We described this intervention and its pilot testing in detail in an earlier


9

journal publication [15]. In order to obtain feedback on the intervention at the end of pilot

implementation in the initial three provinces, we held focus group discussions with the PPHCC

members of these three provinces.

Table 1 illustrates actions planned by the PPHCCs.

Table 1: Examples of activities selected by the PPHCCs to implement their governance action plan

Governing practice Examples of activities

Improve stakeholder engagement Interview patients and health service users

Invite religious, youth, and women leaders to meetings

Provide feedback to consultative assemblies at health facility level

Consult community leaders on a regular basis

Cultivate accountability Share information on resources and performance with communities

and stakeholders

Encourage health workers to share their challenges during joint

monitoring visits

Review health workers’ job descriptions and provide clear service

delivery targets

Set a shared strategic direction Constitute a team of representatives from the community, health

service users, other health system stakeholders, and district health

officers from each district to identify the health needs and challenges

faced by the communities, and to communicate these needs to the

PPHCC for consideration while deciding their strategic direction

Invite health facility shura members to the PPHCC meetings to better

understand community health concerns

Steward resources responsibly Train provincial public health office staff and health workers in ethical

conduct

Recognize health workers with outstanding performance

Involve the community in health facility monitoring

Use data, information, evidence, and technology for decision making

Measures of health system performance

The focus of the study is health system performance, since it has a direct influence on population health

outcomes. The governance intervention was not undertaken to improve governance for the sake of

governance. Instead, the intention was to improve organizational or health system performance through

improvements in governance. For this reason, the outcomes of interest we selected were health system

performance indicators in addition to governance self-assessment scores.

We collected HMIS data on percentage achievement against the MOPH-established targets on nine

indicators of health systems performance: 1) proportion of children less than one year of age who

received the Penta3 vaccine; 2) proportion of pregnant women who received at least one antenatal care

visit; 3) proportion of women who received at least one postnatal care visit; 4) proportion of pregnant

women who received two doses of tetanus toxoid; 5) proportion of deliveries conducted at a health

facility; 6) outpatient visit rate per person; 7) community health worker home visit rate; 8) proportion


10

of new family planning users in target population; and 9) TB case detection rate. In addition, the TB cure

rate was chosen as a health outcome variable because TB is highly prevalent and is a public health

priority in Afghanistan.

Why did we select these indicators? We expected to see a change in maternal and child health

indicators because maternal and newborn care and child health and immunization are the top two

priority elements among the seven elements of the Basic Package of Health Services [17], which is a

mainstay of primary health care in Afghanistan. Likewise, family planning and tuberculosis control are top

public health priorities of the MOPH and provincial health systems in Afghanistan. Any systemic

improvement in the provincial and district health systems, for example its governance, was therefore

expected to be reflected in the indicators related to maternal and child health, family planning and

tuberculosis.

These indicators were also selected because they are proxies of overall health system performance.

Health system strengthening interventions focusing on improving leadership, management, governance,

or health financing will have a direct bearing on the coverage of health services and on access to the

health services the system delivers. Hence, we expected to see improvement in these indicators.

Governance measurement

We developed two self-assessment instruments to measure governance performance and to be used by

committees as a whole. These instruments were based on the extant role of the committees, and also

the expanded role they aspired to take on to make their governance more effective and people-

centered. The two measurement instruments are summarized in Table 2 and are available in full in the

Appendix 1.

One of the two instruments was for self-assessment of performance on the PPHCC governance

responsibilities. The PPHCCs graded their own performance on a 1-10 scale on each responsibility of

the committee. The other instrument helped the PPHCCs self-assess their governance against

established standards in 11 provincial public health core functions. No progress on a standard was

scored 0, 1-25% accomplishment was scored 1, 26-50% was scored 2, 51-75% was scored 3, and 76-

100% was scored 4.

Table 2: PPHCC governance measurement instruments used in the study

#

Name of the

governance self-

assessment scale

Based on Administered to

whom

Frequency of

administration

What is

measured

1

Overall PPHCC

governance

performance

assessment scale

Roles and

responsibilities of

the PPHCC PPHCCs (PPHCC

as a whole did

collective self-

assessment)

Pre- and Post-

intervention

PPHCC

governance

performance

2

Governance

standards- based

scale

PPHCC governance

standards


11

Intervention timeline

Afghanistan has 34 provinces. Of these, 16 provinces implemented the governance intervention.1 The 18

non-intervention provinces served as comparison provinces.2 The MOPH and the U.S. Agency for

International Development’s Leadership, Management, and Governance (LMG) Project, which assists the

MOPH, selected provinces based on where the LMG Project was active. Thus, the selection of

provinces was purposeful instead of random.

Figure 1: Intervention timeline

The provinces were staggered in their implementation of the governance intervention over a period of

27 months—three provinces implemented it from the month of April to October of 2012 [Sawar to

Mizan of Solar Year (SY) 1392], nine provinces did so from the month of July to December of 2013

[Saratan to Qaws of SY 1393], and the remaining four provinces implemented it from the month of

February to July of 2014 [Dalwa of SY 1393 to Saratan of SY 1394]. Figure 1 depicts the time line in

terms of the Gregorian calendar.

5. Sample Size Province-month or the month in which a province was observed is the unit of analysis for analyzing the

impact of intervention on the health system performance indicators. We had 13 months of pre-

intervention data when none of the provinces had the intervention introduced. Similarly, we had eight

months of post-intervention data when all 16 intervention provinces had completed their intervention.

Thus there were a total of 714 province-months in the analysis sample ((16+18) provinces*(13+8)

months).

1 Intervention provinces (16): Badakhshan, Baghlan, Bamyan, Farah, Faryab, Ghazni, Helmand, Hirat, Jawzjan, Kandahar,

Khost, Nimroz, Paktika, Paktya, Takhar, and Wardak 2 Comparison provinces (18): Badghis, Balkh, Dykundi, Ghor, Kabul, Kapisa, Kunar, Kunduz, Laghman, Logar, Nangarhar,

Nooristan, Panjsher, Parwan, Samangan, Sar-e-Pul, Urozgan, and Zabul


12

6. Model We used the simplest set up where outcomes are observed for two groups for two time periods. One

of the groups (intervention provinces) is already exposed to the treatment (provincial health governance

intervention) in the post-period but not in the pre-period. The second group (comparison provinces) is

not exposed to the treatment during either period. Since we observe the same provinces within the

group in each time period, the average gain in the comparison provinces is subtracted from the average

gain in the intervention provinces. This removes biases in second period comparisons between the two

groups that could be the result of permanent differences between those groups, as well as biases from

comparisons over time in the treatment group that could be the result of time trends.

Data varies by province (i), and month (t). Outcome is Yit, and there are two periods: pre and post. The

intervention will occur in a group of provinces. There are three key variables: Interventionit, Postit, and

Interventionit*Postit (or intervention*post).

Yit = β0 + β1 Interventionit + β2 Postit + β3 (Interventionit*Post)it + εit .......................................(1)

where

Interventionit =1 if observation i belongs to the province that will eventually be treated

Postit =1 in the period when the intervention had already occurred

Interventionit*Postit is the interaction term i.e., intervention provinces after the intervention

The coefficient of our interest is β3, which multiplies the interaction term, (Interventionit*Post)it. β3

denotes the true effect of treatment or the effect of treatment on the treated.

We also use the following extended form of this model that controls for more variables:

Yit = β0 + β1 Interventionit + β2 Postit + β3 (Interventionit*Post)it + β4 Povertyit + β5 Labor Participation

Rateit + β6 Female Literacy Rateit + β7 Securityit + εit .................................................................(2)

where

Poverty is the poverty headcount rate in the province, and security denotes the security rating

in the province based on a scale of 1 (worst security) to 5 (acceptable security as decided by

NATO during this time for transition purposes). The poverty headcount rate, labor participation

rate, and female literacy rate were added because of their likely influence on the outcome

variables. Data on these rates were taken from provincial briefs published by the World Bank in

June 2011.

7. Results

Governance action plan implementation

The PPHCCs implemented many specific governance actions. For example, they invited religious, youth,

and women leaders to PPHCC meetings; provided feedback to health facility consultative council

members; ensured they better assessed, understood and addressed community health concerns; shared

information on resources and performance with communities and stakeholders; encouraged health

workers to share their challenges during joint monitoring visits; recognized health workers for

outstanding performance; involved the community in health facility monitoring, and used data,


13

information, and evidence for PPHCC decision making. At the end of the six-month intervention period,

they assessed their performance in implementing these governance actions.

Figure 2 depicts the progress of the intervention provinces in implementing governance action plans at

the end of the initial six-month implementation phase. As mentioned earlier, the provinces were

staggered in their implementation of the intervention. Three provinces that implemented the

intervention in the first phase, five provinces that implemented the intervention in the second phase, and

four provinces that implemented the intervention in the third phase are shown in the chart through

three bars. Four provinces that implemented the intervention in the fourth and final installment did not

carry out the post-intervention self-assessment, and hence do not appear in the chart.

The figure shows that the intervention provinces reported a high level of completion of their

governance development action plans. On average, the intervention provinces completed 55% of their

action plan activities in six months, and an additional 29% were at an intermediate stage of completion.

The committees planned on continuing more than 90% of these actions in the future beyond the original

six-month implementation period.

Figure 2: Progress in implementing governance action plans at the end of the initial six-month

implementation period

58% 58% 50%

55%

20% 13%

15%

16%

10%

14%

16%

13%

6% 4%

8% 6%

7% 12% 12% 10%

0%

20%

40%

60%

80%

100%

Wardak, Khost

and Herat

(Phase 1

provinces)

Takhar, Baghlan,

Badakshan,

Jawzjan and

Faryab (Phase 2

provinces)

Ghazni, Paktya,

Helmand and

Paktika (Phase 3

provinces)

Overall for 12

provinces

Percentage of activities not started

Percentage of activities that did not

progress much

Percentage of activities half

complete

Percentage of activities at advanced

stage

Percentage of activities near

complete or complete


14

Governance measurement results

PPHCCs of the intervention provinces self-assessed their governance performance at baseline and again

after six months of intervention (See Table 3 and Figure 3). PPHCCs of the non-intervention provinces

did not do these self-assessments. Overall, we found significant improvements in self-assessed

governance scores. PPHCCs improved their governance score on average by 23.7% and 22.6% using two

different scales (one based on roles and responsibilities [47 items; = 0.91] and the other based on

governance standards [46 items; = 0.97]). The overall improvement upon adding the scores from both

scales is 23.4%. Improvement was higher in the provinces that began with a lower baseline governance

score.

Table 3: PPHCC governance self-assessment scores in the intervention provinces

# Province PPHCC overall

governance performance

scale based on PPHCC

roles and responsibilities

(Maximum score 450)

Governance standards- based

scale for the PPHCC

(Maximum score 184)

PPHCC overall governance

performance

(Maximum score 634)

Pre Post Percent Point

Change


Change


Change

1 Badakhshan 265 363 22 90 175 46 355 538 29

2 Baghlan 285 415 29 112 119 4 397 534 22

3 Bamyan 306 * 119 * 425 *

4 Farah 205 * 94 * 299 *

5 Faryab 253 346 21 85 145 33 338 491 24

6 Ghazni 153 375 49 66 146 43 219 521 48

7 Helmand 188 381 43 77 129 28 265 510 39

8 Hirat 300 331 7 109 124 8 409 455 7

9 Jawzjan 253 382 29 95 129 18 348 511 26

10 Kandahar 312 * 148 * 460 *

11 Khost 255 351 21 96.5 143 25 351.5 494 22

12 Nimroz 256 * 101 * 357 *

13 Paktika 250 293 10 95 99 2 345 392 7

14 Paktya 213 416 45 101 150 27 314 566 40

15 Takhar 356 347 -2 133 159 14 489 506 3

16 Wardak 256.5 308 11 91.5 132 22 348 440 15

Overall

percentage point

improvement in

governance

23.7 22.6 23.4

*Four intervention provinces, i.e., Bamyan, Farah, Kandahar, and Nimroz did not do the governance self-assessments.


15

The intervention provinces scored higher on both the governance measurement scales. Out of a total

score of 450 of the scale based on PPHCC roles and responsibilities, the intervention provinces scored

higher after going through the governance intervention (359 ± 38) as opposed to before the

intervention (252 ± 52); thus producing a statistically significant increase of 107 (95% CI, 61 to 153),

t(11) = 5.11, p = 0.0002, d=2.34. Effect size is large and significant (Cohen's d = 2.34).

Similarly, out of a total score of 184 of the scale based on PPHCC governance standards, the

intervention provinces scored higher after going through the governance intervention (137.5 ± 20) as

opposed to before the intervention (96 ± 17); resulting in a statistically significant increase of 42 (95%

CI, 25 to 58), t(11) = 5.52, p = 0.0001, d=2.23. Effect size is large and significant (Cohen's d = 2.23).

Figure 3: PPHCC governance self-assessment scores, pre- and post-intervention

Total possible governance score = 634

Scales used: PPHCC overall governance performance scale (maximum score 450) and governance standards- based scale (maximum

score 184)

PPHCCs improved their governance score by an average of 23.4%. Improvement was higher in the provinces that started with a lower

baseline score on governance.

In the study, the governance intervention was a means to improve health system performance.

Having seen the results of governance measurement, let us now turn to the results of health

system performance measurement.

Health system performance

Descriptive statistics

As shown in Table 4, over the 48-month period for which HMIS data was collected, the performance of

provinces (both intervention and comparison) was more than 200% of the MOPH-established target for

the outpatient department (OPD) visit rate per person and tetanus toxoid administration. However, this

figure may be the result of the MOPH establishing a very low target for these indicators. In the

particular case of tetanus toxoid administration, it may also be due to the health staff reporting each of

the two doses separately. Performance was also more than 100% of the MOPH-established target for

antenatal care visits and Penta3 immunization. Provinces performed less than 100% of the MOPH-

0

100

200

300

400

500

600

700

Ghaz

ni

Helm

and

Pak

tya

Far

yab

Pak

tika

Jaw

zjan

War

dak

Khost

Bad

akhsh

an

Bag

hla

n

Hir

at

Tak

har

Pre

Post


16

established target for postnatal care visits, facility delivery, and TB case detection rate and cure rates;

and less than 50% of the MOPH-established target for community health worker home visits and new

family planning users.

Table 4: Indicator-wise average achievement (%) against MOPH-set targets

# Indicator All provinces Intervention Provinces Comparison Provinces

Obs M (SD) 95% CI Obs M (SD) 95% CI Obs M (SD) 95% CI

1 penta3 1632 135 (42) [133, 137] 768 136 (44) [133, 139] 864 134 (39) [132, 137]

2 anc 1632 121 (33) [120, 123] 768 124 (34) [122, 126] 864 119 (32) [117, 121]

3 pnc 1632 84 (29) [82, 85] 768 85 (26) [84, 87] 864 82 (32) [80, 84]

4 delivery 1632 61 (25) [60, 62] 768 64 (25) [62, 65] 864 59 (25) [57, 60]

5 tt2+ 1632 306 (122) [300, 312] 768 295 (115) [287, 303] 864 316 (127) [307, 324]

6 opd 1632 221 (59) [218, 224] 768 209 (51) [206, 213] 864 231 (65) [227, 235]

7 chw hvr 1632 31 (17) [30, 32] 768 39 (16) [38, 40] 864 24 (15) [23, 25]

8 newfp 1632 20 (8) [19, 20] 768 19 (8) [19, 20] 864 21 (8) [20, 21]

9 tbcdr 1632 75 (38) [73, 77] 768 84 (37) [81, 86] 864 68 (38) [65, 70]

10 tbcure 1632 77 (46) [75, 79] 768 81 (47) [78, 84] 864 74 (45) [71, 77]

Notes:

a) Obs is the number of observations in terms of the number of province-months

b) M is the mean and SD is standard deviation. These numbers are the percentage achievement against MOPH-

established targets. For example, Penta3 immunization = 135 indicates that intervention provinces had achieved 135

percent of the target set by the MOPH for immunization of children less than one year of age with three doses of

pentavalent vaccine.

c) Penta3 (penta3) is the percentage achievement among children less than one year of age who received the third dose

of pentavalent vaccine (combined vaccine with five individual vaccines conjugated into one, intended to protect people

from Haemophilus Influenza type B, whooping cough, tetanus, hepatitis B and diphtheria), calculated against MOPH-

established target

d) TT2+ rate (tt2+) is the percentage achievement among pregnant women who received the TT2+ vaccine, calculated

against the MOPH-established target e) Facility delivery (delivery) is the percentage achievement among women who delivered at a health facility, calculated

against MOPH-established target

f) One ANC visit (anc) is the percentage achievement among pregnant women who received at least one antenatal care

visit, calculated against MOPH-established target

g) One PNC visit (pnc) is the percentage achievement among new mothers who have received at least one postnatal

care visit, calculated against MOPH-established target

h) TB CDR (tbcdr) is the TB case detection rate X 100

i) OPD visit rate per person is the total number of OPD visitors in the quarter / Quarterly Target Population

j) CHW home visit rate (chw hvr) is calculated as the number of home visits X 100 / Number of Target visits in a

month

k) New FP (newfp) is New Family Planning users, calculated as the number of New FP Users in the month X 100 /

Monthly Target Population

l) TB Cure Rate (tbcure) is calculated as the number of cases that completed treatment and were smear-negative / the

number that started treatment X 100

As shown in Table 5, eight of ten indicators increased significantly (p < .01) from the pre-period to the

post-period in the intervention provinces. The new family planning user rate and TB cure rate did not

experience significant rises. Similarly, eight of ten indicators increased significantly (p < .01 for seven

indicators and p < .05 for one indicator) in non-intervention provinces. The TB case detection rate and

TB cure rate did not significantly improve. Difference-in-differences was statistically significant for four

variables: antenatal care visits (p < .05); postnatal care visits (p < .01); Penta3 immunization (p < .01),

and the OPD visit rate per person (p < .05).


17

Table 5: Health system performance and health outcomes, pre- and post-period

# Performance

Indicator

Intervention Provinces$

Comparison Provinces$$

Difference-

in-

differences

Confidence

Interval

p-

value

Pre

-

inte

rvention

Post

-

inte

rvention

Diffe

rence

Confiden

ce

Inte

rval

Pre

-

inte

rvention

Post

-

inte

rvention

Diffe

rence

Confiden

ce

Inte

rval

Health System Performance

1 Penta 3 124 (28) 148 (70) 24** [14,35] 129 (31) 136 (37) 7* [1, 15] 17** [4, 29] 0.0087

2 One ANC visit 111 (34) 136 (36) 24** [17,32] 114 (36) 125 (31) 11** [4, 18] 14* [3, 24] 0.0112

3 One PNC visit 69 (23) 98 (27) 29** [24,35] 72 (33) 90 (34) 17** [11, 24] 12** [3, 21] 0.0085

4 Facility delivery 53 (22) 74 (27) 22** [16,27] 50 (23) 66 (27) 16** [11, 21] 5 [-2, 13] 0.1497

5 TT2+ 175 (78) 350 (71) 175** [159, 192]

199 (84) 365 (118) 166** [145, 186]

10 [-17, 37] 0.4684

6 OPD visit 176 (44) 241 (48) 66** [55,76] 208 (62) 256 (70) 48** [34, 61] 18* [0.4,35] 0.0442

7 CHW home visit 36 (14) 42 (17) 6** [2, 9] 21 (14) 29 (16) 8** [5, 11] -2 [-12, 8] 0.6827

8 New FP user 18 (8) 20 (8) 1 [-0.2,3] 19 (8) 22 (7) 3** [1, 5] -1 [-4, 1] 0.2376

9 TB case detection

77 (34) 91 (44) 14** [5, 22] 68 (44) 71 (37) 3 [-12, 6] 11 [-1, 23] 0.0800

Health Outcome

10 TB cure rate 76 (33) 81 ( 43) 4 [-4, 13]

74 (44) 69 (30) -5 [-13, 4] 9 [-3, 21] 0.1297

*p < .05. **p < .01. $208 observations in the pre-period and 128 observations in the post- period (total 336 observations) $$234 observations in the pre-period and 144 observations in the post- period (total 378 observations)

Notes:

a) 124 (28) in the first row should be read as M(SD) (i.e.,, mean=124 and standard deviation=28). These numbers are

the percentage achievement against the MOPH-established target. For example, Penta3 immunization = 124 indicates

that intervention provinces had achieved 124 percent of the target set by the MOPH for immunization of children

below one year of age with pentavalent vaccine during the pre-intervention period. A difference of 24 in the fifth

column of the first row means a difference of 24 percentage points.

b) Indicators 1-8 are calculated as the percentage achievement as against MOPH-established targets, and not counts.

c) p-value was obtained by conducting two-sample equal variance two-tailed t-tests.

We performed a significance test in the pre-intervention period to see if the intervention and

comparison provinces were similar in profile in terms of the ten indicators, i.e., if there was a parallel

trend in the pre-intervention period. Intervention facilities had on average lower achievements in Penta

3 immunization, antenatal visits, postnatal visits, tetanus toxoid administration to pregnant mothers,

outpatient department visits, and new family planning users, while had higher achievements in facility

delivery, community health worker home visits, and tuberculosis case detection and cure. We

performed t-tests to see if these differences were statistically significant. See table 6 for results.

During the pre-intervention period, there is no statistically mean difference, at .05 level of significance, in

the achievements in six areas, i.e., Penta 3 immunization, antenatal visits, postnatal visits, facility delivery,

new family planning users, and tuberculosis cure between intervention and comparison provinces.

There are statistically significant differences, at .05 level of significance, between intervention and

comparison provinces in the achievements in four areas, i.e., tetanus toxoid administration to pregnant


18

mothers, outpatient department visits, community health worker home visits, and tuberculosis case

detection.

Table 6: Results of t-tests in the pre-intervention period

# Outcome variable

Pre-intervention period

Intervention provinces Comparison provinces df t p

Obs M (SD) Obs M (SD) 95% CI

for mean diff.

1 Penta 3 208 124.43 (28.14) 234 129.36 (30.81) -0.60, 10.47 440 -1.75 0.080

2 One ANC visit 208 111.30 (33.59) 234 113.90 (35.91) -3.92, 9.12 440 -0.78 0.434

3 One PNC visit 208 68.97 (22.56) 234 72.34 (32.65) -1.93, 8.68 440 -1.24 0.212

4 Institutional delivery 208 52.62 (21.99) 234 50.20 (23.21) -6.65, 1.83 440 1.11 0.264

5 TT2+ 208 174.68 (78.04) 234 199.02 (83.53) 9.17, 39.51 440 -3.15 0.001

6 OPD visits 208 175.49 (44.14) 234 208.44 (61.74) 22.80, 43.09 440 -6.38 0.000

7 CHW home visits 208 35.86 (14.30) 234 20.99 (13.76) -17.49, -12.24 440 11.12 0.000

8 New FP users 208 18.13 (7.93) 234 18.64 (7.88) -0.97, 1.98 440 -0.67 0.502

9 TB case detection rate x

100 208 77.13 (33.78) 234 68.06 (43.98) -16.46, -1.66 440 2.40 0.016

10 TB Cure Rate 208 76.22 (33.24) 234 73.85 (44.34) -9.77, 5.03 440 0.62 0.529

Notes:

Obs = observations in terms of the number of province-months

M=Mean, SD=Standard deviation, and CI=confidence interval.

t is t-statistic and p is p-value.

We plotted mean monthly achievement rates for the indicators over the 48-month period. See Figures

4-13 for these plots. Months 1-13 are the pre-intervention period and months 41-48 are the post-

intervention period. Note that pre-intervention trends are approximately parallel for the first nine of

the ten plots. Figures 4-13 show the monthly average for each of the ten indicators in terms of

percentage achievement against the MOPH-established targets in the pre-intervention and post-

intervention periods.


19

Figure 4

Figure 5

10

012

014

016

018

0

Perc

ent a

ch

ievem

ent

13 40Months 0-13 is pre-intervention period | Post-intervention period is months 40-48

0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month

Comparison provinces Fitted values

Intervention provinces Fitted values

Penta 3 immunization80

90

10

011

012

013

014

015

0

Perc

ent a

ch

ievem

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month

Comparison Provinces Fitted values


Pregnant women who have received at least one ANC visit


20

Figure 6

Figure 7

60

80

10

0

Perc

ent a

ch

ievem

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



Pregnant women who have received at least one PNC visit

15

017

019

021

023

025

027

0

Perc

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



Outpatient visit rate per person per month


21

Figure 8

Figure 9

45

50

55

60

65

70

75

80

Perc

ent a

ch

ievem

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



Institutional delivery

10

020

030

040

0

Perc

ent a

ch

ievem

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



Tetanus toxoid administration to pregnant mothers


22

Figure 10

Figure 11

20

25

30

35

40

Perc

ent a

ch

ievem

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



Community health worker home visits15

20

25

Perc

ent a

ch

ievem

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



New family planning users


23

Figure 12

Figure 13

50

70

90

11

0

Perc

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



TB case detection rate

50

70

90

11

0

Perc

ent


0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48Month



TB cure rate


24

Difference-in-differences regression analysis

In the full model, we found a statistically and practically significant impact of the intervention on six

indicators. Specifically, the intervention increased a province’s rate of outpatient department visits per

person by an average of 18 percentage points (p<0.01), and achievements in Penta 3 immunization by 17

percentage points (p<0.01), antenatal visits by 14 percentage points (p<0.01), postnatal visits by 12

percentage points (p<0.01), tuberculosis case detection by 11 percentage points (p<0.01), and facility

delivery by 5 percentage points (p<0.01). No impact was detected on the achievements in tetanus

toxoid administration to pregnant mothers and tuberculosis cure. The secular trend was by far a

stronger and more significant predictor of increasing rates of tetanus toxoid administration to pregnant

mothers. Achievements in community health worker home visits and new family planning users

decreased (p<0.01) by 1 and 2 percentage points respectively. The size of the positive effects is large

while that of the negative effects is small.

We conducted difference-in-differences regression analysis. We began with Model 1, which had three

key variables: Interventionit, Postit, and Interventionit*Postit. In Model 2, we added four covariates:

poverty headcount rate of the province, labor force participation rate of the province, and female

literacy rate of the province, and its security categorization. Finally, we added province and time fixed

effects in Model 3.

As mentioned earlier, we had 13 months of pre-intervention data and eight months of post-intervention

data, thus a total of 714 province-months in the analysis sample.

Table 7: Effect of the intervention on health system performance and health outcomes (regression

coefficients)

Variable Model 1

Basic model

Model 2

with covariates

Model 3 with

province and time fixed effects

B SE B β B SE B β B SE B β

Penta3

intervention -4.93 3.93 -0.05 -9.24* 3.96 -0.10* -1.36 6.14 -0.01

post 7.53 4.37 0.08 7.53 4.30 0.08 9.92** 3.29 0.11**

intervention x post 16.81** 6.38 0.15** 16.81** 6.27 0.15** 16.81** 4.53 0.15**

poverty -0.26** 0.08 -0.11** -0.44** 0.16 -0.19**

labor force participation

-0.07 0.12 -0.02 -0.73** 0.24 -0.25**

female literacy -0.68* 0.34 -0.09* -1.35** 0.39 -0.18**

security 4.49** 1.38 0.13** 7.61** 2.56 0.22**

R2 0.0414 0.0796 0.5456

F for change in R2 10.23** 8.72** 17.41**

ANC

intervention -2.59 3.27 -0.03 0.43 3.28 0.00 13.76** 3.28 0.19**

post 10.91** 3.64 0.14** 10.91** 3.56 0.14** 10.68** 3.56 0.14**

intervention x post 13.50* 5.31 0.14* 13.50* 5.19 0.14* 13.50** 5.19 0.14**

poverty 0.20** 0.07 0.10** 0.15 0.07 0.07


25

labor force

participation -0.09 0.10 -0.04 -1.11** 0.10 -0.45**

female literacy 0.84** 0.28 0.13** -1.24** 0.28 -0.20**

security -2.37* 1.15 -0.08* 1.33 1.15 0.04

R2 0.0616 0.1105 0.7127 -0.05

F for change in R2 16.61** 12.53** 39.45**

PNC

intervention -3.37 2.80 -0.05 -2.42 2.75 -0.03 7.23* 2.94 0.11*

post 17.44** 3.11 0.26** 17.44** 2.98 0.26** 16.92** 1.57 0.26**

intervention x post 12.03** 4.53 0.14** 12.03** 4.35 0.14** 12.03** 2.17 0.14**

poverty 0.35** 0.05 0.21** 0.04 0.07 0.02

labor force

participation -0.27** 0.08 -0.12** -1.23** 0.11 -0.56**

female literacy 0.39 0.23 0.07 -0.83** 0.18 -0.15**

security 2.35* 0.96 0.09 4.62** 1.22 0.18**

R2 0.1356 0.2080 0.8015

F for change in R2 37.12** 26.49** 63.60**

Institutional Delivery

intervention 2.41 2.32 0.04 3.25 2.32 0.06 7.06** 2.31 0.13**

post 16.14** 2.58 0.30** 16.14** 2.51 0.30** 15.80** 1.24 0.29**

intervention x post 5.46 3.76 0.08 5.46 3.67 0.08 5.46** 1.71 0.08**

poverty 0.14** 0.04 0.10** -0.41** 0.06 -0.29**


-0.21** 0.07 -0.11** -0.67** 0.09 -0.37**

female literacy 0.61** 0.20 0.13** -0.27 0.14 -0.06

security 0.96 0.81 0.04 -3.45** 0.96 -0.16**

R2 0.1313 0.1712 0.8316

F for change in R2 35.77** 22.04** 71.62**

TT2+

intervention -24.34** 8.39 -0.10** -24.41** 8.44 -0.10** -5.52 11.15 -0.02

post 165.61** 9.33 0.66** 165.61** 9.16 0.66** 183.88** 5.98 0.73**

intervention x post 9.84 13.60 0.03 9.84 13.35 0.03 9.84 8.22 0.03

poverty 0.54** 0.18 0.08** 0.85** 0.29 0.13**


-0.63* 0.26 -0.07* -3.90** 0.44 -0.46**

female literacy -2.98** 0.72 -0.14** -8.74** 0.70 -0.41**

security -5.20 2.95 -0.05 13.44** 4.64 0.14**

R2 0.4737 0.4955 0.8192

F for change in R2 213.00** 99.06** 65.71**

OPD

intervention -32.94** 5.41 -0.25** -27.95** 5.11 -0.21** 33.76** 5.48 -0.26**


26

post 47.80** 6.01 0.36** 47.80** 5.54 0.36** 50.62** 2.94 0.38**

intervention x post 17.71* 8.77 0.10* 17.71* 8.08 0.10* 17.71** 4.04 0.10**

poverty 0.84** 0.10 0.24** 0.84** 0.14 0.24**


0.08 0.15 0.01 -0.84** 0.21 -0.18**

female literacy -2.69** 0.44 -0.24** -4.04** 0.34 -0.36**

security -10.05** 1.79 -0.19** -4.47 2.28 -0.08

R2 0.2252 0.3451 0.8452

F for change in R2 68.79** 53.15** 79.15**

CHW HVR

intervention 14.86** 1.43 0.43** 15.32** 1.36 0.45** 35.61** 1.06 1.05**

post 7.81** 1.59 0.22** 7.81** 1.47 0.22** 7.37** 0.57 0.21**

intervention x post -2.08 2.32 -0.04 -2.08 2.15 -0.04 -2.08** 0.78 -0.04**

poverty -0.20** 0.02 -0.22** -0.37** 0.02 -0.41**


0.20** 0.04 0.17** 0.65** 0.04 0.55**

female literacy 0.38** 0.11 0.13** 0.01 0.06 0.00

security -1.55** 0.47 -0.11** -3.70** 0.44 -0.27**

R2 0.2121 0.3275 0.9158

F for change in R2 63.72** 49.12** 157.61**

New FP

intervention -.50 0.74 -0.03 0.73 0.70 0.04 4.77** 0.72 0.30**

post 2.91** 0.82 0.18** 2.91** 0.76 0.18** 2.99** 0.39 0.18**

intervention x post -1.46 1.20 -0.07 -1.46 1.11 -0.07 -1.46** 0.53 -0.07**

poverty 0.03* 0.01 0.08* -0.11** 0.01 -0.26**


0.20** 0.02 0.37** 0.06* 0.02 0.11*

female literacy 0.51** 0.06 0.38** -0.03 0.04 -0.02

security -0.09 0.24 -0.01 2.62** 0.30 0.42**

R2 0.0255 0.1712 0.8161

F for change in R2 6.20** 20.84** 64.36**

TB CDR

intervention 9.06* 3.79 0.11* 6.91 3.69 0.08 5.50 4.96 0.06

post 2.93 4.22 0.03 2.93 4.00 0.03 5.61* 2.66 0.06*

intervention x post 10.79 6.15 0.10 10.79 5.84 0.10 10.79** 3.66 0.10**

poverty 0.31** 0.07 0.14** 0.05 0.13 0.02


-0.10 0.11 -0.03 -0.04 0.19 -0.01

female literacy -0.43 0.31 -0.06 -1.47** 0.31 -0.21**

security 7.09** 1.29 0.22** 7.39** 2.06 0.23**

R2 0.0397 0. 1393 0. 6805

F for change in R2 9.78** 16.32** 30.89**


27

TB Cure Rate

intervention 2.37 3.66 0.03 0.04 3.64 0.00 9.22 7.01 0.11

post -4.60 4.07 -0.05 -4.60 3.95 -0.05 -8.46* 3.76 -0.10*

intervention x post 9.04 5.93 0.09 9.04 5.77 0.09 9.04 5.17 0.09

poverty -0.34** 0.07 -0.16 -0.33 0.18 -0.16


-0.20 0.11 -0.07 -0.10 0.27 -0.03

female literacy -1.59** 0.31 -0.23** -0.72 0.44 -0.10

security -4.51** 1.27 -0.14** 2.81 2.92 0.09

R2 0.0089 0.0686 0.2916

F for change in R2 2.14 7.43** 5.97**

*p < .05. **p < .01.

Nine indicators increased in the post-intervention period as compared to the pre-intervention period;

the increase was statistically significant (p<0.01 for eight and p<0.05 for one). The TB cure rate

decreased in the post-intervention period (p<0.05).

Model 1 (Basic model)

The coefficient on intervention X post (Intervention*Post) variable is statistically significant for antenatal

and postnatal care visits, Penta3 immunization, and the OPD visit rate per person as dependent

variables. There was no statistically significant difference between the intervention and comparison

provinces in terms of the first three of these outcome variables at baseline. However, the intervention

provinces had a 33% lower OPD visit rate at baseline when compared to non-intervention provinces

(see intervention = -32.94** for OPD in Table 4). There was a statistically significant secular trend of

increase in the antenatal care, postnatal care, and OPD visit rates.

Treatment (i.e., governance interventions) significantly predicted the four outcome variables (see

coefficient on intervention X post variables): Penta3 immunization achievement, R2 = .04, F(3, 710) =

10.23, p < .001; antenatal care visits, R2 = .06, F(3, 710) = 16.61, p < .001; postnatal care visits, R2 = .13,

F(3, 710) = 37.12, p < .001; and the OPD visit rate per person, R2 = .22, F(3, 710) = 68.79, p < .001. It is

important to note that the intervention explained a small proportion of variance in at least two of these

four variables. The intervention explained 23% of the variance in the OPD visit rate, 14% of the variance

of the postnatal care visit achievement, 7% of the variance in antenatal care visit achievement, and 4% of

the variance in Penta3 immunization achievement.

On average, the governance intervention increased a province’s OPD visit rate per person by 18

percentage points, Penta 3 immunization achievement by 17 percentage points, antenatal visit

achievement by 14 percentage points, and postnatal visit achievement by 12 percentage points.

The governance intervention did not have impact on achievements in any of the remaining six variables:

facility delivery, community health worker home visits, tetanus toxoid administration to pregnant

mothers, new family planning users, the TB case detection rate, and the TB cure rate.

There was a strong secular trend of increasing facility delivery, tetanus toxoid administration to

pregnant mothers, and new family planning users. The secular trend was by far a stronger and more

significant predictor of these three outcome variables.


28

At baseline, the achievement of tetanus toxoid administration to pregnant mothers was on average 24%

lower in intervention provinces than in comparison provinces, whereas the TB case detection rate was

9% higher in the intervention provinces. Community health worker home visit achievement was 15%

higher in the intervention provinces at baseline.

Model 2 (Model with covariates)

The main results of Model 1 were maintained after controlling for poverty headcount ratio, labor force

participation rate, female literacy rate, and security categorization of the province. On average, the

governance intervention increased a province’s OPD visit rate per person by 18 percentage points,

Penta 3 immunization achievement by 17 percentage points, antenatal visit achievement by 14

percentage points, and postnatal visit achievement by 12 percentage points. Model 2’s main results are

very similar to Model 1’s main results.

Female literacy had a positive and statistically significant association with achievements in antenatal care

visits, facility delivery, community health worker home visits, and new family planning users. Contrary to

expectation, female literacy was inversely associated with Penta3 immunization achievement,

achievement in administering tetanus toxoid to pregnant mothers, the OPD visit rate per person, and

the TB cure rate.

As expected, deteriorating security had a statistically significant inverse association with antenatal visit

achievement, the OPD visit rate per person, community health worker home visit achievement, and the

TB cure rate. Contrary to what one would expect, deterioration in security was associated with

increase in Penta3 immunization achievement, postnatal care visit achievement, and TB case detection

rate in a statistically significant manner.

Poverty was associated with a decrease in achievement of Penta3 immunization and community health

worker home visits, as well as the TB cure rate. Surprisingly, poverty was associated with an increase in

achievement of antenatal and postnatal care visits, facility delivery, tetanus toxoid administration to

pregnant mothers, the OPD visit rate per person, new family planning users, and the TB case detection

rate.

Higher labor force participation was associated with a decrease in achievements in postnatal care visits,

facility delivery, and tetanus toxoid administration to pregnant mothers in a statistically significant

manner. It was associated with increases in achievement of community health worker home visits and

new family planning users.

Model 3 (Model with province and time fixed effects)

Province and time fixed effects were added in Model 3. The coefficient on intervention X post

(Intervention*Post) variable is statistically significant for achievements in antenatal and postnatal care

visits, facility delivery, Penta3 immunization, community health worker home visits, new family planning

users, and the OPD visit rate per person and TB case detection rate as dependent variables. At baseline,

there was no statistically significant difference between the intervention and comparison provinces in

terms of achievement in Penta3 immunization, tetanus toxoid administration to pregnant mothers, and

the TB case detection rate, and the TB cure rate. However at baseline, the intervention provinces had

higher achievement rates in antenatal and postnatal care visits, facility delivery, OPD visits, community

health worker home visits, and new family planning users. There was a statistically significant secular


29

trend of increase in the achievement in all the indicators except the TB cure rate which showed a

statistically significant secular trend of decrease with time.

Treatment (i.e., the governance intervention) significantly predicted the eight outcome variables (see

coefficient on intervention X post variables): Penta3 immunization achievement, R2 = .54, F(46, 667) =

17.41, p = .0000; antenatal care visits, R2 = .73, F(46, 667) = 39.45, p = .0000; postnatal care visits, R2 =

.81, F(46, 667) = 63.60, p = .0000; facility delivery, R2 = .83, F(46, 667) = 71.62, p = .0000; the OPD visit

rate per person, R2 = .84, F(46, 667) = 79.15, p = .0000; community health worker home visits, R2 = .91,

F(46, 667) = 157.61, p = .0000; new family planning users, R2 = .81, F(46, 667) = 64.36, p = .0000; and

the TB case detection rate, R2 = .68, F(46, 667) = 30.89, p = .0000. It is important to note that the

intervention explained a significantly higher proportion of variance in Model 3 as compared to Model 1.

That is, the intervention explained at least 50% of the variance whereas Model 1 had explained less than

25% variance.

On average, the governance intervention increased a province’s OPD visit rate per person by 18

percentage points, Penta 3 immunization achievement by 17 percentage points, antenatal visit

achievement by 14 percentage points, postnatal visit achievement by 12 percentage points, facility

delivery achievement by 5 percentage points, and the TB case detection rate by 11 percentage points.

On the contrary, the intervention decreased a province’s achievement in new family planning users by 1

percentage point, and community health worker home visits by 2 percentage points.

The governance intervention did not have an impact on the remaining two variables: tetanus toxoid

administration to pregnant mothers and the TB cure rate.

Sensitivity analysis

One crucial assumption for difference-in-differences estimations is the exogeneity of the policy

implementation. A question might be posed whether the governance intervention was exogenous in this

case. The intervention provinces were the ones already being assisted by USAID for a long time. The

comparison provinces were assisted by other donors, the European Union and the World Bank, which

might bias the results. We had no direct way to find out if the better results were driven by the work or

actions implemented by USAID compared with other donors.

To verify whether results were mixing up the effect of the intervention with some other unobservable

trend or confounding factor, we conducted a series of falsification and robustness checks. We found our

results to be robust according to all the checks we carried out.

The pre-intervention period spanned the first 13 months of the data. We conducted a “placebo test" on

this data where the initial six months are treated as the pre-intervention period and months 7-13 are

treated as the post-intervention period. We should not see any impact of the intervention in this

placebo test since, in reality, the intervention had not occurred during this period.

Table 8 shows the results of the placebo test. As expected, we did not find any impact of the placebo

intervention on Penta3 immunization rates. The coefficient on intervention X post (Intervention*Post)

variable was not statistically significant. The result goes against the potential endogeneity of the

governance intervention (getting the causation wrong).


30

Table 8: Placebo test conducted in the pre-intervention period data (Difference-in-differences

regression coefficients)

Variable

Model 1

Basic model

Model 2

with covariates

Model 3 with

province and time fixed effects

B SE B β B SE B β B SE B β

Penta3

intervention -4.85 4.15 -0.08 -7.99 4.13 -0.13 -8.63* 4.10 -0.14*

post -2.70 3.88 0.04 -2.70 3.79 -0.04 -5.57 3.85 -0.09

intervention x post -0.14 5.66 -0.00 -0.14 5.53 -0.00 -0.14 2.85 -0.00

poverty -0.10 0.07 -0.06 0.06 0.10 -0.03


-0.13 0.11 -0.06 -0.72** 0.15 -0.34**

female literacy -0.01 0.30 -0.00 -0.69** 0.25 -0.13**

security 5.60** 1.25 0.24** 7.75** 1.65 0.33**

R2 0.2607 0.0483 0.7469

F for change in R2 1.34 4.20** 29.28**

As a next step, we conducted a robustness check by considering several outcomes that had nothing to

do with the health governance intervention. Firstly, we generated a placebo indicator with random

values within a plausible range and tested the results. Secondly, we serially tested poverty headcount

rate, labor participation rate, and female literacy rate as outcome variables in the model. Again, we

should not see any impact of the intervention either on the placebo or the three real world indicators

(poverty, labor participation, and female literacy) that are not likely to be influenced by the health

governance intervention that too in a period of six months. Indeed, we did not find any impact on any of

the four indicators. Table 9 shows these results.

Table 9: Robustness check using several outcomes that had nothing to do with the health

governance intervention (Difference-in-differences regression coefficients)

Variable Model 1

Basic model

Model 2

with covariates

B SE B β B SE B β

Poverty

intervention -5.07** 1.76 -0.13** -6.87** 1.72 -0.18**

post -0.00 1.96 -0.00 -0.00 1.89 -0.00

intervention x post 0.00 2.85 0.00 0.00 2.76 0.00

labor force

participation -0.07 0.05 -0.05

female literacy -0.57** 0.14 -0.17**

security 2.12** 0.60 0.14**

R2 0.0185 0.0851

F for change in R2 4.45** 10.96**


31

Labor force

participation

intervention -1.53 1.36 -0.05 -3.05* 1.21 -0.10*

post -0.00 1.51 -0.00 -0.00 1.32 -0.00


poverty -0.03 0.02 -0.04

female literacy -1.34** 0.09** -0.54**

security -1.15** 0.42** -0.10**

R2 0.0029 0.2470

F for change in R2 0.68 38.64**

Female literacy

intervention -1.35* 0.55 -0.11* -1.06* 0.43 -0.09*

post -0.00 0.61 -0.00 -0.00 0.47 -0.00


poverty -0.03** 0.00 -0.11**


-0.17** 0.01 -0.42**

security -1.73** 0.13 -0.37**

R2 0.0136 0.4123

F for change in R2 3.26* 82.68**

Random values

intervention 1.11 1.65 0.03 0.59 1.69 0.01

post -2.27 1.83 -0.06 -2.27 1.83 -0.06


poverty -0.00 0.03 -0.00

labor force

participation -0.04 0.05 -0.04

female literacy 0.07 0.14 0.02

security 1.18* 0.59 0.08*

R2 0.2657 0.2323

F for change in R2 1.32 1.33

Power analysis

We tested the power of our statistical model and statistical tests to detect the change in indicators. Our

model and tests were adequately powered; statistical power was 0.80 or above.

Focus group discussions

As stated earlier, we held focus group discussions with the PPHCC members to obtain feedback on the

intervention at the end of pilot implementation in the initial three provinces. The committees reported

many achievements and successes, as summarized in box 2.

There were also goals that committees wanted to accomplish but could not; for example, better

coordination between shuras or committees at different levels and more effective communication with


32

Box 2: Findings of focus group discussion

1. Enhanced transparency and accountability: PPHCC members noted that their meetings had become more

regular; had clear agendas; and minutes of meetings were documented and made public. The members

reported that new sub-committees that oversee accountability, for example financial audit and

transparency subcommittee and governance subcommittee were established at the province level;

information and decisions of the committees were shared through emails, press conferences, magazines,

newsletters and social media websites; private sector pharmacies and food stores that lacked legal

documents or did not follow regulatory standards were closed down; expired medicines were rounded

up through monitoring visits; and judicial proceedings were started against corrupt health facility

personnel.

The members stated that their activities as a committee became more transparent and involved diverse

stakeholders; inter-sector coordination and collaboration improved; there was an improved focus on

patients’ health problems at the community level; community health needs were defined, and discussed;

coordination, communication, and information sharing improved; the frequency of data analysis and

presentation to the committees increased, and more committee decisions were evidence based.

2. Inclusion of stakeholders and stakeholder views: The PPHCC members reported that steps were taken to

recruit more women to community health nursing educator posts; suggestion and complaint boxes were

placed outside health posts and the community complaints were discussed during regular meetings;

attendance of members at the meetings improved; civil society groups, community leaders and

representatives from other sectoral departments were invited to the committee meetings; community

concerns were discussed as a standing agenda item during committee meetings; and vaccination rejection

in some villages was addressed by negotiating with elders and through community mobilization efforts.

3. Setting shared strategic direction: The PPHCC members reported that the MOPH guidelines and policies

were introduced and shared in the PPHCC meetings; service delivery data was analyzed and shared more

frequently with committee members; and annual plans were developed and shared with the stakeholders

in the province and the MOPH.

4. Stewardship of resources: A committee reported success in mobilizing resources to build toilets in the

health facility and completing the overdue construction of a community health center. Health center

buildings and facilities were improved through community support at a few places, and inspections and

enforcement of quality standards also improved.

The members observed that there was a better link between committees at provincial and district levels.

Communities began participating in health facility monitoring, gave feedback on the quality of health

services, and became motivated to contribute to health service delivery improvements.

the public. Some of the actions in the governance action plans needed substantial extra resources that

the committees did not have. Overall, lack of resources emerged as one of the common challenges the

committees faced in completing their action plans. The security situation often prevented engagement of

provincial and district governors, and did not allow for the level of health facility monitoring that the

committees would have liked to achieve. Resources for training and education of committee members

were found to be inadequate. Community expectations on health service delivery rose, and the

committees did their best to meet these expectations by mobilizing community support and resources.

Overall, committees reported notable changes in their knowledge, skills, and behaviors, including feeling

more capable, responsive, and accountable than they were before the intervention. Committee

functioning became more systematic and regular, and members felt more responsible for their decisions.


33

Committees also noticed improvements in their effectiveness; referral of TB cases for treatment

improved in one district, un-served remote areas were identified, and 90% of them were covered

through establishment of mobile teams in another district. One committee reported that antenatal and

postnatal care visits increased, and other maternal and child health services improved. Because of

increased community engagement, committees felt they could solve problems at the health facility level

in collaboration with the local community. This experience showed them that they could build trust with

the communities by working with them.

Committee members said they would continue applying effective governing practices in the future

mainly because they felt their achievements in the short six month period were encouraging. They

became aware of weaknesses in their governing skills and capacity and resolved to improve. Members

thought they gained many benefits at a personal level because of changes in their attitudes and behavior.

They also became aware of their stewardship role and wanted to do more for the communities they

served. The intervention, PPHCC members told us, renewed their commitment to their governance

responsibilities. They observed that periodic governance assessments (because they became aware of

their governance responsibilities and their governance performance) and the overall experience

developed their capacity in discharging their governance role. The committees recommended that the

MOPH should officially introduce the approach in all the provinces and districts, and expressed interest

in sharing their experience with the uncovered jurisdictions.

8. Discussion We found that health systems’ governance can be improved even in fragile and conflict affected

environments. Focus group discussions showed that when health governing bodies in the communities,

the district, and the province worked in coordination, community health concerns could be effectively

represented and addressed and health systems could become more responsive to the community needs

within the available limited resources.

How an organization or health system is governed is a determinant of its performance. Good

governance enables effective and efficient management of people, money, medicines, and information.

Sound management, in turn, facilitates the work of doctors, nurses, and other health workers, enabling

them to provide safe, timely, effective, and efficient care and services that are respectful of and

responsive to individual patient or health service user preferences, needs, and values. We had

hypothesized that the governance intervention would thus have a positive impact on the health system

performance indicators used in the study.

First of all, through the focus groups, we learned how governance could translate into improved health

system performance. In our intervention, we introduced four practices of good governance – cultivating

accountability, engaging with stakeholders, setting a shared strategic direction, and stewarding resources

– that provided an organizing framework for leaders to develop governance improvement plans. When

the Provincial Health Coordination Committee members, who have a predominant governing role,

designed and implemented their governance action plans based on these practices, their governing

behavior, and consequently the governance of their provincial health systems, improved. Inter-sector

and inter-departmental collaboration received a boost; this is relevant in the context of health as the


34

work of many sectors other than health influences the health status of the population.

The PPHCC members designed their action plan in a participatory and consultative manner. This

seemed to foster a sense of responsibility to successfully implement it. The intervention was focused on

people, i.e., health system leaders governing in close partnership with health managers, health providers,

health workers, community leaders, health service users, and governance leaders in other sectors. This

helped ensure the intervention was meaningful for the governance leaders as well as for the community.

As health systems become decentralized, sub-national structures and committees are entrusted with the

responsibility to coordinate, implement, and oversee health services; they are expected to play a

governance role. That they are closer to the people helps. Governing bodies at the community level can

represent the unresolved health needs of their communities to governing bodies at the district and

provincial levels, which may have more resources to address them.

We discovered interventions to improve governance skills are feasible in fragile and conflict-affected

environments, when practices of good governance are consistently applied, periodically assessed and

continuously improved.

Through the quantitative part of the study, we saw a significantly positive impact on six out of nine

indicators of provincial health system performance, a marginally negative impact on two indicators of

provincial health system performance, and no impact on the remaining one indicator of health system

performance and one indicator of health outcomes. We found significant positive impact on achievement

of antenatal and postnatal care visits, facility delivery, Penta3 immunization, outpatient department visits,

and tuberculosis case detection. There was no impact on achievements in tetanus toxoid administration

to pregnant mothers, and the TB cure rate. We also saw marginally negative impact on community

health worker home visits and new family planning users though the size of this negative effect is small;

i.e., less than 2 percentage points.

There could be several reasons for these mixed results – six positive, two null, and two negative.

Firstl, all the ten result areas are important national priorities. However, the PPHCCs in the

intervention provinces might have given the six positive result areas (antenatal and postnatal care, facility

delivery, immunization, OPD visits, and tuberculosis case detection) even higher priority in their

province.

Second, these six positive results might have been “low hanging fruit.” More difficult-to-achieve

indicators (for example, TB cure or new family planning users) could not be accelerated during the six

months of intervention. However, at this time, we don’t know if more time would have made a

difference. It is also possible that the PPHCCs did not sustain their improved governance practices long

enough to have a statistically and also clinically significant impact on the four indicators (tetanus toxoid

administration to pregnant mothers, tuberculosis cure, community health worker home visits, and new

family planning users). Note that the performance in tetanus toxoid administration is already exceedingly

high and there may have been little margin for further improvement.

Third, it might be the case that insecurity posed disproportionate hurdles in the intervention provinces.

Fourth, the governance intervention in and of itself might not be enough to make a difference to the

remaining four intractable indicators. In other words, those indictors may have also needed an

intervention at the health service management level or service delivery level, or perhaps at a


35

combination of both levels. Interventions at the health manager, provider or patient level could

potentially have helped. Finally, all of the intervention provinces have been assisted by USAID for more

than a decade, whereas other provinces were assisted by other donors (the European Union and the

World Bank) for a similar length of time. There might have been a qualitative difference in the support

provided by USAID when compared to the other donors. It is possible that the intervention provinces

were comparatively well-prepared to derive disproportionate benefit from the intervention, or at least

in the six areas where we saw improvements.

Limitations

These findings may not be generalizable to all low- and middle-income countries. Note that Afghanistan

is an archetypal case of a fragile and conflict affected environment. It may not be representative of low-

income countries in general. It might be the case that governance interventions bear impact in systems

that are severely underperforming and have modest goals like Afghanistan. We do not know for sure if

they are as effective in systems that are moderately underperforming and need to reach much higher

standards. Recall that governance improvement in this case was higher in the provinces where the

baseline was lower.

However, our study does invite readers to make connections between its elements and their own

experience. Moreover, the evidence generated by our study is significant given that there is a scarcity of

evidence in this area. The difference-in-differences method has several shortcomings, including possible

endogeneity of the interventions themselves, issues relating to the standard error of the estimate, and

requirement of a parallel trend assumption [18].

Moreover, effects identified by difference-in-differences may not have a causal interpretation if the policy

change occurred in a jurisdiction that derives unusual benefits from doing so [19]. All of the intervention

provinces in our study were USAID-assisted provinces, whereas most of the comparison provinces

were assisted by other donors. We do not know if the intervention provinces were able to derive

unusual benefits from the governance intervention because they were comparatively better-prepared in

health management and/or health service delivery. This might be the case because, in all the intervention

provinces, the provincial public health advisor embedded in the provincial public health office had been

providing public health management support for years before the intervention began. The comparison

provinces did not have such advisors for any considerable length of time.

The simplicity of the difference-in-differences method comes at a price in terms of assumptions. To

obtain impact estimates, the crucial identifying assumption one has to make is - the counterfactual trend

is the same for treated and non-treated units [20]. This assumption can only be tested if more data are

available.

The identifying assumption requires the investigators to assume whatever happened to the control

group over time is what would have happened to the treated group in the absence of the program [21,

22]. It is relatively strong assumption and tends to overlook all other possible differences between the

groups that could have led to the observed outcomes. The difference-in-differences method attributes

any differences in trends between the treatment and control groups to the intervention as long as those

differences occur at the same time as the intervention. The estimation becomes biased if there are other

factors that affect the difference in trends between the two groups [23, 24].


36

To mitigate this methodological limitation, we demonstrated a broadly parallel trend during the pre-

intervention period by plotting mean monthly rates of all health system performance variables during the

48-month period that includes the 13-month pre-intervention period. We also performed t-tests in the

pre-intervention period to see if the intervention and comparison provinces were similar in profile in

terms of the ten indicators.

Another limitation of the study is its short duration. We do not know if six months were enough for the

governance intervention to have an effect on health system performance.

The Hawthorne effect (i.e., governance improved because governance leaders modified their behavior

for the purpose of the assessment and because their performance was being ‘observed’) could also be a

plausible alternative explanation for the results. Governance self-assessments in comparison jurisdictions

could have either refuted or established the Hawthorne effect as the reason for improvement in

governance scores. We could not carry out these assessments due to a lack of time and resources.

PPHCC members measured governance by conducting the self-assessments collectively as a group.

There is an element of subjectivity in the self-assessments. We reduced this bias by designing and

conducting collective self-assessments. Group self-assessments are less vulnerable to subjectivity than

individual self-assessments because group processes can moderate over-rating—if one member of a

group over-rates performance on an item, another group member can bring this to the attention of the

group, which can in turn affect the group’s final rating of that item.

Strengths

Our study has several strong points. It examines the contribution of governance to health system

performance and generates early evidence in this regard using a rigorous quantitative method. It builds

on the Project’s earlier work in the field [15] and establishes the link between the governance of a

health system and its performance.

Our study results are relevant to the situation obtaining in many low-income countries where weak

health systems have been a significant barrier to effectively addressing outbreaks and epidemics and to

providing good quality services to citizens. The international donors and development partners working

in these countries are looking for ways to strengthen the health systems; improving their governance is

one way to do this.

9. Conclusion We conclude that a provincial health governance intervention has the potential to positively impact

health system performance. However, this potential cannot be taken for granted. We still do not know

if the positive impact is experienced every time such interventions are implemented or when

implemented at all different levels of the health systems. We saw beneficial impact only in 67% (six out

of nine) of the health system performance indicators that were included in the study. This is not to imply

that these interventions are not worthy of consideration. Instead, they need to be studied more and the

results documented more systematically.


37

Areas for future research

This study yields directions for future research. Our intervention lasted six months; we recommend that

longer duration governance interventions be the subject of future studies. Governance improvements

might need time to translate into improvements in health system performance at health facility level.

They might need to be implemented over sustained period of time to realize gains in the health system

performance.

Governance is an enabler of management and management is an enabler of service provision. Our

intervention focused on governance from the perspective of the people who govern; it did not directly

involve health managers or service providers. All three groups have a role to play in improving health

governance. In addition, the provinces sit between the national ministry at the top, and the districts,

health facilities, and communities below. Governance happens at all of these levels, but our intervention

only focused on the provincial level. We recommend future studies of interventions at the different

levels, as well as complex multi-dimensional and multi-level studies.

There is also a case to be made for conducting similar interventions and research in different types of

organizations delivering health services - public organizations, civil society organizations and private for-

profit organizations - to examine the link between their governance and their organizational or health

system performance. There is a multitude of sophisticated studies examining the link between corporate

governance and profitability of a firm [25, 26, 27, 28], but we hardly find such studies examining the link

between health systems governance and system performance.

Cost effectiveness studies are also lacking in this arena, including comparing the cost-effectiveness of

implementing a governance intervention on the top of a public health intervention to implementing only

a public health intervention. Finally, this is mainly a quantitative study. More qualitative studies should be

done in the future to open “the black box” that sits between governance and organizational

performance [29].

10. Implication for Practice Our study has an implication for practice. Entry points for improving governance in the health sector

may not be readily obvious to policymakers. Governing bodies have the potential to contribute to

improved health services and the health status of populations. Often they do not get necessary support

in terms of clear mandates, authority, and resources. We have designed and successfully tested an

approach to improving governance of a provincial health system that enhances its performance. This

approach could be of interest to governing bodies in the health sector of low- and middle-income

countries.


38

References

1. Siddiqi S, Masud TI, Nishtar S, Peters DH, Sabri B, Bile KM, Jama MA. Framework for assessing

governance of the health system in developing countries: gateway to good governance. Health Policy.

2009; 90(1):13-25.

2. Veillard JHM, Brown AD, Barış E, Permanand G, Klazinga NS. Health system stewardship of

National Health Ministries in the WHO European region: concepts, functions and assessment

framework. Health Policy. 2011;103(2):191-199.

3. WHO Regional Office for Europe. Governance for health in the 21st century. A study

conducted for the WHO Regional Office for Europe. Smart governance for health and well-being. 2011.

Available at: http://www.euro.who.int/__data/assets/pdf_file/0019/154252/Health-2020-Conf.-

Governance-for-health-....pdf. Accessed 29 Jul 2016.

4. Mikkelsen-Lopez I, Wyss K, de Savigny D. An approach to addressing governance from a health

system framework perspective. BMC international health and human rights. 2011. BMC Int Health Hum

Rights. 2011;11:13.

5. Brinkerhoff D, Bossert T. Health governance: concepts, experience and programming options.

2008.

6. United Nations General Assembly. Resolution adopted by the General Assembly on 25

September 2015 70/1. Transforming our world: the 2030 agenda for sustainable development. 2015.

Available at: http://www.un.org/ga/search/view_doc.asp?symbol=A/RES/70/1&Lang=E. Accessed 29 Jul

2016.

7. Gupta S, Davoodi HR, Tiongson E. Corruption and the provision of health care and education

services. In Political Economy of Corruption. 1st edition. Edited by Jain AK. London and New York:

Routledge. 2002:111–41.

8. Azfar O, Kähkönen S, Meagher P. Conditions for effective decentralized governance: A synthesis

of research findings. College Park: IRIS (Institutional Reform and the Informal Sector) Center, University

of Maryland, Center for Institutional Reform and the Informal Sector. 2001:67-73.

9. Delavallade C. Corruption and distribution of public spending in developing countries. J Econ

Fin. 2006;30(2):222-39.

10. Lindelow M, Serneels P. The performance of health workers in Ethiopia: results from qualitative

research. Soc Sci Med. 2006; 62(9):2225-35.

11. Rajkumar AS, Swaroop V. Public spending and outcomes: Does governance matter? J Dev Econ.

2008; 86(1):96-111.

12. Olafsdottir AE, Reidpath DD, Pokhrel S, Allotey P. Health systems performance in sub-Saharan

Africa: governance, outcome and equity. BMC Public Health. 2011; 11:237-44.

13. Svensson J, Björkman M. Power to the people: evidence from a randomized field experiment of

a community-based monitoring project in Uganda. Q J Econ. 2009;124(2):735-69.

14. Ciccone DK, Vian T, Maurer L, Bradley EH. Linking governance mechanisms to health outcomes:

A review of the literature in low-and middle-income countries. Soc Sci Med. 2014;117(86-95).

15. Anwari Z, Shukla M, Maseed BA, Wardak GFM, Sardar S, Matin J, Rashed GS, Hamedi SA, Sahak

H, Aziz AH. Implementing people-centred health systems governance in 3 provinces and 11 districts of

Afghanistan: a case study. Conflict and health. 2015; 9(1):1-6.


39

16. Wooldridge J. What’s new in econometrics? Difference-in differences estimation, Lecture notes

10 for NBER. Cambridge: NBER Summer Institute. 2007. Available at:

http://www.nber.org/WNE/Slides7-31-07/slides_10_diffindiffs.pdf. Accessed 27 Jul 2016.

17. Newbrander W, Ickx P, Feroz F, Stanekzai H. Afghanistan's basic package of health services: its

development and effects on rebuilding the health system. Global Public Health. 2014;9(Suppl 1):S6–S28.

http://doi.org/10.1080/17441692.2014.916735

18. Epstein AJ. Effects of report cards on referral patterns to cardiac surgeons. J Health Econ.

2010;29(5):718–31.

19. Besley T, Case A. Unnatural experiments? Estimating the incidence of endogenous policies. The

Economic Journal. 2000; 110: F672-94.

20. Dimick JB, Ryan AM. Methods for evaluating changes in health care policy: the difference-in-

differences approach. JAMA. 2014;312(22):2401.

21. Abadie A. Semiparametric difference-in-differences estimators. The Review of Economic Studies.

2005; doi:10.1111/0034-6527.00321.

22. Angrist JD, Pischke J. Mostly harmless econometrics: An empiricist's companion: Princeton

University Press; 2008.

23. Card D, Krueger AB. Minimum Wages and Employment: A Case Study of the Fast-Food Industry

in New Jersey and Pennsylvania. Am Econ Rev. 1994; 84(4):772-793.

24. Bertrand M., Duflo E. and Mullainathan S. How Much Should We Trust Differences-in-

Differences Estimates? Quarterly Journal of Economics. 2004; 119(1):249-275.

25. Black BS, Jang H, Kim W. Does corporate governance predict firms' market values? Evidence

from Korea. Journal of Law, Economics, and Organization. 2006;22(2):366-413.

26. Gompers PA, Ishii JL, Metrick A. Corporate governance and equity prices. Q J Econ.

2003;118:107–55.

27. Black B. The corporate governance behavior and market value of Russian firms. Emerging

Markets Review. 2001:89–108.

28. Durnev A, Kim E. To steal or not to steal: Firm attributes, legal environment, and valuation.

Journal of Finance. 2005:1461–93.

29. Maxwell JA. Using qualitative methods for causal explanation. Field methods. 2004; 16(3):243-64.

doi: 10.1177/1525822X04266831.


40

Appendix 1

Overall Health Governance Self-Assessment Tool for the PPHCC

Directions: For each item, note briefly the internal strengths and weaknesses within the PPHCC as well as

the opportunities and threats that exist in the current environment. You will rate performance on a

scale of from 1 to 10, with 10 being the best possible rating.

# Responsibility Performance (scale 1–10)

Role and responsibilities as per ToR

1 Meeting the MOPH policies, priorities, objectives, strategies, and standards:

a. Overall

b. In relation to the BPHS

c. In relation to the EPHS

2 Coordination of all stakeholders:

a. At the provincial level

b. In the districts and the communities

c. With the MOPH

3 Improvement of services:

a. Overall quality

b. Accessibility

c. Sustainability

4 Input into provincial planning, consolidation of the provincial plan, and monitoring of its implementation

5 Oversight: Development and implementation of a joint supervision plan

6 Coordinate an emergency response and participate in it

7 Coordinate and participate in immunization campaign

8 Provide technical and consultation support to the PPHD

9 Sharing of information related to service delivery, MOPH policies, strategies, standards, new initiatives, and any problems and challenges

10 Identify sites for new health facilities

11 Mediate disputes that arise among stakeholders and ensure shared understanding between governmental organizations and NGOs (cases referred by DPHO)

12 Follow up of the work of committees:

a. HMIS Committee

b. EPI Task Force

c. Emergency Task Force

d. CDC Committee

e. Reproductive Health Committee

f. Maternal and Child Survival Committee

g. Child and Adolescent Health Committee

h. Nutrition Committee


41

i. Pharmacy Committee

j. Environmental Health Committee

k. Any other provincial health-related committee

Expanded role and responsibilities

13 Advocate and ensure openness, transparency, accountability, honesty, and inclusiveness:

a. In the health system of the province

b. In the affairs of the provincial health directorate

14 Set strategic direction:

a. For 3–5 years

b. Strategic plan is based on the strategic direction

15 Advocate and support achievement of health outcomes, responsiveness and patient satisfaction, and patient safety

16 Nurture relationships with the communities and the people served

17

Stewardship of scarce resources:

a. Ethical and best use of available resources for achievement of health outcomes for the

people served

b. Exert influence across different sectors for achieving best health outcomes for the population in province

c. Provide vision and direction for health systems

d. Collect and use information and evidence on health system performance to ensure

accountability and transparency

18 Ensure adequate financial resources

19 Provide financial oversight

20 Monitor performance

21 Support high performance

22 Strengthen health services in the province

23 Continuous improvement of the functioning of the PPHCC

24 Build a competent PPHCC

25 Advocate for decentralization

26 Any other responsibility PPHCC fulfills

Scoring criteria

The maximum score that can be earned is 450.

Score of 338 and above: Outstanding governance

Score of 226–337: Meets most requirements

Score of 113–225: Needs improvement

Score below 113: Unsatisfactory governance


42

Health Governance Standards-based Self-Assessment Tool for the

PPHCC

This instrument is based on 11 PPHO core functions and 46 PPHC governance standards.

Scoring guide

0% 1%–25% 26%–50% 51%–75% 76%–100%

No activity Minimal activity Moderate

activity

Significant

activity Optimal activity

0%, or

absolutely

no activity

Greater than

zero, but no

more than 25% of

the activity

described within

the standard is

met

Greater than

25%, but no more

than 50% of the

activity described

within the

standard is met

Greater than

50%, but no more

than 75% of the

activity described

within the

standard is met

Greater than 75%

of the activity

described within

the standard is

met

Instrument

# ToR 0% 1%–25% 26%–50% 51%–75% 76%–100%

I Oversight for health situation and trend assessment

1 Facilitates access to appropriate resources for community health status monitoring, and mobilizes resources and support for the surveys that are centrally designed

2 Promotes broad-based participation and coordination among all entities active in collecting, analyzing, and

disseminating community health status data

3 Provides oversight and support for community health status

monitoring efforts

4 Guides improvements in the health status monitoring efforts

II Oversight of monitoring and evaluation of health services

1 Facilitates access to the necessary resources to conduct periodic monitoring and evaluations

2 The PPHCC itself evaluates the health services

3 Makes sure that regular supportive supervision, monitoring and evaluation of health services provided in the public and

the private sectors in the province take place

4 Encourages all relevant stakeholders provide input into

monitoring and evaluation processes

5 Reviews evaluation results and utilizes these results to improve health service performance

III Oversight for data and information management

1 Facilitates access to appropriate resources for data and information management

2 Promotes broad-based participation and coordination among all entities active in data and information management tasks


43

# ToR 0% 1%–25% 26%–50% 51%–75% 76%–

100%

3

Reviews data and information management on a quarterly

basis, and provides oversight and support for data and information management efforts

4 Guides improvements in overall data and information management

IV Oversight for health service delivery

1 Oversees public and nongovernmental organizations and the private sector responsible for delivery of health services

2 Allows community monitoring of the delivery of health services

3 Facilitates community input in problem identification and problem solving

4 Conducts periodic reviews of health service delivery with special attention to services for vulnerable populations

V Oversight of coordination, communication, and intersectoral collaboration activities

1 Facilitates access to national, state, and local resources that could be used in support of these activities

2

Establishes and oversees the implementation of policies to

support activities to inform, educate, and empower people about public health issues, and reviews these activities in light of community needs

3 Makes sure that all population subgroups have an opportunity to provide input on health issues and health

services

4 Exerts influence across sectors to protect and promote

health of the community

VI Supporting health service delivery

1

Ensures that the PPHO is supporting the implementing

NGOs through joint visits, assessment of quality of service, and assistance in staff recruitment and training

2 Encourages members from lead and secondary NGOs to surface their issues in a timely fashion

3 Mobilizes community support of the NGOs delivering appropriate services

4 Recognizes NGOs delivering quality services to vulnerable populations

VII Oversight of health resource management

1 Establishes and oversees the implementation of policies designed to assure efficient and effective use of physical,

financial, and human resources and drugs and supplies

2

Establishes and oversees the implementation of policies

designed to assure improvements in workforce, management, and leadership quality

3 Facilitates access to national, state, and local resources available for workforce training, leadership development, and continuing education

4 Provides for the training and continuing education of the

PPHCC

5 Assists in mobilizing resources for the provincial health

system and the public health services


44

# ToR 0% 1%–25% 26%–50% 51%–75% 76%–

100%

VIII Oversight of preventive and clinical services, environmental health services, and forensic medicine services

1

Oversees and supports the delivery of preventive and

clinical services, environmental health services, and forensic medicine services

2 Facilitates the community monitoring of the delivery of these services

3 Encourages community input into the delivery of these services

4 Ensures transparency, accountability, and ethical and moral integrity in the provision of these services

IX Oversight of strategic and annual planning

1 Maintains and annually reviews documentation of its mission

statement

2 Assesses and advocates for adequate resources and the MOPH’s support

3 Supports planning for improvement in the health of the population in the province and works to strategically align

community resources for this purpose

4 Oversees implementation of the annual plan

X Oversight of management of health emergencies

1 Supports planning for emergency response and works to

strategically align community resources for this purpose

2 Facilitates access to appropriate resources for management of health emergencies

3 Promotes broad-based participation and coordination among all entities active in management of health emergencies

4 Provides oversight and support for management of health emergencies

XI Nurturing community relationship and involvement

1 Ascertains people’s preferences, needs, problems, challenges, and issues in health service delivery

2 Mobilizes community input in the planning and

implementation of the health services

3 Mobilizes community input in monitoring, evaluation and

ensuring accountability in health service delivery

4

Provides relevant feedback to its stakeholders and the

communities in

the province


http://www.LMGforHealth.org


Impact of a Health Governance Intervention on Provincial ... of a Health... · A quasi-experimental...

Documents

Transcript of Impact of a Health Governance Intervention on Provincial ... of a Health... · A quasi-experimental...