Document of The World Bankdocuments.worldbank.org/curated/en/489571473188281827/pdf/... · document...

Document of

The World Bank

Report No: ICR00003781

IMPLEMENTATION COMPLETION AND RESULTS REPORT

ON A

CREDIT

IN THE AMOUNT OF SDR 38.3 MILLION

(USD60.0 MILLION EQUIVALENT)

TO THE

SOCIALIST REPUBLIC OF VIETNAM

FOR A

NORTHERN UPLAND HEALTH SUPPORT PROJECT

August 24, 2016

Health, Nutrition and Population Global Practice

East Asia and Pacific Region

CURRENCY EQUIVALENTS

(Exchange Rate Effective February 29, 2016)

Currency Unit = Vietnamese Dong (VND)

VND 22,174.64 = USD 1.00

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank

CNHSP Central North Health Support Project

CPMU Central Project Management Unit

CPS Country Partnership Strategy

DO Development Objective

EC European Community

FA Financing Agreement

FHCIC Free Health Care Insurance for Children under six years

HCFP Health Care Funds for the Poor

HCWM Health Care Waste Management

HEMA Health Care Support to the Poor of the Northern Upland and Central

Highlands

HI Health Insurance

HIC Health Insurance Card

HIS Health Information System

HMIS Health Management and Information System

ICR Implementation Completion and Results Report

IEC Information, Education, and Communication

IMR Infant Mortality Rate

IOI Intermediate Outcome Indicator

IPF Investment Project Financing

ISR Implementation Status and Results Report

KPI Key Performance Indicator

MHI Millennium Development Goal’s Health Indicator

MHSP Mekong Regional Health Support Project

MDG Millennium Development Goal

M&E Monitoring and Evaluation

MMR Maternal Mortality Rate

MoH Ministry of Health

MTR Midterm Review

NMR Neonatal Mortality Rate

NORRED North East Red River Delta Region Health System Support Project

NUP Northern Upland Health Support Project

PAD Project Appraisal Document

PDO Project Development Objective

PPMU Provincial Project Management Unit

QER Quality Enhancement Review

RF Results Framework

SBA Skilled Birth Attendant

SHI Social Health Insurance

SIL Specific Investment Loan

TTL Task Team Leader

UHC Universal Health Coverage

VHI Voluntary Health Insurance

VHLSS Vietnam Household Living Standard Survey

Senior Global Practice Director: Timothy G. Evans

Practice Manager: Toomas Palu

Project Team Leader: Anh Thuy Nguyen

ICR Team Leader: Andre Medici

ICR Primary Author Andre Medici

VIETNAM

Northern Upland Health Support Project

TABLE OF CONTENTS

A. Basic Information…………………………………………………………………....i

B. Key Dates .................................................................................................................... i

C. Ratings Summary ........................................................................................................ i

D. Sector and Theme Codes ........................................................................................... ii

E. Bank Staff ................................................................................................................... ii

F. Results Framework Analysis ...................................................................................... ii

G. Ratings of Project Performance in ISRs ................................................................... ix

H. Restructuring .............................................................................................................. x

I. Disbursement Profile .................................................................................................. xi

1. Project Context, Development Objectives and Design ............................................... 1

2. Key Factors Affecting Implementation and Outcomes .............................................. 6

3. Assessment of Outcomes .......................................................................................... 15

4. Assessment of Risk to Development Outcome ......................................................... 23

5. Assessment of Bank and Borrower Performance ...... Error! Bookmark not defined.

6. Lessons Learned ..................................................................................................... 277

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 28

Annex 1. Project Costs and Financing .......................................................................... 30

Annex 2. Project Outputs (According PAD -Annex 4) ................................................ 31

Annex 2A. Project Output Map……………………………………………………….43

Annex 3. Economic Analysis: Outputs Efficiency, Benefits and Equity Impacts…….48

Annex 3A Estimated Unitary Costs of the Project Outputs ……………………..……59

Annex 4. Bank Lending and Implementation Support/Supervision Processes………..60

Annex 5. Results Framework: Analysis of the PDO Achievement…..………...……..62

Annex 5A. Rating of the Indicators According Achievement………………………...66

Annex 6. Borrowers ICR………………………………………………………….…...71

Annex 6A. Project Risk management (Borrowers View)……………………………..86

Annex 6B. Project Results Framework (Borrowers View)……………………………89

Annex 6C. Achievements by components (Borrowers View)……………………...…92

Annex 7. Summary of the Project Aide-Memoires……………………………………98

Annex 8. List of Supporting Documents……………………………………….…….100

MAP…………………………………………………………………………….…….102

i

A. Basic Information

Country: Vietnam Project Name: Northern Upland

Health Support Project

Project ID: P082672 L/C/TF Number(s): IDA-43980

ICR Date: 08/31/2016 ICR Type: Core ICR

Lending Instrument: SIL Borrower:

SOCIALIST

REPUBLIC OF

VIETNAM

Original Total

Commitment: XDR 38.30M Disbursed Amount: XDR 37.31M

Revised Amount: XDR 38.30M

Environmental Category: B

Implementing Agencies:

Ministry of Health, Vietnam Central Project Management Unit (CPMU)

The Seven Northern Upland Provinces (Cao Bang, Bac Kan, Lao Cai, Ha Giang, Son La, Dien

Bien and Lai Chau) and their respective Provincial Project Management Units (PPMU)

Co-financiers and Other External Partners: No

B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 05/31/2006 Effectiveness: 10/08/2008 10/08/2008

Appraisal: 11/15/2007 Restructuring(s): — 08/29/2014

Approval: 03/13/2008 Mid-term Review: 07/16/2012 07/16/2012

Closing: 08/31/2014 02/29/2016

C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: Moderate

Bank Performance: Satisfactory

Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Central Government -

CPMU: Satisfactory

Quality of Supervision: Satisfactory Regional Governments

PPMU: Moderately Satisfactory

ii

Overall Bank

Performance: Satisfactory

Overall Borrower

Performance: Satisfactory

C.3 Quality at Entry and Implementation Performance Indicators

Implementation

Performance Indicators

QAG Assessments

(if any) Rating

Potential Problem Project

at any time (Yes/No): No

Quality at Entry

(QEA): None

Problem Project at any

time (Yes/No): No

Quality of

Supervision (QSA): None

DO rating before

Closing/Inactive status: Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Central government administration 8 8

Compulsory health finance 15 15

Health 71 71

Sub-national government administration 6 6

Theme Code (as % of total Bank financing)

Health system performance 100 100

E. Bank Staff

Positions At ICR At Approval

Vice President: Victoria Kwakwa James W. Adams

Country Director: Achim Fock (Acting) Ajay Chibber

Practice

Manager/Manager: Toomas Palu Fadia M. Saadh

Project Team Leader: Anh Thuy Nguyen Maryam Salim

ICR Team Leader: Andre Medici —

ICR Primary Author: Andre Medici —

F. Results Framework Analysis

Project Development Objectives (from Project Financial Agreement)

The objective of the Project is to increase the utilization of district health services

especially among the poor and ethnic minorities population of the Northern Upland

iii

Provinces through improving the quality of district-level hospitals and reducing financial

constraints to access to health services.

Revised Project Development Objectives (as approved by original approving authority)

The PDO was not revised. The Project originally had four Key Performance Indicators

(KPIs) and 10 Intermediate Outcome Indictors (IOI) in the Project Appraisal Document

(PAD). However, most of baselines and targets of the project indicators were set during

the first year of implementation according to the result of a Baseline Survey conducted in

June 2009 and published in July 2009. The four KPIs were retained and the number of IOIs

increased from 10 to 13. No other changes were made to the indicators. All KPIs and most

of the IOIs’ targets were achieved by December 31, 2015 (before the project Closing Date

of February 29, 2016).

(a) PDO Indicator(s)

Indicator1 Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target

Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1: Utilization rates of in-patient services in District Hospitals among Decision 139

beneficiaries

Value

(Quantitative or

Qualitative)

0.027 0.033 — 0.096

Date achieved 06/30/2009 08/31/2014 12/31/2015

Comments

(including %

achievement)

Target surpassed (191%).

Indicator 2: Utilization rates of out-patient health services in district hospitals by Decision

139 beneficiaries

Value

(Quantitative or

Qualitative)

0.067 0.075 — 0.247

Date achieved 06/30/2009 08/31/2014 12/31/2015

Comments

(including %

achievement)


Indicator 3: Percentage of households who experience catastrophic healthcare expenditures in

the year prior to the survey.

Value

(Quantitative or

Qualitative)

14.27% 13.23% — 2.00%

Date achieved 06/30/2008 08/31/2014 08/31/2014

Comments Target surpassed (561%). Not measured during the 18-month extension period.

1 Percentage of achievement in this table was calculated as a simple percentage increase of the last actual

on the proposed target.

iv

(including %

achievement)

Indicator 4:

Proportion of district hospitals that provide full set of health services according

to the national norms (Decision 23/205/QB- BYT) adjusted to the Northern

Uplands.

Value

(Quantitative or

Qualitative)

39.1% 70.0% — 80.4%

Date achieved 06/30/2008 08/31/2014 12/31/2015

Comments

(including %

achievement)

Target surpassed (15%). For operational reasons, after the MTR, this indicator

was adjusted to be read as “average percentage of health services covered by the

district hospitals”.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1: Percent of patients satisfied with the health services;

Value

(Quantitative

or Qualitative)

8.5% 10.2% — 84.4%

Date achieved 07/31/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)

Target surpassed (727%). Not measured during the 18-month extension period.

Indicator 2: Adherence of treatment protocols for selected conditions in impatient settings.

Value

(Quantitative

or Qualitative)

— — — —

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)

Measured by sub-indicators 2.1 to 2.9 (all surpassed). Not measured during the

18-month extension period.

Indicator 2.1: Percent of health workers with knowledge of diagnosing and treating Level

A/B/C dehydrated diarrhea

Value

(Quantitative

or Qualitative)

9.7% 14.0% — 95.2%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)


Indicator 2.2: Percent of health workers with knowledge of diagnosing and treating severe

pneumonia

Value

(Quantitative 13.2% 18.5% — 86.9%

v


Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

or Qualitative)

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)


Indicator 2.3: Percent of health workers with knowledge of diagnosing and treating poisoning

Value

(Quantitative

or Qualitative)

26.8% 37.5% — 83.0%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)


Indicator 2.4: Percent of Reasonable diagnoses of severe pneumonia

Value

(Quantitative

or Qualitative)

45.5% 63.7% — 71.1%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(incl. %

achievement)


Indicator 2.5: Percent of Clinical health workers’ reasonable diagnosis of general pneumonia.

Value

(Quantitative

or Qualitative)

19.6% 27.4% — 57.0%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)


Indicator 2.6: Percent of Reasonable diagnoses of Level A dehydrated diarrhea

Value

(Quantitative

or Qualitative)

37.2% 52.1% — 78.9%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)


Indicator 2.7: Percent of Reasonable diagnoses of Level B dehydrated diarrhea.

Value

(Quantitative

or Qualitative)

48.9% 68.5% — 85.1%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including % Target surpassed (24%). Not measured during the 18-month extension period.

vi


Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

achievement)

Indicator 2.8: Percent of Reasonable diagnosis of Level C dehydrated diarrhea

Value

(Quantitative

or Qualitative)

41.2% 57.7% — 80.0%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)


Indicator 2.9: Percent of Reasonable diagnoses of poisoning.

Value

(Quantitative

or Qualitative)

61.1% 85.5% — 86.2%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)

Target achieved. Not measured during the 18-month extension period.

Indicator 3: Percent of eligible district health staff who have successfully completed training

provided by the project

Value

(Quantitative

or Qualitative)

— — — —

Date achieved 06/30/2008 08/31/2014 12/31/2015

Comments

(including %

achievement)

Measured by sub-indicators 3.1 to 3.3 (all surpassed).

Indicator 3.1: Percent of doctors and assistant doctors and pharmacists at district hospitals

trained by the project

Value

(Quantitative

or Qualitative)

0% 80% — 189%

Date achieved 06/30/2008 08/31/2014 12/31/2015

Comments

(including %

achievement)


Indicator 3.2: Percent of health staffs with completed short-term training courses compared to

the plan.

Value

(Quantitative

or Qualitative)

0% 80% — 357%

Date achieved 06/30/2008 08/31/2014 06/30/2014

Comments

(including %

achievement)


vii


Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 3.3: Percent of health staffs completed long-term training courses compared to the

plan

Value

(Quantitative

or Qualitative)

0% 80% — 88%

Date achieved 06/30/2008 08/31/2014 06/30/2014

Comments

(including %

achievement)


Indicator 4: Percent of eligible district hospitals with acceptable operations and maintenance

plans and budget for facility and equipment maintenance

Value

(Quantitative

or Qualitative)

— — — —

Date achieved 06/30/2008 08/31/2014 12/31/2015

Comments

(including %

achievement)

Measured by sub-indicators 4.1 and 4.2 (both surpassed)

Indicator 4.1: Percent of district hospitals having schedule and budget for maintenance of

infrastructure

Value

(Quantitative

or Qualitative)

49.2% 40.0% — 79.7%

Date achieved 06/30/2008 08/31/2014 12/31/2015

Comments

(including %

achievement)

This indicator had a target value of 40% in the PAD. The baseline was measured

after the project approval. The target value was not revised during

implementation to be compatible with the baseline. Consequently, achievement

for this indicator has been calculated over the baseline value of 49.2% instead of

its target value of 40%. Baseline value surpassed (62%)

Indicator 4.2: Percent of district hospitals having schedule and budget for maintenance of

equipment.

Value

(Quantitative

or Qualitative)

77.1% 40.0% — 89.1%

Date achieved 06/30/2008 08/31/2014 06/30/2014

Comments

(including %

achievement)

This indicator had a target value of 40% in the PAD. The baseline was measured

after the project approval. The target value was not revised during

implementation to be compatible with the baseline. Consequently, achievement

for this indicator has been calculated over the baseline value of 77.1% instead of

its target value of 40%. Baseline value surpassed (16%).

Indicator 5: Number of Health facilities constructed renovated, and/or equipped.

Value

(quantitative

or Qualitative)

0 61 — 64

viii


Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Date achieved 06/30/2009 08/31/2014 12/31/2015

Comments

(including %

achievement)

Target achieved. This core indicator was added during project implementation.

Indicator 6: Percent of recently discharged patients satisfied with health services.

Value

(Quantitative

or Qualitative)

8.5% 10.2% — 84.4%

Date achieved 06/30/2009 08/31/2014 06/30/2014

Comments

(including %

achievement)

Target Surpassed (727%). This indicator was added during project

implementation.

Indicator 7: Number of People with access to a basic package of health, nutrition &

reproductive health services.

Value

(Quantitative

or Qualitative)

20% 70% — —

Date achieved 06/30/2009 08/31/2014 12/31/2015

Comments

(including %

achievement)

This indicator was added during project implementation. It cannot be measured

because the baseline and the target (expressed in percentage) were not converted

in the number of beneficiaries with access to the basic package of health,

nutrition & reproductive health services during the project life. Nevertheless, the

number of beneficiaries reached was 270,254 by December 2015.

Indicator 8: Percentage of Decision 139 beneficiaries who have received Health Insurance

Cards

Value

(Quantitative

or Qualitative)

82.1% 70% — 95.2

Date achieved 06/30/2009 08/31/2014 12/31/2015

Comments

(including %

achievement)

Target surpassed (16%). The target of this indicator was established before the

baseline survey. For this reason, the baseline value in the PAD was higher than

the target value. The project team did not revised the target during project

implementation. Consequently, the achievement of this indicator has been

calculated over the baseline instead the target.

Indicator 9: Percentage of 139 Beneficiaries with cards who can correctly identify at least

three benefits covered under the HCFP program

Value

(Quantitative

or Qualitative)

14.8% 75% — 57.4%

Date achieved 06/30/2009 08/31/2014 08/31/2014

Comments

(including %

achievement)

The target was not achieved. Not measured during 18-month project extension

period. However, the following alternate indicator was measured during

implementation “% the poor knowing at least 1 right of health insurance card

holders”. The achievement of this indicator was 95% by December 2015.

ix


Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 10: Percentage of households who identify financial barriers as a main cause for not

seeking health care.

Value

(Quantitative

or Qualitative)

2.0% 1.8% — 1.2%

Date achieved 06/30/2009 08/31/2014 8/31/2014

Comments

(including %

achievement)

The target was surpassed (50%). Not measured during the project extension.

Indicator 11: Project management units at central level and provincial level established; bank

accounts opened, staff and consultant recruited, system established

Value

(Quantitative

or Qualitative)

— — — Achieved

Date achieved 06/30/2008 06/30/2009

Comments

(including %

achievement)

CPMU/PPMUs management, staff and consultants fully board by June 2009.

Project operational and financial systems have been maintained throughout the

project life.

Indicator 12: Project management units prepare adequate plans, meet annual implementation

targets and provide timely financial and activity report.

Value

(Quantitative

or Qualitative)


Date achieved 06/30/2009 12/31/2015

Comments

(including %

achievement)

The project's annual working plans, progress reports, interim financial reports and

audits reports were submitted in time with acceptable quality, except for some

reports on project result indicators toward the last year of project implementation.

Indicator 13: Availability of data for project monitoring and evaluation

Value

(Quantitative

or Qualitative)


Date achieved 02/15/2016

Comments

(including %

achievement)

The project collected data from project related surveys (2009, 2012, and 2014)

and from administrative records by the CPMU. Final project evaluation was

conducted and completed in mid-February 2016.

G. Ratings of Project Performance in ISRs

No. Date ISR

Archived DO IP

Actual

Disbursements

(USD millions)

1 11/21/2008 Satisfactory Satisfactory 0.00

2 01/22/2010 Moderately Satisfactory Moderately Satisfactory 3.03


x


5 04/24/2013 Satisfactory Moderately Satisfactory 28.87







H. Restructuring (if any)

Restructuring

Date(s)

Board

Approved

PDO

Change

ISR Ratings at

Restructuring Amount Disbursed

at Restructuring in

USD, millions

Reason for Restructuring & Key

Changes Made DO IP

08/29/2014 No MS MS

52.1

Level II restructuring: extension of the

closing date from August 31, 2014 to

February 29, 2016. The project

achieved its KPIs’ targets on time and

generated savings. The restructuring did

not change the PDOs and KPIs/IOIs,

but redirected the remaining funds to:

(a) Maximize efficiency and enhance

the sustainability of the Project

investments in the seven provinces, and

(b) Contribute to the achievement of

Vietnam health MDGs in related areas,

especially toward reducing child

mortality and improving maternal

health. Many of the IOIs (especially

those which required a new client

survey) were not monitored because

Government decisions to do not

conduct a project survey after the 18-

month extension period. The end of

project survey was conducted in May-

June 2014 and published in October

2014.

xi

I. Disbursement Profile

1

1. Project Context, Development Objectives, and Design

1.1 Context at Appraisal

1. In the 20 years prior to 2007, Vietnam had one of the highest rates of economic growth

and poverty reduction in the world, despite the persistence of higher inequality. By appraisal

(2007), the country had accumulated a remarkable development performance because of important

economic, social, and political reforms delivered since 1986. These reforms contributed to an

average annual growth rate of 7.4 percent per year between 1990 and 2008, coupled with a fast

reduction in poverty from 58.0 percent to 14.5 percent in the same period, based on the national

poverty line, and a fall in the proportion of people living under a dollar a day from 63.0 percent to

21.5 percent between 1993 and 2006.

2. Despite the fact that an important dimension of the nature of the Vietnamese

economic growth had been its inclusiveness, inequalities and disparities across regions

remained a challenge. In 1993, poverty in rural areas was 2.6 times higher than in urban areas.

By 2008, it was 5.7 times greater. Poverty was concentrated in the Northeast, North, Central Coast,

and Central Highlands—with approximately one-third of the population in those regions living

below the poverty line (mostly associated with ethnic minorities) by the time of project appraisal.

3. In 2007, most Vietnam’s health-related Millennium Development Goal (MDG)

outcome indicators were on track. The under-five mortality rate decreased by 65 percent

between 1990 and 2005 from 53 to 19 per 1,000 births. The maternal mortality ratio fell by two-

thirds, from 250 deaths in 1990 to 85 deaths per 100,000 births in 2007. Cases of Malaria were

significantly reduced, accounting for only 35 deaths in 2007. Tuberculosis programs, since 1997,

had reached and exceeded the global targets for control, detecting 70 percent of new smear-positive

pulmonary tuberculosis cases, curing 85 percent of these detected cases, and leading to a 44 percent

decrease of the incidence rates over the period 1997–2004. No progress was registered in the

reduction of HIV-AIDS cases and the incidence went up from 0.34 percent in 2001 to 0.44 percent

in 2005 among the general population.

4. Despite progress in health-related MDGs, the burden of disease in Vietnam, at project

appraisal, was already concentrated in non-communicable diseases, injuries, and accidents,

accounting for almost three-quarters of reported deaths nationwide. In 2008, hypertension

prevalence had almost doubled in less than 20 years and close to one-third of adults had high blood

cholesterol. Overweight, obesity, and physical inactivity prevalence increased because of changes

in nutrition and the labor market structure and smoking rates for males remained at 56 percent in

the 10 years before the Project appraisal.

5. Improvements in health system protection. Since 1992, Vietnam’s Government

intended to achieve universal health coverage (UHC) by expanding health insurance (HI)

mechanisms. Despite this intention only 49 percent of the Vietnamese population was covered by

HI at the Project appraisal. The main impacts of the expansion of HI in the 15 years before 2007

were increased utilization of health facilities, reduction of health out-of-pocket payment, and

trimming of catastrophic spending risks for families.

2

6. In 2007, Vietnam offered four types of HI for its citizens: (a) Social Health Insurance

(SHI), for those employed in the formal labor markets, retirees, and disabled and meritorious

people, representing only 9 percent of the population; (b) Health Care Funds for the Poor (HCFP),

directed to the poor ethnic minorities in mountainous areas (such as the Northern Upland

Provinces) and inhabitants in disadvantaged communities, enrolling 18 percent of the country’s

population2; (c) Free Health Care Insurance for Children under six years (FHCIC) addressing 11

percent of the population, and; (d) Voluntary Health Insurance (VHI), covering self-employed and

informal sector workers, dependents of the SHI members, and students and children over six years,

covering 11 percent of the entire population. The SHI was financed by payroll contributions while

the HCFP and FHCIC were financed by fiscal revenues. The VHI was financed by premiums

calculated according to the insureds’ ability to pay.

7. Together with the health reforms from the early 1990s to late 2000s, Vietnam

introduced changes in the provision of health services. The most important was the

liberalization of the health care and pharmaceuticals markets. This was accompanied by the

introduction of user fees at public health facilities and the transfer of health workers’ salary

payments from local communities to the central Government. Some of these changes did not

improve the health care delivery process, particularly in the disadvantage regions, given the poor

access and quality of health services in rural areas and district hospitals. Health services were

delivered by public and private hospitals, with the latter mostly providing specialized care in urban

areas, while primary and basic care were provided by public hospitals and community health

centers. A large share of pharmaceuticals was purchased directly by the population from private

vendors with traditional medicine playing a major role, as it is recognized by the Government as

part of the health system and offered by public and private providers.

8. Despite progress resulting from health reforms, coverage and health outcomes were

not uniform within the country. Poverty reduction, HI coverage, and health improvement

remained uneven, with some segments of the population lagging behind the national average and

high disparities in health indicators among regions. For example, in the Northern Upland

Provinces,3 the infant mortality rate (IMR) was 60 per 1,000 live births compared to the national

average of 18 per 1,000 live births. While the majority of births were attended by skilled health

workers, less than 20 percent of births by ethnic minority women were attended by qualified

personnel. In the Northern Upland provinces, the role of district hospitals was compromised by

the lack of adequate and qualified human resources and poor conditions of the physical

infrastructure. The poor and ethnic minorities did not use these hospitals because of poor quality

of the services and difficulty of access. Consequently, these groups were particularly vulnerable

to receiving suboptimal health care and achieving poor health outcomes, particularly mothers and

children.

9. Rationale for World Bank assistance. The Project design was based on (a) best practices

from World Bank health projects and other initiatives in countries similar to Vietnam and (b) the

government health policy to improve equity on achievement of health outcomes at the regional

2 These two insurances—SHI and HCFP—are referred to as Compulsory HI Schemes. 3 The extremely disadvantaged population of the Northern Upland Provinces led the Government to issue the

Decision 139 of 2002 to cover the following beneficiaries: poor, ethnic minorities living in the mountainous

provinces and population living in Government-defined difficult communes. The project initially limited the support

to the poor, but then expanded to the ethnic minorities following the Decision 139.

3

level. Both recommended interventions on the supply side (improving infrastructure, training

human resources, improving management, and maintenance of health infrastructure) and demand

side (providing means to the poor and ethnic minorities to gain access to quality health services).

Therefore, this Project focused on district hospitals and complemented another health project in

the NUP areas approved in June 2008 (Health Care Support to the Poor of the Northern Upland

and Central Highlands - P110251, EC Grant No. TF-091328-VN) and which focused on primary

care service delivery at the commune and village levels. The Project was consistent with the second

pillar of the Country Partnership Strategy (CPS) for Vietnam (Report No. 38236-VN) issued in

January 3, 2007, especially in the area of strengthening social inclusion, assuring economic growth

with social equity by improving social services to the poor and marginalized groups.

1.2 Original Project Development Objectives (PDO) and Key Indicators

10. According the Financing Agreement (FA), the objective of the project was to increase

the utilization of district health services, especially among the poor and ethnic minorities

population of the Northern Upland Provinces through improving the quality of district-level

hospitals and reducing financial constraints to access to health services. The PDO in the

project appraisal document (PAD) had slightly different phrasing, but its main essence was

consistent with the FA. The PDO in the PAD was as follows: Increase utilization of district health

services especially among Decision 139 beneficiaries4 in the Northern Upland Provinces through

(a) strengthening of district hospitals and (b) reducing financial constraints to accessing health

services. For the purpose of this Implementation Completion and Results Report (ICR), this PDO

is split into three parts, all especially aimed at the poor and ethnic minorities’ population of the

NUP: (a) increase utilization of district hospitals services5; (b) improve the quality of district-level

hospitals; and (c) reduce financial constraints to access health services.

11. The four PDO-level indicators (Key Performance Indicators [KPIs]) were defined in

the results framework (RF) of the PAD. The KPIs are the following: (a) utilization rates of

inpatient services in district hospitals among Decision 139 beneficiaries; (b) utilization rates of

outpatient services in district hospitals among Decision 139 beneficiaries; (c) percent of

households which experience catastrophic health care expenditures in the year prior to the survey;

and (d) proportion of district hospitals that provide full set of health services according to the

national norms (Decision 23/2005/QD-BYT) adjusted for the Northern Upland.6 As baseline data

were not available for some indicators at appraisal stage (KPI #4 and Intermediate Outcome

Indicators [IOIs] #1, #2, and #6), the targets were nevertheless set based on the team’s best guess

estimates of the expected percentage of increase/decrease during the project life. Baselines known

at appraisal were adjusted to the outcome of the Baseline Survey of 2009 (KPIs#1, #2, and #3 and

IOIs #3, #4, #5, and #7). However, these changes were not formally recorded through restructuring.

4 The Decision 139 created the HCFP in 2002 to increase access to health care and reduce the financial burden of

health expenditure faced by the poor and ethnic minorities. 5 The first part of the PDO is the overarching objective of the Project. The 2nd and 3rd parts are the means to achieve

this overall objective throughout supply and demand side interventions. 6 According to annex 3 (Project Results Framework) of the PAD, page 32. This indicator was kept but was measured

as the percent of the health services that district hospitals can provide in comparison with the national norms

(Decision 23/2005/QD-BYT).

4

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and

reasons/justification

12. The PDO was not revised during project implementation. However, the original

baselines and targets were revised for KPIs and IOIs during project implementation (without a

formal restructuring) to reflect the outcome of the baseline survey conducted in the first semester

of 2009 and published in July 2009. Baseline values of few indicators were kept as in the original

PAD and, at the end of the project, appeared to be modest if compared with the values achieved

after implementation. See section 2.3 on monitoring and evaluation (M&E) for details.

1.4 Main Beneficiaries

13. The main project beneficiaries were meant to be the poor and ethnic minorities of the

Seven Northern Upland Provinces: Bac Kan, Cao Bang, Son La, Ha Giang, Lao Cai, Dien

Bien, and Lai Chau. These populations were to benefit from project investments by increased

access to quality health services provided by the district-level hospitals and by receiving subsidies

(meals and transportation) to improve their regular access to the district hospitals. Health

professionals and managers of the district hospitals in the NUP provinces were to benefit from

training, improved infrastructure, and additional resources from fees received from the HCFP to

increase financial sustainability of these hospitals.

1.5 Original Components

14. The project consisted of three components aimed at increasing the utilization of district

health services in the NUP provinces (supply-side interventions) and reducing the financial

constraints to accessing health services for the poor and ethnic minorities (demand-side

interventions).

Component 1: Strengthening District-level Health Services (cost estimated at USD42.9

million).

15. This component aimed to improve the quality of district hospitals through the following

three subcomponents: (a) human resources development, to strengthen the health workforce

capacity and increase staff retention, expertise, and technical capacity of district hospitals through

the provision of long-term training for doctors and specialists at level 17; (b) improving the quality

of district hospitals, through the provision of basic medical equipment and selective facility repair

and refurbishment; and (c) improving hospital management, to support the investment in district

hospitals by creating a management environment that will sustain project outcomes measured by

quality of improvement, infrastructure maintenance, and human resources management. This

component involves training district hospital management staff in basic principles of management

of health care institutions, developing hospital maintenance plans as an integral part of the

management improvement program, and creating a management excellence award program for

district hospitals that meet good management practices.

7 Level-1 specialists are doctors with the following specializations: internal medicine, obstetrics, intensive care,

traditional medicine, and imaging.

5

Component 2: Increasing Financial Access to Healthcare Services for Decision 139

Beneficiaries (cost estimated at USD10.0 million)

16. This component aimed to address the demand-side constraints of access to health services

by piloting mechanisms to further reduce financial barriers for the Decision 139 beneficiaries in

the project provinces. The component had three subcomponents. They are: (a) support for direct

catastrophic and nonmedical expenditures of health care for Decision 139 beneficiaries, aimed to

get information on the distribution and pattern of catastrophic expenditures and to support

nonmedical expenditures such as primarily transportation and food for the beneficiaries, which

had a higher proportion (62 percent) of total inpatient out-of-pocket costs in the NUP areas; (b)

strengthening capacity for HCFP, through institutional capacity-building activities to increase the

financial capacity of the district hospitals to enroll beneficiaries and recover the medical expenses

incurred by them. It involves the identification of beneficiaries according to poverty and ethnicity

criteria and issuing and distributing the beneficiaries’ cards among them; and (c) strengthening

local access to health services through promoting health seeking behavior, to increase the

knowledge and understanding of the rights, entitlements, and benefits covered by the HCFP among

the beneficiary population through information, education and communication (IEC) campaigns.

Component 3: Monitoring, Evaluation, and Project Management (Cost estimated at

USD13.1 million, of which USD6.0 million from Government counterpart)

17. This component supported the set up and management of the Central Project Management

Unit (CPMU) and Provincial Project Management Units (PPMUs) through (a) consulting services

to cover technical issues as well as procurement, financial management, and disbursement; (b)

training of project management staff; (c) provision of necessary office equipment; (d) financing of

incremental operating costs; and (d) M&E activities, including baseline data collection, indicator

updates, midterm review (MTR), end-of-project completion report and audits. This component

also supported an initiative to streamline the Ministry of Health (MoH) internal procurement

review and approval process, which was considered a critical initiative in the broader public

procurement reform efforts in the sector and the country.

1.6 Revised Components

18. The project restructuring of August 29, 2014, proposed an 18-month extension of the

project closing date and marginally reallocated the project funds among the components.

With 82 percent of the project funds disbursed, an end project evaluation was conducted from May

to June 2014 in the NUP targeted provinces, revealing that all KPIs and most of the IOIs had

already been achieved by the project before restructuring, saving 18 percent of the project funds.

The restructuring did not propose any review of the project components.

19. The Government proposed to use the remaining funds to continue activities under the

project components. No other significant changes were made to the project. The 18-month

extension was proposed to allow adequate time to: (a) improve the sustainability of the project

outcomes by continuing and adding training courses on specialized techniques and skills such as

the use and maintenance of equipment; (b) ensure that the financing and management of the HCFP

had been transferred to the provincial authorities and finance the entitled activities properly; (c)

improve the efficiency of the project throughout, providing technical assistance in health service

6

delivery, especially related to outpatient services which are essential to improve maternal and child

health and to the achievement of the MDGs in the Project areas until December 2015; (d) ensure

proper use of the equipment; and (e) provide the technical support for the development of policies

to strengthen the district-level health services and support the achievement of the MDGs.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design, and Quality at Entry

20. Soundness of the background analysis. The project team accessed extensive and relevant

background information about the health conditions, government initiatives, and constraints in the

national context and in the project intervention areas. The project benefited from lessons learned

in designing and implementing other health projects in Vietnam, such as the Mekong Regional

Health Support Project (MHSP) and the National Health Support Project. The project was prepared

as a specific investment loan (SIL) inspired by the design of a similar approved project —the

MHSP—which provided the project team important lessons to guide the NUP project preparation,

which was the first of four similar regional programs 8 . The main lessons learned that were

incorporated in the project design were: (a) the need to focus on the demand-side interventions and

underpin quality health coverage to increase utilization of health facilities; (b) the relevance of

addressing regional variations to adapt investments to local conditions; (c) the need to improve

project implementation skills at the local levels through technical support by the central level and

by exchange of experiences among the PPMUs; and (d) the need to build capacity (by training

staff) on World Bank fiduciary procedures and guidelines for the local implementation units, to

avoid implementation delays, particularly with regard to civil works and procurement of

equipment.

21. Assessment of the project design. The PDO, the balance of the activities among the

project components, and project implementation arrangements were formulated realistically and

in line with project complexity. The project design addressed many aspects of the expected

implementation challenges associated with one central and seven local project implementation

units, difficulties in accessing mountainous areas, scattered distribution of the beneficiary

population, and the social (ethnic minority) issues and environmental (associated mostly with the

hospital waste management issues) safeguards.

8 The MHSP (P079663) closed in June 2012 and was the first of a generation of universal coverage projects in

Vietnam that aimed at expanding coverage for the poorest population. The project indicators were achieved and

even exceeded and the project brought huge benefits to the poor, because the enrollment of the poor became a

national policy based on this experience. However, some shortcomings were observed with regard to efficiency,

such as the potential supplier-induced demand in a fee-for-service environment, leading the World Bank to open a

dialogue with the client to implement policy interventions with the Vietnam Social Security Administration to focus

on provider payment reforms. Beside the Mekong and the NUP Project, there are two more similar projects—the

Central North Health Support Project (CNHSP) and North East Red River Delta Region Health System Support

Project (NORRED)—both still in implementation. All projects provided support to the health sector in provinces

that were geographically difficult to access, economically depressed, and had interventions in both demand and

supply sides. Despite that, each project had some different features. The MHSP is for provincial hospitals, while the

NUP focuses on district-level hospitals and the CNHSP deals with district hospitals and community health centers.

The NORRED Project was designed for both provincial and district hospitals with some distinctive interventions.

7

22. Other complementary World Bank health projects and international partner

activities. The PDO did not overlap with other institutional partner support and projects, but

complemented activities supported by another World Bank project (Health Care Support to the

Poor of the Northern Upland and Central Highlands [HEMA] - P110251), which focused on

primary care in three (Son la, Lai Chau, and Dien Bien) of the seven provinces addressed by the

project. HEMA (approved in June 2008) was designed to improve access to primary health care

by the poor at community health centers (not district hospitals). HEMA was trust-funded by the

European Commission and managed by the World Bank. During project preparation, the Asian

Development Bank (ADB) was also financing activities related to the promotion and prevention

in three of the project provinces, complementing other health interventions (such as immunization,

promotion, and preventive interventions) in these provinces. All these activities enhanced the

perspective of an integral health care approach as promoted by the World Bank’s health strategy

in Vietnam and supported the idea to focus the project design on district hospitals.

23. Adequacy of Government’s commitment. The MoH authorities were completely

involved in the project preparation and in development of its design, which followed the MHSP

model. They expressed their full commitment to guarantee adequate budget for communications,

supervision, monitoring, evaluation, and travel during implementation. The provincial health

authorities and district hospital managements were also committed to participate in the project

development. Field visits were organized during preparation to establish links and networks with

the local government health and hospital’s authorities.

24. Project preparation timeline. The project was prepared in 22 months (from concept

review in May 2006 to approval in March 2008). This is considerably longer than the average for

Health Nutrition and Population Global Practice projects of 18 months, but still under the World

Bank’s benchmark for Investment Project Financing of 24 months. The FA was signed four months

after approval (July 10, 2008) and declared effective on October 8, 2008. The project was to be

implemented in six years in view of the institutional implementation conditions and challenges at

the Northern Upland Provinces.

25. Assessment of risks. The overall risk for the project was rated Substantial and mitigation

measures were appropriately described in the PAD. The major risk of not achieving the PDO was

a possible uneven implementation progress across project components. It could prevent the

positive effects of the coordination between the demand side incentives and the supply side

investments on improving health access to the poor during the project implementation. Other risks

were associated with weak procurement capacity at the provincial level and at district hospitals

and the difficulty in retaining trained medical staff at the district hospitals. The CPMU issued an

operational manual that was approved before project effectiveness. During implementation, as

indicated in the project missions, project risks were well identified and managed. Some measures

to mitigate the project risks during implementation are described in annex 6A of this ICR.

26. Quality Enhancement Review (QER) and Decision Review Meeting. The project

underwent a QER in May 2007; the decision review meeting was held in September 2007. During

both meetings, the element of the project that was mostly appreciated by the reviewers was its

good rationale and direction and a solid results framework even though baselines and targets were

to be confirmed at a later stage through the baseline survey. Some of the issues and

recommendations raised during these meetings not only revealed that it was crucial to confirm

8

quality at entry, but were eventually found to be critical during implementation. The main

recommendations to the team were to (a) provide more details on technical aspects of the project

interventions (such as incentives for human resources retention, capacity building at district

hospitals, governance of district hospitals, health promotion, implementation and financing of the

project pilots)9; (b) focus the economic analysis on direct benefits for the target population; (c)

improve the section on lessons learned; (d) improve the description of institutional arrangements

for data collection related to project M&E; and (e) reassess project risk ratings, which appeared to

be overestimated. The PAD addresses all the recommendations, except point ‘d’ on institutional

arrangements for data collection for project M&E, which was not entirely addressed.

2.2 Implementation

27. Project implementation timeline. The project was to be implemented in six years (one

year more than the usual five-year implementation period for SIL projects in the health sector),

because of the difficult institutional implementation conditions in the Project areas. The original

Project closing date of August 31, 2014, was extended by 18 months to February 29, 2016.10 Even

when the PDOs were substantially achieved, project extension was justified by the Government as

a way to use the project savings to ensure sustainability of project interventions and to contribute

to the achievement of the health MDGs in the NUP areas.

28. The July 2012 MTR confirmed that the PDO remained relevant with a satisfactory

performance. A MTR survey was conducted by a local research institute on behalf of the CPMU

from November 2011 to May 2012. Substantial progress was registered in the achievement of the

KPI and IOI targets. By the end of 2012, inpatient utilization of the Decision 139 beneficiaries

increased more than two times and outpatient utilization increased 28 percent compared with the

2009 baseline. According to the survey results, the percentage of poor households suffering

catastrophic health expenditures was reduced from 22.1 percent to 13.3 percent (almost a 9 percent

reduction). The proposed targets in these areas expected about 10 percent to 15 percent

improvement by project closing. Therefore, the project far exceeded these targets, which were

established quite modestly from a low base. The MTR survey also revealed that the district

hospitals in the Project areas were able to deliver 10 percent more services, approaching the

expected number of services to be provided by district hospitals, according the MoH’s regulations.

9 These pilots of ‘incentive packages’ would consist of a combination of different types of incentives (monetary and

non-monetary), which would be designed according to health labor market studies. It would include differentials

payments according to hardship areas, subsidies for continuing education or education costs for the health workers’

children, housing allowances schemes, fast tracked promotion through pay grades for health workers in remote

areas, increased recruitment of students from the project provinces to medical schools, and so on. Paragraph 25 of

the PAD said “Evaluation of the effectiveness of the pilots will be an important part of the project’s overall

effectiveness”. However, this activity was not implemented by the project because of its weak political feasibility in

the context of health human resources policies in the country. 10 The main reasons for project extension were (a) improving the sustainability of the project results by continuing

and adding supplemental training courses on specialized techniques and skills, use and maintenance of equipment,

and other training activities; (b) ensuring that the financing and management of the HCFP have been transferred to

the provincial authorities and are running; (c) improving the project efficiency by strengthening the technical

assistance in health service delivery with particular emphasis on delivery and outreach of essential outpatient

services to maternal and child health care related to the achievement of the MDGs; and (d) distilling lessons learned

and disseminating them.

9

29. At the MTR, delays in procurement and project disbursement led to classify the

overall project implementation as Moderately Satisfactory. In 2012, only one-third of the

credit funds were disbursed. The reasons attributed to this poor performance were: (a) the issuance

of Government Resolution 11 of 2011 limiting some capital investments (vehicles/office

equipment) even if externally funded; (b) inability to translate provincial proposals for non-

training recruitment/retrenchment options into pilot activities; (c) lack of a plan for the utilization

of the technical assistance subcomponent under Component 2 to analyze the barriers to access by

the Decision 139’s beneficiaries; (d) no decisions taken at the time of the MTR on investments

related to the disposal of solid waste treatment at the district hospitals11; (e) delay in the update of

the NUP definition of beneficiary based on the revision of Decision 139 and inclusion of “HEMA”

districts; and (f) slow development of a strategy for IEC activities. These issues were appropriately

addressed after the MTR, resulting in a small reallocation of the project proceeds to Component

1. As a result, the project improved its performance and procurement was classified as satisfactory

in the last project Implementation Status and Results Report (ISR).

30. All four KPIs’ final targets were achieved and surpassed even before the original

project closing date of August 2014. However, the development objective (DO) was rated

Moderately Satisfactory at the time of the original closing date because of internal evaluations of

the World Bank team, which assessed slow progress in the RF, given a reduction of the achieved

values of some indicators from 2012 to 201412. However, by August 2014, the project showed

achievement toward the KPIs and a significant number of IOIs compared with the targets,

including: (a) knowledge of the health workers in diagnosing and treating common illnesses; (b)

knowledge of the benefits of HI by the beneficiaries; and (c) availability of facility and equipment

maintenance plans.

31. The final ISR, of February 25, 2016, rated the DO Satisfactory and the

implementation progress Moderately Satisfactory. All major planned activities were completed

by the closing date. Increased utilization of the district-level health services in the NUP region by

the beneficiaries was noted. The four KPIs used to measure the achievement of the PDO in the

seven provinces were fully achieved, as follows: (a) average inpatient visits per capita in the district

hospitals increased 3.9 times over the baseline and was 2.9 times greater than the end project target

for 2014; (b) average outpatient visits per capita increased 3.7 times over the baseline and was 28

percent greater than the end of project target for 2014; (c) percentage of households experiencing

catastrophic health expenditures was reduced from 14.7 percent in 2008 (baseline) to 2.0 percent

(2014), overachieving the target of 13.2 percent for 2014; and (d) average percentage of health

package procedures covered by the district hospitals according the national norms increased from

39 percent (2008) to 80.4 percent (2015), surpassing the target of 70 percent (2014).

11 In fact, the MoH/CPMU requested to buy the incinerators for solid waste. However, due to the World Bank’s new

regulation not allowing using IDA credit for burning technology, the activities were delayed and then cancelled at

the World Bank’s request. 12 See the Project ISR Sequence 7, issued in May 2014. According to this ISR, “the outpatient utilization increased

to 0.082 from 0.067 visits per capita (a 22 percent increase against the end-of-project target of 10 percent increase);

this was a slight (5 percent) decrease from 2012 of 0.086. In August 31, 2014, the district hospitals supported by the

project could provide 71 percent of the services required according to the national standards, compared with 52

percent at the baseline.”

10

32. Project contribution was crucial in supporting the PDO progress toward the MDGs

on maternal and child mortality. During project implementation, all health workers and

traditional birth attendants of the seven provinces were trained and received technical advice and

monitoring from the MoH’s Department of Maternal and Child Care. All the provinces and district

hospitals were provided with training materials and equipment to improve the performance of

neonatal care units. The hospitals also received clean delivery kits to help improve hospitals’

performance and home-based birth delivery. By December 2015, the Project undertook an

evaluation of maternal and neonatal mortality in the Northern Upland provinces, which indicated

that the Project contributed to the improvement of maternal and child health. The IMR, neonatal

mortality rate (NMR), and MMR decreased from 31.1 per 1,000, 11.2 per 1,000, and 178 per

100,000 in 2007–2008 to 29.4, 10.8, and 106 in 2013–2014, respectively. Data for 2015 was not

processed at the time of this ICR.

33. CPMU commitment and capacity. The CPMU was established before project

effectiveness and was always committed to improve the project performance and achieve its goals.

The CPMU built strong and systematic capacity and was essential in establishing the operational

framework of PPMUs, necessary to implement the project in each of the seven Northern Upland

Provinces. The CPMU was in charge of launching and operating project management systems,

including human resources, fiduciary functions, offices, and equipment throughout the

implementation of the project. During project implementation, the CPMU and the PPMUs

prepared the project’s annual plans, monitored annual implementation targets, and provided timely

interim financial and activity reports and audits. Exception should be made to some reports on

project result indicators toward the last year of project implementation because of some staff

shortness from both CPMU and PPMUs.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

34. Design. Despite the extended time to prepare the project, the Government and the World

Bank agreed that the baseline survey and the completion of the RF would be completed during the

beginning of Project implementation as part of the Project’s Component 3, activity (v)13. Some

baselines and targets for the KPIs and IOIs set in the PAD (annex 3) were to be confirmed or

changed after the 2009 Baseline Survey. The poor quality of the health information system (HIS)

in the country, provinces, and district hospitals at the time of project preparation established

barriers to confirm upfront KPI and IOI baselines and therefore targets. For this reason, the

Government and the World Bank team opted for a baseline survey to be launched after project

effectiveness. The RF would be monitored and evaluated by comparing data from the baseline

survey with the results of a midterm evaluation survey and a project-end survey. The baseline

survey was part of the project design, as indicated in the PAD.

35. The four KPIs were linked to the three parts of the project PDO14 - KPI #1 (increase

utilization rates of inpatient services in district hospitals among Decision 139 beneficiaries) and

KPI #2 (increase utilization rates of outpatient services in district hospitals among Decision 139

beneficiaries)15 were linked with the PDO part 1, increase utilization of district health services;

13 See page 5 of the PAD. 14 See paragraph 10 of this ICR. 15 The baseline inpatient and outpatient rates of district hospitals, apparently low, have to be seen as part of a global

inpatient and outpatient rates that include other health services, such as community health centers and community

11

KPI #4 (proportion of district hospitals that provide full set of health services according to the

national norms) was linked with the PDO part 2, improving the quality of district-level hospitals,

and KPI #3 was linked to PDO part 3, reduce financial constraints to access health services16.

36. The 13 IOIs were fully aligned with the project components. Component 1 (Strengthening

District-level Health Services) was evaluated by six IOIs, addressing the following dimensions:

patient satisfaction (IOI #1), adherence to treatment protocols (IOI #2), human resources training

(IOI #3); maintenance plans and budget for facility and equipment maintenance (IOI #4); health

facilities constructed, renovated, and equipped (IOI #5)17; recent discharged patients (IOI #6).

Component 2 (Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries)

was monitored by four IOIs addressing: people with access to a basic package of health, nutrition,

and reproductive health services (IOI #7)18; percentage of Decision 139 beneficiaries who had

received health insurance cards (HICs) (IOI #8); percentage of Decision 139 beneficiaries with

cards, who can correctly identify at least three benefits covered under the HCFP program (IOI #9)

and households identified with financial barriers for not seeking health care (IOI #10). Component

3 (Monitoring, Evaluation, and Project Management) was monitored by the following tasks:

project management units at the central and provincial level established and maintained, bank

accounts opened, staff and consultants recruited and systems established (IOI #11); availability of

data for project M&E in years three and six (IOI #12); and project management units provided

with adequate plans, annual implementation targets met, and timely financial and activity reporting

(IOI #13).

37. Implementation. During the Baseline Survey in 2009, the Government and the World

Bank team considered that, given operational reasons, KPI #4 was adjusted to measure “the

average percentage of health services provided by the district hospitals.” Baseline values were

available for all KPIs after project approval. The baselines for KPI #119 and #2 were calculated as

of June 2009, while baseline values for KPI #3 (from VHLSS) and #4 (from hospital administrative

records) were calculated as of June 2008. The 2009 Baseline Survey conducted during the first

semester of 2009 provided baseline values for the NUP and HEMA Projects.

38. Baseline values for some IOI’s received special treatment during project

implementation. IOIs #1, #2, and #5 to #10 baselines were calculated as of June 2009 while

baselines for IOI # 3 and #4 were calculated as of June 2008. No baselines were established to the

three IOIs related to Component 3 (IOIs #11, #12, and #13) because they were process indicators.

Given their complexity, IOIs #2, #3, and #4 were split into several sub-indicators to be

health stations (for outpatients) and provincial hospitals, national hospitals, and private hospitals for inpatient rates.

According to the Vietnamese Household Living Standard Surveys (VHLSS) 2014, in the Northern Upland

Provinces, the number of total inpatient visits per inhabitant was 0.116 (0.063 in district hospitals) and the number

of outpatients visits was 0.462 per inhabitant (0.074 in district hospitals). 16 Baselines and follow up data for this indicator was obtained from household surveys (VHLSS). 17 IOIs #5 and #6 did not appear in the PAD and were added during project implementation. 18 IOI #7 did not appear in the PAD and was added during project implementation. However, it cannot be measured

because the baseline and target (expressed in percentage) were not converted by the number of beneficiaries as

referred to in the title of the indicator. This indicator is therefore not a measure for the purpose of this ICR.

However, by December 2015, the number of beneficiaries reached was 270,300. 19 Baseline for KPIs #1 and #3 and for IOI #9 were initially established in annex 3 of the PAD as based in the

VHLSS 2004. However, the baseline of these indicators was revised to June 30, 2009 (KPI #1 and IOI #9, using the

baseline survey) and to June 2008 (KPI #3, using the VHLSS 2008), respectively.

12

appropriately measured. IOI #2 (adherence to treatment protocols) were split into nine sub-

indicators of three relevant health conditions affecting population in the NUP areas—diarrhea,

pneumonia, and poisoning20. A similar process was used to measure IOI #3 (percent of eligible

district health staff who have successfully completed training provided by the project), measured

by three sub-indicators21, and IOI #4 (proportion of district hospitals with acceptable operations

and maintenance plans and budget for facility and equipment maintenance) measured by two sub-

indicators22. All baselines for the IOI #2 to IOI #4 sub-indicators were calculated as of June 2009.

39. Supplemental M&E indicators. To improve the monitoring of project achievements, the

CPMU created and monitored additional indicators that were not part of the PAD and legal

document (Supplemental Letter #2), but were reported in project ISRs. These indicators were

recently discharged patients (IOI #6); number of health facilities constructed, renovated, and/or

equipped (IOI #5), and; number of inpatient beneficiaries with access to a basic package of health,

nutrition, or reproductive health services (IOI #7). The latter was crucial to support the

achievement of the health MDGs in the NUP areas upon project closing. However, no baseline

was established for these indicators.

40. MDG’s health indicators (MHIs) monitored after project extension and measured by

the Government. The level II restructuring of August 2014 to extend the project by 18 months

did not use that opportunity to formally revise the RF and establish the baselines and targets from

the 2009 Baseline Survey. Considering the need to strengthen the capacity to improve health

MDGs by the district hospitals in the NUP areas, some additional indicators (not followed by the

World Bank) were followed by the CPMU and the PPMUs to compare positive variations between

June 2009 and December 2015. These indicators were the following: percentage of district

hospitals providing caesarean section surgeries (MHI #1); percentage of district hospitals

providing blood transfusion (MHI #2); percentage of district hospitals having continuous positive

airway pressure systems (MHI #3); percentage of district hospitals having light for jaundice

phototherapy treatment (MHI #4); percentage of district hospitals having oxygen breathing

systems (MHI #5); and percentage of district hospitals having newborn resuscitator (MHI #6).

Several other indicators were part of this M&E block, but without baselines.

41. Final targets for the project RF’s indicators. All KPIs and IOIs (except IOIs #11 to #13,

which required yearly monitoring) considered August 31, 2014 as the final target date. By

agreement between the Government and the World Bank, the CPMU did not extend the final target

indictors as part of the project restructuring of August 2014, but as mentioned before, the project

missed the opportunity to formally revise the 2009 baselines and update targets to 2015. Even

keeping the original date for the final targets, some of the project indicators were measured by

20The description of the 9 sub-indicators could be found in Annex 5. According the provincial health authorities,

these three conditions represented the majority of the demand for health care in district hospitals. 21 The sub-indicators are (a) percentage of doctors and assistant doctors at district hospitals trained by the project;

(b) percentage of health staffs who completed short-term training courses compared to the plan and; (c) percentage

of health staffs who completed long-term training courses compared to the plan. 22 The sub-indicators are (a) proportion of district hospitals with acceptable operations and maintenance plans and

budget for facility maintenance and (b) proportion of district hospitals with acceptable operations and maintenance

plans and budget for equipment maintenance.

13

December 31, 2015 by the independent evaluation of the project, conducted by the CPMU during

the project extension period. These indicators are KPIs #1, #2, and #4; IOI #3, #4, #5, and #8.

42. Intermediate targets and M&E processes. Given the hard conditions to obtain health

information associated with the NUP areas and district hospitals, the project established its own

administrative records at the PPMUs and associated its evaluations to midterm and final surveys

with the same methodology used in the 2009 Baseline Survey. The current Government’s surveys

are improving, but they still could not be used as the main source of information for the project’s

M&E processes because of lack of reliability and availability of the data, especially for project

areas where the sample surveys are not representative. During the project life, three surveys

associated to the M&E process were done: for the baseline (first semester of 2009); for the MTR

(November 2011–May 2012), and at the project original closing date (before the extension, May–

June 2014). Intermediate targets are measured only once (May 2012) and final targets in June 2014.

District hospitals’ administrative records and eventual regional surveys provided additional

information that was recorded in the project’s ISRs in the interim.

43. Utilization of M&E data. Given that district hospitals’ HIS had weak capacity in the first

years of the project implementation, the utilization of M&E data by the CPMU and PPMUs was

limited during part of the project life. Some difficulties remained in the use of definitions and in

the calculation of some project indicators by the PPMUs and therefore building the aggregated

data for some indicators was delayed. As a result, many data and information were not recorded

timely and accurately, leading to delays in the submission of M&E reports. The MTR evaluation

was useful to refocus the project, in the needed areas, with more support, such as providing

technical assistance of qualified consultants to support the provinces to improve data reporting and

accurate calculation of the project indicators. At the project end, the RF was reported regularly by

the consultants, creating the database for the final project evaluation.

2.4 Safeguard and Fiduciary Compliance

44. There were two types of safeguard policies triggered by the project: social and

environmental. The World Bank’s safeguard policy on Indigenous People (OP/BP 4.10) was

triggered given that the majority of project beneficiaries were ethnic minority groups. Since

its design, the project was expected to have positive impacts for these beneficiaries, which

represented 82 percent of the NUP population. As a result, the project increased demand and

utilization of health services by the poor and ethnic groups, with positive impacts on reducing the

financial burden on health care for beneficiary households through increasing coverage and

strengthening the implementation of the HCFP. The project execution was considered satisfactory

on implementing the indigenous people policy, given that ethnic minorities were part of the project

primary target population.

45. The project hospitals achieved remarkable improvements in the waste management

process compared with the baseline (2007), when most of them did not comply with health

care waste management regulations. The project design included the development of a health

care waste management (HCWM) plan to provide district hospitals with proper disposal of the

medical wastes. It included the preparation of specific HCWM plans for each district hospital

under the project; training and use of IEC materials on HCWM for district hospital staff;

procurement of HCWM equipment and supplies; and; the introduction of wastewater treatment

14

facilities at district hospitals. The implementation of the HCWM plans experienced some delays

during the project life and was classified in the last ISR as moderately satisfactory, because of

difficulties encountered in the procurement of equipment and corresponding staff training and

changes in the World Bank’s policy on the procurement of some HCWM that were included in the

original procurement plans, such as incinerators.

46. By the end of the project, most of the district hospitals had a well-prepared HCWM

plan and monitoring program. Out of the 65 project hospitals, 59 have trained their HCWM

officers. All project hospitals fully complied with the regulations on waste segregation and

collection. Regarding waste storage, 45 percent of project hospitals fully complied and 51 percent

of project hospitals partly complied with the related regulation. By project closing, around 60

percent of project hospitals were treating their hazardous waste and 38 percent of project hospitals

were treating wastewater in a proper manner. Because none of the hospitals had relevant

experience in waste management processes at project onset, the project was quite successful in

that regard and in the CPMU’s ability to make sure that all district hospitals would continue

implementing HCWM plans after project closing.

47. The project provided good solutions and implementation of waste management

processes at district hospital level. Basic training on regulations related to health care waste

management and nosocomial infection control were provided for the district hospital managers

and the staff. The CPMU and PPMUs received guidance for planning and implementing measures

for health care waste management. Consumables and equipment for health care waste management

were procured by the PPMUs and distributed to the project district hospitals. The availability and

proper use of waste containers, transportation, and cooling devices resulted in significant

improvements in health care waste separation, collection, storage, and final disposal in district

hospitals.

48. Financial management. The financial management and counterpart’s fund commitment

performance moved from satisfactory, in the first years of project implementation to moderately

satisfactory in the last three years of implementation because of some delays in the audit processes.

The CPMU and PPMUs were appropriately staffed during the great part of project implementation.

Financial reports were delivered with satisfactory quality and audit reports were provided, but with

some delays. Despite the fact that project implementation went faster than project disbursement,

the 18-month project extension lead to the use of 97 percent of the loan funds (USD58.4 million).

About USD1 million equivalent of the project funds were unused and returned to the World Bank.

49. Procurement. During the major part of the project life, procurement performance was

rated as moderately satisfactory because of some delays in the delivery of procurement packages,

but it was improved during project implementation. The Government and the World Bank were

proactive in solving many of the problems and in reviewing and adjusting procurement plans. Most

of the packages were implemented on schedule or slightly behind schedule. Post review of

procurement found no major deviation or non-compliance. At closing, procurement plans were

respected with no complaints and progress was rated as satisfactory.

15

2.5 Post-completion Operation/Next Phase

50. Project sustainability is likely, by the commitment of provincial government’s

budgets, to finance transport and meals subsidies for the poor/ethnic minorities to access

health care services at district hospitals, maintain the equipment, and retain skilled human

resources in district hospitals. The MoH issued national norms to create Provincial Health Funds,

establishing incentives to keep health care providers in the mountainous provinces. MoH Decision

38, issued in 2012, allows the use of district hospital revenues to maintain and buy new equipment.

In addition, Decision 14, also issued in 2012 by the Prime Minister, allows the Provinces to use

their fiscal revenues to support transportation and meals to the poor and near poor for their visit to

district hospitals. Both government’s decisions were taken as the positive outcomes of the NUP

project23. Four of the seven provinces have established provincial HCFP to finance meals and

transportation for the poor and ethnic minorities’ medical visits. The provinces which have not yet

established HCFP - Dien Bien, Bac Kan, and Cao Bang - are in the process of doing so with support

from the MoH. However, during the project extension phase, between 2014 and 2015, the number

of district hospitals with budgets to maintain infrastructure and equipment had a slight reduction

by 20 percent and 10 percent, respectively.

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design, and Implementation

Rating: Substantial

51. Relevance of Objectives: High. The project was fully aligned with Vietnam’s CPS 2007–

2011 (Report 38236-VN) and with Vietnam’s CPS 2012–2016 (Report 85986-VN) in terms of

priorities for the health sector and also with the country’s future health strategies. The project was

essential to support the country’s health sector along with its transition from low- (IDA) to middle-

income country (IBRD), improving quality coverage for essential district hospital services to the

poor and ethnic minorities in the Northern Upland Provinces, especially to mothers and children.

The project was also a priority of the MoH and provincial government by its relevance to reduce

health coverage inequities and strengthen district hospitals’ performance, contributing to attending

the health needs of the mountainous areas under the Vietnamese Health Reform priorities. The

project was consistent with the country’s Health Sector Development Plan (2011–2015) and the

Health Sector Strategy, with a vision to 2020. The PDO remains relevant now and in the coming

years. In 2016, the Vietnamese Government and the World Bank launched the Vietnam 2035

Agenda, which is completely in line with the project objectives, especially the objective of

increasing quality health services access to the poor24.

23 Center for Environmental and Health Studies. 2014. Report on the Final Evaluation of the Northern Uplands

Health Support Project, Hanoi, July 2014. 24 The World Bank and the Ministry of Planning and Investment of Vietnam. 2016. Vietnam 2035: Toward

Prosperity, Creativity, Equity and Democracy, Hanoi, February 2016. According this document ‘the major policy

challenge facing Vietnam’s health system over the next 20 years will be to achieve universal health coverage that is,

to ensure that everyone has access to high quality services without suffering financial hardship. The objective of

universal health coverage is closely linked to Vietnam’s overall equity agenda, both in ensuring access to services to

promote social inclusion and in reducing poverty due to out-of-pocket payments for health care’.

16

52. Relevance of Design: Substantial. The project design drew from projects in other poor

regions of the country (such as MHSP, CNHSP, and NORRED) aimed at ensuring equity in the

coverage and access to health services for the poor and ethnic minorities. The project design was

partially based from the MHSP design. It also was influential in developing the design of the

CNHSP and NORRED projects—both still under implementation. The project design

appropriately addressed both demand and supply sides to improve coverage and access to health

services provided by district hospitals in the mountainous areas.

3.2 Achievement of Project Development Objectives

Rating: High

53. The overall efficacy rating is the result of the assessment of the achievement of the

indicators associated with the three parts of the PDO, which are rated High (part 1), High (part

2), and Substantial (part 3) (See annex 5, Analysis of the PDO Achievement).

54. Achievement of the project PDO part 1: High. The PDO part 1, related to the supply-

side interventions, was increased utilization of district health services (in the project case, district

hospitals). The utilization level of district hospitals could be measured majorly by two indicators:

inpatient utilization rates (KPI #1) and outpatient utilization rates (KPI #2). The targets for these

two KPIs were significantly surpassed as the project created effective access to district hospitals

services for the poor and ethnic minorities. Targets were exceeded by a set of health services that

were broader than those originally defined as the project target.

55. Achievement of the project PDO part 2: High. The PDO part 2, related also to the supply

side, was improving the quality of district-level hospitals. This part of the PDO was measured by

the proportion of district hospitals that provide the full set of health services according the national

norms (KPI #4), which was surpassed, and seven IOIs (#1 to #7, six surpassed and one achieved).

These IOIs addressed quality aspects of health care at district hospitals, such as client satisfaction,

adherence of treatment protocols for select health conditions, and interventions for inpatients;

training of district hospitals staff on maintenance schedule, and budget allocation for hospital

infrastructure and equipment; construction/renovation and equipment of health facilities; and

patients discharged.

56. Achievement of the project PDO part 3: Substantial. The PDO part 3, related to the

demand side, was reducing financial constraints to access health services. This part of the PDO is

measured by the percentage of households experiencing catastrophic health care expenditures in

the year before the survey (KPI #3), which was surpassed, and three IOIs: #8 and #10 (surpassed)

and #9 (not achieved)25. These IOIs addressed aspects such as the possession of HICs by the

beneficiaries, their knowledge about how to use/access the HI benefits and the percentage of

households identifying financial barriers as the main cause for not seeking health care.

Achievement of this part of the PDO is considered substantial according the methodology used to

rate the PDO’s efficacy. IOI #9 (percentage of Decision 139’s beneficiaries with cards who can

correctly identify at least three benefits covered under the HCFP program) was not achieved

25 The CPMU manifested to the World Bank team, after the MTR that the achievement of the IOI #9 should be

ambitious, but did not suggest to change the indicator or its targets during the project implementation.

17

according to the criteria used to evaluate PDOs26. One possible reason, among others, for this

failure may be the need for more appropriate IEC activities to increase the understanding of the

HIC benefits to the ethnic minorities.

Table 1. Achievement of Project Indicators

Target Achievement Status KPI IOI

IOI

Sub-

indicators

Total

Target surpassed 4 9 12 25

Target achieved or substantially achieved (>=85

percent met) 0 2 2 4

Target partially achieved (65%–84% met) 0 0 0 0

Target not achieved (<65% met) 0 1 0 1

Not considered 0 1 0 1

Total 4 13 14 31

% surpassed and achieved/substantially achieved 100 80 100 94

Source: ICR Datasheet, pages iii–ix. This table does not include progresses in the MHI because they are not part of

the PAD or the legal agreement and neither had established targets.

57. An overall analysis of the 31 project indicators shows that 94 percent of the project

indicators were surpassed, achieved, or substantially achieved by the end of project

implementation. As can be seen in table 1, the project had 4 KPI’s, 13 IOI’s (three of them split

into sub-indicators) and 14 sub-indicators used to measure three IOI’s (#2, #3, and #4). Table 1

shows the level of achievements of project indicators. Part of this good performance could be

explained by modest targets attributed to the KPIs and IOIs at the design stage of the project RF.

The lack of information to foresee how the project investments should affect the district hospital

performance made it difficult for the Government and the World Bank to increase the project risk

by using challenging targets for the project indicators. During the project restructuring of 201427,

both— Government and the World Bank—should have formally agreed on setting more ambitious

targets for some of the KPIs and IOIs for the following 18-month extension, based on the

accomplishments verified by the 2014 end-of-project survey. However, this was not done because

many original project activities were not continued during the extended period and were replaced

by new training activities and the provision of equipment/consumables for maternal and child

health care for achieving the MDGs.

58. The project showed excellent performance on the achievement of the RF targets.

There are two reasons for this: first, the uncertainty about the results of the project investments

during the project design and beginning of implementation led the Government and the project

team to be cautious about the KPIs and IOIs targets. Second, as it happens in other international

contexts (see the efficiency analysis section of this ICR), projects that drive the investments to

26 See annex 5, paragraphs 2 and 3 and annex 5A. 27 The project had problems that should lead to require an earlier project restructuring, such as adjustments in the

indicators baselines and others. However, the decision process to do a project restructuring in Vietnam is complex. It

requires six months of anticipation and needs to be approved by the country’s prime minister. The MoH did not

considered adjustment in the project indicators a relevant reason to require a project restructuring. Only subjects

considered more relevant, such as a project extension, should be reasons to lead the sectoral ministers to submit to

the prime minister a project restructuring in Vietnam.

18

simultaneously solve supply- and demand-side constraints are proven to be more efficient in

achieving good performance in outputs and outcomes.

59. Assessment of achievement of the project activities and outputs. Another way to verify

the PDO’s achievements is evaluating the completion of the project outputs by activity,

subcomponent, and component (see annex 2). According to the Project Output Map (annex 2A),

the project was structured in three components, six subcomponents, 19 activities, and 29 outputs.

Two outputs were not in the PAD, but were added during project implementation (long-term

training for pharmacists and short-term training for health staff on MDG-related activities). The

other 27 outputs could be split into two groups: 10 with quantitative results and 17 with qualitative

results. For the first group, the same rating categories could be used to classify the KPIs and IOIs

in the PDO analysis (surpassed, achieved, substantially achieved, partially achieved, and not

achieved)28. Assessment for the second group is based on the ISR records. This exercise allowed

the ICR team to confirm that 9 out of 10 outputs in the first group were surpassed and one was

substantially achieved. These outputs are related to long-term and short-term training activities,

techniques and skills transfers, medical equipment provision and facilities

construction/improvement, management training, and hospital maintenance plans. In the second

group of 17 outputs, 9 were achieved, 7 were partially achieved, and 1 not achieved and cancelled

(the issuance of a management excellence award program), because the MoH created a systematic

process to award health services and personnel with excellent performance which was not related

to the NUP. This system has not been followed up by the CPMU and the project. The achievements

(full or partial) are related to support capacity improvement of the HCFP, implementation of

studies on the benefits’ incidence of the program, provision of office equipment to PPMUs, and

others.

60. Substantial progress has been verified in KPIs and IOIs during the 18-month project

extension. Table 2 shows the achievements during the project extension period: three KPIs, two

IOIs, and a few MHI indicators were achieved. At project closing, three of the four KPIs were

measured and were found to have shown improvements. Short- and long-term training for health

staff at the district hospitals level have also improved with increases from 20 percent to 115 percent.

Some problems were identified with regard to the health budgets and the maintenance and

operations of district hospitals’ infrastructure and equipment, which could have indicated

commitment toward sustainability of project interventions. Substantial progress could be

registered in the health MDG indicators, especially in the reduction of maternal mortality, given

that it was one of the main reasons used by the Government to request the 18-month extension of

the closing date.

Table 2. Indicators Achievements during the Project Extension Period

Indicator Baseline

(2009)

End

Project

Survey

(2014)

Extension

Phase

(2015)

Percent

Increase

During the

Project

extension

period

(2014–2015)

KPIs

28 See description of the indicators rating in paragraph 2 of annex 5.

19

Indicator Baseline

(2009)

End

Project

Survey

(2014)

Extension

Phase

(2015)

Percent

Increase

During the

Project

extension

period

(2014–2015)

#1 - Increase utilization rates of inpatient services in

district hospitals among Decision 139 beneficiaries (per

100 inhabitant)

0.025 0.085 0.096 13

#2 - Increase utilization rates of outpatient services in

district hospitals among Decision 139 beneficiaries (per

100 inhabitant)

0.067 0.206 0.247 20

#4 - Proportion of district hospitals that provide full set

of health services according to the national norms (%)

39.1 71,4 80,4 13

IOIs

#2.1 - Percentage of health staffs who completed short-

term training courses compared to the plan

0 357.3 427.3 20

#2.2 - Percentage of health staffs who completed long-

term training courses (specialty level-1 doctors)

compared to the plan

0 88.0 189.2 115

#3.3 - Total number of health staffs trained in short-term

and long-term courses

0 8,929 11,868 33

#4.1 - Percentage of district hospitals with acceptable

operations and maintenance plans and budget for facility

maintenance

49.2 99.3 85.9 −13

#4.2 - Percentage of district hospitals with acceptable

operations and maintenance plans and budget for

equipment maintenance

77.1 99.2 93.8 −5

MHIs

#1 - Percentage of district hospitals providing caesarean

section surgeries 74.6 81.2 92.1 13

#2 - Percentage of district hospitals providing blood

transfusion

56.7 67.2 82.8 23

#3 - Percentage of district hospitals having CPAP

devices 25.0

75.0 79.7 6

#4 - Percentage of district hospitals having light for

jaundice phototherapy treatment 21.4 82.1 87.5

7

#5 - Percentage of district hospitals having oxygen

breathing system

42.9

75.0

78.1

4

#6 - Percentage of district hospitals having newborn

resuscitator

46.4 92.9 92.2 −1

Note: CPAP = Continuous Positive Airway Pressure.

3.3 Efficiency

Rating: Substantial

61. A quantitative cost-benefit analysis was not included in the PAD because there was

no empirical basis for estimating the project’s impact on health outcomes at that time.

However, project investments improved the coverage and quality of health services,

especially among the poor and ethnic minorities population of the NUP, increasing the

efficiency in both the supply and demand side of health care. Annex 3 presents the project’s

economic analysis based on (a) rationale of the government investment in the region; (b) the

http://www.webmd.com/sleep-disorders/sleep-apnea/continuous-positive-airway-pressure-cpap-for-obstructive-sleep-apnea

20

efficiency on achieving quality access for district hospitals by the NUP population (project

outputs); (c) the project contribution to improve health benefits (project outcomes) and; (d) the

impact of the project interventions in the equity on health access and health spending in benefit of

the poor and ethnic minorities. On the supply side, the project effectiveness was associated with

improvements in the outputs of district-level health services by providing medical equipment and

better trained personnel. On the demand side, effectiveness was associated with increases in the

utilization of cost-effective health care by poor and ethnic minorities, whose previous levels of

health care utilization were considerably lower than those of other social groups in the country.

62. The Government’s choice to invest in this region was based on the unfavorable

socioeconomic conditions and the difficult access and generally poor quality of health

services, especially at district health hospitals. The Northern Upland is the region with the

highest concentration of ethnic minorities, which is also part of the Decision 139’s beneficiaries

targeted by the project. The central Government sought to improve the health status of the

population and reduce the health gap between the NUP provinces and the rest of the Vietnamese

population. The project achieved this by improving the efficiency on delivering health outputs,

reducing maternal mortality in higher proportion than the country’s average, and reducing the

equity gap in out-of-pocket health spending between the poorest and the richest income quintiles.

63. The main conclusions of the economic analysis are the following: (a) the project was

efficient in delivering its outputs, reducing costs for training, equipment installation, and civil

works. This led to the savings being used, during the project extension period, for new activities

related to the MDGs in the project area and (b) the project interventions and subsidies to the poor

and ethnic minorities were efficient by contributing to reducing maternal, neonatal, and infant

mortality and by improving the equity on assessing health care and reducing health spending to

poor and ethnic minorities.

64. The project has significantly contributed to the improvement of equitable access of

health care in the NUP areas. Project benefits resulted in health improvements of the poor and

ethnic minorities as they represent a high share of the Northern Upland populations in addition to

the project demand-side subsidies targeted to the poor and ethnic minorities.

Allocative efficiency

65. Improving access to mother and child health interventions at the community-level

hospitals is recognized by literature as a highly cost-effective investment. Most of the health

interventions provided by district hospitals are focused on reducing maternal, newborn, and child

morbidity and mortality. A recent Diseases Control Priorities Program Third Edition publication29

shows high economic return rates for interventions aimed at increasing coverage of services where

good evidence exists for demand-side interventions to motivate service uptake. Following this

evidence, the project interventions contributed to the reduction of infant and maternal mortality

29 Black, Robert E., Ramanan Laxminarayan, Marleen Temmerman, and Neff Walker, Editors. 2016., Reproductive,

Maternal, Newborn, and Child Health, Disease Control Priorities, Third Edition (Volume 2), Ed. Washington (DC):

The International Bank for Reconstruction and Development/World Bank; Apr 5, 2016. ISBN-13: 978-1-4648-

0348-2ISBN-13: 978-1-4648-0368-0.

21

rates in the NUP regions, which were reduced from 31.1 to 29.4 per 1,000 births alive and from

178 to 106 per 100,000 births alive between 2008 and 2014, respectively.30

66. Improving health infrastructure and skilled professional attendance at district

hospitals were identified as the main factors to remove barriers to health service availability

for women and children in the poor mountainous areas of the Northern Upland provinces.

Therefore, closing the gap in health care access required targeting resources on the identified

barriers and on ethnic minority’s populations. The return on infrastructure investments has clear

long-term benefits, which facilitate improvements in the provision of quality services. In addition

to infrastructure investments, the project funded some important and highly cost-effective demand-

side interventions under Component 2.

Efficiency of project preparation and implementation

67. No major delays were registered during project preparation. The project was prepared

in 22 months, which is slightly longer than the average for health projects in the World Bank, by

4 months, and longer than the average time for IPF preparation of 18 months for all sectors.

However, considering that no previous experience and information were available to inform the

project design at the concept stage, several documents were elaborated or assessed during project

preparation, which required substantial time from the project team and country staff31.

68. The PDO was achieved within the originally planned period. The project’s

implementing efficiency is associated with the fact that, by July 2014, the project had substantially

accomplished the KPIs and IOIs using only 82 percent of the credit amount. Given the need to

improve MDGs’ outcomes and consolidate the project beneficiary gains, the Government

proposed an 18-month extension to use the remaining funds. By its closing in February 2016, the

94% of the project indicators (KPIs and IOIs) were achieved and surpassed and 97 percent of the

credit proceeds had been disbursed.

69. The efficiency associated with the project closing date extension period could be

considered moderate. The investments in training and equipment to improve mother and child

care (see table 2) were appropriate and contributed to further achieve project outcomes. However,

the provincial budget’s consolidation to guarantee the maintenance of health infrastructure was

30 The impact on maternal mortality reduction is directly associated with the quality of birth attendance provided by

better access and quality birth delivered at the district hospitals. The impact on infant mortality appears to be more

modest, because it should be associated to other non-hospital and nutritional interventions at the community levels. 31 The main reports produced to feed the project preparation are: Health status in the seven provinces of Northern

Upland; Health care service use and accessibility status in the seven provinces of Northern Upland: Son La, Dien

Bien, Lai Chau, Cao Bang, Bac Can; Health Human Resource Analysis in the seven provinces of Northern Upland;

Health care for the Poor: Identification of the needs and proposal of investment for capacity building and

management capacity development; Assessment of the HCFP in the seven provinces of Northern Upland; Health

care for the poor: management according to the Decision 139 in seven provinces of Northern Upland (most difficult

provinces); Health System Assessment for seven provinces of Northern Upland; Inventory of medical equipment in

hospitals of seven provinces of Northern Upland; Output indicators after analysis; Socioeconomic, Demographic,

Cultural Geographic and Health Status indicators: Morbidity, Mortality, Crude Death Rate/IMR, Under 5-child

nutrition status; List of medical staff to be trained and; Cost estimate and Cost Table for training component.

22

slightly reduced from 99.3 percent in 2014 to 79.7 percent in 2015, indicating that efforts may be

needed to ensure proper maintenance of project investments32.

Fiscal impact and sustainability

70. The fiscal impact of the project was considerable at the provincial and district

hospitals’ level. The project had significant fiscal impact in the additional recurrent expenditures

needed to maintain project investments in infrastructure and equipment of district hospitals,

enabling these hospitals to receive funds from the HCFP. For example, from 2008 to 2013, the

average revenue of the 28 district hospitals supported by the project increased by 336 percent.33

On the other hand, the MoH supported the creation of the Provincial Health Funds, establishing

the financial basis to support the recurrent costs of district hospitals in the future, guaranteeing the

sustainability of project investments beyond the project life with a significant fiscal impact.

Vietnam’s rapid economic growth, expected over the coming years, will ensure that provincial

health budgets and district hospitals sustainability are likely to continue to grow at a high rate.

3.4 Justification of Overall Outcome Rating

71. The overall outcome rating is Satisfactory. This rating takes into account the project

remained relevant throughout implementation and beyond and the activities financed by the project

being efficient in providing supply- and demand-side interventions to improve quality access to

health care at the district hospitals. In addition, the overall achievement against the PDOs is

considered substantial.

Table 3. Summary of Outcome Ratings

Project Outcome Ratings

Relevance Substantial

Efficacy High

Efficiency Substantial

Overall Outcome Rating Satisfactory

3.5 Overarching Themes, Other Outcomes and Impact

(a) Poverty Impacts, Gender Aspects, and Social Development

72. The project specifically targeted the poorest mountainous regions, home of ethnic

minorities and isolated communities, and focused on the most vulnerable—women and children,

achieving excellent outcomes in the reduction of maternal mortality and improving the coverage

of HI and utilization of district hospitals by poor and ethnic minorities. The equity dimension was

important in conceptualizing and implementing the project, which contributed to social inclusion

32 The CPMU considered that district hospitals did have plans and minimum budgets for infrastructure and medical

equipment maintenance and repairing at the project end. These types of plans and budgets are submitted yearly by

the provincial Departments of Health to the MoH. However, in 2015 (the last year of the project implementation),

the buildings, and equipment supported by project were still in good condition. Because of that, part of these budgets

was not spent because it was not necessary. 33 Center for Environment and Health Studies. 2016. Report on the Final Evaluation of the Northern Uplands Health

Support Project in the Extension Phase, Hanoi, February 2016.

23

and promoted a solid basis to increase access to better quality health services by providing

economic subsidies, such as transportation costs and meals to encourage medical visits by poor

and ethnic minorities. More information on the equity impact of the project and the district

hospitals access improvement for the poor and ethnic minorities is addressed in Annex 3.

73. Strengthening local levels’ facilities and staff based on social consultations. The project

showed progress in building capacity at the Provincial Health Services Management in the NUP,

benefiting government representatives and different categories of health professionals. The project

investments were defined according to the findings of extensive consultations over a period of

eight weeks in 2007 by the Government team, involving site visits to 10 district hospitals, focus

group discussions with hospital staff at provincial and district levels, 20 in-depth interviews with

health workers, and 40 direct interviews with patients at the district hospitals. According to the

end line project evaluation results, realized by the end of 2015, more than 90 percent of the

interviewed district hospital staff said that they were entirely satisfied with the training received

and its appropriateness to job demands.

74. Environmental contribution of the project. The project improved the awareness of

environmental issues at district hospitals by training personnel on appropriated treatment of

medical and hazardous hospital waste, and contributed to improving the environmental quality and

reducing the risk of hospital infection for the community.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

75. At the project original closing date and at the end of the extension phase, the CPMU

contracted a consulting firm to conduct beneficiary surveys and collect administrative data

comparable with the 2009 Baseline Survey and the 2012 MTR survey34. The main findings of

the beneficiary surveys highlighted that the support and interventions of the project had a strong

influence on changing health care services seeking behavior of the poor/ethnic minorities. The

beneficiaries perceived a continuous improvement of service delivery capacity of district hospitals,

which attracted more people to use the district hospitals’ health facilities. In addition, the project

financing support policy to the poor increased the opportunity to use inpatient services at the

district hospitals by the poor/ethnic minority patients. Moreover, the coverage of HIC increased

and the people’s perception about the district hospitals improved, contributing to create positive

changes in the poor/ethnic minorities’ behavior of seeking health care services.

4. Assessment of Risk to Development Outcome

Rating: Moderate

76. The project strengthened health policies and institutions at the district level,

improved health techniques and technology, provided new managerial tools, and created

financial sustainability of district hospitals in the NUP areas. District hospitals that participated

in the project strengthened quality of the services, developed qualified health workforce and will

34 The assessment tools of these surveys included questionnaires for health staffs, households, and in/outpatients at

district hospitals, in-depth interviews with leaders/staffs at CPMU/PPMUs, leaders at provincial health departments

and social insurance units, leaders/staffs at provincial/district general hospitals, heads of commune health centers,

trainers and trainees of the project, and focus group discussions with patients who completed their treatment at

district general hospitals in the last three months.

24

continue to satisfy the local people’s needs based on sustainable mechanisms for service provision

at the district and commune levels. Supported health care policies for the poor and ethnic minority

groups are based on better management capacity of the resources received from HCFP. The project

contributed significantly to the implementation of policies on strengthening human health

resources, especially for health care at the district level. The district hospitals developed plans to

maintain and use the health staffs recruited from the local human resources after training and plan

to maintain a regular training system to strengthen the capacity of health staff. On the financial

side, all project provinces had specific commitments to provide funds for the maintenance of

equipment and upgrading and repairing of facilities after the project phases out. Besides the

commitments and specific plans, the localities still face a limited budget, creating, despite the

government provisions and regulations to assure the future sustainability of the project investments,

some uncertainty regarding the adequacy of the resources to guarantee the continuation of project

outcomes after closing.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry

Rating: Moderately Satisfactory

77. The project preparation time (22 months) was a bit longer than the average for Health

Nutrition and Population Global Practice projects, but still under the World Bank’s

benchmark for IPFs. The signing of the financial agreement happened 4 months after Board

approval (July 10, 2008) and the project effectiveness took 3 additional months (October 8, 2008),

totaling 29 months between the issuance of the project concept note and the effectiveness. The

project was proposed to be implemented in six years given the institutional implementation

conditions at the NUP, but despite achievement of the PDO within its planned time frame, the total

implementation time was about eight years after project extension. The World Bank team

supported the preparation and transferred to the government team the appropriate technical skills

to prepare the project and used the lessons learned for previous World Bank projects in the country.

The project design was totally consistent with the CPS Government Strategy, described in

paragraph 9 of this ICR. Given the experience in previous projects, the Government and the World

Bank, during preparation, concentrated their efforts on preparing the institutional arrangements to

implement the PDOs and mitigate the main challenges that could prevent the achievements. The

project Operational Manual was prepared and approved by the World Bank before effectiveness,

which contributed to guide the project implementation at the CPMU and the PPMUs.

78. Despite the good and focused design, the project could have improved efficiency in its

implementation if the baseline survey had been carried out during the project preparation

period. The project RF was well designed. However, the baseline survey was designed during

implementation as part of the project Component 3, as can be read in the PAD35. During the

project’s QER, no recommendation was provided to change this implementation arrangement, but

the project could have sped up its implementation if the team pushed to set up the baseline survey

35 in the paragraph 97(page 50) of the PAD is written the following: “This component will support the set up and

management of the Central Project Management Unit (CPMU) and PPMUs through: ….(v) M&E activities,

including the baseline data collection, indicator updates, MTRs, end of project completion report, and audits”.

25

as part of the project preparation instead of completing it in the first semester of 2009 when the

project had been under way for almost one year of implementation. If the baseline survey had been

carried out during project preparation, the World Bank team would have had more room to assist

the Government in establishing a sound and rigorous project M&E system.

(b) Quality of Supervision

Rating: Satisfactory

79. The task team conducted 11 implementation support visits during the project life.

Financial management and safeguards missions were performed adequately. In addition to formal

missions, the project benefited from having during most of the implementation a field-based task

team leader (TTL) who was in constant communication with the CPMU36. The TTL and the team,

including the fiduciary technical support were readily available to solve issues and explore the best

solutions to address technical and administrative problems, and the borrowers’ report (annex 6)

recognizes the capacity of the World Bank team in addressing promptly the demands from the

client. Records of ISRs and aide memoires were kept adequately and the project documents were

easily made available. The team composition, staff resources, and budget to supervision missions

were well used during the project implementation. The quality of the aide memoires was adequate

for supervision purposes. Annex 7 summarizes the project aide memoires.

80. The ISRs reported progress on most of the indicators, but a complete view of these

indicators was only available after the MTR survey. The team was aware of the outdated

baseline and targets during project implementation, but the RF depended on administrative records

and surveys that sometimes were not available or delayed. The project team made many efforts to

improve data availability of the RF, and many district hospitals implemented IT systems to

improve their administrative records as part of the tasks proposed by the project. However, the

response was sometimes slow because it depended on the local HIS implemented by the PPMUs

at the provincial level and their support to the district hospitals.

81. Environmental and social safeguards were well monitored resulting in considerable

improvements in the waste management systems at the district hospitals and a change of

perception of the quality of these hospitals by poor and ethnic minorities. The World Bank

team’s performance was crucial to guarantee the procurement for the hospital waste management

equipment and to help the country design adequate solutions for district hospitals.

(c) Justification of Rating for Overall Bank Performance

Rating: Satisfactory

81. The project was implemented in a very proactive manner by the Bank team, in terms of

technical assistance, good project reports and follow up of the project implementation results.

The client was satisfied with the Bank assistance during implementation and the Bank team try to

move the project positive agenda in the best way, recognizing the limitations and constraints of

the government health policy in the country. Given all features described in paragraphs 78 to 80,

36 Along the implementation, the project had four TTLs, but from 2010 to 2016, local TTL’s were in charge of

implementation.

26

combined with project results in terms of relevance, efficacy and efficiency, the overall Bank

performance could be rated as satisfactory.

5.2 Borrower Performance

(a) Government Performance

CPMU Performance: Satisfactory

82. The CPMU was established with sufficient human resources (including the national

consultants) and good capacity to manage and implement the activities and to use the

financial resources. As expressed in paragraph 23, the MoH authorities were totally committed

to project preparation and to development of the project design. After the project be approved, they

worked to guarantee adequate budget for communications, supervision, M&E, and travel during

implementation. The provincial health authorities and district hospital managements were also

committed to participate in the project development. Field visits were organized during preparation

to establish links and networks with the local government health and hospital’s authorities. The

CPMU has promoted an effective coordination with the functional departments of the MoH,

accelerating the processes for evaluation and approval of the project needs in the higher

government levels. The handbook for guiding the implementation of the project was compiled and

issued by the decision of the Minister of Health right at the beginning of the project, contributing

to speeding up its implementation. The Government established a project steering committee that

works satisfactorily with regular meetings to solve issues during the project implementation.

83. The CPMU was able to promote the effort, commitment, and responsibility of the

provincial leaders to increase the efficiency of the project on the economic and social

development of the provinces. The CPMU also provided capacity building for the PPMUs’ staff

to improve project management, including financial, accounting, and procurement issues; assets

management; civil works; training on HCFP assessment; and M&E activities. The CPMU

performance was essential to implement the project on time and exceed the targets. The CPMU

assistance to the provinces and district hospitals was highly effective, providing significant support

to the health management of the NUP beneficiaries.

(b) Implementing Agency or Agencies Performance

Implementing Agency Performance Rating (PPMUs): Moderately Satisfactory

84. In the provinces with high attention and strong support from the local governments,

the PPMUs performed the project more efficiently than in others. The progress on the project

implementation should be considered uneven among the PPMUs. Given this situation, some delays

in auditing and reporting happened with differences in performance of health indicators among

them. Additionally, many PPMUs did not provide stable human resources that could guarantee

constant improvements in the project management and results M&E, especially close to the end of

the project when staff departure was observed in some PPMUs.

(c) Justification of Rating for Overall Borrower Performance: Satisfactory

85. The overall borrower performance could be classified as satisfactory, given that

despite the weakness of part of the PPMUs on M&E, the CPMU provided all elements to fix these

problems by contracting local consultants and provide technical assistance to produce data and

27

good reports that fed the final project evaluations. On the other hand, the CPMU direction

increased the managerial autonomy of the district hospitals and assist the provincial health

departments on improve the general management of the local health systems in the project areas

6. Lessons Learned

86. Investments in hospital infrastructure, equipment, improved skills of health staff, and

incentives to retain qualified human resources at grassroots levels (provinces, districts, and

communities) are important factors to achieve basic health goals (such as the MDGs) and

improve access to health services for achieving universal health coverage. The project assisted

localities to develop their health plans, to implement preparation steps for maintaining the

activities after the project ended, and to continue the issuance of policies and plans on health

support for the northern mountainous provinces. Two aspects could be highlighted: (a) the health

workforce development policy and (b) the retention and training of teams of village midwives to

serve remote areas where home birth delivery is necessary.

87. The project supported progressive improvements in the financing mechanisms for

provincial health services, but financial sustainability at the district hospitals in the NUP

areas remains a challenge. National and provincial hospitals have more ability to increase their

revenues and sustain and balance their budgets than district hospitals, which face lower levels of

funding, contributing to increasing the risk of poor quality services. However, in Vietnam, HI

payments and user fees are the dominant source of finance for public (non-district) hospitals. The

Government progressively is introducing mechanisms to strengthen the sustainability of the district

health services, including the use of government bonds, which, according to a MoH study, reached

91 percent of district hospitals by 2011. However, the Government needs to set up mechanisms

that could allow these hospitals to have adequate financing as well as higher autonomy and less

dependency on the provincial revenues. Part of this effort should be to increase and adjust the price

of the health services provided by the district hospitals (and paid by the HI) according to their real

costs, especially considering that the district hospitals received lower fees than those paid to

regional and national public hospitals.

88. Balanced interventions and health investments in both the supply and demand sides

enable district hospitals to improve coverage, access to health services, and the range of

services offered in deprived areas. Although district hospitals in Vietnam have the ability to

provide about 75 percent–85 percent of their assigned service list, these hospitals in the Northern

Upland provinces at the beginning of the project only provided 35 percent of the list because of

lack of medical equipment, lack of health professionals, particularly specialists, and the weakness

of provider payment mechanism and service price list, which do not support or encourage district

hospitals to provide services. The project increased the range of services from 35 percent to 80

percent addressing investments in medical equipment and human resources, but failed in

addressing innovative provider payment mechanisms to increase incentives for adequate health

human resources retention.

89. Support for transport and meals are efficient mechanisms to increase the utilization

of the health services as these were the key barriers for the poor and ethnic minority

population. Once understanding their benefits and availability of health services, the poorest

populations living in distant areas need the means to travel and sustain themselves when seeking

28

health care at district hospitals. Further, the support policy must be monitored closely to avoid

abuse or overuse by both the service providers (health facilities) and users (patients and their

family) such as unnecessary longer stay for inpatients, unnecessary hospitalization of the patients

who can be treated as outpatients, and so on.

90. Despite the investments and incentives implemented by the project, district hospitals

still have difficulty to retain staff. They have less capacity to generate incentives as well as

reward health staff and improve their salaries. Thus, the provider payment mechanism should be

changed and revised to encourage hospitals to produce better services. The project was able to

solve partially problems of staff, benefiting from rotation of health workers from provincial

hospitals and supplemental salaries to skilled doctors and health specialists working at the district

hospitals. Some district hospitals in Vietnam are using capitation as an incentive to retain staff and

pay for performance, but this kind of arrangement was not used in the district hospitals of the NUP

provinces.

91. The institutional capacity of the PPMUs and CPMU plays an important role in

implementing the project. Staff in implementation units should be well trained on all aspects of

project management and work closely with the technical support from the central project team.

Staff turnover should be limited. During original project implementation time, the quality of

project implementation by the PPMUs was granted, but during the extension period, the transition

of the project staff to the Health Provincial Services was not completed and some of the

administrative functions, such as M&E, were missed. Although CPMU had only 20 staff (less than

many other projects of the same scale), project activities were well implemented, the schedule and

work plan were always on track, and the targets/results were surpassed compared to the planned

outputs.

92. M&E systems and plans should be built ahead of the project effectiveness with

standard forms for collecting data at the implementing sites. This will allow future projects to

have good data and reference sources from the start, improve monitoring of projects achievements

and get the accurate information to set more realistic baselines for some project indicators.

93. Strengthened coordination to assure financial sustainability is key to guarantee the

project continuity at the provincial and local levels. The PPMUs should closely follow up with

Provincial People’s Committees to approve annual financial plans at the earliest, so activities can

be implemented at the beginning of each year.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

94. The borrower issued a report, annexed to this ICR in full, where the main comments

and issues raised are summarized as follows:

(a) The project has successfully achieved its overall development objective. The

utilization of district health services (by the poor and vulnerable population defined

accordingly with the Decision 139) has sharply increased after IEC campaigns,

improving the effectiveness of the Government’s priority to UHC, both by improving

the geographical accessibility to quality health services at district hospitals and by

29

reducing the financial burden in accessing health services for the poor and ethnic

minorities.

(b) The project has been successful on implementing the health care policies for the

poor, increasing their access to quality health services. The number of poor people

receiving financial assistance (meals and travel costs) to visit district hospitals from

NUP has increased yearly, along with the significant increase in the rate of use of

district health services among the general population, especially vulnerable groups

such as poor and ethnic minorities. The average number of inpatient and outpatient

visits per capita per year of ‘HN’ (Ho Ngheo or Poor Household in English) in the

district hospitals increased 390 percent (from 0.0247 to 0.096) and 369 percent (from

0.067 to 0.247) from 2009 to the end of 2014/2015, respectively, exceeding the

project’s expectations. The awareness and confidence of the vulnerable population on

health care services has improved. The percentage of households who experienced

catastrophic health care expenditures at the project end was substantially lower than

the baseline.

(c) The project was designed and implemented based on the needs and

recommendations of the local authorities and communities. The MoH

accumulated experience in implementing projects with similar components and

activities. Therefore, the implementation of the project was quite favorable and the

risk management process was well conducted and did not menace significantly the

project performance.

(d) The project implementation was provided with sufficient human resources

(including national consultants) and good capacity to manage and implement the

activities and to use the financial resources. The CPMU provided capacity building

for the PPMUs’ staff on project management, including financial, procurement,

accounting, assets management, civil works, training, and M&E activities.

(e) The success in implementation of the project was also the consequence of good

management and positive support from the World Bank team. Close monitoring

of the project activities, timely issuance of no objections for procurement and work

plan, suggestions for important solutions and recommendation for speeding up

implementation progress, and participation in technical missions were some of the

valuable contributions of the World Bank team to the success of the project.

(f) For the improvement of future projects, the M&E system and M&E plan should

be built ahead of effectiveness with standard forms for collecting data from the

PPMUs and implementing sites. This will allow future projects to have good data

and reference sources from the start and improve monitoring on the projects’

achievements. The PPMUs should closely follow up with Provincial People’s

Committees to approve annual financial plans at the earliest, so activities can be

implemented at the beginning of each year.

30

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD, Million equivalent)

Components Appraisal Estimate

(USD Millions)

Actual/Latest

Estimate (USD

Millions)

Percentage of

Appraisal

1. Strengthening District-level

Health Services 42.90 40.40 94.0

2. Increasing Financial Access to

Health Care Services for

Decision 139 Beneficiaries

10.00 10.03 101.0

3. Monitoring, Evaluation and

Project Management 13.10 9.71 61.1

Total Baseline Cost 66.00 60.14 88.5

Total Financing 66.00 60.14 91.1

Note: * Exchange rate loss is USD 1.59 million. Total IDA credit is USD58.41 million at project closing.

(b) Financing

Source of Funds

Appraisal

Estimate

(USD Millions)

Actual/Latest

Estimate

(USD millions)

Percentage of

Appraisal

Borrower 6.00 1.73* 28.8

International Development Association 60.00 58.41 97.3

Total Financing 66.00 60.14 91.1

Note: *Without in kind contribution.

31

Annex 2. Project Outputs

Summary

1. The NUP project implementation presented a relatively good performance. The project had

3 components, 6 subcomponents, 19 identified activities, and 29 identified outputs. Annex 2A

provides the project output map before the extension period and during the 18-month extension

phase, identifying project achievements compared with planned or expected outputs. Of the 29

identified outputs, 8 outputs were surpassed, 9 were achieved, 9 were partially achieved, 1 was not

achieved, and 2 were added during project implementation as supplemental outputs37.

2. Project outputs of Components 1 and 2 are related to the PDOs, while outputs of

Component 3 are related with project administrative tasks. Giving that, this annex analyzes the

achievements related to Components 1 and 2. Outputs for Component 3 are found in the project

output map (Annex 2A).

A: Achievements during Original Project Implementation Period (October 2008–August

2014)

Component 1: Strengthening District-level Health Services

3. Component 1 was organized in three subcomponents (a) human resources

development; (b) improving quality of the district hospitals and; (c) improving hospital

management. The first subcomponent tackled training activities for health workers; the second

included activities related to basic medical equipment minor repairs and upgrading in

infrastructure and the last included subjects related to training of district hospitals management

staff, developing hospital maintenance plans, and a management excellence award program. This

component has 14 outputs: 8 were surpassed, 2 were achieved, 2 are supplemental outputs, 1 was

partially achieved, and 1 was not achieved.

(a) Subcomponent 1: Human resources development

4. General conditions. This subcomponent has five activities: (a) long-term training for

health workers; (b) short-term training for health workers; (c) technical handover/skills transfer;

(c) innovative incentive schemes to retain health workers (Phase 1) and; (d) innovative incentive

schemes to retain health workers (Phase 2). All activities were implemented by 10 outputs: 5 were

surpassed, 2 achieved, 1 partially achieved, and 2 are supplemental outputs (without targets

defined upfront).

5. The project financed long- and short-term trainings, covering the costs of

examination and tuition fees, living allowances and per diem, training materials, and

transportation. The training activities were based on plans prepared by district hospitals and the

PPMUs, which were revised and approved by the CPMU and the World Bank. As can be seen, the

37 The achievement of the supplemental outputs is difficult to measure given that no baselines were established at

the onset.

32

number of trained staffs exceeded the original training plans. This subcomponent had 8 outputs: 5

were surpassed, 1 was achieved, and 2 were supplemental outputs (without defined targets upfront).

6. Long-term training: level-1 specialists. Table 2.1 shows that under the project funds, 377

level-1 specialist doctors were supported: 246 at the district level and 131 at the provincial level.

This represents an increase of 206 percent of the original project plans. Out of those 377 level-1

doctors, 309 graduated before December 2013. The participation of doctors from ethnic minorities

was 48 percent. Around 97 percent of the trained doctors returned to work at the original district

hospitals, representing a significant percentage of retention of qualified doctors.

Table 2.1. Number of Level-1 Doctors Trained in Project Provinces According to Plan

Project

Provinces

Level-1

Doctors

as

Planned

Level-1 Doctors Graduated Level-1 Doctors who Returned

to Work at Localities Number

Supported

by Project

Number of

Person

Years under

Project

Support

Total Female

(%)

Ethnic

Minority

(%)

Total Female

(%)

Ethnic

Minority

(%)

Cao Bang 23 56 58.9 92.9 54 59.3 92.6 56 (243%) 102 (222%)

Bac Kan 31 36 50.0 100.0 34 50.0 100.0 50 (161%) 96 (155%)

Ha Giang 38 89 31.5 14.6 86 31.4 14.0 96 (253%) 176 (232%)

Lao Cai 22 38 39.5 18.4 35 37.1 28.6 51 (232%) 97 (220%)

Lai Chau 22 17 35.3 47.1 17 35.3 47.1 19 (86%) 37 (84%)

Dien Bien 19 47 34.0 42.6 47 34.0 42.6 50 (263%) 95 (250%)

Son La 28 26 34.6 38.5 26 34.6 38.5 55 (196%) 102 (182%)

Total 183 309 40.5 47.2 299 40.1 48.2 377 (206%) 705 (193%)

District level 104 — — — — — — 246 (236%) 457 (220%)

Province level 79 — — — — — — 131 (166%) 248 (157%) Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project, Hanoi, 2014.

7. Long-term training: assistant doctors to become medical doctors. The project

supported 1,467 assistant doctors attending the four-year additional training, exceeding 98 percent

of the agreed target on the original project plans. Of this number, 54 percent were ethnic minorities.

Additionally, the project trained 98 assistant pharmacists, attending the four-year additional

training to become pharmacists (32 percent ethnic minorities). This additional outcome was not

originally planned under the project.

8. Short-term trainings: curative care. The project sponsored 3,187 doctors to receive

short-term training in several specialties (table 2.2), compared with 1,041 originally planned by

district hospitals in agreement with the PPMUs (306 percent increase). The CPMU surveyed the

opinions of 226 health staffs who participated in short-term trainings. According to the survey, the

participants highly appreciated the training methods and contents of the courses. Approximately

91 percent and 93 percent of respondents were satisfied with the teaching methods and contents of

the courses and 75 percent reported that the training durations were appropriate.

Table 2.2. Number of Doctors Receiving Short-term Trainings according to the Type of Trainings

Course Plan Total Percentage of Plan

Anesthesia 92 135 147

Lab test 95 149 156

Rehabilitation 105 121 115

Surgery 101 143 142

Pediatrics 103 214 208

Internal medicine 111 194 175

33

Course Plan Total Percentage of Plan

Image diagnosis 165 404 244

Obstetrics 102 191 187

Communicable diseases 80 165 206

X-ray 87 103 118

Pediatrics emergency n.a. 681 —

Obstetrics emergency n.a. 687 —

Total 1,041 3,187 306 Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project,

Hanoi, 2014.

9. Short-term trainings: preventive care. The project trained 335 staffs (156 at the

provincial level and 179 at the district level) focusing on the planning, implementation, evaluation,

and monitoring of preventive medicine activities at the primary level. This number exceeded the

originally planned target of the PPMUs, by 57 percent. The trainings included the preventive and

counseling aspects of several diseases including HIV, tuberculosis, and malaria, which are targeted

at the MDGs.

10. Other short-term trainings: hospital management, health systems management,

information systems, and maintenance of medical equipment. Table 2.3 shows that the number

of trained staffs on these specific short-term trainings exceed the original plans agreed with the

district hospitals and the PPMUs. The number of personnel trained on medical waste management

shows that a huge part of the district hospital staff was involved in the accomplishment of the

project’s environment safeguard, given that almost 3,000 staffs were trained in this specific subject.

Table 2.3. Number of Hospital Staffs Trained and

Comparison between the Planed and Achieved

Field of training Number of Staff

Training Planed

Staff

Trained

in 2014

Percentage Plan Achievement

Hospital management 303 477 157 Surpassed

Medical waste management 215 2,922 1,300 Surpassed

Health Management and Information

Systems (HMIS) 358 397 11

Surpassed

Maintenance of equipment 134 169 126 Surpassed

Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands

Health Support Project, Hanoi, 2014.

11. Incentives to retain human resources in district hospitals: The PAD included, as one

of the project activities, the use of incentive schemes to attract and retain the newly trained health

workers at district hospitals because of a severe shortage of health staff in the NUP areas. To

address this issue, the CPMU and the PPMUs discussed and proposed solutions in workshops that

were held to seek sustainable solutions to maintain adequate health workforce for disadvantaged

areas in the NUP. The workshops discussed national and international experiences on enrollment

and training of health staff. However, the solutions and recommendations focused mainly on

training contents and less on financial support, sustainability, and budget feasibility of the

proposed incentives. To tackle the staff shortage in the NUP areas, during the health workers’

training period, the World Bank agreed that the provinces could hire retired doctors to work at the

district hospitals and health centers. Some provinces, including Cao Bang, Lao Cai, and Bac Kan

had used this hiring mechanism to partially overcome staff shortages with positive results.

34

12. District hospitals gained expertise on many specialties such as surgery, internal

medicine, obstetrics, pediatrics, intensive care, traditional medicine, nursing, and anesthesia,

focusing on emergency techniques and surgery. The project financed the technical support to

the district hospitals’ doctors by assigning provincial doctors to guide the former on every step of

these specific techniques and helping them practice these techniques themselves. In 2014, over

670 provincial doctors participated in the transfer of these techniques to district hospitals

exceeding 167 percent of the PPMU project’s plans. This activity was planned as a result of the

project MTR in 2012.

(b) Subcomponent 2: Improving quality of district hospitals

General conditions. When compared with the original plans, two outputs of this subcomponent

were surpassed.

13. Equipment for district hospitals was procured in two phases (as recommended in the

PAD): Phase 1 in 2011–2012, and Phase 2, in 2013–2014. Table 2.4 shows the number of

equipment acquired in the two phases in the seven provinces, which reached 4,400 units (109

percent of the original procurement plans). Phase 1 included ambulances to address the transfer of

emergency patients in remote districts and four groups of medical equipment: laboratory

equipment, treatment equipment, monitoring devices and ventilators, and infection control team

equipment. Phase 2 included equipment of high-value technology and trained staffs capable of

using this equipment. The project combined investment and infrastructure facilities and human

resource staffing and training in a synchronous manner to avoid under use and depreciation of the

equipment.

Table 2.4. Number of Project-supported Equipment for the District Hospitals at Seven NUP (2012–2014)

Equipment Son

La

Ha

Giang

Bac

Kan

Cao

Bang

Dien

Bien

Lai

Chau

Lao

Cai Total Plan

Percentage

Obtained

Lab test devices 181 131 128 127 83 74 98 822 664 123.8

Intensive care equipment 117 145 126 195 93 78 119 873 1255 69.6

Monitoring equipment 21 24 28 23 14 11 21 142 157 90.4

Ventilators 22 19 14 25 14 4 14 112 217 51.6

Infection control equipment 29 28 22 34 14 10 23 160 173 92.5

Incinerators 0 0 0 0 0 0 0 0 37 0.0

Ambulance care 8 5 5 14 6 6 8 52 52 100.0

Ultrasound machine 24 14 16 16 11 11 17 109 77 141.6

Endoscopic machine — — — — — — — — 37 —

Emergency care devices 59 69 43 75 37 36 47 366 390 93.8

Internal medicine 111 40 59 59 25 18 35 347 209 166.0

Surgical room equipment 85 46 55 65 40 36 55 382 225 169.8

Surgical equipment 158 82 94 120 62 72 67 655 548 119.5

Examining equipment, 69 55 53 68 37 35 48 365 296 123.3

X-ray machine 18 18 16 12 14 9 15 102 78 130.8

Pediatric/ obstetrics surgery 19 7 3 31 25 68 21 174 0 —

Intensive care in obstetrics 15 11 3 31 20 45 35 160 0 —

Total 936 694 665 895 495 513 623 4,821 4,415 109.0 Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project,

Hanoi, 2014.

14. However, a small portion of the equipment provision was delayed. Some of the

equipment supplied by the project faced difficulties in the import procedures and by 2014, few

35

bidding processes were ongoing. The delivery of part of the acquired equipment to hospitals in

remote areas was challenging at times because of difficult weather conditions.

15. The equipment installed by the project achieved high level of utilization. The 2014

project results survey shows that 95 percent of the equipment was delivered and installed into the

district hospitals and was frequently used for diagnosis and treatment. Only 1.4 percent of the

installed equipment was broken and 3.9 percent was unused because of lack of technical skills or

utility/infrastructure associated problems. Table 2.5 shows that in January 2014, the installed

equipment achieved a high level of monthly utilization in the respective district hospitals.

Table 2.5. Number of District Hospitals’ Patients using Equipment in the NUP Areas (January 2014)

Group of Equipment Number of Cases using the Equipment/Month

Surgical room 1,116

Obstetric monitoring 159

Cardiograph 49

Semiautomatic biochemical analyzer 1,981

Automatic hematological analyzer 600

Ultrasound 418

X-ray (high voltage) 466

Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern

Uplands Health Support Project, Hanoi, 2014.

16. The project implemented the upgrading or new construction of 18 district hospitals

compared to 10 that were originally planned. All civil works were carried out in 6 provinces

between 2012 and 2013. The only province where civil works were delayed was Lao Cai, where

two hospitals were expected to have minor repairs. These hospitals were supposed to be upgraded

further using public resources.

(c) Subcomponent 3: Improving hospital management

17. General conditions. This subcomponent had three outputs: one was achieved, one partially

achieved, and one not achieved.

18. Many hospital managers and provincial authorities were trained in hospital

management programs by recognized institutions. The project has signed an agreement with

training units such as the School of Public Health, Bach Mai Hospital, and Hanoi Medical

University to organize management training for 477 hospital leaders and heads of departments in

provincial and district hospitals. The training contents were focused on human resources

management, health financial management, health-financing analysis, procurement and bidding

processes, and quality control of hospital performance. Considering that the planned goal was the

training of 303 hospital managers, the original target was exceeded by 57 percent. The trainees

evaluated the quality of the courses very positively. Survey results from in-depth interviews with

some hospital leaders showed that the management skills that were learned were effectively used

in the preparation of the district hospitals annual plans, plans for human resource development,

and other activities of the hospital. Overall, the project achieved its goals and the project funds

were used effectively.

36

19. Most of the district hospitals under the project developed and implemented

maintenance plans for facilities and equipment and increased the budgets to implement these

plans in 2013. The results of the 2014 survey revealed that, when compared with the early project

implementation years, the proportion of district hospitals having annual plans for maintenance

increased, but with some shortcomings by the end of the project. In December 2013, 99 percent of

the hospitals developed maintenance plans. However, at the end of 2014, only 80 percent of the

district hospitals kept these plans active. The executed budget for facilities and equipment

maintenance at the district hospitals achieved 87 percent of the total estimated costs in 2013,

compared to 57 percent in 2008.

20. Excellence award for human resources. The project planned to implement an excellence

award for human resources, but this initiative was the only activity that was not implemented, as

a project activity, during the project life. The CPMU did not collect information on this activity.

However, annually, the provincial health departments follow health units and individuals who

performed well. They are rewarded and apprised according to the Emulation and Reward

Regulation.


Beneficiaries

21. Component 2 was organized around three subcomponents: (a) support for direct

catastrophic and nonmedical expenditures of health care for Decision 139 beneficiaries; (b)

strengthening capacity for HCFP; and (c) strengthening local access to health services

through promoting health seeking behavior. The first subcomponent addressed the support for

nonmedical expenditures to poor and ethnic minorities’ beneficiaries. The second supported

institutional capacity building for district hospitals to identify target beneficiaries; to provide them

HICs and to institute mechanisms to receive medical expenses payments. The third subcomponent

developed IEC campaigns to increase awareness and knowledge from ethnic minorities about their

rights, entitlements, and benefits from HI. This component had four outputs and, at the project end,

three were achieved and one was partially achieved

(a) Subcomponent 1: Support for direct catastrophic and nonmedical expenditures of health care

for Decision 139 beneficiaries.

22. The number of project assistance recipients and the value of the monetary allowances

increased significantly during the project. In 2009, the project was committed to support

730,183 (only the poor) beneficiaries (or 25 percent of the population in the seven provinces), but

in 2013 this number jumped to 3,185,341 beneficiaries, given the inclusion of the ethnic minorities

(82 percent of the population in the same provinces). This support covered recurrent costs for travel

and meals for the population seeking health services at the district hospitals. From June 2009 to

August 2011, the project covered these expenses for the population under the poverty threshold.

Since September 2011, the project coverage was expanded for all ethnic minorities (including

those above the poverty threshold) because they are quasi-poor and represented the majority of the

NUP population. Until May 2012, the value of the allowance was VND 15,000 per day for

inpatient meals. After May 2012, the allowance value for inpatient meals increased to VND 25,000

per day. The allowance for travel varied according to the distance (VND 60,000 under 100 km and

37

VND 100,000 for 100 km or more). Table 2.6 shows the expansion of the coverage of the

nonmedical expenditures for poor and ethnic minorities from 2009 to 2013 in the seven NUP areas.

Table 2.6. Number of Targeted Beneficiaries of NUP, 2009–2013

Province

Number

of

Districts

with

NUP

Support

Persons in

2010 (below

the poverty

threshold)

Number of

Persons in

2011 (below

the poverty

threshold)

Number of

Persons since

August 2011

(poor + ethnic

minorities of

communes 30a)

Number of

Persons since

June 2012

(added from

HEMA Project)

Number

of

Persons

in 2013

Dien Bien 9 29,399 50,008 83,426 466,000 411,405

Lai Chau 7 50,279 78,916 133,933 300,000 263,390

Son La 9 157,653 103,054 339,126 700,000 700,000

Cao Bang 13 136,497 204,305 427,453 427,453 425,342

Bac Kan 8 73,748 87,949 282,154 282,154 263,390

Ha Giang 10 155,522 318,243 607,181 607,181 647,194

Lao Cai 8 127,085 313,736 424,145 424,145 471,082

Total 64 730,183 1,156,211 2,297,418 3,206,933 3,185,341 Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands Health Support Project,

Hanoi, 2014.

23. The number of inpatients’ coverage increased substantially, after the implementation

of the policy, extending the allowance for all ethnic minorities. According to table 2.7, in 2009

only 1,054 district hospitals’ inpatients were supported by the project, compared to 244,801 in

2013. The accumulated number of inpatients who benefited by the project from 2009 to 2013

reached 544,000, expending 86 percent of the resources planned for this activity in the project.

The household survey results at the project end line (2014) showed that 77 percent of the

respondents reported that they were supported on travel cost for inpatient treatment at the district

hospitals and 78 percent of those said that the NUP project supported meal costs for the poor/ethnic

minority people during their treatment at the district hospitals. The project contributed significantly

to promote the increased access to district hospitals for the poor and ethnic minorities in the seven

provinces.

Table 2.7. Total District Hospitals’ Inpatients Supported by Provinces, 2009–2014

Provinces 2009 2010 2011 2012 2013 First Three Months

of 2014

Dien Bien 411 1,779 3,814 16,889 30,058 7,524

Lai Chau — 1,264 4,205 13,605 18,132 4,716

Son La — 875 2,617 24,856 44,195 10,106

Cao Bang 186 4,853 18,129 44,248 45,044 10,533

Bac Can — 3,654 12,769 26,975 27,756 7,088

Ha Giang 30 3,457 17,727 38,781 39,702 11,231

Lao Cai 427 3,508 20,252 35,345 39,914 8,020

Total 1,054 19,390 79,469 200,709 244,801 59,218

Source: Center for Environment and Health Studies (2014) Final Report: End-line evaluation of the Northern Uplands

Health Support Project, Hanoi, 2014.

38

(b) Subcomponent 2: Strengthening capacity for HCFP

24. Given improvements and gains of scale and efficiency in the management of this

subcomponent, 100 percent of the objective was achieved with a substantial cost reduction.

Since May 2012, the management functions of the HCFP were extended to the HEMA Project

provinces, increasing the total number of HCFP beneficiary districts up to 64 districts compared

to the initial 50 districts. Two out of five of the HEMA Project provinces (Dien Bien and Lai Chau)

upgraded to manage HCFP were part of the NUP project. Therefore, the planned budget of this

subcomponent was reduced by 71 percent compared to the plan in the project document, (from

USD 872,282 to USD 255,925). On the other hand, simplifications to transfer the HCFP funds to

provinces with poor and ethnic minorities reduced the cost of the procedures that were expected

to be created at the provincial HCFP management units.

25. In March 2014, all activities of this subcomponent were completed and high efficiency

in strengthening the management capacity of the HCFP was achieved. These activities

(including training courses for the fund management unit, support of equipment, and monitoring

of the fund) were completed, disbursing USD 191,652.59 (75 percent of the planned budget after

adjustment).

(c) Subcomponent 3: Strengthening local access to health services through promoting health

seeking behavior

26. IEC-related activities, such as assessment needs, staff training, and provision of IEC

equipment paved the way to raise awareness and changes in behavior, thus promoting higher

utilization of health care services for the poor and ethnic minority people. From 2010 to 2014,

the project implemented several IEC activities such as (a) conducting an IEC needs assessment in

the project provinces as a basis to develop the IEC strategy and prepare IEC materials for

community and training for educators; (b) training 411 district and 9,070 commune staffs in the

seven provinces, for IEC activities; (c) providing essential IEC equipment, such as digital video

cameras, non-linear editing systems, loudspeakers, cameras, recorders, portable speakers, image

editing kits, and the so on to the provincial IEC centers established by the project; and (d) printing

and distributing materials to the provincial centers and district rooms to conduct IEC activities for

the community, according to the plan that was proposed by the PPMUs and approved by the CPMU.

27. The local health staffs and community reported that the IEC activities of the project

substantially reached out to the poor and ethnic minority people. According to the final report

of the CPMU, in 2013, 880 out of 899 IEC project communes in the provinces had IEC activities

implemented, reaching 98 percent of the original plan. The total number of local people benefiting

from the IEC activities of the project was 25,744.

28. The effectiveness of the IEC activities of the project was highly appreciated based on

the results of the household survey at the end line as compared to the end-line target of the

project. The end-line evaluation results show that most NUP inhabitants who had HICs could

name at least one right of the HICs (IOI #9). The rate of respondents who could name at least three

rights of the HICs increased significantly from 14.8 percent to 57.4 percent.

39

29. The implementation of the IEC subcomponent led to savings in the project. As of

March 31, 2014, disbursement of the subcomponent reached USD1.13 million (86 percent of

adjusted plan). However, by March 31, 2014, IEC activities continued to be implemented until

December 31, 2014, for an amount of USD 186,625.

B: Project Achievements during the 18-Month Project Extension Period (August 2014–

February 2016)

Component 1: Strengthening District-level Health Services

(a) Subcomponent 1: Human resources development

30. Long-term training: level-1 specialists. During the project extension, 57 level-1

specialists graduated before December 2015, bringing the total number of graduated doctors to

367. Ten additional level-1 specialists were expected to finish graduation after project closing.

Thus, by the end of the extension phase, there was no change in the number of level-1 specialists

supported by the project and an addition of 57 new graduated doctors. At the late extension phase,

nearly 97 percent of the trained level-1 specialists, who graduated to doctors, returned to their

former workplace: 57 percent of the level-1 trained doctors are ethnic minorities and 53 percent

are female. This reflects increased sustainability of the human resources supply in the provinces.

31. Long-term training: assistant doctors to become medical doctors. During the project’s

extended phase, 324 additional assistant doctors graduated as medical doctors using the project

funds, totaling 1,058 assistant doctors who transformed to medical doctors along the project. In

total, it represents 160 percent of the original plans’ targets. Additionally 40 pharmacists graduated

in the extension phase (see table 2.8).

Table 2.8. Number of Four-year Additional Doctors and Pharmacists who Graduated by the Evaluation Time

Health Professionals Trained Cao

Bang

Bac

Kan

Ha

Giang

Lao

Cai

Lai

Chau

Dien

Bien

Son

La Total

Support for four-year added doctors

Total doctors graduated (#) 131 53 213 145 142 192 182 1,058

Doctors graduated in regular project time (#) 104 42 130 106 91 142 119 734

Doctors graduated in project extension phase (#) 27 11 83 39 51 50 63 324

Doctors returned to work at their units (#) 122 44 208 96 136 193 179 978

Plan achievement (%) 147 144 192 179 128 179 148 160

Rate of ethnic minority doctors (%) 99 100 36 36 15 34 44 46

Rate of female doctors (%) 99 50 32 41 43 29 40 45

Support for four-year additional pharmacist training

Total pharmacists graduated (#) 13 7 15 23 13 16 11 98

Pharmacists graduated in regular project time (#) 6 6 9 10 6 13 8 58

Pharmacists graduated in project extension phase (#) 7 1 6 13 7 3 3 40

Rate of ethnic minority pharmacists (%) 92.3 71.4 20 26.1 7.7 6.0 18.2 30.6

Rate of female pharmacists (%) 86.4 71.4 60 82.6 69.2 50 72.7 70.4 Source: Center for Environment and Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The Extension

Phase, Hanoi, 2016.

32. Short-term training: curative care. The extended phase contributed to complete the

plan’s achievement for some important short-term trainings (emergency care and x-ray) and to

exceed the plan in other short-term trainings that were already accomplished in the regular project

40

time, adjusted according to the needs from district hospitals (anesthesia, and diagnose imaging).

During the extension phase, additional training was provided in areas that were not present in the

project’s regular time (ear, nose, throat and dental care, and nursing managing). The extension

phase added 18 percent additional short-term trainings in this specific activity. Therefore, at project

closing, the short-term trainings completed was 2.6 times the original plans’ goals, as seen in table

2.9.

Table 2.9. Number and Rate of Doctors Attending Short-term Training in the Extension Phase

Course Plan

Regular

Project

Time

Extension

Phase

Percentage

Added in

Extension

Phase

Total in

Project

Life

Percentage

Plan Increase

Anesthesia 117 135 22 19 157 134

Teasing 95 149 0 — 149 156

Emergency care 140 121 31 22 152 109

External medicine 101 143 0 — 143 142

Pediatrics 103 214 0 — 214 208

Internal medicine 111 194 0 — 194 175

Diagnostic imaging 200 404 32 16 436 218

Obstetrics 102 191 0 — 191 187

Communicable diseases 80 165 0 — 165 206

X-ray 107 103 18 17 121 113

Ear, nose, and throat, and dental 108 0 70 65 70 65

Nursing management 70 0 63 90 63 90

Pediatric emergency n.a. 681 0 — 681 —

Obstetric emergency n.a. 687 0 — 687 —

Total 1,334 3,187 236 18 3,423 257 Source: Center for Environment and Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The Extension

Phase, Hanoi, 2016.

33. Additional short-term training on MDG related activities in the project extension.

Along the regular project time, training for obstetric and pediatric emergencies was important to

prepare health staff to tackle maternal and neonatal mortality for ethnic minorities in the NUP

mountainous areas. During the project extension, the CPMU organized additional training to

certify health staff on appropriate interventions to increase the accessibility and utilization of

maternal and child health care services in villages, community health centers, and district hospitals.

The CPMU worked with the related stakeholders (Maternal and Child Health Department,

Provincial Departments of Health, and so on) to establish training and certification for skilled birth

attendants (SBA) (directed to general doctors and other health workers) and for specialists to work

on emergency in gynecological and obstetric care. Specific training was delivered to midwives at

the village level. As result, between August 2014 and December 2015, the following professionals

were trained and certified: (a) 794 health staffs working on obstetric and gynecologic services were

certified as SBAs; (b) 307 general doctors working on obstetric and gynecologic services at the

district/commune levels were certified as SBAs; (c) 110 medical doctors were trained on

emergency care for obstetric and gynecologic services at district level; (d) 252 village midwives

graduated; and (e) 79 clinical staff at neonatal units of the district hospitals, and other minor

trainings were provided according to the needs.

41

34. Short-term training: preventive care. Given that the training in this area was achieved

and exceeded during the project’s original phase, no additional training was provided during the

extension phase.

35. Short-term training: hospital management, medical waste management, HMIS, and

maintenance of medical equipment. During the regular project time, the goals for these four

kinds of short-term trainings were already achieved and surpassed. In the project extension time

(table 2.10), these short-term training courses were also delivered with 75 percent more staff over

the original plans (761 staff) trained. At the end of the project, the number of professionals trained

in hospital management was 2.5 times more than what was originally planned, focusing on areas

such as human resources management, health financial management, health financial analysis,

procurement management, bidding, and hospital quality management. In the area of medical waste

management, no additional training was delivered in the project extension phase, but it already

achieved 13.6 more times of staff training than planned. Regarding HMIS, the extension time

added 125 percent of trained staff. Regarding the maintenance of equipment, the achievement was

28 percent of additional trained staff. In summary, the extension phase provided additional trained

staff to the already achieved goals. The overall achievement of short-term training for health staff

was almost five times higher than planned.

Table 2.10. Health Staffs Attending Hospital Management, Health Systems Management, Information

Systems, and Maintenance of Medical Equipment Short-term Training in the Extension Phase

Course Plan

Regular

Project

Time

Extension

Phase

Percentage

Added in

Extension

Phase

Total in Project Life Percentage

Plan Increase

Hospital management 303 477 276 91 753 249

Medical waste management 215 2,922 0 — 2,922 1,359

HMIS 358 397 448 125 845 213

Maintenance of equipment 134 169 37 28 206 154

Total 1,010 3,965 761 75 4,726 468 Source: Center for Environment and Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The Extension

Phase, Hanoi, 2016.

(b) Subcomponent 2: Improving quality of district hospitals

36. Given that this activity was accomplished in the regular project time, few new

equipment was provided during the extension period to the district hospitals. Only new

equipment related to improving the quality of care for mother and children undergoing surgeries

and intensive care were acquired during that period. Equipment included the model of delivery

attendance instruction and neonatal resuscitation and accompanied appendixes; clean birth

delivery packages; instrument bags for village midwives; equipment for infant weight and length

measurement and kits for newborn resuscitation (including vacuum and suction pipe, metals

collection box, heating lamp, mask, and oxygen ventilation). Therefore, during the extension

period, the project’s new purchases were mainly focused on equipment for neonatal units and

village health teams, including instruments and consumable supplies for neonatal units. In total,

the project, with the extension phase, purchased 5,065 medical equipment; 14.7 percent more than

what was originally planned.

42

37. By the end of 2015, all equipment acquired by the project achieved high levels of

utilization in the district hospitals. Figure 2.1 shows the levels of utilization of almost 100

percent in three of the seven provinces. The only province where the level of utilization was lower

than 90 percent was Cao Bang, which was expected due to the small demand for hospital inpatient

facilities when compared with other project district hospitals. No new constructions and civil

works in the district hospitals were financed by the project during the extension phase.

Figure 2.1. Percentage of the Equipment Acquired by the Project that are in Use in the District Hospitals of

the Seven NUP Provinces.

Source: Center For Environment And Health Studies (2016) Final Evaluation Of The Northern Uplands Health Support Project In The

Extension Phase, Hanoi, 2016.

(c) Subcomponent 3: Improving hospital management

38. Short-term trainings on hospital management and maintenance plans for facilities

and equipment were kept as project priorities during the project extension phase for

sustainability purposes. However, the district hospitals and health provincial authorities at the

provincial level are still struggling with the perspectives of increasing budgets to build long-term

sustainability of project interventions. One of the lessons learned in this area is that policy

development is a process that requires a lot of time and involvement of ministries and provincial

people's committees in Vietnam. In this case, the project could be considered successful, given

that between 2014 and 2015 (a) the percentage of district hospitals with plans to maintain physical

infrastructure increased from 80 percent to 86 percent and the budget for these activities increased

by 11 percent and (b) the percentage of district hospitals with plans to maintain medical equipment

increased from 91 percent to 94 percent and the budget for these activities increased by 47 percent.


Beneficiaries

39. Three out of the four outputs of this component were achieved and one partially

achieved before the extension period. However, no additional information about the indicators

of this component has been provided during the extension phase, making it difficult to know if the

achievements remain sustained. There are indirect evidences (such as budget allocations)

confirming that the support to health care nonmedical expenditures for the poor, by providing

meals and transportation subsidies in the seven provinces, worked during the project extension

phase. However, IEC activities were not carried out during the extended period (only until

December 2015).

93.799.6

93.887.9

100 99.7 97.3 95.1

0

20

40

60

80

100

Son La Ha Giang Bac Kan Cao Bang Dien Bien Lai Chau Lao Cai Total

43

Annex 2A. Project Output Map

Subcomponent Related Activities Planned Outputs

Output Achieved

Before Extension

(October 2008–

August 2014)

Achievement as

Percentage of the

Planned Output

before Extension

Output Achieved

during Project

Extension Period

(February 2016)

Additional

Achievement as

Percentage of the

Planned Output

Total Percentage of

Achievement of the

Planned Output

during the Project

Life

Component 1: Strengthening District-level Health Services (USD42.9 million)

(a) Human

resources development

Long-term training

activities for health workers

Train 183 level-1

specialists

309 level-1

specialists trained 169

58 level-1 specialists

finished training*1 32 201 (Surpassed)

741 assistant doctors to become medical

doctors

734 assistant doctors became medical

doctors

99 324 additional assistant doctors became medical

doctors*2

44 143 (Surpassed)

No planned outputs for assistant

pharmacists

58 assistant pharmacists became

pharmacists —

40 additional assistant pharmacists became

pharmacists —

Complementary

Output

Short -term training activities for health

workers.

1,334 medical doctors to receive

short-term training in

medical techniques on curative care

3187 medical doctors received short term

training in medical

techniques on curative care

238

236 additional medical doctors received short-

term training in medical

techniques on curative care

18 256 (Surpassed)

No planned

additional short-term

training on MDG-related activities

—

—

1542 health staffs

received short-term

training on MDG-related activities*3

— Complementary

Output

213 health staffs to

receive short-term training in preventive

care

336 health staffs

received short-term training in preventive

care

157 — — 157 (Surpassed)

1,010 health staffs to receive short-term

training on hospital

management, HMIS, and maintenance of

medical equipment

3,965 health staffs received short-term

training on hospital

management, HMIS, and maintenance of

medical equipment

393

761 additional health staffs received short-

term training on

hospital management, HMIS, and maintenance

of medical equipment

75 468 (Surpassed)

Techniques/skills

transfer

Promote knowledge

sharing between provincial and

district level

facilities (402 times of technique

transfers at the

district hospitals)

Over 670 times of

technique transfers were implemented in

the district hospitals

at regular project time

167

The project gave

priority to transfer techniques/services for

emergency of obstetrics,

pediatrics, that district hospitals have not had

ability to do as required

by the MoH.

—

167 (Surpassed)

Transfers of techniques were

achieved establishing

sustainable mechanisms to

develop capacity in

service provision, especially at district

and commune levels

Innovative incentive schemes to retain

health workers

Study on the characteristics of

health workers

In 2009, the CPMU conducted a study on

‘Health Human


44


Output Achieved

Before Extension

(October 2008–

August 2014)

Achievement as

Percentage of the

Planned Output

before Extension

Output Achieved

during Project

Extension Period

(February 2016)

Additional

Achievement as

Percentage of the

Planned Output

Total Percentage of

Achievement of the

Planned Output

during the Project

Life

Phase I - Rapid labor

market assessment

currently working in

the NUP and job

characteristics in the NUP provinces

(salaries and

opportunities)

Resource Analysis in

7 Provinces of

Northern Upland’. The study was used

to plan human

resources health needs in the NUP and

establish training

goals and incentives

Innovative incentive

schemes to retain

health workers

Phase II - Developing

and implementing innovative incentive

schemes

Develop potential

incentive

schemes and study their likely impact on

recruitment and

retention

Backup arrangement

for health staffs when

being trained;

Hiring doctors who

are retired to work for the

hospitals/district

health centers.

— — —

This output was

partially achieved,

The provinces need to continue training

and developing

mechanisms and policies to support

retention of health

staffs at district level.

(b) Improving quality of

district hospitals

Basic medical equipment

Acquire and install 4,415 medical

equipment

4,821 medical equipment were

acquired and installed

109 244 additional medical equipment were

acquired and installed

6 115 (Surpassed)

Minor repairs and

upgrading

10 district hospitals

to have new

construction and

upgrades

18 district hospitals

had new construction

and upgrades 180 — — 180 (Surpassed)

(c) Imp1roving

hospital

management

Training of district

hospital management

staff

303 health staffs to

attend hospital

management training

477 health staffs had

attended hospital

management training 157

276 additional health

staffs had attended

hospital management training

91 248 (Surpassed)

Developing hospital

maintenance plans

All district hospitals

to have infrastructure

and equipment maintenance plans

80% of the district

hospitals had

maintenance plans in December 2014

80

86% of the district

hospitals had

maintenance plans in December 2016

86 86 (substantially

achieved)

Management

excellence award program

The manager

excellence award program does not

appear in the

program execution documents*4

— — — — Not achieved

Component 2: Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries (USD10.0 million)

(a) Support for

direct

catastrophic and

nonmedical

Study of distribution

and patterns of catastrophic

expenditures in the

first year

Develop a survey in

the first project year to establish a

baseline for the KPI

The survey was done

in the first year of project execution and

the baseline for

Achieved

Two other similar

surveys were developed in the MTR and in the

original project closing

to follow up the project

Achieved Achieved*5

45


Output Achieved

Before Extension

(October 2008–

August 2014)

Achievement as

Percentage of the

Planned Output

before Extension

Output Achieved

during Project

Extension Period

(February 2016)

Additional

Achievement as

Percentage of the

Planned Output

Total Percentage of

Achievement of the

Planned Output

during the Project

Life

expenditures of

health care for

Decision 139 beneficiaries

and IOI related to the

project Component 2

Component 2 KPI

and IOI set

Component 2 KPI- and

IOI-related targets.

730,183 beneficiaries (25% of the

population in the

seven provinces) were targeted to be

supported according

to the 2009 survey

3,185,341 beneficiaries (82% of

the seven provinces

population) were supported in

December 2013

according to the end project survey

Achieved — — Achieved*6

(b) Strengthening capacity for

HCFP

The HCFP capacity is measured by their

capacity, at the provincial level, to provide the

support for transportation and food allowance.

All activities were

completed with a

disbursement rate of 75% of the budget

schedule after the

adjustment. Achieved

At the time of the

extension-phase project

evaluation, three provinces (Dien Bien,

Bac Kan and Cao Bang)

have not yet established the health care for the

poor fund because they

could not balance the funds for this activity.

— Partially achieved

(c) Strengthening

local access to

health services

through promoting

health seeking

behavior

IEC activities, such as

(a) searching for

assessment needs, (b)

training staff in IEC

and, (c) providing IEC equipment

899 communities in

the seven provinces

to receive IEC

activities

880 communities in

the seven provinces

received IEC

activities addressing

25,744 local people in December 2013.

98

No IEC activities were

identified as performed

in the project extension

phase. — Achieved*7

Component 3: Monitoring, Evaluation, and Project Management (USD13.1 million)

No subcomponent

has been identified

Strengthen procurement, financial

management, and disbursement

The CPMU

contributed to increase local

capacity (PPMUs) for

fiduciary procedures. Some delays in

procurement of

medical equipment

were identified. At

the end of the first

phase, disbursement was 88% of project

loan.

Partially achieved

The project

management was effective, focused on

promoting the project

results, and enhanced the sustainability. The

procedures and

processes of project

implementation were

relatively clear and no

significant gaps were identified.

— Partially achieved*8

Training of project management staff The CPMU, with

World Bank support, provided training for

the project

Achieved

In the extension phase

the PPMUs reduced staff and increased

rotation creating some

Partially achieved Partially achieved

46


Output Achieved

Before Extension

(October 2008–

August 2014)

Achievement as

Percentage of the

Planned Output

before Extension

Output Achieved

during Project

Extension Period

(February 2016)

Additional

Achievement as

Percentage of the

Planned Output

Total Percentage of

Achievement of the

Planned Output

during the Project

Life

management staff at

the PPMUs

gaps on trained staff,

especially for M&E

purposes

Provision of necessary office equipment Office equipment

was provided to the

CPMU. However, some PPMUs were

difficult to be fairly

equipped.

Achieved

Office equipment was

provided to the CPMU.

However, some PPMUs had difficult to be fairly

equipped.

Achieved Achieved

Financing of incremental operating costs The project activities were financed

properly. Achieved

Some PPMUs had constraints to finance

operational costs during

the extension phase.

Partially achieved Partially achieved

M&E activities Baseline data

collection

As scheduled, this

was completed in the

first semester of 2009

Achieved — — Achieved

Indicators’ update Indicators’ update was done based on

surveys (2009, 2012,

and 2014) and project administrative

records

Achieved

Some indicators were not updated after the

project extension Partially achieved Partially achieved

MTR

MTR was done in 2012

Achieved — — Achieved

End-of-project

completion reports

The first end-of-

project completion report was done in

July 2014 (regular

project time)

Achieved

The second end of

project completion report was done for the

extension phase in

February 2016 (project extension phase)

Achieved Achieved

Audits Project financial

audits were done for

procurement and financial

management with

some delays*9

Partially achieved — — Substantially

achieved

Source: Center for Environment and Health Studies (2014) FINAL REPORT: End-line evaluation of the Northern Uplands Health Support Project, Hanoi, 2014, and Center for Environment and Health

Studies (2016) FINAL EVALUATION OF THE NORTHERN UPLANDS HEALTH SUPPORT PROJECT IN THE EXTENSION PHASE, Hanoi, 2016.

(*1) After the project closing in February 2016, there were 10 remaining Level 1 medical doctors expecting to be graduated.

(*2) After the project closing in February 2016 there were 409 remaining assistant doctors having training to be graduated medical doctors. So, the total assistant doctors supported by the project were

1467. (*3) Training was offered at provincial, district and village level according the specialty.

(*4) The Government had their own long standing system and mechanism for awarding well-performing hospitals. They did it every year as a routine activity. However this did not appear as a project

related activity and the results of this award mechanism were not reported to the Bank in the project documents.

47

(*5) The last survey was related with the original project closing. During the extension project phase a new survey was not planned given that the extension was not planned to follow-up the Project Component 2.

(*6) This achievement is related to the project regular time. The increase of the number of beneficiaries entitled to receive benefits is related to the inclusion of all ethnic minorities as project

beneficiaries by May 2011. In the first three months of 2014 the district hospitals under the project provided subsidies to 59,182 beneficiaries. From 2009 to 2013, the number of ethnic minorities’ inpatients receiving subsidies increased from 1054 to 244,181 totalizing more than 545 thousand inpatient along this period.

(*7) Despite the 98% of the target achievement compared to the plan in the regular project time, the Project Evaluation Report of the Extension phase recommend strengthening of the IEC activities

under the project. (*8) The implementation progress of some training courses and provision of medical equipment were slightly slow according the project registers.

(*9) The delays in the project financial audits were registered in the beginning of the project. In the last 3 years of execution and during the project extension, the audit reports were submitted on time.

48

Annex 3. Economic Analysis: Output Efficiency, Benefits, and Equity Impacts

Introduction

1. This annex addresses the project’s efficiency by analyzing (a) the rationale of the

Government interventions in the project and in the Northern Upland Provinces; (b) the

efficiency in achieving quality access for the district hospitals by the NUP population (project

outputs); (c) the project contribution to improve health benefits (project outcomes); and (d)

the impact of the project interventions on the equity on health access and health spending in

benefit of the poor and ethnic minorities.38 According to the project economic analysis (annex

9 of the PAD), the expected benefits were associated with health outcome improvements for the

NUP areas’ population and to narrow the health spending gap between the poor and the average

population. The benefits would be achieved by the following main interventions: (a) improve

efficiency on the health services delivery by increasing supply of skilled human resources and

refurbishing and equipping the district hospitals; (b) improve efficiency for the poor on assessing

health services by removing financial barriers to increase health care services utilization; and (c)

reduce the risk of impoverishment for the NUP poor and ethnic minorities populations.

Rationale of the Government Interventions

2. As stated in the PAD, the Northern Upland Provinces constitute the most

disadvantaged region in Vietnam, with an unusually high concentration of poor and ethnic

minorities living in sparsely populated, mountainous localities under difficult circumstances.

The Government’s choice to invest in this region was based on the unfavorable socioeconomic

conditions and the difficult access and generally poor quality of the health services, especially at

the district health hospitals. The Government seeks to improve the health status of the population

and reduce the health gap between the NUP and the rest of the Vietnamese population. The project

achieved this by improving the efficiency on delivering health outputs, reducing maternal mortality

in higher proportion than the country’s average, and reducing the equity gap in out-of-pocket

health spending between the poorest income quintile and the richest income quintile.

Efficiency on Achieving the Project Outputs

3. The project surpassed all relevant planned output targets increasing efficiency by

reducing unitary costs. Annex 3A shows the project’s planned and achieved output targets and

38 According the PAD, a quantitative economic analysis based on costs was not feasible for the project because there

was no empirical basis for estimating the project’s health outcome costs. Neither an economic rate of return nor a

net present value of the benefits of the project was calculated/forecasted upfront during the project appraisal. The

PAD highlighted the difficulty to estimate costs of the proposed interventions. Given the special conditions to

implement project activities in the NUP areas, the economic costs may differ significantly from their financial costs

or budget expenditures. For example, under Component 1, the opportunity cost of the personnel sent for training is a

significant economic cost of the project that may not be completely reflected in the project budget. It has been the

experience of other projects (for example, the ADB-supported Health Care in the Central Highlands Project) that

sending large numbers of personnel for training from facilities that are already understaffed imposed a serious

budgetary burden on the public health system. Another example is associated with the reimbursement of the travel

and food costs of poor hospital inpatients, which is an income transfer, not an economic cost. Economic costs in this

case would be limited to the cost of administering the transfers and the cost of any additional health care utilization

that might result from this support.

49

the correspondent budget allocations. These outputs, related to training human resources and

equipping, repairing, and upgrading infrastructure in the district hospitals, had targets that were

surpassed during the project execution with simultaneous reductions in the costs, thus increasing

savings that allowed the project to execute additional activities during the extension phase. For

example, the implicit unitary cost for the level-1 specialists and the transformation of assistant

doctors to medical doctors were, at the end of the project, 50 percent and 37 percent lower than

the original plans, while the number of individuals trained as level-1 specialists and medical

doctors were 101 percent and 50 percent higher than planned.

Analysis of the Contribution to the Project Outcomes

a) Benefits associated with reducing mortality

4. The analysis of the project interventions and PDO achievements (annexes 2 and 5 of

this ICR) evidenced improvements in the access and utilization of health services in the NUP

areas. This contributed to the reduction of maternal and child mortality rates, which are two major

problems in the burden of diseases of the NUP areas. The project interventions were intended to

improve the quality and efficiency in delivering prenatal care, birth delivery, and childcare in the

first 12 months of life. So, the reduction of maternal and child mortality rates along the project

implementation should be relevant benefits reflecting improvements in the health status of the

NUP areas’ population.39

5. Current literature evidences that the project used the right interventions to reduce

maternal and child mortality rates. Most maternal and newborn deaths can be prevented using

existing, proven, cost-effective interventions, such as clean delivery packages, composed of

antibiotics, sterile blades for cutting umbilical cords, drugs that prevent and treat postpartum

hemorrhage, resuscitation, immediate and exclusive breastfeeding, and education and

communication to the mother to keep the newborn warm with skin-to-skin contact and

breastfeeding.40 Increasing access to mothers to deliver their babies at first-level facilities and/or

by SBAs was the focus of the project, providing an opportunity to expand quality services around

the time of birth. All these initiatives were used by the project interventions, providing a solid

ground to start a process to reduce maternal, neonatal, and infant mortality in the northern

mountainous provinces in Vietnam.

6. As can be seen, between 2007 and 2014, the maternal mortality reduction was

remarkable in the NUP areas when compared with the average reduction verified in the

country, but it is still very high in the region compared with the national benchmark and

international standards. During the project regular time implementation, MMR in the NUP areas

reduced 40 percent compared with 7 percent in the whole country. However, MMR in the seven

NUP provinces is still nearly two times higher than the country average, as seen in table 3.1.

39 However, given that both—maternal and infant mortality—are affected by many factors, such as water and

sanitation conditions, nutrition patterns, and others, it is difficult to attribute the results (for good or for bad) only to

the health interventions. 40 Bill and Melinda Gates Foundation. 2016. Maternal, Newborn and Child Health Strategy Overview.

http://www.gatesfoundation.org/What-We-Do/Global-Development/Maternal-Newborn-and-Child-Health


50

Table 3.1. Maternal, Neonatal, and IMR Vietnam and the NUP: 2007–2014

Years MMR

(per 100,000 born alive)

NMR


IMR


Vietnam NUP Vietnam NUP Vietnam NUP

2007 58 178 13.0 11.2 21 31.1

2014 54 106 12.0 10.8 18 29.4

Percent of

reduction

−6.9 −40.4 −7.7 −3.6 −14.3 −5.4

Source: Vietnam: World Bank Data (http://data.worldbank.org/indicator/SP.DYN.IMRT.IN?locations=VN;

http://data.worldbank.org/indicator/SH.STA.MMRT?locations=VN;

http://data.worldbank.org/indicator/SH.DYN.NMRT?locations=VN). Government of Vietnam: NUP Project

Surveys 2009 and 2014.

7. The project opened the floor to continue achieving progress on reducing MMR in the

region and the relevance to keep it sustainable along the time. A study on maternal and neonatal

mortality financed by the CPMU as part of the project closing evaluation activities found that in

2014, the still high MMR in the NUP areas is related to: (a) the high proportion of home birth

deliveries, representing 38 percent of the total birth deliveries in the NUP41; (b) delays in detection

and decision to seek care by women and their families; (c) delays in reaching care, mainly due to

difficult geographic conditions and lack of transportation means; and (d) delays in receiving health

care and appropriate treatment at the health facilities, mainly due to shortcomings in compliance

with the process of care and monitoring of pregnant mothers, especially during and after delivery.

The reduction registered in the MMR between 2007 and 2014 was not uniform among the seven

provinces and in one of them (Bac Kan) the MMR increased during the project implementation

(figure 3.1).

Figure 3.1. MMR in Six Northern Upland Provinces

Source: Ministry of Health of Vietnam: Survey on Maternal and Neonatal Mortality in the Northern Upland Provinces: 2007-2008 and

2013-2014.

8. Though there is no information related to MMR in the 2015 report, the project

extension phase provided training for midwives to reduce maternal mortality at home, thus

providing safer birth delivery. The administrative registration of the NUP health services showed

41 According to the study, the risk of maternal mortality in a home delivery is 3.7 times higher than when deliveries

are performed at health facilities.

409

215

167143 133

46

178

83

150

104 95

128

75

106

0

50

100

150

200

250

300

350

400

450

Dien Bien Lai Chau Son La Cao Bang Lao Cai Bac Kan Total

2007-2008

2013-2014

http://data.worldbank.org/indicator/SP.DYN.IMRT.IN?locations=VN

http://data.worldbank.org/indicator/SH.STA.MMRT?locations=VN

http://data.worldbank.org/indicator/SH.DYN.NMRT?locations=VN

51

that between 2014 and 2015, the proportion of mothers giving birth at home supported by village

midwives increased from 6.4 percent to 16.5 percent and the proportion of clean delivery packages

provided to the mothers giving birth at home increased from 34 percent to 43.1 percent in the same

period respectively. The continued training of village midwives and policies aimed at maintaining

a network of village midwives show that the possibilities to increase the project benefits in the

extension phase were very high, given that these solutions are crucial for safe motherhood in the

mountainous areas.

9. The reduction of neonatal and infant mortality in the NUP areas was lesser than the

country average. Table 3.1 shows that IMR and NMR reduced by 5.4 percent and 3.6 percent in

the NUP between 2007 and 2014, compared with 14.3 percent and 7.7 percent in the national

average, respectively. These results were also influenced by the high proportion of home birth

deliveries in the NUP areas. Neonatal mortality among newborns delivered at home was 14.2 per

1,000 live births, almost twice as much as in health facilities, according to the end-of-project

survey.

10. Figures 3.2 and 3.3 show that the NMR and IMR increased in some of the provinces

along the project execution period, despite the decreasing average. In fact, in the case of NMR,

only Bac Kan and Son La presented decreases in NMRs and IMRs. In the case of IMRs, decreases

were verified in Ha Giang, Cao Bang, Son La, and Bac Kan. Given that most of the data is from

administrative registrations, maybe the reporting system in some provinces, such as Lai Chau and

Dien Bien, had improved sharply during (and because) the project implementation, increasing the

confidence in the administrative records of NMR and IMR in 2014, compared with 2008, when

the mortality data were not captured well.

Figure 3.2. NMR in Six Northern Uplands Provinces

Source: Ministry of Health of Vietnam: Survey on Maternal and Neonatal Mortality in the Northern Upland Provinces: 2007-2008 and

2013-2014.

11.5

13.8

12.3

9.1

7.4

11.711.2

17.1

14

12.812.1

5.6 5.9

10.8

0

4

8

12

16

20

Lai Chau Lao Cai Cao Bang Dien Bien Bac Kan Son La Total

2007-2008

2013-2014

52

Figure 3.3. IMR in Six Northern Uplands Provinces

Source: Ministry of Health of Vietnam: Health Statistical Yearbooks 2008 and 2014.

11. The disadvantaged situation of the MDGs in the NUP areas, revealed during the

project implementation by the improvement of the medical records, lead the Government to

focus the project extension on reducing the MMR and IMR, striving to achieve the MDGs on

health, by providing clean birth delivery packages, supporting basic tools for newborn care,

equipment for neonatal units at the district hospitals, and related training. Despite the outputs

related to these tasks being positively recorded (see annex 2), their impact on the reduction of IMR

and NMR after the project extension was unknown at the time this ICR was prepared.

b) Benefits associated to improved supply and quality of health services

12. The project interventions (health staff training, management improvement, and

investment in infrastructure, equipment, and maintenance) increased the availability and

quality of the essential health services delivery in the district hospitals benefiting the

population in the NUP areas. The number of inpatient and outpatient visits per capita delivered

in the district hospitals increased by 290 percent and 270 percent, between 2009 and 2015,

respectively, according to the project administrative records.

13. However, more important than the number and coverage of the services delivered is

the quality of these services. Table 3.2 shows indicators associated with the benefits provided by

the project in improving the quality and capacity to deliver health services to mother and children

by district hospitals. The number of health services that district hospitals can according to the

national norms increased by 105%. Hospitals that have equipment needed for quality procedures,

such as positive airway pressure machines, phototherapy for newborn machines, oxygen breathing

systems and newborn resuscitators increased remarkably. High improvements were achieved by

following the national quality norms, enhancing diagnosis capacity, and having skilled personnel

and equipment to implement appropriate techniques for regular and emergency care services.

40 40

33 33

28

2321

31.1

34.2

24.3

34.4

42.6

23.2

29.2

17.9

29.4

0

10

20

30

40

50

Ha Giang Cao Bang Dien Bien Lai Chau Son La Lao Cai Bac Kan Total

2008

2014

53

Table 3.2. Quality Improvements of the Services Delivered by the District Hospitals in the NUP Areas:

Project Baseline (2009), Regular Closing Date (2014), and Extended Closing Date (2015)

Health Services Quality Indicator

Project

Baseline

(%)

Project

Regular

Closing

Date

(%)

Project

Extension

Closing Date

(%)

Quality

Improvements

at Extension

Closing Date

(%)

Percentage of health services that district

hospital can provide according to the national

norms

39 71 80 105

Percentage of medical records with appropriate

diagnosis of acute respiratory infections 18 39 62 244


diagnosis of acute diarrhea 41 69 82 200


diagnosis of acute poisoning 61 86 89 46

Percentage of district hospitals equipped with

skilled staff to implement appropriate pediatric

techniques

69 75 85 23

Percentage of district hospitals equipped with

skilled staff to implement appropriate obstetric

techniques

65 84 88 35

Percentage of district hospitals having capacity

for caesarean section. 86 89 92 7

Percentage of district hospitals having capacity

for blood transfusion 64 75 83 137

Percentage of district hospitals having breathing

machines with continuous positive airway

pressure

25 75 80 220

Percentage of district hospitals having light for

jaundice phototherapy treatment 21 82 88 219

Percentage of district hospitals having oxygen

breathing systems 43 75 78 81

Percentage of district hospitals having newborn

resuscitators 46 93 92 100

Source: Center for Environment and Health Studies (2016), Final Evaluation of the Northern Uplands Health Support Project in The Extension

Phase, Hanoi, 2016.

c) Benefits associated with increased HI coverage and health services demand to prevent

catastrophic expenditures for the poor and ethnic minorities

14. The project increased not only the coverage, but also the demand for health services

utilization by providing transportation and meals subsidies to the poor and ethnic minorities.

Because of adjustments, the number of project beneficiaries increased significantly. From 2009 to

2011, only the poor were covered by the project, but given that the majority of the ethnic minorities

were quasi-poor, they were included as beneficiaries of the subsidies since 2012. The number of

project recipients, were 730,183 persons in 2009 (25 percent of the NUP population) when

compared with 3,185,341 in 2013 (82 percent of the NUP population). Since 2011, the project

implemented a communication campaign (IEC) to increase the beneficiaries’ awareness about HI

rights and the processes to achieve their subsidies. The number of beneficiaries knowing at least

three HI rights increased substantially.

54

15. The number of beneficiaries receiving direct monetary subsidies for transportation,

meals, and health expenditures, increased several times along the project implementation, as

can be seen in figure 3.5.42 They were only 1,000 in 2009 but increased to almost 245,000 in 2013.

From the project start in 2009 to January 2014, the number of poor/ethnic minority inpatient who

received support to use medical services from the project totaled 545,423, with registered

expenditures of USD7,398,693.69 (around USD13.56 per inpatient beneficiary). As of January

2014, 86 percent of the budget for this activity has been spent, but the sustainability of these

subsidies was granted during the project extension and hopefully beyond, according the CPMU

information.

Figure 3.4. Number of Poor/ethnic Minorities Benefited by Inpatient Heath Subsidies (transportation, meals,

health expenditures) by the NUP Project 2009–2013

16. The subsidies paid to increase coverage and access to health services were crucial to

reduce out-of-pocket health-related spending, prevent catastrophic expenditures, and

increase the coverage of HI in the NUP areas. According to the CPMU sponsored surveys,

between 2009 and 2014, the percentage of households which experienced catastrophic health care

expenditures in the NUP had a remarkable reduction: from 15 percent to 2 percent, respectively.

The percentage of population living in these areas with HICs increased from 82 percent to 95

percent between June 2009 to December 2015, mostly among the poor and ethnic minorities

populations.

Analysis of the Project Impact on Equity Pro-poor and Ethnic Minorities

(a) Methodological considerations

17. The equity analysis of the project interventions will consider the impact of the project,

mostly from 2012 to 2014. During 2009 and 2011, most of the efforts of the NUP Project were

concentrated on planning and implementing the process to provide training for health staff,

42 The project’s 2009 Baseline Survey showed that, of the inpatients in the NUP areas who visited the district

hospitals in 2008 to assess health care, 96 percent paid for food, 88 percent paid for transportation, and 5 percent

paid for other costs, such as medication, exams, tests, and so on. The inpatient average daily cost paid by the

families was VND 122,224 (USD 7.46 at the 2008 exchange rate) and the average stance was 4.4 days, representing

an average cost of USD 32.80 per inpatient. To finance these costs, 33 percent of the families got loans, 22 percent

asked for family support, and 5 percent sold assets or means of production.

1054 1939079469

200709

244801

2009 2010 2011 2012 2013

Number of Beneficiaries

55

procurement for investments in infrastructure and equipment of the district hospitals, and the

institutional arrangements to finance incentives to the poor and ethnic minorities to increase the

access of health services and utilization of district hospitals. From 2012 to 2015, the project was

able to measure the impact of these investments for improving the equity on accessing the project

benefits, especially to the poor and ethnic minorities.

18. This equity analysis will consider the progress on equity indicators related to (a)

affiliation to HI; (b) household income spent on health; and (c) household income spent on

inpatient visits. It is based on the data collected through the VHLSS of the respective years. Equity

impact will be calculated on variables such as gender and age of the beneficiaries, ethnicity,

residency status (rural or urban), and income (poorest, near poor, middle, near richest, and richest

income quintiles).

19. Main hypotheses. The main hypotheses used in this analysis are the following: (a) the

project improved the equity on accessing inpatient visits, disproportionally benefiting women and

children over five years; (b) the project reduced the relative spending of the poor and ethnic

minorities to access inpatient visits; (c) the project increased the number of inpatient visits of the

poor in the district hospitals, increasing the equity of the access to hospital services43; (d) the

project increased the proportion of the poor and ethnic minorities having HI; and (e) the project

reduced the participation of health spending in the poor families’ out-of-pocket expenses,

increasing the equity of the health spending.

(b) Equity impact on inpatient medical visits for women, children, and rural population

20. Given that the project was focused on achieving the health MDGs, it was expected

that the proportion of inpatient visits related to mother and children would increase faster

than for other groups, increasing the gender and age equity on the access to the district

hospitals. The equity impact is measured by the equity ratio.44 As shown in table 3.3, between

2010 and 2014, women inpatient visits increased 32 percent as a proportion of men inpatient visits,

and children inpatient visits increased 181 percent as a proportion of total inpatient visits at the

district hospitals. Rural population inpatient visits also increased 19 percent as a proportion of

urban medical visits during the same period. So equity in accessing inpatient services at district

hospitals was improved for women (because women at a reproductive age always need more

services than men because of reproductive health issues), for children until they are five years old

(where the associated mortality risks are higher), and for the rural population, who were previously

underserved by the health system.

43 Considering the nature of the Project investments, the impact on health services utilization in equity is more

sensitive to the inpatient than to outpatient visits to the district hospitals. The Project influenced the Provincial

Departments of Health to functionalize the health referral process, according the levels of complexity in the health

care provision, inducing the population to do not use district hospitals for unnecessary outpatient services for that

level of complexity. So, many outpatient visits that in the past went to district hospitals (especially among the center

village populations) were redirect to Community Health Centers. So this analysis is pertinent only for inpatient

services. 44 The equity ratio for inpatient visits is defined as the proportion of the visits per capita in the category where the

number of visits per capita is expected to increase disproportionally as a proportion of the complement (for example,

women compared to men) or the total visits per capita (poor compared to the total population). If this ratio has

positive variations, the equity impact was achieved, but if the ratio has negative variation, the equity was reduced.

56

Table 3.3. Equity Gap on Annual Per Capita Inpatient Visits at the District Hospitals

NUP Provinces: 2010–2014

Annual Per Capita inpatient Visits:

Equity Ratios for Different

Population Groups

2010 2012 2014

Increase

2010–2012

(%)

Increase

2012–2014

(%)

Increase

(2010-2014)

(%)

Gender Equity Ratio

Women visits/men visits 1.14 1.46 1.50 28 3 32

Child Visits Equity Ratio

0–5 years old visits/average visits 1.00 1.13 2.81 13 149 181

Place of Residency Equity Ratio

Rural visits/urban visits 0.94 0.78 1.12 −17 44 19 Source: VHLSS, 2010, 2012, and 2014.

(c)Equity on the impatient visits to the district hospitals benefiting the poor and near poor

21. The project did not collect data for the near poor, because its focus was the poor and

ethnic minorities. However, the information of VHLSS use the classification of poor and near

poor in two ways: (a) The global poverty line (the World Bank criteria); and (b) by income quintile,

which means that the poorest should be considered the first quintile and the near poorest are the

second quintile. Table 3.4 shows the impact of the project in the inpatient visits to the district

hospitals by poverty status and income quintiles.

Table 3.4. Inpatient Visits to the District Hospitals According Poverty Status and Income Quintiles

Poverty Status and

Income Quintiles

Inpatient Visits to the District Hospitals per inhabitant

2012 2014 Increase (2012-2014)

(%)

Poverty Status (according World Bank Global Poverty Line)

Poor 0.037 0.054 45.9

Near-Poor 0.088 0.070 -20.5

Non-Poor 0.056 0.075 33.9

Income Quintiles

Poorest 0.028 0.044 57.1

Near-Poorest 0.034 0.061 79.4

Middle 0.048 0.063 31.3

Near-Richest 0.083 0.093 12.0

Richest 0.047 0.053 12.8

Average 0.048 0.063 31.2 Source: VHLSS, 2012 and 2014.

22. Table 3.4 shows that the inpatient visits in the district hospitals, between 2012 and

2014, had a higher increase for the poor than for the average population and the non-poor.

This is a strong evidence of the positive impact of the Project in the equity of the health services

utilization at the district hospitals level. The inpatient visits per inhabitant increased in average

31% compared with 57% in the poorest quintile and 46% among the poor. Regarding the near poor,

the income quintile approach shows an increase of 79% in the near poorest quintile. However,

using the global poverty line approach, the data shows a reduction of 20% of the number of

57

inpatient visits between 2012 and 2014, which could be associated with methodological

differences in the way that the information associated with each year was captured45.

(d) Equity on the relative spending of the poor and ethnic minorities to access inpatient visits

23. Another way to verify the equity impact is the proportion of the out-of-pocket

expenditures of mothers and children and the poor and ethnic minorities on inpatient visits

to the district hospitals compared to other groups, along the project implementation. The

VHLSS data shows that the spending per inpatient visit at the district hospitals as a share of project

target groups was relatively reduced. As can be seen in table 3.5, the equity ratio for the inpatient

visit spending improved for women compared to men; for children compared to the average

population; for ethnic minorities compared to the Kin/Hoa ethnicity; for rural populations

compared to urban, and especially for the poorest quintile compared with the average population.

The relative reduction of the out-of-pocket spending with inpatient visits verified for the poor was

supported by the project subsidies for meals, transportation, and medical expenses, the last covered

by the HI, during the project implementation.

Table 3.5. Equity Gap in Out-of-Pocket Spending for Inpatient Visits to District Hospitals NUP Provinces:

2010–2014

Out-of-Pocket Spending per

Inpatient Visit: Equity Ratios for

Different Population Groups

2010 2012 2014

Increase

2010–2012

(%)

Increase

2012–2014

(%)

Increase

(2010–2014)

(%)

Gender Equity Ratio

Women/men spending 1.01 0.89 0.91 −12 2 −10

Child Visits Equity Ratio

0-5 years old/average spending 0.72 0.79 0.53 10 -33 −26

Ethnic Minorities Equity Ratio

Ethnic minorities/Kin-Hoa spending 1.19 0.56 1.13 −53 102 −5

Place of Residency Equity Ratio

Rural/urban spending 1.34 1.89 1.11 41 −41 −17

Income Equity Ratio

Poorest quintile/average spending 1.04 0.69 0.60 −34 −13 −42 Source: VHLSS, 2010, 2012, and 2014.

(e) Equity impact of the project in increasing HI coverage

24. The VHLSS data also shows relevant impacts in the coverage of HI during the project

implementation time, especially for the poor. The total NUP population without HI, decreased

slightly between 2010 and 2014 (from 7.4 percent to 7.2 percent, respectively) and the HI coverage

for the poor improved. The percentage of the poorest economic quintile affiliated to the HI

increased from 98.6 percent to 99.2 percent between 2010 and 2014 and for the near-poor (second

poorest quintile) the HI coverage increased from 93.5 percent to 98.9 percent, according the

VHLSS data. As part of the affiliation to the HI mechanisms for the poor, the project spent USD

8.5 million in subsidies for the poor and ethnic minorities, contributing to pay for transportation

45 The data for 2012 used the international poverty line (in purchasing parity power –PPP) of and income of USD

1.25 a day. However, this line was upgraded to US$ 1.90 in the 2014 data analysis, creating difficulties to compare

both years using the World Bank global poverty line criteria.

58

and meals for 596,700 poor and ethnic minorities’ inhabitants of the NUP areas, with an average

expenditure of USD 14.27 per medical visit.

(f) Equity impact on reducing the share of out-of-pocket expense in health for the poor and

ethnic minorities

25. During project implementation, the poorest quintile reduced the health spending as

a share of the out-of-pocket expense in the NUP areas. From 2010 to 2014, the share of out-of-

pocket health spending of the poorest quintile was reduced slightly from 6.7 percent to 6.5 percent,

while the average out-of-pocket family spending on health increased from 7.9 percent to 8.3

percent in the NUP areas. Probably health subsidies to the poor and ethnic minorities for visiting

district hospitals and improvements in the health care assistance at the villages, sustained partially

by the project, had positively affected the family budgets, contributing to avoiding the risk of

catastrophic health expenditures for these populations. That is one of the reasons why the

proportion of the NUP families with catastrophic health expenditures reduced from 10.4 percent

to 2.0 percent during the project execution, as is demonstrated by the project KPI #3.

Final Considerations

26. The present economic analysis intends to demonstrate the following: (a) the project

was efficient in delivering its outputs, reducing unitary costs for training, equipment installation,

and civil works during implementation, compared with the original implicit costs. This allowed,

savings from the original implementation time to be used during the project extension in new

activities related to improving the MDGs in the project area and (b) efficient project interventions

and subsidies to the poor and ethnic minorities contributed to reducing maternal, neonatal, and

infant mortality and improved the equity in assessing health care and reducing health spending for

the poor and ethnic minorities.

59

Annex 3A. Estimated Unitary Costs of the Project Outputs According to the Original Plan (2009) and Closing Implementation

(2016)

Component and Output

Project Outputs Budget (USD, thousands) Estimated Unitary Cost (USD)

Planned Implemented Variation

(%) Planned Implemented

Variation

(%) Planned Implemented

Variation

(%)

Trained Level-1 specialists 183 367 101 691.7 687.8 −1 3,779.78 1,874.11 −50

Assistant doctors trained as

doctors*1 741 1,156 56 6,236.0 6,085.6 −2 8,415.65 5,264.36 −37

Short-term-doctors trained in

curative care techniques*2 1,334 4,965 272 2,409.3 1842.3 −24 1,806.07 371.00 −79

Short-term-doctors trained in

preventive care 213 336 58 211.0 200.1 −5 990.61 595.53 −40

Skills techniques transfer to

district hospitals*3 402 670 67 365.6 296.9 −19 909.45 443.13 −51

Basic medical equipment

(number of units installed) 4415 5165 17 26,698.5 25,648.5 −4 6,047.23 4,965.83 −18

Minor repairs and upgrade of

district hospitals (number) 10 18 80 3,465.9 3,465.9 — 346,590.00 192,550.00 −44

Training of district hospital

management staff 303 753 248 729.5 493.4 −32 2,407.59 655.25 −73

Note: *1 Includes 98 trained pharmaceutical assistants transformed to pharmacists which was not planned at the beginning of the project. *2 Includes 1,542 health staffs completing short-term training on MDG-related activities during the extension phase. *3 Times of skill transfers. The budget includes expenses to support mobilized staff.

60

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit

Lending

Bukhuti Shengelia Task Team Leader (left the World Bank)

Mai Thi Nguyen Team Member GED02

Nga Quynh Nguyen Senior Program Assistant EAPDE

Maryam Salim Team Leader MDI

Hoi Chan Nguyen Country Counselor (retired)

Samuel S. Lieberman Task Team Leader (retired)

Kelichi Ohiri Health Specialist/Team Member (already left the

World Bank

Lingzhi Xu Senior Operation Officer GHN03

Marko Vujicic Economist (left the World Bank)

Lan Thi Thu Nguyen Safeguards Specialist GEN2B

Hung Viet Le Financial Management Specialist EAPCO

Supervision/ICR

Mai Thi Nguyen Team Member GED02

Kari L. Hurt Team Leader GHN06

Anh Thuy Nguyen Team Leader GHN02

Bukhuti Shengelia Task Team Leader (left the World Bank)

Andre C. Medici ICR Author GHN04

Hoang Xuan Nguyen Procurement Specialist GGO08

Mai Thi Phuong Tran Senior Financial Management GGO20

Sang Minh Le Environment Safeguards Specialist GHN02

Giang Tam Nguyen Social Safeguards Specialist GSU02

Nga Thi Anh Hoang Program Assistant EACVF

Nghi Quy Nguyen Social Development Specialist GSU02

Trang Phuong Thi Nguyen Safeguards Specialist EASVS

Maryam Salim Team Leader MDI

Nguyen Hoang Nguyen Procurement Specialist GGODR

Quynh Xuan Thi Phan Financial Management Specialist GEFPO

Maya Razat Program Assistant GSP

Minh Thi Hoang Trinh Program Assistant AFCNG

Nga Quynh Nguyen Senior Program Assistant EAPDE

Duong Minh Duc Public Health Consultant

61

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of Staff Weeks US$, thousands (including

travel and consultant costs)

Lending

FY06 19.89 96.00

FY07 31.35 246.50

FY08 34.80 145.50

Total: 86.04 488.00

Supervision/ICR

FY09 24.50 83.70

FY10 26.50 96.80

FY11 17.00 75.00

FY12 14.00 53.50

FY13 22.00 70.20

FY14 17.30 49.50

FY15 14.40 42.00

FY16 26.80 100.50

Total: 162.50 571.00

62

Annex 5. Results Framework - Analysis of the PDO Achievement

Introduction

1. The PDO has three parts: (a) PDO 1 - increase the utilization of district hospital services;

(ii) PDO 2 - improve the quality of district-level hospitals; and (iii) PDO 3 - reduce financial

constraints to access health services. The objective of this annex is to attribute the KPIs and IOIs

to the three parts of the PDO and to rate the PDO (and its parts) according of the achievement of

the corresponding indicators.46

2. The methodology to define the indicators’ ratings is the following: For quantitative

indicators, the achievement at the end of the project is compared with its end target. If the result is

above 105 percent the indicator was surpassed; if it is between 95 percent and 104 percent, it was

achieved. If it is between 85 percent and 94 percent, it was substantially achieved. If it ranges

between 65 percent and 84 percent, it was partially achieved, and if it is lower than 65 percent, it

was not achieved. For qualitative indicators, the classification is only achieved (if the qualitative

target was accomplished) and not achieved (if it was not accomplished). The IOI #7, with no

reliable information about baselines and targets of values of achievement will not be considered

as part of the PDO rating.

3. The criteria used to calculate the indicator achievement is the following: (a) if the baseline

is not zero, it is calculated on the difference between what was intended (baseline) and actual47

and divided by the difference between the target and the baseline; (b) if the baseline is zero it is

calculated on the coefficient between the actual and the target; and (c) if the target is lower than

the baseline, it is calculated on the coefficient between the actual and the baseline.

4. Annex 5A presents a table calculating the rating of each indicator according to

achievements recorded during project implementation. This table has the following columns: (a)

original indicators (according to the PAD); (b) indicators added during project implementation;

(c) value and date of the indicator baseline; (d) value and date of the indicator target; (e) value and

date of the indicator at the project’s original closing date of August 31, 2014; (f) percentage of

target achieved at the project’s original closing date; (g) value of the indicator at the project’s

revised closing date of February 29, 2016; (h) percentage of target achieved at the project’s revised

closing date; and (i) indicator achievement rate according to the methodology presented in

paragraph 2.

5. The rating of the PDO is attributed to the proportion of the indicators’ values that have

been surpassed, achieved, or substantially achieved as a share of the total project indicators. It is

high, when more than 95 percent of the indicators’ target values have been surpassed, achieved, or

substantially achieved; substantial, from 75 percent to 94 percent of achievement; modest, from

50 percent to 74 percent, and negligible when less than 50 percent of the indicators have met their

target values.

46 PDOs and IOIs will have the same weight to classify the PDO’s achievement. 47 A= (Ia-Ib)/(It-Ib), where A is achievement, Ia is indicator’s actual; Ib is indicator’s baseline, and It is indicator’s

target.

63

6. This annex has three tables. Table 5.1 shows the KPIs and IOIs distributed according to

the three parts of the PDO. Table 5.2 summarizes the results found in annex 5A, and table 5.3

summarizes the PDOs’ rating to measure the project efficacy.

Project KPIs and IOIs related to the PDO

7. Number of indicators. According to the PAD, the project had 4 KPIs and 10 IOIs. All

KPIs and IOIs #1 to #7 were related to the PDO. The other IOIs (#8 to #10) were designed to

measure project management performance. Some IOIs were complex to be measured through just

one indicator. Then, during the project implementation, the Government and the World Bank

agreed on creating subindicators to measure these complex and multidimensional IOIs.

Accordingly, IOIs #2, #3, and #4 were measured by nine, three, and two subindicators, respectively.

Additionally, other IOIs were included during the project execution without a formal project

restructuring. Other administrative indicators were also included using similar processes. The last

project ISR,48 issued in February 2016, lists a total of 25 KPIs (4) and IOIs (21). Table 5.1 shows

the PDO parts 1, 2, and 3 and the corresponding KPIs and IOIs. PDO part 1 was measured by 2

indicators, PDO part 2 by 19 indicators and subindicators, and PDO part 3 by 4 indicators.

Table 5.1. Distribution of the Project Indicators (KPIs and IOIs) according to the PDO parts.

48 The project documentation does not reflect when all these indicators and the corresponding baselines were set and

included. They are not in the PAD and start to appear only in ISR #6 (issued in August 2013). From 2008 (project

starting) to 2013, the five project ISRs do not have clear information on the project M&E and RF.

PDO’S PARTS KPI (*) IOI * IOI Sub Indicators **

PDO #1:

Increase

utilization of

district health

services.

(2 indicators)

KPI #1: Increase

utilization rates of

inpatient services in

district hospitals among


KPI #2: Increase


outpatient services in



PDO #2:

Improve the

quality of

district-level

hospitals

(19 indicators)

KPI #4: Proportion of

district hospitals that

provide full set of health

services according to the

national norms (Decision

23/205/QB- BYT)

adjusted to the Northern

Uplands

IOI #1: Percentage of patients

satisfied with the health services

IOI #2: Adherence of treatment

protocols for selected conditions

in inpatient settings (based on

six subindicators)

IOI #2.1: Percentage of health workers

with knowledge of diagnosing and

treating Level A/B/C dehydrated

diarrhea



treating severe pneumonia



treating poisoning

IOI #2.4: Percentage of reasonable

diagnoses of severe pneumonia

IOI #2.5: Percentage of clinical health

workers’ reasonable diagnosis of general

pneumonia

64

Note: * According to page 34–36 of the PAD.

** According to project ISRs Sequence #6 to #11 (last).

8. Indicators’ baseline. Despite the fact that annex 3 of the PAD presents the project RF with

the project indicators (KPIs and IOIs), the RF was incomplete and most of the baselines and targets

were revised in 2008 (VHLSS) and 2009 (Baseline Survey). Different dates for project baselines

could be found. Some indicators incorporated during project implementation do not have baselines.

Some indicators should report progress twice during the project life (Year 3 and Year 6). This is

the case for KPIs #1, #2, and #3 and IOIs # 3, #5, and #6. The survey developed for the MTR


diagnoses of Level A dehydrated

diarrhea


diagnoses of Level B dehydrated

diarrhea


diagnosis of Level C dehydrated

diarrhea


diagnoses of poisoning

IOI #3: Percentage of eligible

district health staffs who have

successfully completed training

provided by the project (based

on three subindicators).

IOI #3.1: Percentage of doctors and

assistant doctors at district hospitals


IOI #3.2: Percentage of health staffs

with completed short-term training

courses compared to the plan

IOI # 3.3: Percentage of health staffs

completed long-term training courses

compared to the plan

IOI #4: Percentage of eligible

district hospitals with acceptable

operation and maintenance plans

and budgets for facility and

equipment maintenance (based

on two subindicators).

IOI #4.1: Percentage of district hospitals

having schedule and budget for

maintenance of infrastructure

IOI #4.2: Percentage of district hospitals

having schedule and budget for

maintenance of equipment

IOI #5: Number of health

facilities constructed, renovated,

and/or equipped

IOI #6: Percentage of recently

discharged patients satisfied with

health services

IOI #7: Number of people with

access to a basic package of

health, nutrition, and

reproductive health services

PDO #3:

Reduce

financial

constraints to

access health

services.

(4 indicators)

KPI #3: Percentage of

households who

experienced catastrophic

health care expenditures

in the year prior to the

survey

IOI #8: Percentage of Decision

139 beneficiaries who have

received HICs

IOI #9: Percentage of 139

beneficiaries with cards who can

correctly identify at least three

benefits covered under the

HCFP program

IOI #10: Percentage of

households who identify

financial barriers as a main cause

for not seeking health care

65

captured the results of these indicators at Year 3 of project implementation. All other indicators

had to be measured at project end, according to the PAD, but some were followed by the CPMU

using the project administrative registries.

9. Results of the PDO’s achievement evaluation: Substantial. Project efficacy could be

considered substantial according to the ratings obtained in each one of the parts of the methodology.

Efficacy of PDO 1 is High, given that all KPIs surpassed their target values. Efficacy of PDO 2 is

considered High, given that the percentage of indicators surpassed or achieved the targets is 100

percent. Efficacy of PDO 3 is considered Substantial, given that 75 percent of the indicators

surpassed their target values.

Table 5.2. Summary Table of Indicator’s Achievement

Rating Categories KPI’s IOIs Total

PDO Part 1 - Increase utilization of district health services - High

Surpassed (>105%) 2 — 2

Achieved (95%–105%) — — —

Substantially achieved (85%–104%) — — —

Partially achieved (65%–84%) — — —

Not achieved (<65%) — — —

Not considered — — —

Total PDO Part 1 2 — 2

PDO Part 2 - Improve the quality of district-level hospitals - High

Surpassed (>105%) 1 13 14

Achieved (95%–105%) — 4 4



Not achieved (<65%) — — —


Total PDO Part 2 1 17 18

PDO Part 3 - Reduce financial constraints to access to health services - Substantial

Surpassed (>105%) 1 2 3

Achieved (95%–105%) — — —



Not achieved (<65%) — 1 1


Total PDO Part 3 1 3 4

Grand Total 4 20 24

Table 5.3. Summarized Rating for Project Efficacy

PDOs Parts Efficacy Rates Based on the Achievement of the Indicators

PDO Part 1 High

PDO Part 2 High

PDO Part 3 Modest

Overall Rating Substantial

66

Annex 5A. Status of the Indicators According to Achievement

Original Indicators

(According to the

PAD)

Indicators Added

during Project

Implementation

Value and

Date of the

Indicator

Baseline

Value and

Date of the

Indicator

Target

Value and

Date of the

Indicator at

Project

Original

Closing Date

(Aug 31, 2014)

Ratio of

Achievement

at Project

Original

Closing Date

Value of the

Indicator at

the End of

Project

Extension

Period

(February

29, 2016)

Ratio of

Achievement

at the End of

the

Project

Extension

Period (*)

Rating of

the

Indicator

Key Performance Indicators (KPIs)

KPI #1: Increase


inpatient services in


Decision 139

beneficiaries

— 0.027

(June 2009)

0.033

(August

2014)

0.081

(December

2013)

9.00

0.096

(December

2015)

11.50 Surpassed

KPI #2: Increase


outpatient health

services in district

hospitals among

Decision 139

beneficiaries

—

0.067

(December

2009)

0.075

(August

2014)

0.082

(December

2013)

1.87

0.247

(December

2015)

22.50 Surpassed

KPI #3: Percentage of

households who

experienced catastrophic

healthcare expenditures

in the year prior to the

survey

— 14.27%

(June 2008)

13.23%

(August

2014

2.0%

(August 2014) 11.80 Not measured — Surpassed

KPI #4: Proportion of

district hospitals that

provide full set of health

services according to the

national norms

(Decision 23/205/QB-

BYT) adjusted to the

Northern Uplands

— 39.1%

(June 2008)

70%

(August

2014)

71.4%

(October 2014) 1.05

80.4%

(December

2015)

1.34 Surpassed

Intermediate Outcome Indicators (IOIs)


patients satisfied with

the health services

— 8.5%

(July 2009)

10.2%

(August

2014)

84.4%

(October 2014) 44.71 Not measured — Surpassed

67

Original Indicators

(According to the

PAD)

Indicators Added

during Project

Implementation

Value and

Date of the

Indicator

Baseline

Value and

Date of the

Indicator

Target

Value and

Date of the

Indicator at

Project

Original

Closing Date

(Aug 31, 2014)

Ratio of

Achievement

at Project

Original

Closing Date

Value of the

Indicator at

the End of

Project

Extension

Period

(February

29, 2016)

Ratio of

Achievement

at the End of

the

Project

Extension

Period (*)

Rating of

the

Indicator

IOI #2: Adherence of

treatment protocols for

selected conditions in

inpatient settings (based

on six subindicators)

IOI #2.1: Percentage of

health workers with

knowledge of diagnosing

and treating Level

A/B/C dehydrated

diarrhea

9.7

(June 2009)

14.0%

(August

2014)

95.2%



health workers with


and treating severe

pneumonia

13.2%

(June 2009)

18.5%

(August

2014)

86.9%



health workers with


and treating poisoning

26.8%

(June 2012)

37.5%

(August

2014)

83.0



reasonable diagnoses of

severe pneumonia

45.5%

(June 2009)

63.7%

(August

2014)

71.1%



clinical health workers’

reasonable diagnosis of

general pneumonia

19.60%

(June 2009)

27.44%

(August

2014)

57.00%




Level A dehydrated

diarrhea

37.2%

(June 2009)

52.1%

(August

2014)

78.9%




Level B dehydrated

diarrhea

48.9%

(June 2009)

68.5%

(August

2014)

85.1%



reasonable diagnosis of 41.2%

(June 2009)

57.7%

(August

2014)

80.0%

(October 2014) 2.35% Not measured — Surpassed

68

Original Indicators

(According to the

PAD)

Indicators Added

during Project

Implementation

Value and

Date of the

Indicator

Baseline

Value and

Date of the

Indicator

Target

Value and

Date of the

Indicator at

Project

Original

Closing Date

(Aug 31, 2014)

Ratio of

Achievement

at Project

Original

Closing Date

Value of the

Indicator at

the End of

Project

Extension

Period

(February

29, 2016)

Ratio of

Achievement

at the End of

the

Project

Extension

Period (*)

Rating of

the

Indicator

Level C dehydrated

diarrhea



poisoning

61,1%

(June 2009)

85.5%

(August

2014)

86.2%

(October 2014) 1.03 Not measured — Achieved


eligible district health

staff who have

successfully completed

training provided by the

project


doctors and assistant

doctors and pharmacists

at district hospitals


0

(June 2008)

80

(August

2014)

102

(December

2013)

1.27

189

(December

2015)

2.36 Surpassed


health staffs completed

short-term training

courses compared to the

plan *1

0

(June 2008)

80

(August

2014)

357

(June 2014) 4.46 — — Surpassed

IOI # 3.3: Percentage of

health staffs completed

long-term training

courses compared to the

plan *1

0

(June 2008)

80

(August

2014)

88

(June 2014) 1.10 — — Surpassed


eligible district hospitals

with acceptable

operations and

maintenance plans and

budget for facility and



district hospitals having

schedule and budget for

maintenance of

infrastructure *2

49.2

(June 2009)

40.0

(August

2014)

99.3

(October 2014) 2.02

79.7

(December

2015)

1.62 Achieved


district hospitals having

schedule and budget for

maintenance of

equipment. *2

77.1

(June 2009)

40

(August

2014)

99.2

(October 2014) 1.29

89.1

(December

2015)

1.16 Achieved

IOI #5: Number of

health facilities

constructed, renovated,

and/or equipped

0

(June 2008)

61

(August

2014)

63

( August 2014) 1.03

64

(December

2015)

1.05 Achieved

69

Original Indicators

(According to the

PAD)

Indicators Added

during Project

Implementation

Value and

Date of the

Indicator

Baseline

Value and

Date of the

Indicator

Target

Value and

Date of the

Indicator at

Project

Original

Closing Date

(Aug 31, 2014)

Ratio of

Achievement

at Project

Original

Closing Date

Value of the

Indicator at

the End of

Project

Extension

Period

(February

29, 2016)

Ratio of

Achievement

at the End of

the

Project

Extension

Period (*)

Rating of

the

Indicator


recently discharged

patients satisfied with

health services

8.5

(June 2009)

10.2

(August

2014)

84.4

(August 2014) 44.65 — — Surpassed

IOI #7: Number of

people with access to a

basic package of health,

nutrition, and

reproductive health

services *4

20%

(June 2009)

70%

(August

2014)

244,801

(August 2014) —

270,274

(December

2015)

— Not

considered


Decision 139

beneficiaries who have

received HICs *5.

— 82.1 (June

2009)

70

(June 2013)

94.3

(June 2012) 1.15

95.2

(December

2015)

1.16 Surpassed


139 beneficiaries with

cards who can correctly

identify at least three

benefits covered under

the HCFP program. *6

— 14.8

(June 2009)

75.0

(August

2014)

57.4

(August 2014)

0.57

— —

.Not

Achieved


households who identify

financial barriers as a

main cause for not

seeking health care.

— 2.0

(June 2009)

1.8

(August

2014)

1.2

(August 2014) 4.03 — — Surpassed

Note: *1 This indicator was not listed in the PAD. The information was obtained from the Center for Environment and Health Studies (2014) Final Report: End-line evaluation

of the Northern Uplands Health Support Project, Hanoi, 2014, and Center for Environment and Health Studies (2016) Final Evaluation of the Northern Uplands Health

Support Project in the Extension Phase, Hanoi, 2016. *2 The target of this indicator was established before the baseline survey. For this reason, the baseline value in the PAD was higher than the target value. However, the project

did not revise the target during project implementation. For this reason, the achievement for this indicator had been calculated over the baseline instead of the target. Even

using these criteria, the actual value for this indicator was twice the baseline value by August 2014, but was only 62 percent over the baseline by the end of the project

extension period, which could bring some doubts on the sustainability of infrastructure maintenance. Despite this issue, the ICR team considered the target achieved. *3 According to the PAD, this indicator’s provisory baseline was 20.14 percent in 2014. An actualization of this indicator was expected to update the baseline after start the

project implementation.

70

*4 This indicator cannot be measured because the baseline and the target were not converted in the number of beneficiaries with access to the basic package of health,

nutrition, and reproductive health services during the project life. *5 The target of this indicator was established before the baseline survey and was based according to the preparation team best guess estimate. However, the project did not

revise the target during project implementation. For this reason, the achievement for this indicator had been calculated against the baseline instead of the target. Despite this

issue, the ICR team considered the target achieved. *6 The Government additionally followed the indicator ‘Percentage of Decision 139 Beneficiaries with cards who can correctly identify at least three benefits covered under

the HCFP program’. The achievement of this indicator was 95 percent in December 2015.

71

Annex 6: Summary of Borrower's ICR

Northern Uplands Health Support Project

1. Context

1. The World Bank supported the Vietnamese Government with a loan to implement the

Northern Uplands Health Support Project (hereafter called the NUP). The project was

implemented in eight years (2008–2015) in the seven northern provinces of Cao Bang, Bac

Kan, Lao Cai, Ha Giang, Son La, Dien Bien, and Lai Chau.

2. The general objective of the project was to improve the health status of the poor ethnical

minorities in these provinces, which required strengthening the capacity of the health care

system, providing better quality health services, and improving access. The legal background

of the project is composed of:

(a) the general policies and decisions of the party and the state during the period

2001–201049 to promote health care and protect people’s health;

(b) the master plan to develop Vietnam’s health care system up to 2010 and vision to

2020;50 and

(c) the creation of the HCFP in 2002 (known as Decision 139) to increase access to

health care and reduce the financial burden of health expenditure faced by the poor

and ethnic minorities.

3. To achieve this general objective, the Project supported (a) the upgrade of district

hospitals in these seven provinces, by training health staff, developing human resources for

health care, renovating these hospitals, providing medical equipment to achieve better health

care services, creating mechanisms and skills to repair infrastructure and medical devices; and

(b) the increase of health care services’ access for the poor and the ethnical minorities by

providing economic subsidies for transportation and meals and ensuring equity in protecting,

caring, and improving people’s health.

4. The seven NUP provinces constitute the most disadvantaged regions in Vietnam, with

high concentrations of poor and ethnic minorities living in sparsely populated, mountainous

localities, under difficult circumstances. As a result of these unfavorable socioeconomic

conditions, and despite somewhat higher public health expenditure per capita than in other

regions, health services in these provinces are difficult to access and generally poor in quality,

while the health status of the population is significantly worse than the rest of the Vietnamese

population.

49 Declared at Decision No. 35/2001/QD/TTg (dated 03/19/2001) and the Resolution No. 46-NQ/TW (dated

02/19/2005). 50 Decision No. 153/2006/QD-TTg of the prime minister.

72

5. The health system in these mountainous provinces was weak with regard to the quantity

of qualified human resources, infrastructure, medical equipment, and health financing. The

investments provided by the project for these provinces significantly improved the health

supply and the health status of local people, contributing toward the achievement of the MDG

targets in health, in accordance with the party and the state policies. It also contributed to

developing Vietnam’s health sector toward fairness and efficiency.

6. After seven years of project implementation, the health indicators of the Northern

Upland Provinces have improved significantly. The inpatient services’ utilization of the poor

ethnical minorities (beneficiaries under Decision 139) increased almost four times between

2009 and 2015. The outpatient services’ utilization rates increased two times in the same period

and the proportion of households with catastrophic health expenditures reduced from 10

percent to 2 percent, between 2009 and 2014.

7. Health human resources (weak and inadequate in the past) have been strengthened. All

health staff in district hospitals (111,800 workers) were trained. Health facilities were repaired

and upgraded in parallel with investments in infrastructure. Equipment provided to district

hospitals have been upgraded and used effectively.

8. The project design draws on best practice examples and lessons learned from other

international and Vietnam health investment projects. The project supported interventions on

both the ‘supply’ and ‘demand’ sides. On the supply side, the project supported district

hospitals by providing training and developing health human resources and repairing and

upgrading district hospitals’ infrastructure and equipment. On the demand side, the

contribution was facilitated, by providing economic subsidies to the poor, access to good

quality services, and thereby, increasing the probability of success and efficiency of

investments to improve the population’s health.

9. The project design is well articulated with other donor-supported activities in the region

and therefore does not cause any duplication. It only complements other ongoing initiatives.

There are some projects also implemented in the NUP, such as Project 225, financed by the

Government, to upgrade the provincial district hospitals. Between 2005 and 2007, Project 225

has invested around US$10 million (VND 169 billion), which only met 20 percent of the total

health investment needs for the district level in these seven provinces. HEMA (sponsored by

the European Commission) supported health care investments for three of the seven NUP

provinces (Dien Bien, Son La, and Lai Chau). The HEMA Project was focused on the

community health services. The Global Alliance for Vaccination and Immunization Project

(2007–2010) supported training for village health workers in 10 provinces, including 4 of the

Northern Upland provinces (Ha Giang, Cao Bang, Bac Kan, and Dien Bien). The ADB

financed a project for development of provincial preventive medicine system (by providing

equipment and training for laboratories). This project was also financed by several bilateral

donors and nongovernmental organizations with limited funds.

10. The PDO risks were well managed and limited at the lowest level with appropriate

measures from the project. To avoid the abuse of inpatient services (admitted with mild cases)

at district hospitals in the project areas, the project provided trainings for the officers on topics

73

such as management, operation of the fund for the poor, and verifying the admission cases.

The audits (in collaboration with HI officers) of using funds for the poor were regularly

conducted, to ensure that the admission cases were appropriate, inpatient cases received

treatments, and inpatient’s medical records were maintained. The monitoring activities, which

focus on support for the poor, from the CPMU and the PPMUs to the hospitals had been made

periodically. The results of the supervisions and audits showed that management of the HCFP

was done in accordance with the regulations of the Government and the project.

11. The risk of the non-poor group also benefiting from the interventions of the project,

leading to rising inequality, was well managed by different interventions. The project

developed a good management system to control these risks by coordinating well with Vietnam

Social Insurance in the provinces, for review and synthesis of the project beneficiaries annually.

The list of the poor and ethnic minorities who were qualified for support from the NUP was

provided to district hospitals. To receive the NUP support, the patients should present the

insurance card with the code ‘HN’ or the certification of poor from the communal people’s

committee

12. Many officers/staff were sent for trainings causing short-term shortage of health

workers and affecting the availability and quality of services. However, various activities were

implemented to alleviate these difficulties. The implementation of training courses was spread

throughout the project duration. District hospitals had plans for replacing the staff who

attended trainings. The leaders, physicians, and assistant doctors working at ancillary

departments (management board, departments of planning, financial, or infection control) at

the hospital also shared the responsibility of treating and caring for patients. Medical staffs are

required to work in night shifts more frequently. The project allowed district hospitals to sign

contracts/hire retired medical doctors or temporarily transfer staff among hospitals to help each

other when their staff were attending the training. The project risk management is described in

annex 6A.

2. Achievement of the PDOs

13. The project has successfully achieved its overall DO. The utilization of district health

services (by the poor and vulnerable population defined accordingly with Decision 139) has

increased sharply after IEC campaigns, improving the effectiveness of the Government’s

priority to UHC, both by improving the geographical accessibility of quality basic health

services at district hospitals and by reducing the financial burden in accessing health services

for the poor and ethnical minorities.

14. The project has been successful in implementing the health care policies for the poor

and increasing their access to quality health services. The number of the poor receiving

financial assistance (meals and travel costs) to visit district hospitals from the NUP has been

increased yearly, along with a significant increase in the rate of using district health services

among the general population, especially among vulnerable groups such as the poor and ethnic

minorities. The average number of inpatient and outpatient visits per capita per year of ‘HN’

(Ho Ngheo or Poor Household in English) in district hospitals, increased 390 percent (from

0.0247 to 0.096) and 369 percent (from 0.067 to 0.247) from 2009 to the end of 2014,

74

respectively, exceeding the project’s expectations. The awareness and confidence of the

population vulnerable to health care services has been improved. The percentage of households

who experienced catastrophic health care expenditures in the year before the survey reduced

81 percent in comparison to the baseline. Detailed data is presented in annex 6B.

15. The activities related to the project components51 have remarkably improved the supply

and quality of the health services offered by district hospitals in the seven provinces. The

percentage of health techniques (based on the national list of techniques/protocols defined by the

MoH) in the seven provinces’ district hospitals increased from 39.1 percent at the baseline (2009)

to 80.4 percent at the end of 2015. The percentage of patients who were satisfied with health care

services increased from 8.5 percent at the baseline to 84.4 percent at the end line. Around 90

percent of patients were satisfied with the qualification of medical staff, facility infrastructure,

medical equipment, and drugs used for treatment (annex B).

16. District hospitals could provide more complex clinical techniques, especially

emergency surgeries and various endoscopic surgeries. The capacity of obstetric and newborn

care in district hospitals has improved significantly, with more than 75 percent of the district

hospitals providing cesarean section and blood transfusion services. The strengthened hospital

capacity and improvement of staff quality, has as a consequence, reduced the transferring of

patients to a higher-level hospital, and average length of inpatient stay by a half of day, in

comparison to the baseline (6.6 days down to 6.1 days).

17. At the end of the project, the number of health staffs with higher qualifications (general

medical doctors and medical doctors-level-1 specialists) was doubled. The knowledge and

skills of health workers to manage common health episodes were significantly raised (annexes

6B and 6C). Besides district hospitals’ infrastructure and equipment were upgraded by the

project, thereby improving district hospitals’ efficiency by absorbing a higher number of

medium to complex cases and by reducing referral to provincial hospitals. These positive

effects in the district hospitals’ performance are widely recognized and have long-term effects,

increasing the sustainability of the provincial health systems. However, the hospitals still lack

specialized doctors (in surgery, trauma, and in specialized departments such as eyes and dental)

which requires appropriate measures to attract qualified human resources to fulfill this need at

district hospitals.

3. Analysis of the Results Framework (baselines, targets, results)

18. There were no major changes in the Project Development Indicators (PDI’s) and

Intermediate Indicators (II’s) with regard to concept or calculation formula during the project

execution. The only exception is the indicator of proportion of district hospitals that provide

full set of health services according to the national norms (Decision 23/205/QD-BYT), which

was adjusted for better measuring the improvements of the district hospitals’ capacity. Some

51 Such as strengthening capacity in provision and quality of health care service at district level; comprehensive

interventions of the NUP in supporting human resources; investment in upgrading infrastructure; procurement,

purchasing, and installment of medical equipment; and improvements in hospital management have remarkably

improved the health services (quality and quantity, diversify the types of services) in the seven provinces.

75

key maternal and child health indicators were presented to reflect the impacts toward the

achievement of MDGs targets at the project provinces.

19. The evaluation results showed that 9 out of 10 KPIs were achieved and even exceeded

several times the objectives set in the project document. One indicator where the goal was not

achieved is the percentage of ‘HN’ beneficiaries knowing at least three beneficiaries’ rights

guaranteed by the HI system. This indicator reached 57.4 percent of the expected target at the

end of the project. The results also indicated that the trainings and equipment support improved

the district hospitals’ capacity of providing obstetric/pediatric emergency services,

contributing to improving the MDG performance in the region. In the extension period, in

collaboration with Mother and Child Health centers, PPMUs and CPMU focused on the

trainings that aimed at improving the capacity for newborn care, maternal health care,

traditional birth attendants, and providing clean delivery kits for the traditional birth attendants.

The results of the survey in 2015 showed that the maternal mortality ratio, adjusted in seven

provinces, is 98 per 100,000 live births, significantly reduced in comparison to the 2008 survey

data. The IMR was 10.6 percent, similar to the 2008 survey results in 2015. The MMR and

IMR in the seven project provinces are still higher than those of the whole country, which set

out the needs to continue communications on safe motherhood to the local people, especially

ethnic minority groups, develop the traditional birth attendants’ networks in remote villages,

and strengthen the management of pregnancy and antenatal care at the commune health centers,

to achieve the MDGs of the country. The project RF is presented in annex B.

4. Achievements by Components

20. The performed activities and interventions are the same as defined in the project

document. The outputs of the components have surpassed the targets that were set. See detailed

information in annex 6C.

4.1 Strengthening District-level Health Services (disbursement rate=90 percent (USD

38,584,300/USD 42,880,362)

21. The project has supported long-term and short-term trainings for health staff in the

NUP in many fields. The training programs fulfilled actual needs of the locality and were

implemented in accordance with guidelines, policies, and strategies of the MoH. The training

activities improved staff skills and reduced health workforce shortages, providing

professionals who were better prepared. It also contributed to reduce the unbalanced health

skills in district hospitals and provide a stable workforce to attend their needs in the long term

in the seven provinces. The project has supported the development of 377 level-1 doctors,

achieving 266 percent of the original target, and 56 percent are ethnic minority doctors. About

1,500 assistant doctors (46 percent with ethnical minority background) were trained and

converted to principal doctors. The project has also trained 98 intermediate pharmacists.

22. Overall, 95 percent of the staff who graduated through the project training activities

returned to work at the host hospitals. Since 2011, the PPMUs decided to enroll assistant

doctors from commune health stations as part of the long-term training activities and to include

76

important new specialties in the level-1 training, such as image diagnoses, tuberculosis

treatment, clinical pathology, anesthesiology, and others.

23. The short-term training courses on specialized contents have improved knowledge and

practical skills on disease treatment to address the shortage of professional capacity and

strengthen the decentralized techniques in the district hospital. Besides the clinical areas,

trainings on preventive medicine and health management and related areas (hospital

management, health information systems, medical equipment repair, and medical waste

management) were also provided.

24. To improve maternal and child health status and support the achievement of the health

MDGs by 2015, relevant training for nurses and emergency care for maternal and newborn

babies were also added in the training programs. The project also conducted short-term training

courses on safe motherhood, focusing on maternal and newborn health care.

25. The implementation of non-training human resources activities such as ‘technology

transfer; (in collaboration with the Government 1816 program)52 and ‘rotation of doctors’

(temporary placement of a doctor from a higher-level facility at a district hospital) had partially

alleviated the shortage of doctors and other staff in the district hospitals, providing capacitation

on specific techniques while emphasizing more practical skills. Detailed training results are in

tables 6.1–6.6 of annex 6C.

26. The project also performed activities to repair and upgrade 18 district hospitals (100

percent working plan, completed in 2010–2012) providing the installation and effective use of

the medical equipment supported by the NUP (table 3.7, annex 6C).

27. The project provided 52 ambulances and 5,065 essential medical equipment for 64

district hospitals (average of 80 devices per hospital achieving 102 percent of the working

plan). It included valuable equipment such as high-tech x-ray machines (102), endoscopic

systems (37), ultrasound systems (109), ventilators (112), and monitor tracking devices (142)

(table 3.8, annex 6C).

28. The equipment provided is strongly based on the hospitals’ demands, possible because

of carefully reviewing and assessing the needs before starting the purchasing process. The

equipment that was received immediately had a positive effect on providing health care for the

people and offering favorable conditions for the health staffs to practice the knowledge and

skills that they had gained from the long-term and short-term training programs.

29. The maintenance of this equipment had complied with the requirements. Health staff

were fully trained and instructed to adequately use and explore the equipment. The specific

teams responsible for maintaining and repairing the medical equipment were established in

each of the seven provinces.

52 1816 Program is a program of the Government of Vietnam on ‘Sending the professional staff from higher-

level hospitals to support lower-level hospitals professionally to improve the quality of the health care services’.

77

4.2 Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries

(disbursement rate=100.5 percent (USD 10,048,816/USD 10,000,000)

30. To increase the use of health services for the poor and minorities, the project financed

travel expenses, meals, and direct costs of the beneficiaries who were not supported by HI or

other health care funds. This support helped the poor/minorities access and use quality basic

health care services, ensure fairness in the health financial protection, and improve the health

of the beneficiary population. It sustained the effective implementation of the provincial health

care policy for the poor/minority, defined in Decision 139. From 2009 to 2014, around 65,000

inpatients’ medical visits received support. (Table 6.9 annex 6C). The percentage of poverty

households which experienced catastrophic health care expenditures (according to the World

Health Organization criteria) reduced from 10.4 percent to 2.0 percent between 2009 and 2014

surpassing the target of 9.4 percent. (Annex 6B).

31. The final project survey showed that the rate of households which experienced

expenditures for health care services decreased five times and three times compared to the

baseline and MTR, respectively. In 2015, the HCFP (supported by Decision No. 14/2012 / QD-

TTg) was operated in four provinces (Lao Cai, Ha Giang, Son La, and Lai Chau). Total 269,400

inpatient visits were supported in four provinces in the 18 months of the project extension

period (table 6.10, annex 6C). Three other provinces—Cao Bang, Bac Kan, and Dien Bien—

had not implemented this decision as the provincial budgets were not adequate or allocated to

continue these supports. During the extension period, the number of inpatients in these three

provinces was slightly reduced in comparison to the previous period. Without support on

transport and food allowances, poor households tended not to go to the hospitals as they could

not afford travel and food during their hospitalization period. This suggested that the financial

barrier is still one of the main reasons that limit access to and use of health care services to

households, especially poor households, so the NUP’s support for the poor in accessing health

services is very important.

32. The project also promoted health-seeking behavior for the poor by developing IEC

activities to increase the target population’s knowledge about their benefits under the HI

scheme and the additional support provided by the project. As part of the activities related to

this area, IEC material was distributed to the households, IEC messages were announced in the

community and newspapers, movies, videos, and comedies were produced and broadcasted on

mass media (TV, speakers, and radio) and posters were displayed in the hospital

departments/rooms. The project provided some essential communication equipment to be used

in the provincial centers for the IEC activities. Nearly 5,000 meetings and campaigns were

conducted at crowded places and the IEC teams visited around 8,000 households.

33. The IEC activities increased access and utilization of health care services for the

poor/ethnic minority groups in the district hospitals. The final project evaluation showed that

94.7 percent of the poor/ethnic minority people could tell at least one right of HI cardholders,

94.3 percent of the poor/ethnic minority people in the project areas had HICs and the number

of inpatients at district hospitals sharply increased.

78

34. To strengthen capacity for the HCFP, the project provided training for the health

provincial teams and basic equipment (computers, printers, and photocopiers) to district

hospitals. The PPMU implemented systematic supervisions in the HCFP and guaranteed that

the management of the fund activities was in compliance with the provided guidance.

4.3 Monitoring, Evaluation, and Project Management (disbursement rate=85.3 percent

(USD 6,075,917/ USD 6,744,638)

35. The project maintained an adequate management structure and implementation

arrangements. The Project Steering Committee at the MoH was established to directly lead

project management. The CPMU and PPMUs were the key players in implementing and

managing the project. Monthly meetings with the PPMUs gave district hospital leaders the

chance to express their needs, comments, and suggestions for more appropriate and informed

implementation. The coordination between the MoH, CPMU, and PPMUs were systematic and

effective.

36. The project was implemented faster than previously scheduled, in the original plan.

The project outputs have reached and exceeded the outputs agreed in the project document.

After the project closing, the rate of disbursement was estimated at 96 percent. The goals and

targets are completed and beyond the schedule, at a cost lower than originally expected,

suggesting that the project was very cost-effective as well. The MoH authorities are satisfied

with the project achievement results.

5. Restructuring: Project Extension for 18 Months

37. After five years of execution, the project was implemented on schedule and the outputs

were achieved and exceeded the original plan, with savings in the original budgets (mainly

through procurement and tendering activities). However, there were a number of new

activities—non-planned in the original project design—that could be implemented with the

saved funds to promote efficiency and enhance the future sustainability of the project. The

main arguments to explain the rationale for the project extension are

• although the district has invested in training and upgrading of facilities and

equipment, the needs for medical support at the seven NUP provinces were still

huge because the district and commune levels were still facing many difficulties;

• the project investment has significantly helped the quality of services, increased

the number and type of health services provided, and increased accessibility and

the use of health services by the population at the district level. However, the NUP

team identified additional needs to be implemented in the short term (12–18

months) to guarantee the project’s long-term efficiency of investments and

maximize the effective use of the upgraded health facilities and the acquired

equipment; and

• the health indicators in general, especially those related to the health MDGs in the

NUP are the lowest compared to other regions in the country. The prime minister

79

issued Resolution No. 05/NQ-CP, dated 01/13/2014, on accelerating the

implementation of the health MDGs. Therefore, the support to accomplish this

resolution in the NUP areas (especially the goal of reducing child mortality rate

and MMR) was very urgent, especially considering the budget cuts that occurred

during this period.

38. Given these arguments, the DO and the scope of the interventions for the extension

phase did not change in comparison with the original stated in the project document. In addition,

the extension phase of the project looked for ways to

• maximize efficiency and enhance the sustainability of the project investments in

the seven provinces; and

• contribute to the achievement of Vietnam’s health MDGs in the related areas,

especially by reducing child mortality and improving maternal health.

39. During the extension phase, the project has achieved the objectives and implemented

the planned activities such as training on maternal and childcare and procuring equipment for

newborn health units and consumables to ensure safe birth delivery. The project management

was effective, focused on achieving the project results and promoting the sustainability of the

investment.

40. Special attention was dedicated to improve inpatient and outpatient services utilization

and capacity to implement clinical techniques at the district hospitals. The project has

contributed significantly to reach the MDGs (especially in reducing maternal and child

mortality) through capacity-building activities on obstetric and neonatal emergency at the

health facilities, particularly at district hospitals. The project contributed to the development

of village midwives’ teams and to increase the percentage of women giving birth at home, with

SBAs’ assistance, by providing clean delivery packages for pregnant women. During 2008–

2014, the maternal and child mortality rates declined remarkably, as can be seen in the annex

6B: Project Results Framework, MDG indicators.

6. Beneficiaries

41. The main project beneficiaries, as stated in the project document, were the poor and

ethnic minorities in the Northern Upland Provinces. These population groups were supported

by reducing their financial barriers to access and make better use of quality health services.

Project investments focused on the district level, an appropriate level at which the poor and

ethnic minorities had the ability to access their health care needs.

42. Other project beneficiaries were the health workers of the district hospitals in the seven

NUP areas who participated in trainings in many fields, such as (a) training level-1 doctors for

treatment; (b) doctors with four-year added training; and (b) short-term training courses for

clinical, preventive medicine, and health management. Since 2011, the project extended the

training support to pharmacists and assistant doctors at the community health centers.

80

43. Last, but not the least, the project benefited the health management agencies at

provincial and district levels (Department of Health, Division/District Health Centre) by

improving their managerial capacity. The district hospitals’ management teams were also

benefited by trainings that ensured better organization and means to provide health care for the

poor and ethnic minorities.

7. Evaluation of Other Project Aspects (risks, safeguards, fiduciary)

44. The project was designed based on the needs and recommendations of the local

authorities and communities. There were several rounds of referendums, for finalizing project

objectives and activities, with different local stakeholders (the provincial people committees,

department of health, district hospitals, and specialists), using different methods, including

participatory rapid assessment, consultation workshops, and direct consultations. The MoH

accumulated experience in implementing projects with similar components and activities.

Therefore, the implementation of the Project was quite favorable. The risk management

process was well conducted and did not significantly impede the project’s performance.

45. Environment safeguards. The project provided good solutions and implementation of

waste management processes at district hospitals. Basic training on regulations related to

HCWM and nosocomial infection control was provided for the district hospital managers and

staff. The CPMU and PPMUs received guidance for planning and implementing measures for

HCWM. Consumables and equipment for HCWM 53 were procured by the PPMUs and

distributed to the project district hospitals. The project hospitals achieved remarkable

improvements in the waste management process compared to the baseline (2007), when most

of them did not comply with the HCWM regulations. By 2013, a well-prepared HCWM plan

and monitoring program started to be implemented and all district hospitals had strengthened

institutional arrangements for that. The availability and proper use of waste containers,

transportation, and cooling devices resulted in significant improvements in health care waste

separation, collection, storage, and final disposal in district hospitals.

46. Social safeguards. The project beneficiaries included the poor and ethnic minorities

living in disadvantaged areas. Given the typical geographical condition of the NUP provinces,

the proportion of ethnic minorities is quite high. Data reported from PPMUs and surveys are

always disaggregated by Kinh (the majority of the population) and ethnical minorities to

indicate that the support from the NUP is considerable to ethnical minorities and the poor.

Most of the ethnical minority health staff received long- and short-term trainings, which

contributed to the increased ratio of ethnical minority doctors in the seven project provinces.

The percentage of beneficiaries who are ethnical minority is higher than 75 percent. In some

provinces, ethnical minority staff account for a higher percentage of total participants (such as

91 percent in Cao Bang). The project tailored IEC campaigns with the linguistic and cultural

characteristics of ethnic minorities. The IEC materials were translated into the local languages

and, to the extent possible, verbal and graphical means of information transfer were used. The

key success was the development and implementation of a strategy that reaches the diverse

53 It includes 23,000 kg of color-coded plastic bags, 25,000 sharp boxes, together with fixation frames, more

than 500 waste containers, 12 waste on-site transportation devices, and 64 cooling devices.

81

and dispersed beneficiaries in the Northern Upland Provinces, given the ethnical diversity of

the region and the difficult geographical terrain.

47. Fiduciary aspects. The project did not register major fiduciary problems (regarding

procurement of civil works or medical equipment acquisition). The counterpart funds from the

MoH and provincial budgets were allocated/provided on time and met the demand for timely

implementation of the management activities of the project.

8. Bank and Borrower Performance

48. The project implementation had sufficient human resources (including national

consultants) and good capacity to manage and implement the activities and use the financial

resources. The CPMU provided capacity building for PPMUs staff on project management,

including financial, procurement, accounting, assets management, civil works, training, HCFP,

and M&E activities. In addition, the project had international experts in the related fields. The

project was implemented on time and exceeded the set targets. Project interventions were

highly effective, providing significant benefits to health care of the NUP beneficiaries.

49. Project management activities were carried out with high demand for quality

management. Due to the design (simultaneous interventions on both the ’supply’ and demand’

sides) the project interventions were completed and exceeded the targets.

50. The basic elements for managing the project (Project Manual, job description for each

position, and financial management software) were available right from the start of the

implementation. The accounting software (installed at the central and provincial levels) met

the regulations of the Ministry of Finance and the project requirements. The financial

management system provided accurate and timely information on whether the credit proceeds

were used for the intended purposes. Cash accounting were conducted monthly. Financial

statements were prepared quarterly by the PPMUs and sent to the CPMU for consolidation and

submission to the World Bank. The procurement processes and procedures strictly followed

the requirements and regulations laid down by the Vietnamese Government and the World

Bank.

51. The involvement of stakeholders (CPMU, related departments of the MoH, PPMUs,

Department of Health, provincial/district hospitals) in the planning process indicated that the

plans were carefully considered before approval. The project plan and adjustments/revisions

were made based on actual demands and need of localities, in accordance with the project

objectives so as not to impede the progress of the project. The procedures and processes of

approval were agreed by both the MoH and World Bank to harmonize both institutional

perspectives. Therefore, in general, the progress on implementing the activities satisfied the

request of the World Bank and the MoH.

52. The operation implementation was sequenced as follows: staff training, hospital repair,

upgrade and acquisition of new medical equipment or supply. Communication activities were

carried out simultaneously to encourage people to seek medical care and treatment at the health

82

facility. The project activities were carried out in accordance with the approved working and

financial plans.

53. The M&E process was implemented according to the project M&E framework. The

baseline survey provided the initial and final targets of the project indicators. The MTR and

final project evaluation reviewed all the targets. The progress monitoring missions collected

outcomes at the localities with the appropriate tools. The data on indicators and performance

coming from provinces was aggregated, analyzed, and used to adjust project activities as

needed (some adjustments to increase the quotas and additional medical staff training,

enhanced communication activities to change behavior, and others were decided and

implemented in the middle of the project cycle).

54. The CPMU also provided training on monitoring, evaluation, and reporting and data

quality assurance for the PPMUs staff.

55. The success in implementation of the project is also the consequence of good

management and positive support from the World Bank team. Closed monitoring of project

activities, timely issuance of ‘no objections’ for procurement and work plan, suggestions for

important solutions and recommendation for speeding up the progress of the project, and

participation in technical missions were some of the valuable contributions of the World Bank

team to the success of the project.

56. For the improvement of future projects, the M&E system and M&E plan should be

built at the beginning, with the standard forms for collecting data from the PPMUs and

implementing sites. This will allow future projects to have good data and reference sources

from the beginning, improving the follow-up of the projects’ achievements. The PPMUs

should closely follow up with provincial people’s committees for approval of annual financial

plans as soon as possible, so the activities can be implemented at the beginning of each year.

9. Arrangements for Sustainability of Results

57. The NUP project investment presents a high level of sustainability and has contributed

to strengthening the NUP health systems. The project interventions were in line with the

policies and priority of the health sector and consistent with local needs. The seven provinces

are committed to continue to perform efficiently and maintain the sustainability of the project.

Many interventions were recognized to guarantee long-term sustainability to provide and

promote access of health services to the poor and ethnic minorities, such as the following:

• Training/developing human resources. The doctors (who received support from

NUP) will receive funds (from the provincial, host hospitals) to further continue

long-term trainings to become specialized doctors in surgery, traumatic, and

subordinate departments. This will certainly help hospitals to use more techniques

at district hospitals. For the policies’ implementation to strengthen health

workforce, especially for health care at the district level, the localities recruited

local staff and organized plans to maintain regular trainings to strengthen the

capacity of health staff.

83

• Quality of care. The model of technology transference among different health

complexity levels has paved a sustainable mechanism to develop capacity in the

provision of health care services and to constantly improve the quality of the

health care services at all levels, especially at the district level.

• Upgraded medical equipment. All provinces participating in the project

provinces have specific commitments to provide funds to maintain equipment and

upgrade and repair infrastructure, after the project closing. The infrastructure and

equipment will continue to be used and will ensure the quality in providing health

care services for the local people at district hospitals.

• Funding access to health for the poor. The project’s high sustainability was also

demonstrated through the health care policies for the poor/ethnic minorities

currently being applied. Infrastructure, equipment, and human resources training

supported by the project have worked well in providing quality health care

services to beneficiaries. Many provinces have mobilized funds to support the

local people to access and use health care services at district hospitals. Four of the

seven provinces have established provincial HCFPs. In the remaining three

provinces, where the Decision 1454 is not implemented, the Government allocated

partial funds (of transportation and meals) for the ethnical minorities and the poor

who need to be served by district hospitals.

• Other aspects. The IEC materials developed by the project and the increased

management capacity of the HCFP have been essential to support the provinces to

implement the program effectively and transparently. These are considered as

sustainable contributions for health care activities for the poor.

10. Lesson learned

58. The NUP project is designed with specific activities after careful consultations with

local stakeholders in the seven provinces. Right after coming in to effect, the project activities

are implemented, without any delays. Some projects are designed in the form of a project

framework; the project started with need assessments and specific activities are designed after

that, consequently, the project is implemented slowly.

59. The project is built bottom-up, based on local needs, and referred to the criteria and

standards issued by the MoH and Ministry of Finance (list of medical equipment, contents, and

training curricula). The NUP project does not develop its own cost norm or project

implementation guidance, but applies the existing regulations, so that project implementation

is fast, convenient, and highly sustainable.

60. Interventions of the NUP project are designed on both the supply and demand sides,

including supporting the provision of services (trainings, hospital renovations, providing

54 Decision 14, issued by the prime minister in 2012 on the revisions to Decision 139, issued in 2008 for the

health care for the poor.

84

equipment) and using services. This design brings high efficiency: the improvement in the

quality of health service, interacting with the support for the poor, and increase in the access

of the poor to health services.

61. The NUP’s support focused on northern mountainous region, with comprehensive

investments for district hospitals, and created equal development for the health sector in the

northern mountainous region. Therefore, the project implementation is far more favorable in

comparison with the projects where targets are scattered in different regions across the country.

62. The NUP project focused on the district level, at the grassroots level of health of

Vietnam (from the district level and below). This is one of the few investments in health that

support the district level in Vietnam. The project targeted the districts of the most difficult

areas in Vietnam, focused on primary health care, maternal, and child health care, which are

very important factors that contribute to pursuing the MDGs and the UHC in Vietnam today

and up to 2035. The effectiveness of these investments is significantly observed. The project

interventions (the support for long-term training, the IEC activities on health and support for

the poor, and so on) should be continued, using regional and local funds to ensure that the

project achievements will be strengthened and sustained.

63. The project assisted localities to develop their health plans and implement preparation

steps for maintaining the activities after the project ended and to continue the issuance of

policies and plans on health support for the northern mountainous provinces. Particularly, two

aspects could be highlighted: (a) the health workforce development policy and (b) the retention

and training of a village midwives team to serve remote areas where home birth delivery is

necessary.

64. A number of activities, for example, procurement of equipment or training of staff,

although the work plan has been approved a year before, are still reconfirmed with local needs

(quantities and types of training/equipment) before deployment. This ensures that the project

investments are appropriate and meet the real needs from district hospitals.

65. The capacity of PPMUs plays an important role in implementing the project. The staff

of the PPMUs should be stable, be well trained in project management, and work closely with

the technical support and regular supervisions from the central project. Although the CPMU

has only 20 staff (much less than many other projects of the same scale), the project activities

were well implemented, the schedule and work plan were always on track, and the

targets/results were surpassed in comparison to those assigned at the start of the project.

66. The support for the poor to access health care services is very well implemented in the

seven NUP provinces. Experience in implementing and critical results gained from the NUP,

is evidenced, and driven to replicate this policy nationally, through Decision No. 14 of the

Government. The policy of providing HICs to the poor should be continued to achieve

universal HI.

67. The very close collaboration and helpful support from the World Bank office in Hanoi,

by quickly exchanging information and solving the difficulties, which was the responsibility

85

of the TTL and relevant World Bank staff greatly contributed to the achievements and success

of the NUP.

86

Annex 6A. Project Risk Management (Borrower’s View)

Risks Level of

Risks Measures to Overcome

The non-poor group could

benefit from the intervention

of the project, leading to

rising inequality

Medium The project developed a good management system to

control these risks by coordinating well with Vietnam

Social Insurance in the provinces for review and

synthesis of the project beneficiaries annually. The list of

the poor and ethnic minorities who were qualified for

support from the NUP was provided to the district

hospitals. To receive the NUP support, the patients

should present the insurance card with the code ‘HN’ or

the certification of poor from the communal people’s

committee

The project implementation in

the district hospitals may not

be synchronized with the

commune health centers,

which could misuse the funds

of the HI system.

High Because the project investment and health care support

for poor people was focused at the district level, a large

part of the poor, instead of seeking health checkups at the

commune health center, tended to go to the district

hospital to access and use the support of the project,

compromising the use of the HI fund at the commune

level due to the impact from the abovementioned patient

flow. The project had controlled this risk well by

supporting only inpatients and setting the cap for the

length of stay (10 days) to avoid overuse of the support.

Uneven implementation

progress across project

components undermines the

integrated health systems

development approach

Potential mismatch between

the timing of demand-side

interventions that are expected

to boost demand for services

and investments

The project developed a yearly work plan to ensure that

the activities were implemented in an appropriate

sequence. Closely monitoring the implementation

progress to ensure coordination between components.

The completion of civil works in the first two years has

created a favorable condition for installing the medical

equipment. The trainings for health staff on using and

maintaining the different types of equipment were

conducted before and immediately after the equipment

was provided. A four-year training to upgrade assistants

to become medical doctors for commune health stations,

was implemented in 2011, when many doctors from the

district level graduated and covered for the commune

heath center. The IEC support for the poor and training

for district hospitals to manage funds for the poor were

implemented in parallel with others.

Staff not qualified for

entrance exams for level-1

specialists

Medium The project supported the staff preparation for entrance

exams.

The staff did not return to the

district hospitals to work after

graduation

Medium Recruitment of staff training was done in accordance with

local needs and selection criteria of the project. Staff

signed a terms of responsibility to return to their origin

district hospitals’ workplaces before going for training.

The certificate was issued only after staff completed the

training and returned to work in the district hospital.

Too many officers/staffs were

sent for training, affecting the

health services delivery at

health facilities

Medium The district hospital managed to schedule plans before

sending staff for training. Training courses were

organized to be spread throughout the project duration,

thus avoiding problems that could compromise the

service delivery process at the district hospitals.

87

Risks Level of


Otherwise, the project supported temporary staff transfers

among hospitals to help each other when their staff was

attending the training.

Delays in hospital civil works

could compromise the

investment plans

Medium Civil works investments were selected in district

hospitals where the area for construction is ready and

where PPMU and CPMU capacity to develop the

fiduciary tasks exist.

Project delays due to

specialized procurement,

tendering, and construction

tasks

High Procurement tasks followed the existing bidding laws and

instructions. The MoH conducted administrative reforms

in procurement, creating favorable conditions for faster

implementation. Project staff, and especially the team

working with the World Bank are trained and

accumulated experience in procurement before the

project started.

Medical equipment provided

to the district hospital by the

project are not used

effectively

Medium During the implementation, the team identified

investment needs to match the acquired equipment with

the social and epidemiological needs and organized

training and human resource development to use

equipment from the first year of the project. The team

also trained staff to carry out minor repairs, creating

favorable conditions for the installation and efficient

operation of the equipment provided.

Equipment is inadequately

maintained throughout the life

of the project

The civil works investments

in facility repairs and

refurbishments were not

maintained, leading to rapid

degradation of the facilities

Medium The project provides trainings for the hospital leaders on

hospital management, including the section of managing

the equipment. The hospital had plans and allocated

funds for maintaining/repairing equipment and facilities

based on the actual hospital needs. These tasks are

gradually improved because the hospitals are aware of the

important roles of regularly maintaining the equipment.

Unrecognized barriers

continue to limit access for the

poor, despite improved

financial access

Medium— The project regularly collected data on inpatient visits

who received support from the NUP. Data showed that

the number of beneficiaries gradually increased. The rate

of occupied hospital beds has increased to average 6%

per year. Another activity is the promotion of health-

seeking behavior through information dissemination and

other outreach activities regarding benefits and

entitlements under the HCFP. The improvement of

quality of the services also was a factor to attract the poor

who went to the hospitals for treatments

Coordinating capacity

strengthening for HCFP

management that builds on

existing systems and does not

create a duplicate

administrative system

supported by the project

Low The project has guidance for PPMUs and district

hospitals, which clearly states the roles and

responsibilities of the staff administering HCFP. The

management of HCFP has been strengthened during the

project life, through various support, including training

courses for the fund management unit, and support of

equipment to help fund management at the hospitals more

conveniently and rapidly. The examination and

monitoring of the health care for the poor has been

carried out in a systematic way, to ensure that the

88

Risks Level of


implementation is in accordance with the project’s

regulations.

The beneficiaries, not being

adequately informed of their

benefits, do not make

adequate use of their HICs

Low The project improved the quality of medical services and

conducted various types of behavior change

communication interventions to include messages geared

toward educating beneficiaries on health education, the

scheme and their benefits (support indirect costs in

addition to support from the state through the HCFP) to

poor families who went for the services.

89

Annex 6B. Project Results Framework

No. Outcome Indicators

Baseline

Survey

(2009)

MTR

(2012)

Final

Survey

(2014)

Extension

Phase

(2015)

Disparity

2009–

2014/2015

Targeted

Year 6

Indicator 1:

(original)

Utilization rates of inpatient

services in district hospitals

among Decision 139

beneficiaries (according to

household survey)

0.063 0.054 0.074 n.a. 117%

10%

increase

(supplement) Utilization rates of inpatient


among Decision 139


reports of district hospitals)

0.0247 0.049 0.085 0.096 389%

10%

increase

Indicator 2:

(original)

Utilization rates of outpatient


among Decision 139


household survey)

0.016 0.071 0.032 n.a. 200%

>15%

increase

(supplement) Utilization rates of outpatient


among Decision 139


reports of district hospitals)

0.067 0.162 0.206 0.247 369%

>15%

increase

Indicator 3: Percentage of households

which experienced

catastrophic health care

expenditures in the year prior

to the survey

10.4 7.1 2.0 n.a.

−81%

> 10%

decrease

Indicator 4: Percentage of health services

according to the national

norms that district hospitals

can implement

39.1 47.5 71.4 80.4 206%

70%

Result Indicators for Each Component

Component 1: Strengthening District-level Hospitals

Indicator 5: Percentage of patients

satisfied with health care

services at district hospitals

8.5 48.8 84.4 n.a. 993%

Increased

by 20%

Indicator 6: Adherence to treatment

protocols in treatment of

three common diseases

(MOH)

— — — — —

Increased

by 40%

Percentage of health staffs

with correct knowledge of

diagnosis and treatment for

ARIs

0.4 41.6 83.6 n.a. 20,900% —




severe pneumonia

13.2 58.1 86.9 n.a. 658% —


with correct knowledge of 9.7 65.5 95.2 n.a.

981%

—

90


Baseline

Survey

(2009)

MTR

(2012)

Final

Survey

(2014)

Extension

Phase

(2015)

Disparity

2009–

2014/2015

Targeted

Year 6


diarrhea level A/B/C




poisoning

26.8 53.8 83.0 n.a. 310% —

Percentage of ARIs medical

records with appropriate

diagnosis

18.5 38.6 62.2 n.a. 336% —

Percentage of severe

pneumonia medical records

with appropriate diagnosis

45.5 54.5 71.1 n.a. 156% —

Percentage of pneumonia

medical records with

appropriate diagnosis

19.6 20.9 57.0 n.a. 291% —

Percentage of diarrhea level

A/B/C medical records

appropriate diagnosis

41.1 69.4 81.6 n.a. 199% —

Percentage of poisoning

medical records appropriate

diagnosis

61.1 86.2 88.8 n.a. 145% —

Indicator 7: Percentage of district health

staffs trained by the project — — — — 145% 80%


completed short-term

training courses compared to

the plan

0.0 102.0 357.3 427.3 — —


completed long-term training

courses (specialty level-1s

doctors) compared to the

plan

0.0 54.9 88.0 189.2 — —

Total number of health staffs

trained in short-term and

long-term courses

0 2,664 8,929 11,868 — —

Indicator 8: Proportion of district

hospitals with acceptable

operations and maintenance

plans and budget for facility

and equipment maintenance

— — — — — 40%

Proportion of district



plans and budget for facility

49.2 30.8 99.3 85.9 175% —

Proportion of district



plans and budget for


77.1 61.5 99.2 93.8 122% —

Component 2: Increasing Financial Access to Healthcare Services for Decision 139 Beneficiaries

91


Baseline

Survey

(2009)

MTR

(2012)

Final

Survey

(2014)

Extension

Phase

(2015)

Disparity

2009–

2014/2015

Targeted

Year 6

Indicator 9: Percentage of Decision 139

beneficiaries provided with

HI cards

94.3 86.8 94.3 95.2 101% 70%

Indicator 10 Percentage of 139

beneficiaries with cards, who

can correctly identify at least

3 rights of the health

insurance card holders

14.8 19.8 57.4 n.a. 388% 75%

Indicator 10

(supplemental)

Percentage of 139

beneficiaries with cards, who

can correctly identify at least

one right of the health

insurance card holders

74.0 98.1 94.7 n.a. 128% —

Indicator 11: Percentage of households

who identify financial

barriers as a main cause for

not seeking health care

2.0 0.9 1.2 n.a. −40%

>10%

decrease

MDG indicators 2009 2014 2015 (%)

1 Percentage of district hospitals providing

caesarean section 74.6 82.1 92,2 124

2 Percentage of district hospitals providing blood

transfusion 56.7 67.2 82,8 146

3 Percentage of district hospitals having neonatal

units meeting standards — — 49,9 —

4 Capacity of district hospitals in obstetric and

pediatric emergency care — — — —

5 Having Continuous Positive Airway Pressure

devices 25.0 75.0 79.7 319

6 Having light for jaundice phototherapy treatment 21.4 82.1 87.5 409

7 Having oxygen breathing system 42.9 75.0 78,1 182

8 Having newborn resuscitator 46.4 92.9 92.2 199

MDG indicators 2007–2008 2013–2014 (%) 8 IMR 31.1 29.4 −5

9 NMR 11.2 10.8 −4

11 MMR 178 106 −40

92

Annex 6C. Achievements by Components

Table 6.1. Status of Health Human Resources at District Hospitals in Seven Project Provinces over the

Evaluation Periods

Health Human Resources Baseline

(2008)

Final

(2013)

Extension

Phase

(2015)

Disparity

Baseline –

Extension

Phase

Number of Health

Staffs who Returned

to Work after Being

Trained under

Support of the Project,

till 2015

Number of health staffs 599 1,097 1,286 687 (2.2 times) 1,329

Master, MD 1 11 17 16 0

Level-2 doctors 0 9 11 11 0

Level-1 doctors 148 325 396 248 (2.7 times) 351

General doctor 450 752 818 368 (1.8 times) 978

Postgraduate or above 27 87 116 89 (4.3 times) 33

Pharmacist (college,

secondary school) 207 486 554 347 0

Midwife 355 525 612 257 0

Nurse 1,124 2,059 2,238 1,114 0

Number of district

hospitals 61 67 68 — 68

Source: Reports of district hospitals.

Table 6.2. Number of Level-1 Doctors Trained in Project Provinces

Province Plan

Level-1 Doctors who Graduated

Level-1 doctors who

Continue to Work at

Localities

Total of

Level-1

Doctors

Supported

by the

Project

Total of

Person-

Years55 Late

2013

Late

2015

%

Female

%

Ethnic

Minority

Total %

Female

%

Ethnic

Minority

Cao

Bang 23 56 56 57 91 56 57 89 56 (243%)

112

(243%)

Bac Kan 31 36 50 50 98 44 57 95 50 (161%) 100

(161%)

Ha

Giang 38 89 98 68 59 95 69 57 98 (258%)

196

(258%)

Lao cai 22 38 50 38 20 49 39 24 51 (232%) 101(230%)

Lai Chau 22 17 18 28 50 18 28 50 19 (86%) 37 (84%)

Dien

Bien 19 47 50 50 44 50 50 44 50 (263%)

100

(263%)

Son La 28 26 53 42 21 53 42 21 53 (189%) 106

(139%)

Total 183 309 367 53 57 351 55 56 377(206%)

752

(205%)

93

Table 6.3. Number of Four-year Added Doctor and Pharmacist Graduated by the Evaluation Time

Cao

Bang

Bac

Kan

Ha

Giang

Lao

Cai

Lai

Chau

Dien

Bien

Son

La

Total

Support for four-year added doctor training

Total 185 82 302 208 182 250 258 1467

Number of doctors who graduated by late

2014

104 42 130 106 91 142 119 734

Person years supported by late 2014 505 191 788 544 577 714 690 4009

Converted to number of persons with full

four-year added training

126 48 197 136 144 178 173 1002

Number of doctors who graduated by late

2015

131 53 213 145 142 192 182 1058

Number of doctors who returned to work at

their units

122 44 208 96 136 193 179 978



four-year added training

147 58 240 162 167 206 207 1186

Plan achieved (%) 147 144 192 179 128 179 148 160

Rate of ethnic minority doctors 99.5 100 36.1 36.1 15.4 33.6 44.2 46.1

Rate of female doctors 98.9 50 32.1 40.9 43.4 29.2 39.9 45.1

Support for four-year added pharmacist training

Total 13 7 15 23 13 16 11 98

Rate of ethnic minority pharmacists (%) 93 86 31 17 7 6 18 31



four-year added training 7.8 2.8 7.0 14.3 7.3 7.3 4.8 51.0

Graduated by 2015 7 1 6 13 7 3 3 40



four-year added training 11 4.5 10.25 20.5 10.5 11.25 7.5 75.5

Rate of ethnic minority pharmacists 92.3 71.4 20 26.1 7.7 6.0 18.2 30.6

Rate of female pharmacists 86.4 71.4 60 82.6 69.2 50 72.7 70.4

Table 6.4. Number and Rate of Clinical Staff Attending Short-term Training Courses on Health

Examination and Treatment in the Extension Phase

Course Plan Total Percentage versus Project

Document

Anesthesia 117 157 134

Testing 95 149 156

Emergency care 140 152 109

External medicine 101 143 142

Pediatrics 103 214 208

Internal medicine 111 194 175

Diagnostic imaging 200 436 218

Obstetrics 102 191 187

Communicable diseases 80 165 206

X-ray 107 121 113

Ear, nose, and throat, and dental 108 70 65

Nursing management 70 63 90

Pediatric emergency n.a. 681 —

94

Course Plan Total Percentage versus Project

Document

Obstetric emergency n.a. 687 —

Total 1.334 3.423 257

Table 6.5. Number and Percentage of Doctors/Assistant Doctors Participating in Short-term Training

Courses on Mother and Child Health Care in the Extension Phase

Training Course

Planned Quantity

(Number of

Trainees per

Course)

Actual

Quantity

(Number of

Trainees per

Course)

%

Training to standardize the health workers who work

on obstetrics/gynecology be certified as SBA 35 22 80

Training to standardize the health workers who work

on obstetrics/gynecology be certified as SBA 600/30 courses 794/30 courses 113

Training to standardize the general doctors who work

on obstetrics/gynecology at district/commune be

certified as SBA

420/21 courses 307/12 courses 118

Emergency care for obstetrics//gynecology medical

doctors at district level 154/7 courses 110/5 courses 69

Village midwives 240/12 courses 252/12 courses 105

Training for clinical staff at neonatal units - district

hospitals

70–100/7–10

courses 79/6 94

Training for service providers on Integrated

Management of Childhood Illness 350/14 courses 140/6 courses 40

Table 6.6. Number of Doctors Trained on Management, Comparison between the Planned and the Actual

Achieved

Type of Training Planned

Quantity

Actual

Quantity % Achieved

Hospital management 603 753 125 Exceeded

Medical waste management 215 2,922 1,300 Exceeded

Health system management and HMIS 806 845 105 Exceeded

Maintenance of medical equipment 171 206 120 Exceeded

Effectively using the medical equipment provided by

the project — — — —

Effectively using laboratory equipment 35 35 100 Achieved

Exploring and using equipment for emergency care,

fluid vacuum, aerosol 7 7 100 Achieved

Effectively using disinfection equipment 1 31/1 100 Achieved

95

Table 6.7. Upgrading of 18 District Hospitals

No. Name of Facilities, Location Start

Date

Total Investment

(VND)

Completed

Date

Settlement

Date

1 Renovate, minor repair of Pac

Nam district hospital September

2010

884,641,819 January 2011

January 2012

2

Renovate, minor repair of

Ngan Son district hospital 780,595,945

3

Renovate technical house of

Bao Lam general district

hospital

March

2010

694,900,000

June 2010

June 2012

4

Renovate technical house and

toilet - Tra Linh general district

hospital

859,000,000

5

Inpatient house for external

and internal department -

Quang Uyen general district

hospital

3,750,000,000 March 2010–

March 2011

6

Inpatient house for external,

internal, and traditional

medicine department - Hoa An

general district hospital

July 2011 10,354,000,000 July 2012 July 2013

7 Renovate Muong Cha district

hospital

August

2009 2,480,318,000 June 2012 June 2012

8 Renovate Dien Bien Dong

district hospital

December

2010 11,469,000,000

December

2011

December

2012

9

Meo Vac general district

hospital: technical house and

supporting facilities

April

2010

4,531,446,000

April 2011 April 2012 10

Quan Ba general district

hospital: technical house,

toilet, and water tank

3,602,837,534

11

Hoang Su Phi general hospital:

outpatient examination house

and administrative

5,020,981,190

12 Minor repair of Phong Tho

district hospital

November

2009

163,940,587

December

2009

December

2010 13

Minor repair of Than Uyen

district hospital 617,135,748

14 Minor repair of Tan Uyen

general district hospital 163,194,151

15 Build high-tech house for Than

Uyen District Health Centers May 2011 8,525,000,000 May 2012 May 2013

16

Repair and upgrade building

for technical, pharmacy

department, nutrition

department of Moc Chau

general district hospital

November

2011 12,937,587,000

November

2012

November

2013

17

Renovate consultation

department and emergency

resuscitation department of

Mai Son general district

hospital

December

2011 907,320,000

September

2012

September

2012

96

No. Name of Facilities, Location Start

Date

Total Investment

(VND)

Completed

Date

Settlement

Date

18

Renovate technical house of

Muong La general district

hospital

December

2012 1,249,290,000

December

2012

September

2012

Total 68,991,187,974

Table 6.8. Bids Procured by CPMU

No Code Name of Bids Quantity Total Cost

(US$) Notes

1 ICB 01 Equipment and technology (6 types) 278 1,839,775.25

2 ICB 02 Equipment for intensive care (8 types) 873 1,019,763.17

3 ICB 03 Monitor (3 kinds) 142 845,600.00

4 ICB 04 Ventilator (2 kinds) 112 1,281,178.00

5 ICB 05 Sterilization device (3 kinds) 160 1,449,028.38

6 ICB 06 Furnaces for medical waste treatment 39 Canceled

7 ICB 07 Ambulance (52 pieces) 52 3,071,809.16

8 ICB 08 Working cars (8 pieces) 8 908,794.77

9 ICB 09 Communication equipment (11 kinds) 140 524,408.00

10 ICB 10 Ultrasound (2 types) 109 1,471,900.00

11 ICB 11 Endoscopic systems (2 types) 77 Canceled

12 ICB 12 Cardiopulmonary resuscitation (2 types) 366 386,805.00

13 ICB 13 Medical equipment (4 categories) 337 956,065.00

14 ICB 14 Operating room equipment (6 types) 396 4,257,544.49

15 ICB 15 Surgical instruments (7 types) 655 532,867.05

16 ICB 16 Special equipment (6 types) 365 2,198,896.00

17 ICB 17 X-ray (2 types) 102 1,309,275.00

18 ICB 18 Laboratory equipment (9 categories) 565 1,095,884.64

19 ICB 19 Basic equipment (19 kinds) 115 467,885.41

20 ICB 20 Monitor device (5 categories) 94 764,210.00

21 NCB 01 Ambulance (7 pieces) 7 393,180.20

22 ICB 21 Pediatric surgical equipment and

gynecology (6 types) 174 589,105.60

23 ICB 22 Emergency resuscitation equipment and

obstetrics (7 types) 160 529,963.96

24 ICB

01/2015

Model for trainings on obstetric care and

clean package for delivery, tool bag for

traditional birth attendants s

1.276.708,92

Total: 25,893,939.08

97

Table 6.9. Total Times of Inpatients Supported by NUP, by Provinces, 2009–2014

Province 2009 2010 2011 2012 2013 QI/2014

Dien Bien 411 1,779 3,814 16,889 30,058 7,524

Lai Chau — 1,264 4,205 13,605 18,132 4,716

Son La — 875 2,617 24,856 44,195 10,106

Cao Bang 186 4,853 18,129 44,248 45,044 10,533

Bac Can — 3,654 12,769 26,975 27,756 7,088

Ha Giang 30 3,457 17,727 38,781 39,702 11,231

Lao Cai 427 3,508 20,252 35,345 39,914 8,020

Total 1,054 19,390 79,469 200,709 244,801 59,218

Table 6.10. Number of Poor/ethnic Minority People being supported from the HCFP in the Project

Provinces - Extension phase

No. Contents Son La Ha Giang Lai Chau Lao Cai Total

1 Total times

poor/ethnic

minority

people

were

supported

52,597 82,676 46,147 87,985 269,405

Supported

travelling

expenses

51,174 82,676 305 57,354 191,509

Supported

meal

expenses

52,597 82,676 36,907 87,985 260,165

Supported

direct

expenses

for health

care

3,896 — 8,935 45,683 58,514

2 Total

expenses

supported

by the

HCFP

(VND)

16,175,884,199 33,397,067,456 30,185,983,270 19,128,174,711 98,887,109,636

Supported

travelling

expenses

5,007,080,521 — 1,138,446,108 2,213,686,966 8,359,213,595

Supported

meal

expenses

10,843,276,112 — 4,998,535,291 16,203,882,250 32,045,693,653

Supported

direct

expenses

for health

care

325,527,566 — 24,049,001,871 710,605,495 25,085,134,932

98

Annex 7. NUP (P082672) Project Supervision Missions Datasheet

Year

Number of

Missions, Dates, and

TTLs

Provinces

Visited During

the Missions

Main Issues Raised and Solved during the Missions

2009 2 (May and

November)

TTL: Bakhuti

Shengelia

2 provinces

Ha Giang, Dien

Bien

May: Review project implementation progress by components; discuss the baseline survey; discuss on how to

accelerate implementation of work plan; review arrangements for the implementation of ‘Health Care for the Poor’

component at the district level; and finalize training arrangements.

June: Review the implementation of 2009 plans; discuss 2010 action plans, related procurement plans, and the

measures to improve the effectiveness of implementation; assess the participatory planning process under the HEMA

Project and review M&E arrangements; conduct post review of procurement activities in 2009 and the financial

management practices; and conduct an implementation assessment in Dien Bien province.

2010 2 (June and

December)

TTL: Bakhuti

Shengelia

2 provinces: Lai

Chau, Son La

(for financial

management

review)

June: Review the implementation progress of 2010 annual work plans; review implementation of the

recommendations of November 2009 supervision mission; review procurement and financial management processes

at the CPMU and PPMUs; update implementation and procurement plans as needed; and assess developments in the

health policy field that may have implications for the projects.

December: Review the implementation progress of 2010 annual work plans; review implementation of the

recommendations of November 2009 supervision mission; review procurement and financial management processes

at the CPMU and PPMUs; update implementation and procurement plans as needed; and assess developments in the

health policy field that may have implications for the projects.

2011 2 (May and October)

TTLs: Bakhuti

Shengelia and Kari

Hurt

Note: An additional

safeguard review

mission in December

2011

4 provinces

Bac Can, Cao

Bang, Lai Chau,

Lao Cai

May: Review the general assessment of the project progress by component and activities; review procurement,

financial management, and environmental safeguards; and review social safeguards.

October: Discuss the progress toward achieving the DOs of increased utilization particularly by the poor in the project

provinces and strengthen district hospitals; review the implementation progress of 2011 annual work plans and look at

the priorities for the 2012 annual work plans; document the progress against the recommendations of December 2010

supervision mission; assess procurement and financial management processes at the CPMU and PPMUs; update

implementation and procurement plans as needed; and support progress of the current project implementation issues,

particularly concerning the HCWM plan and investments in support of the hospital plans and the implementation of

pilots for retention of rural health professionals in the project provinces.

2012 1 (July - MTR

mission)

TTL: Kari Hurt

3 provinces

Ha Giang, Son

La, and Dien

Bien (for

financial

management

only)

Review the general assessment of the project progress by component and activities; review procurement, financial

management and environmental safeguards, review social safeguards. Evaluation of the Project Results Matrix as part

of the MTR.

99

Year

Number of

Missions, Dates, and

TTLs

Provinces

Visited During

the Missions

Main Issues Raised and Solved during the Missions

2013 2 (May and

December)

TTL: Thuy Anh

Nguyen *December mission

for financial

management review

only.

4 provinces

Son La (May),

Son La, Lai

Chau, Dien Bien

(December for

financial

management

review)

May: Review the general assessment of the project progress by component and activities; time remaining for

disbursement, M&E, procurement medical equipment, financial management, and environmental safeguards; and

review social safeguards.

December: Update the World Bank’s understanding on the financial management arrangements for the project since

last financial management supervision missions (Son La in November 2012, Lai Chau in May 2013, and Dien Bien in

April 2012) which covers all areas of financial management, including planning and budgeting, disbursement, funds

flows, accounting system and software, reporting, and auditing. The mission will also follow up on all outstanding

issues raised during the previous mission and in the 2012 audited financial statements.

2014 2 (January and

September**)

3 provinces:

Dien Bien

(April),

Cao Bang, Bac

Can (September

2013)

January: Review the general assessment of the project progress by component and activities; time remaining for

disbursement, M&E, procurement medical equipment, financial management, and environmental safeguards; and

review social safeguards.

September: interim mission. No official announcement was sent.

2015 2 (May and

December**)

No site visit May: Key issues for ministry and project management attention, progress and key issues by component and

subcomponent; M&E; financial management and disbursement arrangements; procurement, safeguards

December: Review the implementation progress of the project by components and actions agreed and concluded in

the aide memoires from mission in May, 2015, to discuss and agree the final actions for project closing in February;

to discuss the preparation for the final implementation support mission combined with the ICR mission that is planned

in late February/early March 2016.

2016 1 (February) Lai Chau; Lao

Cai

Closing Mission - Discussion of the ICR assessment

Notes:

*There are also procurement post review missions, at least one every year.

** No aide memoires for September 2014 mission because of the TTL’s sudden sickness and December 2015 mission as it may be combined with the ICR

mission which is two months later.

100

Annex 8. List of Supporting Documents

1. Bill and Melinda Gates Foundation.2016. Maternal, Newborn and Child Health Strategy

Overview, in http://www.gatesfoundation.org/What-We-Do/Global-

Development/Maternal-Newborn-and-Child-Health.

2. Ekman, B; L. Thanh, H. A. Duc, and H. Axelson. 2008. “Health insurance reform in

Vietnam: A Review of Recent Developments and Future Challenges.” Health Policy and

Planning 23:252–263, doi:10.1093/heapol/czn009.

3. Center for Environmental and Health Studies. 2014. Report on the Final Evaluation of

the Northern Uplands Health Support Project, Hanoi, July 2014.

4. Center for Environmental and Health Studies. 2016. Report on the Final Evaluation of

the Northern Uplands Health Support Project in the Extension Phase, Hanoi, February

2016.

5. Fosberg, L.T,.2011. The Political Economy of Health Care Reform in Vietnam, Oxford –

Princeton Global Leaders Fellow, Woodrow Wilson School of Public and International

Affairs, Princeton University, 2011.

6. World Bank. 2011. Country Partnership Strategy for the Republic of Vietnam, IBRD,

IDA, Report No. 62500-VN.

7. World Bank. 2006. Mekong Regional Health Support Project. Project Appraisal

Document. Washington DC, February 9, 2006.

8. World Bank. 2012. Mekong Regional Health Support Project Implementation

Completion Report Results, Washington DC, December 2012.

9. World Bank. 2008. Northern Upland Regional Health Support Project, Project Appraisal

Document, The World Bank, Washington DC, February 2008.

10. Government of Vietnam, Ministry of Health. Annual Health Statistics Year Book, Hanoi,

Series from 2007–2014.

11. Government of Vietnam. 2012. Ministry of Health. Master Plan on Universal Health

Insurance Coverage, draft, Hanoi, June, 2012;

12. Government of Vietnam, Ministry of Health. 2015. Success Factors for Women’s and

Children’s Health, Ed. WHO, Geneva, 2015

13. Lieberman, S. and Wagstaff, A. 2009. Health Financing and Delivery on Vietnam –

Looking forward. Health, Nutrition and Population Series, The World Bank, Washington

DC, 2009.

14. Tiberti, L. 2000. Health Sector Reform in Transition Economies: The Case of Vietnam,

University of Florence, Florence, 2000.



101

15. Toan, Tran Khanh .2012. Antenatal and Delivery Care Utilization in Urban and Rural

Contexts of Vietnam: A study in two health and demographic surveillance sites, Doctoral

thesis at the Nordic School of Public Health NHV, Gothenburg, Sweden, 2012.

16. World Health Organization .1999. Mother-Baby Package Costing Spreadsheet, Geneva,

Switzerland, 2012.

REPORTS PRODUCED BY THE GOVERNMENT TO SUBSIDIZE THE PROJECT

PREPARATION

1. Health status in the 7 provinces of Northern Upland

2. Health care service use and accessibility status in the 7 provinces of northern upland: Cao

Bang, Bac Kan, Lao Cai, Ha Giang, Son La, Dien Bien and Lai Chau)

3. Health Human Resource Analysis in the 7 provinces of northern upland - Health care for

the Poor: Identification of the needs and proposal of investment for capacity building and

management capacity development

4. Assessment of the Healthcare Fund for the Poor in the 7 provinces of northern upland -

Health care for the poor: management according to the Decision 139 in 7 provinces of

northern upland (most difficult provinces)

5. Health System Assessment for 7 provinces of northern upland - Inventory of medical

equipment in hospitals of 7 provinces of northern upland

6. Output indicators after analysis - Socioeconomic, Demographic, Cultural Geographic and

Health Status indicators: Morbidity, Mortality, CDR/IMR, Under 5-child nutrition status

7. List of medical staff to be trained and cost estimate and cost table for training component

102

Map

Source: World Bank Maps

Document of The World Bankdocuments.worldbank.org/curated/en/489571473188281827/pdf/... · document...

Documents

Transcript of Document of The World Bankdocuments.worldbank.org/curated/en/489571473188281827/pdf/... · document...