documents.worldbank.org · Web view2016/06/29 · APLAdaptable Program Loan. AUSAseguramiento...

Peru - Second Phase of Health Reform Program - Implementation Completion and Results Report

Document of The World Bank

Report No: ICR00003691

IMPLEMENTATION COMPLETION AND RESULTS REPORT(IBRD-76430)

ON A

LOAN

IN THE AMOUNT OF US$15.0 MILLION

TO THE

REPUBLIC OF PERU

FOR A

SECOND PHASE OF THE HEALTH REFORM PROGRAM

June 30, 2016

Health, Nutrition and PopulationBolivia, Ecuador, Peru and Venezuela Country Management UnitLatin America and the Caribbean Region

CURRENCY EQUIVALENTS

(Exchange Rate Effective December 31, 2015)

Currency Unit = Peruvian Nuevo Sol

PEN 300.35 = US$ 1.00

US$ 1.00 = PEN 3.33

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

APLAdaptable Program Loan

AUSAseguramiento Universal en Salud – Universal Health Insurance

CPSCountry Partnership Strategy

CREDControl de Crecimiento y Desarrollo

CRVSCivil Registration and Vital Statistics

DIADerecho a la Identidad y Aseguramiento

DIRESADirección Regional de Salud

EAEnvironmental Assessment

EDAExtreme Diarrhea

ENDESEncuesta Demográfica y de Salud Familiar

EPPESEstrategia de Promoción de Práctica y Entornos Saludables

FSFeasibility Study

GOPGovernment of Peru

IADBInter-American Development Bank

ICRImplementation Completion and Results Report

IMRInfant Mortality Rate

INTDepartment of Institutional Integrity

IOIIntermediate Outcome Indicator

IPPIndigenous People Plan

ISRImplementation Status and Results Report

KPIKey Performance Indicator

M Million

M&EMonitory and Evaluation

MAManagement Agreement

MEF Ministerio de Economía y Finanzas – Ministry of Economy and Finance

MINSAMinisterio de Salud de Perú – Ministry of Health of Peru

MMRMaternal Mortality Rate

PADProject Appraisal Document

PARSALUD Programa de Apoyo a la Reforma del Sector Salud

PDOProject Development Objective

PIUProject Implementation Unit

PRONIS Programa Nacional de Inversión de Salud

QUALYQuality Adjusted Life Years

RFResults Framework

SIAFSistema Integrado de Administración Financiera

SISSeguro Integral de Salud – Comprehensive Health Insurance

SNIPSistema Nacional de Inversión Publica

SUNASA Superintendencia Nacional de Salud

Senior Global Practice Director:Timothy G. Evans

Practice Manager:Daniel Dulitzky

Project Team Leader:Andre Medici

ICR Team Leader/Author:Federica Secci

PERU

Second Phase of the Health Reform Program (P095563)

TABLE OF CONTENTS

Data Sheet

B. Key Datesiv

C. Ratings Summaryiv

D. Sector and Theme Codesv

E. Bank Staffv

F. Results Framework Analysisv

G. Ratings of Project Performance in ISRsx

H. Restructuringx

I. Disbursement Profilexi

1. Project Context, Development Objectives and Design1

2. Key Factors Affecting Implementation and Outcomes5

3. Assessment of Outcomes11

4. Assessment of Risk to Development Outcome21

5. Assessment of Bank and Borrower Performance22

6. Lessons Learned26

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners27

Annex 1. Project Costs and Financing28

Annex 2. Outputs by Component29

Annex 3. Economic and Financial Analysis38

Annex 4. Bank Lending and Implementation Support/Supervision Processes44

Annex 5. Beneficiary Survey Results46

Annex 6. Stakeholder Workshop Report and Results47

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR48

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders69

Annex 9. List of Supporting Documents70

Annex 10: Analysis of PDO Achievement71

Annex 11: Loan Amount Allocation75

Annex 12: Organization of PARSALUD II Project Implementation Unit (PIU)76

MAP77

Data Sheet

A. Basic Information

Country:

Peru

Project Name:

PE- (APL2) Health Reform Program

Project ID:

P095563

L/C/TF Number(s):

IBRD-76430

ICR Date:

06/30/2016

ICR Type:

Core ICR

Lending Instrument:

APL

Borrower:

GOVERNMENT OF PERU

Original Total Commitment:

USD 15.00M

Disbursed Amount:

USD 11.98M

Revised Amount:

USD 15.00M

Environmental Category: B

Implementing Agencies: PARSALUD

Cofinanciers and Other External Partners: Inter-American Development Bank (IADB)

B. Key Dates

Process

Date

Process

Original Date

Revised / Actual Date(s)

Concept Review:

11/02/2005

Effectiveness:

12/14/2009

12/15/2009

Appraisal:

12/11/2008

Restructuring(s):

06/20/2011

08/25/2014

Approval:

02/17/2009

Mid-term Review:

10/30/2012

02/26/2013

Closing:

01/31/2015

12/31/2015

C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes:

Moderately Satisfactory

Risk to Development Outcome:

Moderate

Bank Performance:


Borrower Performance:


C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank

Ratings

Borrower

Ratings

Quality at Entry:

Moderately Unsatisfactory

Government:

Moderately Unsatisfactory

Quality of Supervision:


Implementing Agency/Agencies:


Overall Bank Performance:


Overall Borrower Performance:


C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance

Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No

Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

No

Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original

Actual

Sector Code (as % of total Bank financing)

Compulsory health finance

4

4

Health

62

62

Public administration- Health

34

34

Theme Code (as % of total Bank financing)

Child health

30

30

Health system performance

30

30

Indigenous peoples

10

10

Population and reproductive health

30

30

E. Bank Staff

Positions

At ICR

At Approval

Vice President:

Jorge Familiar

Pamela Cox

Country Director:

Alberto Rodriguez

Carlos Felipe Jaramillo

Practice Manager:

Daniel Dulitzky

Keith E. Hansen

Project Team Leader:

Andre C. Medici

Fernando Lavadenz

ICR Team Leader:

Federica Secci

ICR Primary Author:

Federica Secci

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document)

Framed within the long-term objective of the Health Reform Program (APL in all its phases) to reduce maternal and infant mortality rates in Peru’s nine poorest regions of the country (Amazonas, Huánuco, Huancavelica, Ayacucho, Apurimac, Cusco, Cajamarca, Ucayali, and Puno); the specific APL 2 Project Development Objectives are to: (i) improve family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three; (ii) strengthen health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three; and (iii) support MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment (same as in the Loan Agreement).

Revised Project Development Objectives (as approved by original approving authority)

The PDOs, the Key Performance Indicators (KPIs) and the outcome targets were not revised. However, some targets were revised during the 2014 project restructuring for the Intermediate Outcome Indicators (IOIs) (see below).

(a) PDO Indicator(s)

Achievement of targets was assessed as follows:

· Target Surpassed: the value of the indicator at the end of the project (December 2015, based on 2014 data) is higher than the target

· Target Achieved: the value of the indicator at the end of the project (December 2015, based on 2014 data) is equal to the target, or it is equal to or greater than 85% of the target

· Target Partially achieved: the value of the indicator at the end of the project (December 2015, based on 2014 data) is equal to or greater than 65% and lower than 85% of the target

· Target Not Achieved: the value of the indicator at the end of the project (December 2015, based on 2014 data) is equal to or lower than 64% of the target

· Not evaluated: no data is available to assess the achievement of the target and/or the value of the indicator.

Indicator

Baseline Value

Original Target Values (from approval documents)

Formally Revised Target Values

Actual Value Achieved at Completion or Target Years

Overall program indicator: Reduce infant mortality rate

Value:

42

25

17

Date:

2005

January 2015 (2013 data)

December 2015

(2014 data)

Comments: Target Surpassed

Overall program indicator: Reduce chronic malnutrition of children under the age of 5

Value:

38.2

30.2

23.7%

Date:

2005


December 2015

(2014 data)


KPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2013) – proxy indicator for Maternal Mortality

Value:

44%

78%

74.2%

Date:

2005


December 2015

(2014 data)

Comments: Target Achieved (89%).

KPI #2: Reduce the prevalence of anemia among children under age in the nine regions from 69.5% (2005) to 60% (2013)

Value:

69.5%

60%

57.3%

Date:

2005


December 2015

(2014 data)


KPI #3: Increase from 64% to 80% the share of children in the nine selected regions who are exclusively breastfed until 6 months of age

Value:

64%

80%

87%

Date:

2005


December 2015

(2014 data)


KPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2013)

Value:

41.5%

35%

36.4%

Date:

2005


December 2015

(2014 data)

Comments: Target Partially Achieved (78%).

KPI #5: Reduce the hospital lethality rate among neonates in the nine selected Regions from 9.5% (2005) to 5% (2013)

Value:

9.5%

5%

5%

Date:

2005


December 2015

(2014 data)

Comments: Target Achieved (100%).

KPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2013)

Value:

20%

45%

69.1%

Date:

2005


December 2015

(2014 data)


(b) Intermediate Outcome Indicator(s)

The rating scale is the same as the one above used for the PDO Indicators.

Indicator

Baseline Value

Original Target Values (from approval documents)

Formally Revised Target Values

Actual Value Achieved at Completion or Target Years

IOI #1: Percentage of SIS affiliated children who received growth and development controls (CRED) according to their age

Value

34%

66%

56.8%

Date

2005


December 2015

(2014 data)

Comments: Target Partially Achieved (71%).

IOI #2: Percentage of health facilities with improvement in infrastructure (minor construction, rehabilitation and/or new equipment)

Value

0

104

Dropped

104

Date

2005


2014

December 2015

(2014 data)

Comments: Target Achieved (100%) (dropped at the 2014 restructuring but reintroduced for the ICR, based on government’s data).

IOI #3: Percentage of SIS affiliated rural pregnant women with laboratory tests on hemoglobin, urine and syphilis

Value

37%

80%

53.5%

69%

Date

2005


December 2015

(2014 data)

December 2015

(2014 data)

Comments: Surpassed against revised target; 74% achievement against original target.

IOI #4: Percentage of pregnant women under SIS that receive iron and folic acid supplements

Value

37%

80%

60%

55%

Date

2005


December 2015

(2014 data)

December 2015

(2014 data)

Comments: Revised Target Partially Achieved (78%); 42% achievement against original target.

IOI #5: Percentage of women satisfied with the services in selected facilities by confidence index

Value

N/A

25%

Dropped

74.4%

Date

2005


2014

December 2015

(2014 data)

Comments: Surpassed (dropped at 2014 restructuring, but reintroduced for the ICR). Based on ISR data, the target was 25% and the value after the survey was conducted was 74.4%.

IOI #6: Percentage of cesareans in SIS affiliated pregnant rural women

Value

3%

5%

10%

9.5%

Date

2005


December 2015

(2014 data)

December 2015

(2014 data)

Comments: Revised Target Achieved (93%); surpassed against original target.

IOI #7: Percentage of references among SIS-affiliated women (during pregnancy and puerperium) and neonates

Value

N/A

5%

Dropped

N/A

Date

2005


2014

December 2015

(2014 data)

Comments: Not evaluated. Indicator not monitored throughout the project and dropped at restructuring (2014)

IOI #8: Percentage of SIS affiliated households that make out-of-pocket expenditures in medicines

Value

67.3%

25%

55%

56%

Date

2005


December 2015

(2014 data)

December 2015

(2014 data)

Comments: Revised Target Achieved (92%); 27% achievement against original target.

IOI #9: Number of accredited health establishment by type of resolution

Value

80

169

Dropped

N/A

Date

2005


2014

December 2015

(2014 data)


IOI #10: Number of Management Agreements in place

Value

N/A

N/A

Dropped

N/A

Date

2005


2014

December 2015

(2014 data)


IOI #11: Number of health personnel and community health workers trained within the behavior change campaign (EPPES)

Value

0

758

1178

Date

2005


December 2015

(2014 data)

Comments: Surpassed (added for the ICR - based on Government data)

IOI #12: Number of health facilities improved

Value

0

73

69

Date

2005


December 2015

(2014 data)

Comments: Achieved (95%) (added for the ICR - based on Government data). This indicator measured the number of facilities that were either built, re-built or expanded by the project (i.e. major constructions).

IOI #13: Norms and regulations to improve efficiency and equity of the health delivery system prepared

Value

0

27

32

Date

2005


December 2015

(2014 data)


IOI #14: Clinical pathways and corresponding financing systems designed

Value

0

11

16

Date

2005


December 2015

(2014 data)


IOI #15: Periodic evaluations of the performance of the health networks prepared

Value

0

17

19

Date

2005


December 2015

(2014 data)


G. Ratings of Project Performance in ISRs

No.

Date ISR

Archived

DO

IP

Actual Disbursements

(USD millions)

1

06/19/2009

Satisfactory

Satisfactory

0.00

2

12/11/2009

Satisfactory

Satisfactory

0.00

3

06/28/2010

Satisfactory

Satisfactory

0.00

4

02/23/2011

Satisfactory

Satisfactory

0.86

5

08/05/2011

Satisfactory

Satisfactory

1.20

6

01/23/2012

Satisfactory

Satisfactory

3.48

7

09/22/2012

Satisfactory

Satisfactory

4.05

8

05/10/2013

Satisfactory


5.65

9

12/21/2013

Satisfactory


7.40

10

07/12/2014

Satisfactory


7.96

11

12/19/2014

Satisfactory


9.27

12

06/17/2015

Satisfactory


9.73

13

12/30/2015



11.98

H. Restructuring

Restructuring Date(s)

Board Approved PDO Change

ISR Ratings at Restructuring

Amount Disbursed at Restructuring in USD millions

Reason for Restructuring & Key Changes Made

DO

IP

06/20/2011

No

S

S

1.20

Reallocation of funds to increase funds allocated to Component 2 and reduce those for Components 1 and 3.

08/25/2014

No

S

MS

8.58

Based on the MTR, the restructuring (a) revised the results framework; (b) extended the Closing Date of the Project to Dec 31, 2015; (c) increased the threshold for firm contracts to US$300,000; and (d) changed the disbursement estimates.

I. Disbursement Profile

xi

1. Project Context, Development Objectives and Design 1.1 Context at Appraisal

1. Peru was a growing economy, but with persistent inequalities, and undergoing a demographic transition. At appraisal, it had registered strong economic growth, with a 7.6% GDP growth in 2006, 9.0% in 2007 and 9.3% in 2008. This progress contributed to a substantial reduction of poverty. The national poverty rate dropped from 48.6% in 2004 to 39.3% in 2007, while extreme poverty fell from 17.1% to 13.7%. However, inequalities and disparities across regions remained a challenge. Extreme poverty was 3.5% in urban areas and 32.9% in rural areas in 2007. From being heavily rural in 1950 with 33% of the population living in urban areas, by 2007 this proportion had increased to 76%. In terms of age structure, of a population of more than 27 million in 2007, 33% were less than 15 years old and 4.8% over 65.

2. By appraisal, Peru had advanced on some health-related MDG outcome indicators; yet, improvements were not uniform ― across all socio-economic groups, regions, and between rural and urban settings, revealing persistent inequalities. Despite its overall decrease, in 2006 the infant mortality rate (IMR) varied from 5 per 1,000 live births in the richest quintile to 45 in the poorest; Lima had a low IMR of 20, but Cusco has the highest at 84. While mortality in the post-neonatal period decreased, the relative share of perinatal mortality as a cause of infant deaths increased. This was due to conditions related to both demand and supply side factors (e.g. low institutional delivery rate, lack of immediate attention for newborns) and strongly linked with maternal malnutrition – more than a quarter of pregnant women, age 15 to 49, suffered from anemia (ENDES 2000-1). Despite progress on nutrition outcomes, one-quarter of Peruvian children under five suffered from chronic malnutrition, while 69% of children under two suffered from anemia. Located at high altitude, the regions of Huancavelica, Huánuco, and Ayacucho, among the poorest of Peru, were the ones with the highest stunting levels (more than 40 percent). At 164 deaths per 100,000 live births, Peru’s maternal mortality ratio (MMR) was almost double the Latin American average. In 2006, institutional delivery in urban areas was 92% and only 44% in rural areas. Finally, financial obstacles still represented a significant barrier to access. In the poorest quintile, 34% of individuals reported they had no access to health care for lack of money, while in the richest quintile only 6% did (ENAHO 2006).

3. Evolving health system. The Ministry of Health (Ministerio de Salud, MINSA) had taken some steps to strengthen accountability within a fragmented health care system in an increasingly decentralized environment. Management Agreements (MAs) were adopted to set goals for the Regions/municipalities’ health networks and results-based budgeting was increasingly used. The Comprehensive Health Insurance (Seguro Integral de Salud, SIS), created in 2001 and covering over 16% of the population, reimbursed MINSA public providers based on agreed upon health plans and covered predominantly vulnerable population living in poverty or extreme poverty, although not all in need.

4. Rationale for Bank assistance. The project evaluated in this ICR was part of a two-phase Adaptable Program Loan (APL) to support the Government of Peru (GOP)’s Health Reform Program (Programa de Apoyo a la Reforma del Sector Salud – PARSALUD). In both its phases, the overall PARSALUD program aimed at improving maternal and child health outcomes in Peru. The APL series was built on an ongoing dialogue with the Government of Peru (GOP) and on prior analytical work (e.g. RECURSO) and lending operations in the health sector (e.g. Basic Health and Nutrition Project – P008048).

5. The first phase (APL 1) in support of PARSALUD I was the Mother and Child Insurance and Decentralization of Health Services Project – P062932, which started in July 2001 and closed in June 2006. The planned investment under PARSALUD I amounted to US$239 M, jointly funded by the International Bank for Reconstruction and Development (IBRD, i.e. the Bank) (US$ 87 M), the Inter-American Development Bank (IADB) (US$ 87 M), the GOP (US$ 64.3 M), and the OPEC Fund (US$ 8 M); the total actual investment was US$232 M ― GOP (US$ 176.80 M), Bank (US$ 27 M), and IADB (US$ 28 M). The objective of the first phase of the program was to improve maternal and child health and to help reduce morbidity and deaths of the poor from communicable diseases and environmental conditions. The specific objective of the APL 1 was to increase access of the poor to better quality health programs and services. PARSALUD I was successful in reducing perinatal mortality and IMR and increasing skilled birth attendance by strengthening the demand and improving the quality of the supply of health programs and services. APL 1 contributed to the success of the overall program and its ICR (ICR000073) rated efficacy as substantial and the overall project as satisfactory ― confirmed by the IEG evaluation. Finally, a set of nine triggers was agreed on to demonstrate readiness for transition from phase I to phase II. All triggers were met at the end of phase I, with the exception of one, due to changes in regulatory framework, independent from PARSALUD[footnoteRef:1]. [1: A summary of triggers is reported in the PAD, pp 10-11. The trigger that was not achieved was the separation of financing and service provision within the Social Security Fund (ESSALUD). Changes in the regulatory framework granted more autonomy to ESSALUD and reduced the ability of MINSA to influence its institutional processes. Eventually, ESSALUD started piloting a separation of functions, but this was decided independently from the PARSALUD trigger.]

6. The project evaluated by this ICR (P095563) supported the second phase of the GOP’s PARSALUD and intended to sustain the achievements of the first phase. The overall objective of the second phase of the program was to reduce maternal and infant mortality and reduce chronic malnutrition of children under the age of 5. The specific objective of the APL 2 are described in section 1.2. The total planned investment under PARSALUD II amounted to US$162.40 M, funded largely by GOP (US$ 132.40 M) and co-funded by the Bank and the IADB (US$ 15 M each); the total actual investment was US$165 M ― GOP (US$ 138 M), Bank (US$ 11.98 M), and IADB (US$ 15 M). The project represented a small portion of domestic financing; yet, there was considerable demand from the GOP for the Bank’s support to policy reforms and for its fiduciary contributions to leverage an expeditious and efficient execution of policies and investments. The project also did provide additional funding for regional governments for investments and interventions not covered by budgetary allocations. Finally, the project was aligned with the national and sector policies and with the Bank’s Country Partnership Strategy (CPS) for Peru for FY12-16, as well as with the Ministry of Economy and Finance (MEF)’s efforts towards results-based budgeting.

1.2 Original Project Development Objectives (PDO) and Key Indicators

7. This project was the second of the APL series to continue supporting the broader GOP program (PARSALD II). The specific program objective was to reduce maternal and infant mortality and reducing chronic malnutrition of children under the age of 5. The program indicators were: MMR[footnoteRef:2], IMR and chronic malnutrition of children under 5. [2: Due to difficulties in monitoring MMR, institutional delivery was used as a proxy for maternal mortality by PARSALUD.]

8. The stated objectives of APL2 were to continue supporting the Borrower’s effort to reduce maternal and infant mortality rates in intervened rural areas in Selected Regions in the Borrower’s territory, in particular through: (i) the improvement of family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three; (ii) the strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three; and (iii) the supporting of MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment. The objectives were aligned between the Loan Agreement and the PAD.

The PDO-level indicators (Key Performance Indicators, KPIs #1-6) were:

a) Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2013)

b) Reduce the prevalence of anemia among children under age 3 in the nine regions from 69.5% to 60%

c) Increase from 64% to 80% the share of children in the nine selected regions who are exclusively breastfed until 6 months of age

d) Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2013)

e) Reduce the hospital lethality rate among neonates in the nine selected Regions from 9.5% (2005) to 5% (2013)

f) Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2013).

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification

9. Neither the PDO nor the PDO indicators were revised. However, as per restructuring paper dated August 20, 2014, the Results Framework (RF) was revised to reflect the new proposed end date of the project, which was moved from January 31, 2015 to December 31, 2015. The targets of some Intermediate Outcome Indicators (IOIs) were revised and some IOIs were dropped (IOIs # 2, 5, 7, 9, 10). No additional indicators were added.

1.4 Main Beneficiaries,

10. The main beneficiaries were meant to be families (preponderantly rural) with pregnant women and children under age of three in the nine poorest regions in the country: Amazonas, Huánuco, Huancavelica, Ayacucho, Apurimac, Cusco, Puno, Cajamarca and Ucayali. The last two regions were also prioritized due to slow advances on IMR and MMR. Particular attention was to be targeted on indigenous populations as part of vulnerable and poor groups. Other beneficiaries included health professionals who benefited from training and improved infrastructure; local health managers, who benefited from training, tools, and technical assistance; and MINSA who benefited from technical assistance for enhancing government capacities, regulatory framework for local services, and M&E functions.

1.5 Original Components

11. The project consisted of four components:

Component 1 (total estimated costs US$6.00 million). Improving health practices at the household level for women (during pregnancy, delivery and breastfeeding) and children under the age of three in rural areas of selected Regions (demand-side interventions), by: a) design, implementation and monitoring of a behavioral change communication and education program to promote healthy practices at the household level, including increased demand for health services (Estrategia de Promocion de Practica y Entornos Saludables, EPPES); and b) promotion of SIS enrollment rights and identity rights of the targeted population (Derecho a la Identidad y Aseguramiento, DIA).

Component 2 (total estimated costs US$142.30 million). Increasing the capacity to provide better maternal and child health services for the poor (supply-side interventions); through: a) the improvement of the quality of services in health facilities of the nine regions; and b) the provision of support for the integrated health delivery model and the development of support systems to raise the efficiency and effectiveness of health networks.

Component 3 (total estimated costs US$5.20 million). Strengthening government capacities to offer more equitable and efficient health system in a decentralized environment (governance and financing) by: (a) supporting a regulatory framework and increasing quality in the provision of health services, (b) expanding the health insurance system (SIS) enrollment; (c) strengthening data monitoring and accountability in the system; and (d) supporting the decentralization of health services.

Component 4 (total estimated costs US$8.9 million) Project Coordination and Monitoring and Evaluation (M&E), through the provision of technical assistance, financing of incremental operating costs, and external and concurrent audits.

1.6 Revised Components

12. During the 2011 restructuring, the reallocation of Loan proceeds among disbursement categories was revised, as shown in Annex 11. The contribution to Components 1 and 3 was halved (from 50% of program costs to 25%), while the contribution to Component 2 was increased (from 6% of program costs to 8%). No changes were made to Component 4.

1.7 Other significant changes

13. The project underwent two level 2 restructurings. The first, in June 2011, changed funding allocations among components and supported more timely disbursement. It was motivated by the fact that, at the onset of the project, the GOP used domestic resources to advance expenditures for technical assistance activities originally planned to be financed with the loan funds, given that the budget allocation for the loan had not yet been approved by the borrower.

14. The second restructuring, in August 2014:

a) Extended the Closing Date to December 31, 2015, to complete all Project activities and to ensure full disbursement of loan proceeds, focusing on component 2 and 3;

b) Revised the RF to increase clarity, improve the accuracy of indicator definitions and data, and revised project targets in line with available evidence and feasibility of achieving targets. The indicator target dates were also adjusted to the new Closing Date;

c) Changed disbursement estimates to reflect the new action plan and respective procurement plan, and

d) Increased procurement threshold for prior review for consulting firms from US$100,000 to US$300,000.

2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry

15. Project Design. The project design was based on the lessons of APL 1 outlined in its ICR (ICR000073). In particular, it was emphasized that: (1) the lack of a clear implementation strategy that would ensure results, among other factors, caused difficulties in the first phase of the program, which were overcome by using evidence-based research to focus on results; (2) political volatility was detrimental on project implementation and impact and, while ministerial influence was necessary; it was also important to implement project activities in coordination with the areas of MINSA responsible for specific project components; (3) there was a need to build institutional and managerial capacity at regional level before transfer funds directly to these local management units; and (4) intercultural strategies were vital when beneficiaries belonged to diverse indigenous groups and community participation was key to ensure sustainability. In addition, the second phase of the program, including its activities and the Results Framework (RF), was informed by a comprehensive feasibility study finalized in 2008 (with a baseline for the indicators taken in 2005).

16. Due to a strong interest of the GOP at the time of appraisal on infrastructure investment, the project was largely focus on upgrading health facilities in the nine regions, where lack of infrastructure was identified as an important barrier to service access. Yet, these infrastructure investments were used as entry point to promote broader sectoral reforms and the use of evidence-based policy. These “soft interventions”, which were at the core of PARSALUD I, were less prominent in PARSALUD II; however, they are fundamental in ensuring sustainability of progress.

17. Project Preparation. Preparation of APL 2 started promptly, even before the closing of APL 1. As was the case for the first phase, APL 2 was also co-financed by the World Bank and the IADB, and therefore, project preparation was conducted in close collaboration with the IADB team[footnoteRef:3]. However, a three years gap stands between the end of phase I and the effectiveness of the project supporting phase II. Changes in political priorities and leadership due to frequent changes in Government were the main causes of this delay. When a new Minister of Health was appointed in October 2008, an opportunity window opened up and project preparation regained traction. The project was negotiated in December 2008 and approved by the Board of Directors on February 17, 2009. The signing of the Loan Agreement only happened 9 months later, on November 16, 2009 and the project was declared effective on December 15, 2009. [3: The IADB project and the Bank project in support of PARSALUD II were aligned in terms of development objective, timeline and resources committed (US$15 million each). The results frameworks of the two projects slightly differed on some of the KPIs and IOIs. While the Mid-Term Review was conducted jointly and communications between the two teams was maintained throughout the life of the projects, supervision missions were largely carried out separately by the two co-founders.]

18. Quality Enhancement Review (QER) and Decision Review Meeting (DM). The project underwent a QER in May 2006; the DM was held in December 2008. During both meetings, the element of the project that was mostly appreciated by the reviewers was the cultural adaptation of all the planned activities. Some of the issues and recommendations raised during these meetings not only revealed to be crucial to determine quality at entry, but were eventually found to be critical during implementation. These included:

a) Establishing clearer links with the previous APL and, more broadly, better explaining how the new operation would fit within the Peruvian institutional environment, which had changed from the end of the first phase of the program and, even more, from its original conceptualization;

b) Strengthening the M&E system by: (i) reducing the number of indicators, but establishing a clear results chain from activities to outputs and outcomes; (ii) reducing the number of data sources from which the indicators would be derived; and (iii) ensuring that the counterpart had the capacity to monitor the RF;

c) Assessing more realistically the risks deriving from the institutional environment which are outside the scope of the project, especially with regard to the links with SIS and the dependence upon approvals from the National Investment System (Sistema Nacional de Inversión Pública, SNIP). During implementation, it became clear that SIS had gain much more independence than anticipated and coordination with PARSALUD gradually reduced. SNIP, which was created in Peru in the early 2000s, was rather rigid in its conceptualization of investment projects, with a non-participatory decision-making process and long approval time of programs, thereby compromising the possibility to efficiently make any changes to the projects during implementation. Furthermore, its clear preference for financing infrastructure investment left very little margin to incorporate other investment approaches, such as those adopted by APL 1, despite their proven effectiveness.

2.2 Implementation

19. Implementation of the project did not suffer from any major complications. The project had a slow start. While activities related to Components 1 and 3 started immediately, delays in disbursement and execution related mainly with the infrastructure investments under Component 2. Once construction works began and medical equipment started being purchased in 2011, project disbursement picked up towards the end of 2012.

20. As a result of the nature of the activities, project implementation was characterized by a high volume of transactions. Given the commitment to co-finance all civil works of the PARSALUD program with 6% funding coming from loan resources (i.e. 3% from the Bank and 3% from the IADB) and the remaining 94% from domestic resources, all procurement processes followed the Bank’s procedures. This was highly desirable from the perspective of MINSA, given the stricter Bank procurement guidelines, and it reflected a general trend in Peru at that time ― when the GOP was seeking external resources to fund rather small portions of broader national investment programs to benefit from streamlined procurement processes and technical assistance. Yet, this posed stress on the task team for the supervision of all transactions related to the program, efforts that were, therefore, disproportionate with respect to the resources committed with the loan.

21. While the program was successful in achieving its broader goals of reducing maternal and infant mortality and chronic malnutrition in children under 5, progress on the indicators in the RF was mixed during the life of the project. Some of the targets were achieved even before the MTR in February 2013, while others had a more fluctuating trajectory. To some extent, this was related to the delays in construction works, which shortened the time horizon available to see the impact of the infrastructure investments on the selected health outcomes and outputs. In other cases, the trend of some indicators reflected those at the national level, such as in the case of the prevalence of anemia among pregnant women. The delays in progress on some indicators motivated the downgrade of the Overall Project Implementation (IP) rating from satisfactory to moderately satisfactory in 2013, rating which was then kept in consideration of the disbursement delays ― eventually, the project disbursed 80% of the planned amount.

22. The Mid-Term Review (MTR) in February 2013 identified some of the challenges and correcting measures, including the needs to better coordinate with other relevant units within the MINSA and regional governments, strengthen M&E, increase loan disbursement (by then only 35%), modify the RF to reflect the actual starting and end date of the project, and support the reform process within MINSA by increasing capacity of and coordination with the regional and levels. Some of these recommendations were immediately taken on board, including for example strengthening the M&E function within PARSALUD. Others were only partially addressed during the life of the project.

Overall, the following implementation strengths were identified:

(i) Project Implementation Unit (PIU)’s capacity and commitment. The PARSALUD PIU was reconstituted in 2009, after a period of two years from its closure after the program’s first phase. The capacity of the PIU for project coordination was assessed as satisfactory and the unit was generally appropriately staffed (see Annex 12). The role of project coordinator was stable (two main coordinators and two acting for a very brief period of time). The project coordinators and many of the key personnel were very committed and remained within the team throughout project implementation, ensuring continuity and supporting improved capacity of the PIU. The PIU was found to be very proactive in a number of areas. For example, the PIU accepted all communications and documents to be sent electronically to the IADB and the Bank, which is not common practice in Peru. The PIU developed and made available checklists to constructors to ensure environmental safeguards were abided by and monitored compliance independently. Finally, the PIU worked closely with each Direccion Regional de Salud (DIRESA) to support local and regional-level interventions (such as the EPPES) and to strengthen capacity at the regional and local level based on the needs of each region.

(ii) Cultural adaptation of interventions. The systematic strengthening of health rights and empowerment of the population in rural areas about social participation in health through the EPPES and DIA campaigns were very well received by the local communities. These initiatives, together with the prior consultations held with local communities before the start of all civil works under PARSALUD, helped regional administrations build capacity for intercultural strategies, including communication in local languages (e.g. Quechua and Aymara). As a result of those culturally-sensitive interventions, coverage from SIS increased in the lowest quintiles in the Project areas.

(iii) Contribution to the Identity and Insurance Rights movement. PARSALUD was very active in catalyzing efforts to support the Derecho a la Identidad y Aseguramiento (DIA) for the health sector. This was an intersectoral initiative, in collaboration with the RENIEC and civil society, and with a strong regional and local commitment, to which PARSALUD contributed by supporting a campaign to promote the issuance of the Live Birth Certificate to children under 3 years of age and the National Identity Document to pregnant women and mothers.

The following implementation weaknesses were identified:

(i) Political changes within MINSA and regional governments. Four Ministers of Health changed during project implementation. Despite posing some concerns about stability of political commitment for the project and its key staff, eventually political changes at the national level did not substantially impact on project implementation. On the other hand, frequent political changes in the regional governments meant that PARSALUD PIU had to constantly re-engage with new administrators and staff, requiring duplication of efforts for capacity building at the local level.

(ii) Delays in civil works and consultancies. As of December 2012, after 2 years of implementation, about 27% of project's civil works were delivered, with heterogeneous patterns among the regions. Logistic difficulties for timely civil works completion were associated with difficult climate and access to sites, as well as with a few cases of collusion, properly addressed by the counterpart. Delays in hiring consultants were due to, among others, frequent changes in regional administration, scarcity of professionals adequately qualified for some tasks, delays in approval processes from the regions, and political attention diverted to sudden public health emergencies (e.g. pneumonia in Puno and dengue in Ucayali). Despite all delays, almost all planned civil works were completed before the project closing date, with a few being delivered in 2016.

(iii) Underestimation of the impact of other programs or initiatives on progress on the PARSALUD indicators. In some cases, project indicators showed irregular progress. This was partially due to other government programs providing incentives contrasting with those provided by PARSALUD (e.g. distribution of formula milk which at times affected exclusive breastfeeding of infants), and partially with changes in procedures for the registration with SIS (e.g. requirement of national identification document to register with SIS, which negatively affected the number of affiliates).

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

23. Design. The project’s Results Framework (RF) was derived from the PARSALUD program RF, which was based on a feasibility study (FS) finalized in 2008[footnoteRef:4] and approved as the program proposal by the SNIP. [4: Segunda fase del Programa de Apoyo a la Reforma del Sector Salud - PARSALUD II. ESTUDIO DE FACTIBILIDAD. Nov 2008]

24. The indicators in the RF reflected all parts of the PDO. The PDO-level indicators (Key Performance Indicators, KPIs) focused on improving practices at the household level (PDO 1) and on strengthening the health service network (PDO 2). The Intermediate Outcome Indicators (IOIs) focused on PDO 2 and 3 (supporting MINSA’s government functions). The first three components of the project were also aligned with the three parts of the PDO. Given the funds allocation, KPIs and IOIs focused largely on PDO 2 and Component 2 (strengthening of health services networks – supply side). Component 3 (governance and financing) is reflected by a small number of IOIs, most of which were dropped in 2014.

25. Baseline data as of 2005 was available for all but three indicators, based on the FS. Targets were set against that baseline by imposing improvements greater than the expected improvements based on historical trends. However, delays in project preparation and effectiveness made the baseline and the targets outdated, but neither of these were revised. If unable to change the baseline and/or the targets at the time of appraisal, due to time constraints and lengthy government processes, within the context of the first restructuring in 2011, when data from 2009 was becoming available, these should have been used to update the baseline and the targets.

26. Implementation. The project indicators were monitored using government’s systems and surveys. This had the advantage of not creating a parallel system. However, the PIU had to request or download data from different institutions responsible for data collection before being able to analyze it. There was a time lag of one year between the data collection and the data availability, so that data for a given year became available only in the following calendar year.

27. Due to unavailability of data at the time of the ISRs, two KPIs only started being monitored in December 2014 (ISR 11, with data of December 2013). Given that they had not been monitored, five IOIs were dropped during the restructuring in 2014. However, data was available for two of those, which were reintroduced for the purpose of the ICR. The other three indicators had not been clearly defined and monitoring was therefore problematic. Unfortunately, all three of those IOIs measured progress against the same part of the PDO (3). Since the PARSALUD PIU monitored more indicators than those monitored by the Bank, additional intermediate indicators might have introduced to replace the problematic IOIs to better assess improvements on PDO 3 ― for example during the first restructuring in 2011 or immediately after the MTR in 2013.

28. Within the PIU the M&E Unit originally included two specialists, one focused on Monitoring and the other on Evaluation. Following suggestions from the MTR, the M&E function was strengthened and the original unit was split into two to focus and strengthen each area ― supervision of program performance, and management of scientific evidence for enhanced effectiveness of the overall program (see Annex 12).

29. Utilization. Once data was processed, the M&E team analyzed data for each indicator and informed the technical team of the trends. If needed, the local coordinators within the DIRESA were contacted to understand the reasons for the variation in the indicators, especially with regard to birth, death and maternal anemia. However, given the difficulties in monitoring the full results chain and the delay in obtaining information on the indicators, data was not used by the local level to inform decision-making or revise practices and procedures in real time. It was used by the PIU to promote studies that supported increased knowledge and evidence-base policy, informing the design of technical guidelines and regulations that improved MINSA’s regulatory capacity.

2.4 Safeguard and Fiduciary Compliance

30. Safeguards. Given that the project (environmental category B) triggered the Environmental Assessment (OP/BP/GP 4.01) and the Indigenous People Safeguard Policies (OP 4.10), environmental and social safeguards were monitored. The Environmental Assessment was conducted in 2005; the Indigenous People Plan (IPP) was prepared in 2006. In 2013, specific missions assessed the compliance with environmental and social safeguards and proposed corrective recommendations where needed. More targeted safeguards supervision missions were conducted in 2013 to review implementation of the action plans in line with the EA and the IPP. The implementation of the activities under an intercultural approach related to Component 1 (including the EPPES, the DIA, and the prior consultations to IP for infrastructure construction) were rated as satisfactory. Similarly, the implementation of environmental safeguards was rated satisfactory, given the proactivity of the PIU in promoting the use of checklists for solid waste and water management during infrastructure construction under Component 2.

31. Financial Management. The counterpart’s financial management performance was considered generally satisfactory. The PIU was appropriately staffed from the start and, despite delays in systematically adopting the official system for managing transactions used in all implementation units in the public sector (Sistema Integrado de Administración Financiera, SIAF), financial reports were timely and were found to be of satisfactory quality. Audit reports were provided on time and there were no qualified opinions. Until 2013, disbursement was very slow (39% in August 2013). Problems were related to a number of factors, including previous delays in civil works as a consequence of inadequate planning and due to the remote project locations, which did not offer incentives for enterprises to participate in the bidding process; and the contractors’ noncompliance with contractual clauses. The GOP established a condition of Pari-Passu for all civil works (6% of external resources), which limited the scope for accelerating disbursement of the loan. Finally, deferred payments for civil works at the end of the project (Ocongate Health Center and Health Center Chuquibambilla) and the cancelation of the scheduled execution of large amounts of consulting services (i.e. Diplomado APS PROFAM and Sistema Nacional de Sangre Segura) expected to be contracted before the project closing and executed during the project grace period, negatively impacted disbursement. As a result, total disbursement reached 80% at project closing.

32. Procurement. Given that the entire Government program followed the Bank’s procurement guidelines, the Bank reviewed and approved a large volume of transactions. The initial delays due to lack of planning by the counterpart and long processing time for No Objections by the Bank were addressed and resolved after the first two years of project implementation. Procurement delays remained associated mainly with the scarcity of qualified contractors and remoteness of the work sites.

2.5 Post-completion Operation/Next Phase

33. Building on the PARSALUD know-how, a follow on GOP investment program, the Programa Nacional de Inversion de Salud (PRONIS) was approved in 2015 and is now operational. In line with the decentralization, PRONIS allows regions to set their own priorities and request funding from the central level. The possibility of a follow-on Bank-funded operation to support this new investment strategy has been discussed. MINSA and PARSALUD presented concrete proposals to MEF and the Bank (including a logic framework for the new operation). Discussions are still ongoing until the political situation stabilizes after the national elections which are taking place between April and June 2016.

3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation

The Overall Relevance Rating is: Substantial.

34. Relevance of Objectives: Rating High. The project was fully aligned with the CPS for FY12-16 for Peru (February 1, 2012; p.21) and with the priorities of MINSA for an investment program with a strong focus on reducing maternal mortality, infant mortality and undernutrition, and an interest in improving equity in access to services. Today, reduction of maternal mortality and especially of infant mortality remain important, while non-communicable diseases are increasingly capturing attention and funding due to increased burden. Malnutrition and anemia still remain very high on the Government agenda given that progress has been fluctuating, not only in the project areas, but more generally across the country. In addition, poverty and inequalities, especially in the project areas, are still far from being resolved. The World Bank is currently undertaking the first Systematic Country Diagnostics for Peru (the review meeting will be held before the end of FY16) and will start working on the Country Partnership Framework in the first quarter of FY17. Addressing inequalities, support to effective decentralization, and improved nutrition, all of which were at the core of PARSALUD, still remain highly relevant.

35. Relevance of Design: Rating Substantial. Given that this project supported phase II of the program and all triggers were met at the end of phase I, the APL design was maintained. MINSA gained experience in managing APL fiduciary rules and believed these would help reinforce accountability, expressing preference for this lending instrument.

36. The project design appropriately aimed to address both demand and supply side factors to improve maternal and child health outcomes in the nine regions, as well as to increase the capacity of the government at the national and local level. It had a strong intercultural footprint, which was very appropriate given the areas of focus under the program. It was also designed in parallel and to build synergies with another Bank-financed project aimed at improving nutrition outcomes by using conditional-cash transfers, the Juntos Results for Nutrition Project (P117310), approved in 2011.

37. As mentioned above, the RF was aligned with the project components and reflected the different parts of the PDO. At approval, the design of the RF still reflected baseline data of 2005, which could have been updated with more recent data, either at that stage or at the first restructuring in 2011. Data for some IOIs was not available and five indicators were dropped at the 2014 restructuring. More effort could have gone to incorporate in the project’s RF some IOIs included in the program’s RF to better assess progress with Component 1 and 3; to compensate for these weaknesses, several IOIs were included at the time of the ICR.

3.2 Achievement of Project Development Objectives

The Efficacy Rating is: Substantial. The overall efficacy rating is the result of the assessment of achievement of the three parts of the PDO, all of which are rated substantial.

Assessment of achievement of the program’s objectives

38. The project objective was to contribute to the overall GOP’s effort to reduce maternal and child mortality and improve chronic malnutrition in children ― and, as explained below, the project did contribute to this. A comprehensive impact evaluation would be needed to demonstrate the attribution of the achievement of the program objectives and of the PDO-level indicators the project. However, the case for project contribution to the achievement of the PDO, is supported by progress on the intermediate outcome indicators that are more directly attributable to the project.

39. In terms of the overall results of the program, between 2009 and 2014, Peru was able to reduce IMR from 42 to 17 against a target of 25, and chronic malnutrition of children under 5 years of age from 38.2% to 23.7%, against a target of 30.2% (last PARSALUD progress report). MMR per se was not monitored by the program due to uncertainty about the reliability of data; however, the indicator on institutional deliveries was used as a proxy. As seen below, this indicator moved from 62.2% to 74.2% against a target of 78%, between 2009 and 2014. The total number of maternal deaths was also reduced from 186 to 159 in the project areas, against a national average of 481 and 411 between 2009 and 2014. Hence, overall, PARSALUD was successful in reducing MMR and IMR and chronic malnutrition in children under 5 years of age. More detailed program results are found in Annex 3.

Assessment of achievement of the project’s objectives

40. In terms of the three specific project objectives, the ICR assesses the original indicators as well as additional indicators added at the time of the ICR to introduce additional evidence in support of the results chain to the PDO. All indicators were related to the three parts of the PDO statement, as follows:

(i) Improvement of family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three:

KPIs# 1, 2, 3, 4, 6; IOIs # 1, 11

(ii) Strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three:

KPIs# 1, 2, 4, 5, 6; IOIs # 2, 3, 4, 5, 6 and 12 (added)

(iii) Supporting MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment

IOIs # 7, 8, 9, 10; and 13, 14, 15 (added).

41. IOIs #2 and 5 (PDO 2), 7, 9 and 10 (PDO 3) were dropped at the 2014 restructuring. At the time of the ICR data was only available to support IOIs #2 and 5, which were therefore reintroduced. Additional qualitative data was collected to support the analysis of PDO 3.

42. When assessed against the original baseline, all the indicators have surpassed, achieved or partially achieved the targets at the end of the project, which supports achievement of all parts of the PDO (Table 1). Also, notably, the final percentage of achievement does not reflect that roughly half of all indicators surpassed their targets. Therefore, the actual level of achievement is far greater than the table suggests.

Table 1: Achievement of PDO (targets against original 2005 baseline)

Program level

PDO level

Intermediate

PDO 1*

PDO 2*

PDO 3

Target surpassed

2

3

6

4

4

3

Target achieved or substantially achieved (>=85% met)

2

4

1

5

1

Target partially achieved (65%-84% met)

1

2

2

2

0

Target not achieved (<65% met)

0

0

0

0

0

Unknown

0

3

0

0

3

Total

2

6

15

7

11

7

% surpassed and achieved

100%

83%

83%

71%

82%

100%

* Note: The table double-counts some indicators to assess PDO 1 and 2.

43. Given the fact that the baseline for this project which was approved in 2009 actually used 2005 data (for reasons explained above), the ICR team reviewed achievement of project indicators against the 2009 baseline (see Annex 10). Notably, many of the indicators already registered improvements in 2009 compared with the baseline values of 2005. This overall trend continued during project implementation, with all but one indicator (prevalence of anemia in pregnant women) following the positive trajectory. This supports the positive contribution of the project to the achievement of the PDO. In addition, many of the indicators for the project regions showed performance at least in line with the national average (see Annex 3), a very positive result considering the geographical and socio-economic conditions of the project regions.

44. The assessment below is based on the official 2005 baseline data, and when appropriate, also uses revised targets for IOIs.

PDO 1: improvement of family care practices for women (during pregnancy, delivery and breast-feeding), and children under the age of three. Rating: Substantial.

KPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2014) – Achieved

KPI #2: Reduce the prevalence of anemia among children under the age of 3 in the nine regions from 69.5% (2005) to 60% (2014) – Surpassed

KPI #3: Increase from 64% to 80% the share of children in the nine selected regions who are exclusively breastfed until 6 months of age – Surpassed

KPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2014) – Partially Achieved

KPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2014) – Surpassed

IOI #1: Percentage of SIS affiliated children who received growth and development controls (CRED) according to their age – Partially Achieved

IOI #11: Number of health personnel and community health workers trained within the behavior change campaign (EPPES) – Surpassed

45. The project contributed to the increase of breastfeeding practices, one of the most important indicators for Component 1. Exclusive breastfeeding has been low in Peru, although in the project areas it was higher than the national average in 2009 (82.2% vs. 68.5%). The project greatly contributed to the uptake of breastfeeding practices in the nine regions, since the share of exclusive breastfed children in project areas rose to 87%, while the national average remained flat at 68.4%. This was in line with other GOP programs such as Juntos, although other programs aimed at providing food and formula supplements to poor families; therefore, achievements under the project seem to be remarkable.

46. Progress on anemia in the nine regions has been slow, reflecting national trends. Anemia has traditionally been a problem in Peru, often associated with lack of knowledge. Anemia in children under 3 has been stable between 2009 and 2014 in the project areas, while it has slightly declined as a national average (50.4 and 46.8). Anemia in pregnant women declined between 2005 and 2014 but it did not follow a steady trajectory. The prevalence of anemia, higher in the nine regions compared to national average, was relatively stable at the beginning of the project, at around 30%. However, after dropping to 24.3% in 2013, well below the national average of 28%, it started rising again to levels higher than the national average (36.4% vs 32.5%). In the project regions, the reasons for this trend seems to be related more to demand-side factors than supply-side factors. The availability of iron supplements at facility level has increased over time and the proportion of women who receives those supplements has also increased. According to PARSALUD staff, women admit that they are not taking iron supplements due to their bad taste, color and smell. New supplements have recently been purchased to overcome this issue; their effectiveness is yet to be assessed once the data for 2015 becomes available. The slow progress on the prevalence of anemia in women in Peru requires additional efforts to ensure stable and sustainable improvements.

47. The achievement of PDO 1 is supported by successful progress on a number of IOIs. All facilities that needed to be provided with audiovisual equipment to support the behavior change campaigns were in fact provided one (1,423). The project trained more a total of 1,178 health personnel and community health workers within the communication and behavior change campaign (400 more than planned). According to SIS data, the biggest increase in the number of services provided to SIS-enrolled beneficiaries between 2011 and 2015 was for primary care services, which demonstrates greater accessibility of basic services for the poor, with a sharp increase in prevention as opposed to curative services. Finally, the campaign promoting identity and insurance rights (Derecho a la Identidad y Aseguramiento, DIA) was conducted twice in every region for 3 months each time, for a total of 6 months of campaing in every region ― 3 months less than originally planned.

48. PDO 2: strengthening of health services networks with capacity to solve obstetric, neonatal and infant emergencies and to provide comprehensive health services to women (during pregnancy, delivery and breast-feeding) and children under the age of three. Rating: Substantial.

KPI #1: Increase the proportion of institutional deliveries in rural areas of the nine selected Regions from 44% (2005) to 78% (2014) –Achieved

KPI #2: Reduce the prevalence of anemia among children under age in the nine regions from 69.5% (2005) to 60% (2014) – Surpassed

KPI #4: Reduce the prevalence of anemia among pregnant women in the nine Regions from 41.5% (2005) to 35% (2014) – Partially Achieved

KPI #5: Reduce the hospital lethality rate among neonates in the nine selected Regions from 9.5% (2005) to 5% (2014) – Achieved

KPI #6: Increase in the proportion of pregnant women of the nine regions with at least 1 prenatal control during the first trimester of pregnancy from 20% (2005) to 45% (2014) – Surpassed

IOI #2: Percentage of health facilities with improvement in infrastructure (minor construction and/or equipment) -- Achieved

IOI #3: Percentage of SIS affiliated rural pregnant women with laboratory tests on hemoglobin, urine and syphilis – Surpassed

IOI #4: Percentage of pregnant women under SIS that receive iron and folic acid supplements – Partially Achieved

IOI #5: Percentage of women satisfied with the services in selected facilities by confidence index – Surpassed

IOI #6: Percentage of cesareans in SIS affiliated pregnant rural women – Achieved

IOI #12: Number of health facilities improved – Achieved

49. Health infrastructure improved in the nine regions. The project supported the construction and/or renovation of 69[footnoteRef:5] out of the planned 73 hospitals and basic health centers that offered obstetric and neonatal services; the remaining 4 are being completed in 2016, financed by GOP. It provided all of the 104 pre-identified health centers with medical equipment for maternal and child care and installed IT systems in 55 centers to support the implementation of the e-Health plans (planned 54). [5: More precisely, one health center underwent two types of renovations and received two sets of equipment; hence although technically the number of health centers renovated and equipped was 68 and 103 respectively, the number of renovation works done and sets of equipment delivered was 69 and 104 respectively. ]

50. The project delivered capacity building programs for different groups of health professionals using a culturally-sensitive approach. The project contributed to the training of 956 health workers (nearly 300 more than planned) on maternal and child care and specifically of 282 doctors and midwives on the vertical delivery (Parto Vertical) in 7 health facilities located in 4 regions within PARSALUD. Together with improved infrastructure and training, cultural adaptation of health facilities for the provision of vertical deliveries and support to Maternal Waiting Homes (Casas Maternas) were key in improving rates of institutional deliveries among indigenous populations. This includes the creation, dissemination, and systematization of knowledge around vertical delivery practices among health care providers and the institutionalization of this method in both regions: in the Amazon and Cusco, 54% and 33% of all deliveries were vertical, respectively, in 2012. The Casas Maternas, communal space managed, built, and maintained by communities and local governments, helped address the physical and cultural barriers faced by indigenous women residing far from health centers. There are 475 operational Casas Maternas throughout the country, with Cusco, Puno, Huancavelica and Apurimac the most important areas of reference.

51. The project strengthened the networks of services and improved quality of care and access to emergency services. In order to ensure a more effective network of services, all 9 regions elaborated a plan for the improvement of the referral and counter-referral system. The project also trained key health personnel in management of human and financial resources (1,336 actual vs. 1,143 planned).

52. On quality of care, the project funded a specific training and implementation program to reduce maternal pre- and post-partum bleeding through the introduction of a supervised medical protocol based on the use of specific evidence-based interventions and medicines, which has been shown to reduce post-partum hemorrhage. According to a survey done in 2012, the reported satisfaction among women who used the services was 75%.

53. The rates of C-sections in women affiliated with SIS increased in line with the intention of ensuring that women in need were actually able to access this service. While the national average has increased to 12%, in the PARSALUD region it seems to have been stable since 2012. This seems to suggest that while excessive use of those services might be starting in Peru, as it is already common practice across Latin America, in the project area those services are provided only when required.

54. PDO 3: supporting MINSA's governance functions of regulation, quality, efficiency and equity for improving the new health delivery model of maternal and child health care in a decentralized environment. Rating: Substantial.

KPI #8: Percentage of SIS affiliated households that make out-of-pocket expenditures in medicines – Partially Achieved

KPI #13: Norms and regulations to improve efficiency and equity of the health delivery system prepared – Surpassed

KPI #14: Clinical pathways and corresponding financing systems designed – Surpassed

KPI #15: Periodic evaluations of the performance of the health networks – Surpassed

55. The achievement of the IOIs suggests that progress towards PDO 3 was significant. PARSALUD II developed 32 proposals for norms and legal acts to improve efficiency and equity of the health system for approval by MINSA, when they had only planned for 27. The design of the clinical pathways and corresponding financing system was also finalized by the project (16 actual vs 11 planned). The capacity of analyzing performance of the health system also improved through the completion of regular performance reports (19 produced vs. 17 planned) – although the majority of those were released in 2015, rather than being released constantly throughout the project and periodically updated.

56. PARSALUD II supported MINSA in reformulating its regulatory framework to ensure better quality of the health delivery system, in particular on purchasing, prescribing, and monitoring the quality of pharmaceuticals. Peru’s consolidation of a purchasing system for essential medicines lowered their costs for the country. In addition, a comprehensive training system in essential drug prescriptions was implemented in the project’s health facilities, and support was provided for improving the regulatory framework for quality control using SUNASA (Superintendencia Nacional de Salud).

57. The project strengthened the regional management capabilities on MCH through specialized training in neonatal and obstetric competencies for health professionals and managerial capacities for the Health Regional Directorates (DIRESA). Budget executions for functions related to maternal and child care and especially nutrition improved in the nine regions following capacity building interventions between 2012 and 2014 (e.g. Cusco moved from 88.7% in 2012 to 97% budget execution in 2014 on maternal and child health, while Amazonas moved from 77.5% in 2012 to 91.6% in 2014 on nutrition)

58. Additional qualitative information supports significant progress on PDO 3. This information was collected during the ICR mission and triangulated with findings by PARSALUD evaluations derived from focus group discussions and individual interviews conducted with informants at MINSA, PARSALUD team, regional administrators, health professionals, and beneficiaries. The technical assistance provided by PARSALUD supported a cultural change within the regional administrations and health professionals towards an approach that recognizes the human right of pregnant women and children to be treated fairly and with dignity, which has been appreciated by the beneficiaries. The project assisted the decentralization process by providing targeted assistance to the regions that needed more specific support with the regulation function. Topics of focus varied depending on the needs of each region, ranging from human resources to supply chain management. Progress were made on enhancing transparency, with a portal and website created for PARSALUD. In addition, central planning for civil works was strengthened as delays in starting and concluding civil works at the beginning of the project reduced over time. Overall, this evidence suggests that PARSALUD promoted significant progress in improving the government capacity on regulation, quality, equity, and efficiency.

3.3 Efficiency

The Efficiency Rating is: Modest

59. Rather than carrying out a separate economic analysis, the PAD referred to the one done in the FS. The ICR could not update the analysis done in the FS; instead, a more qualitative assessment of project efficiency and cost-effectiveness was undertaken (see Annex 3).

60. Cost-effectiveness was overall substantial, especially considering that the Bank loan leveraged a much more significant amount of domestic resources; on balance, in consideration of the delays and the disbursement level, efficiency is considered modest.

Allocative efficiency

61. Maternal, newborn and child health (MNCH) interventions are recognized by the literature as highly cost-effective investments. Particularly effective interventions in MNCH packages would be management of labor and delivery, care of preterm births, and treatment of serious infectious diseases and acute malnutrition [footnoteRef:6] ― all key focus of PARSALUD. Such MNCH interventions, targeted to those most in need, can prevent maternal and infant deaths and reduce the healthy years of life lost due to disability, thereby benefiting the health sector and society as a whole [footnoteRef:7],[footnoteRef:8],[footnoteRef:9]. [6: Black R, Levin C, Walker N, Chou D, Liu L, and others. 2016. "Reproductive, Maternal, Newborn, and Child Health: Key Messages from Disease Control Priorities, 3rd Edition." The Lancet. Published online 9 April 2016 ] [7: World Health Organization the case for Asia and the Pacific, Investing in Maternal Newborn and Child Health, Geneva, Switzerland. ] [8: Bill and Melinda Gates Foundation, Maternal, Newborn and Child Health Strategy Overview, www.gatesfoundation.org. ] [9: Black R, Laxminarayan R, Temmerman M, and Walker N. editors. 2016. “Reproductive, Maternal, Newborn, and Child Health. Disease Control Priorities, third edition, volume 2”. Washington, DC: World Bank. doi:10.1596/978-1-4648-0348-2. License: Creative Commons Attribution CC BY 3.0 IGO]

62. Lack of infrastructure and professional attention, under registration of beneficiaries on the public health insurance (SIS) and poor managerial and logistic capacity on local health units were identified as barriers to health service availability for women and children in rural areas. Therefore, closing the gap in health care access required targeting resources on the identified barriers and on vulnerable rural populations. The return on infrastructure investments have clear long-term benefits, which facilitate improvements in the provision of quality services. In addition to infrastructure investments, the project funded some important and highly cost-effective demand-side interventions under Component 1. These evidence-based interventions supported the growing health service demand, improved the quality of care and were at the core of PARSALUD I; a stronger focus on similar interventions would have further increased the cost-effectiveness of PARSALUD II.

Efficiency of project preparation and implementation

63. Project preparation was characterized by delays leading to a three-year gap between the end of APL 1 and APL 2 effectiveness. Delays were largely due to the political economy in the country. Once the project gained political traction, it was rapidly appraised and negotiated.

64. The project was implemented within the time period originally planned. The project was extended by 11 months, which balanced the 10-month gap between Bank approval and effectiveness. However, despite the extension, the project disbursed 80% of the planned amount, due to delays in procurement and civil works[footnoteRef:10]. [10: Until three months before the project closing date, the PARSALUD team assured that the project would disburse 100% of the loan. However, this did not materialize due to legal problems with regard to a big consultancy contract and delays in civil works, which are being funded by domestic resources.]

65. Given the nature of the project, the evaluation of Project efficiency should consider the Bank’s overall program financial contribution. In this sense, the project was extremely cost-effective, since by supporting less than 10% of the total program’s costs, it leveraged nearly US$ 140 million of domestic resources, in addition to the US$ 15 million provided by the IADB. Despite the initial civil works delays and the 80% final loan disbursement ratio, the program’s actual costs were fundamentally in line with the amount of resources identified at appraisal.

66. The implementation efficiency of the overall Government program was leveraged by the project funds and the use of Bank fiduciary safeguards that provided cover for the weaker Government systems and processes. The Bank project team reviewed procurement bidding documents and contracts for civil works, consultancy services and provision of training for the overall program. This effort increased the leverage of technical and fiduciary implementation support beyond the loan funds and enhanced the technical skills of the PARSALUD technical and fiduciary team.

Fiscal impact and sustainability

67. The fiscal impact of the project was marginal, limiting sustainability concerns. Project implementation did not have a major impact on the MINSA budget, as it weighed an average of 0.08 percent throughout the period analyzed. Once the facilities were built, improved or better equipped through the program’s funds, the responsibility for the management of those was transferred to the regional government. By signing the Convenios (contracts between PARSALUD and each of the nine regions), regional governments committed to allocate budget for maintenance of the upgraded facilities as a condition for starting civil works in pre-identified facilities in that specific region.

3.4 Justification of Overall Outcome Rating

The Overall Outcome Rating is: Moderately Satisfactory. This rating takes into account that the project is the second in a series of two APLs to support a broader GOP program that has been largely successful in achieving its goal of improving maternal and child health.

Table 6: Summary of Outcome Ratings

Project Outcome Ratings

Relevance

Substantial

Efficacy

Substantial

Efficiency

Modest

Overall Outcome Rating


3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development

68. The project targeted specifically nine of the poorest regions, home of indigenous people and isolated communities, and within those communities focused on the most vulnerable groups ― women and children in the early years of life. The equity dimension was important in conceptualizing the project, which contributed to social inclusion and promoted a solid basis for personal identification and for the affiliation to the SIS. The project was mindful of local identities and practices and successfully built on those to select interventions that were culturally appropriate for a specific context, such as the parto vertical. This was key in ensuring that people were more comfortable in accessing the services, with the aim of sustaining improvements in access to basic services.

(b) Institutional Change/Strengthening

69. The project showed progress in building capacity at the central level and most importantly at the regional and local levels, benefiting government representative and different categories of health professionals. For example, PARSALUD II contributed to the local and regional capabilities to conduct social dialogue with various stakeholders prior to project implementation. The prior consultations on project’s infrastructure (to build, improve and equip the health facilities for obstetric and neonatal care), was a coordinated effort involving indigenous populations, regional and local governments, which strengthened the ability of local and regional actors to seek consensus on health infrastructure building.

(c) Other Unintended Outcomes and Impacts (positive or negative)

70. The project contributed to increased inclusion of indigenous communities and to provide space for communities and especially women to be heard, increasing accountability of the system. At the end of 2013, the project organized a South-South Knowledge workshop on intercultural interventions in health aimed to showcasing PARSALUD’s best lessons learned and bringing regional experiences that informed and strengthened PARSALUD’s intercultural agenda. The main attendants to the workshop were representatives of the Pan American Health Organization (PAHO), the National Autonomous University of Mexico, the Ministry of Health of Venezuela and the Ministry of Cultures of Peru, among others.

71. The project had a pivotal role in fostering and guiding the identity rights movement in the health sector and in creating a platform for dialogue of different actors at the national and local levels. The systematic introduction of birth certificates as the first means of personal identification provided the legal basis for the registration to the SIS as well as other GOP programs and access to the related benefits.

72. The project improved awareness of environmental issues, helped create a conducive institutional environment to ensure environmental hazards are considered and dealt with, and built capacity at the central, regional and local level.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

Not available.

4. Assessment of Risk to Development Outcome

Risk to Development Outcome Rating: Moderate

73. The follow-on GOP program, Programa Nacional de Inversion de Salud (PRONIS), built on the lessons learned and know-how from PARSALUD. Financial sustainability of the program does not seem to be a concern, given that PARSALUD was already largely funded by domestic resources and funds have already been allocated for the new program investments. While the Project Coordinator and the Technical Coordinator have been transferred, many of the key PARSALUD personnel are now working on PRONIS, ensuring greater continuity of the follow-on investment program with PARSALUD. PRONIS is also supervising the completion of the outstanding PARSALUD consultancies and civil works to be delivered in 2016.

74. PRONIS focuses on 748 key health centers and facilities, many of which PARSALUD rehabilitated or improved. However, contrary to PARSALUD, PRONIS is not specifically targeted to the poorest regions and it does not support an explicit, a priori objective or topic. The regions will identify the specific outcomes of focus for the investments depending on their demographic and epidemiological characteristics and political priorities.

75. PARSALUD put in place systems to ensure sustainability at the local level, by promoting and supporting decentralization of responsibilities and functions and building capacity for better use of resources for maternal and child services, which would benefit the regions even in the context of PRONIS. However, the high turnover of staff in the regional administrations and the irregular progress of some of the indicators pose some questions on sustainability of achieved results if a systematic strategy to address those is not adopted. Nutrition and anemia in particular remain problematic at the national level. Therefore, ensuring more systematic progress on those indicators, especially among the most vulnerable population, would require alignment of national, regional and local commitment.

5. Assessment of Bank and Borrower Performance 5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry

Quality at Entry rating is: Moderately Unsatisfactory.

76. The project suffered from a long preparation time. The Concept Note review was held in November 2005, but appraisal and negotiations were only conducted in December 2008. However, the time gap between the end of PARSALUD I and the beginning of phase II was mainly due to changes in the political arena an

documents.worldbank.org · Web view2016/06/29 · APLAdaptable Program Loan. AUSAseguramiento...

Documents

Transcript of documents.worldbank.org · Web view2016/06/29 · APLAdaptable Program Loan. AUSAseguramiento...