Public Disclosure Authorized IMPLEMENTATION...

Document of The World Bank

Report No: ICR0000862

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-71760)

ON A

LOAN IN THE AMOUNT OF (US$52.5 MILLION)

TO

THE REPUBLIC OF PERU

FOR A

RURAL EDUCATION PROJECT IN SUPPORT OF THE FIRST PHASE OF THE

RURAL EDUCATION PROGRAM

June 25, 2008

Human Development Sector Management Unit Bolivia, Ecuador, Peru and Venezuela Country Management Unit Latin America and the Caribbean Regional Office

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 25, 2008)

Currency Unit = Peruvian Nuevo Sol (PEN) 1.00 = US$ 0.33

US$ 1.00 = PEN 3.00

FISCAL YEAR January 1 – December 31

ABBREVIATIONS AND ACRONYMS

APAFA Parents Association (Asociación de Padres de Familia) APL Adaptable Program Lending CAS Country Assistance Strategy CEI Initial Education Centers for 3-5 year-old children (Centros de Educación

Inicial) COMPFE

Comisión de Coordinación y Administración para la Ejecución de Proyectos con Financiamiento Externo del Ministerio de Educación also known as the PCU

CONEI Institutional School Council (Consejo Educativo Institucional) CPS Country Partnership Strategy DfID United Kingdom’s Department for International Development DRE Regional Education Directorates (Dirección Regional de Educación) DGPM General Director of Public Sector Multi-Annual Programming (Dirección

General de Programación Multianual) ESMED Secondary Distance Education Model (Educación Secundaria con

Metodología a Distancia) FMRs Financial Management Reports FONCODES National Social Development and Compensation Fund (Fondo Nacional de

Compensación y Desarrollo Social) GOP Government of Peru ICR Implementation Completion and Results Report IDB Inter American Development Bank ISR Implementation Status Results and Reports LA Loan Agreement MED Ministry of Education (Ministerio de Educación) MEF Ministry of Economy and Finance (Ministerio de Economía y Finanzas) PAD Project Appraisal Document PCN Project Concept Note PCU Project Coordination Unit PDO Project Developmental Objectives PEAR Rural Education Project (Proyecto de Educación en Areas Rurales)

PEI Institucional Educational Project (Proyecto Educativo Institucional) POA Annual Operational Plan (Plan Operativo Anual) PHRD Japan Policy and Human Resources Development PRONAMACHCS National Program for Hidrological Basin Management and Soil Conservation

(Programa Nacional de Manejo de Cuencas y Conservación de Suelos) PSRL Programmatic Social Reform Loan QAG Quality Assurance Group QEA Quality at Entry Assessment REACT DPL Results and Accountability Development Policy Loan TTIs Teacher Training Institutions PRONOEI Non Formal Inicial Education for 3-5 year-old children (Programa No

Escolarizado de Educación Inicial) SINEACE National Educational Assessment and Certification System (Sistema

Nacional de Acreditación y Certificación Educativa) SNIP National Public Investment System (Sistema Nacional de Inversión Pública) TTL Task Team Leader UGEL Local Educational Management Units (Unidad de Gestión Educativa Local) USAID United States Agency for International Development

Vice President: Pamela Cox Country Director: Carlos Felipe Jaramillo

Sector Director: Evangeline Javier Sector Manager: Eduardo Velez-Bustillo

Project Team Leader: Livia Benavides ICR Team Leader: Livia Benavides

ICR Primary Author: Juan Prawda

COUNTRY RURAL EDUCATION PROJECT

IN SUPPORT OF THE FIRST PHASE OF THE RURAL EDUCATION PROGRAM

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design............................................... 12. Key Factors Affecting Implementation and Outcomes ............................................ 133. Assessment of Outcomes .......................................................................................... 204. Assessment of Risk to Development Outcome......................................................... 275. Assessment of Bank and Borrower Performance ..................................................... 276. Lessons Learned ....................................................................................................... 367. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 38Annex 1. Project Costs and Financing.......................................................................... 39Annex 2. Outputs by Component ................................................................................. 41Annex 3. Economic and Financial Analysis................................................................. 47Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 49Annex 5. Beneficiary Survey Results ........................................................................... 52Annex 6. Stakeholder Workshop Report and Results................................................... 53Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR..................... 54Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders....................... 55Annex 9. List of Supporting Documents ...................................................................... 56

MAP

i

A. Basic Information Country: Peru Project Name: Rural Education ProjectProject ID: P055232 L/C/TF Number(s): IBRD-71760 ICR Date: 06/26/2008 ICR Type: Core ICR

Lending Instrument: APL Borrower: GOVERNMENT OF PERU

Original Total Commitment:

USD 52.5M Disbursed Amount: USD 29.0M

Environmental Category: C Implementing Agencies: Ministerio de Educacion Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s)

Concept Review: 11/28/2001 Effectiveness: 04/15/2004 04/15/2004 Appraisal: 04/02/2002 Restructuring(s): Approval: 05/29/2003 Mid-term Review: 10/16/2006 12/01/2006 Closing: 12/31/2007 12/31/2007 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Unsatisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Unsatisfactory Borrower Performance: Moderately Unsatisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Unsatisfactory Government: Moderately

Unsatisfactory

Quality of Supervision: Moderately Unsatisfactory

Implementing Agency/Agencies:

Moderately Unsatisfactory

Overall Bank Performance:


Overall Borrower Performance:


C.3 Quality at Entry and Implementation Performance Indicators

Implementation Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project Yes Quality at Entry Satisfactory

ii

at any time (Yes/No): (QEA): Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:


D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 20 16 Pre-primary education 20 6 Primary education 20 71 Secondary education 20 5 Tertiary education 20 2

Theme Code (Primary/Secondary) Education for all Primary Primary Indigenous peoples Secondary Secondary Rural services and infrastructure Secondary Secondary E. Bank Staff

Positions At ICR At Approval Vice President: Pamela Cox David de Ferranti Country Director: Carlos Felipe Jaramillo Marcelo Giugale Sector Manager: Eduardo Velez Bustillo Marito H. Garcia Project Team Leader: Livia M. Benavides Livia M. Benavides ICR Team Leader: Livia M. Benavides ICR Primary Author: Juan Prawda F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Program will support the Peruvian government's objective of improving the quality and equity of basic education through focused investments in rural areas with the lowest education indicators and through systemic reforms of teacher policy and education management. This Project is the first one of a three-phase APL. Under Phase I, financing would be provided to support the implementation of key reforms of teaching and system

iii

management, initiate the investments in rural school infrastructure and quality enhancements, and pilot and evaluate alternative models of distance education at the secondary level. Specifically, the Project will: (a) expand access to formal and non-formal initial and pre-school programs in rural areas; (b) pilot and evaluate three different modalities of distance education in 116 secondary schools in the targeted rural areas; (c) improve primary school quality in rural areas through specially designed professional development programs for rural teachers focused on multigrade and bilingual education, distribution of improved learning materials, and rehabilitation of infrastructure where needed; and (d) raise sector efficiency through the implementation of the first phase of teacher policy reforms, restructuring of regional offices, initiating a decentralization process leading to more autonomous schools, with community voice in management, and organizing rural school support network. To monitor quality and guide corrective strategies, the new national system of student assessment would be strengthened. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Percentage of 11 year-old children enrolled in sixth grade or higher in rural areas increases by 0.6 percentage points.

Value quantitative or Qualitative)

37.08% (rural) 44.27% (project area)

37.66%


Date achieved 12/01/2003 12/01/2007 12/31/2007 Comments (incl. % achievement)

770% (rural) 990% (project area)

Indicator 2 : Percentage of 11 year-old children enrolled in grades 5 and lower will not increase


67.1% 67.1% (or less) NA


Data not available

Indicator 3 : Rural 11 year-old children who do not go to school decreases by 0.7 percentage points

Value quantitative or 7.41% 6.71% 3.87%

iv

Qualitative) Date achieved 12/01/2003 12/31/2007 12/31/2007 Comments (incl. % achievement)

-505%

Indicator 4 : 375,000 children received bilingual materials and their teachers trained Value quantitative or Qualitative)

0 375,000 239,520


68%

Indicator 5 : Rural pre-school coverage (0-2 years) increases Value quantitative or Qualitative)

1% 4.7% NA


Data not available

Indicator 6 : Rural pre-school coverage (3-5 years) increases by 5.5 percentage points Value quantitative or Qualitative)

48.09% 53.59% 54.79%


122%

Indicator 7 : Secondary school gross enrollment ratio in rural areas increases by 8 percentage points


71,39% 79.39% 77.74%


79%

Indicator 8 : Secondary school coverage at grade 1 for 12 or less year-old youngsters in rural areas increases by 3.3 percentage points


37.08% (rural) 44.27% (PEAR districts)

40.3% (rural)



140% (rural) 180% (PEAR districts)

v

Indicator 9 : Learning outcomes of children in target Project Areas showed improvement on national assessment at the primary level


2.3% (language) 1.3% (math) NA

Date achieved 12/01/2004 12/31/2007 Comments (incl. % achievement)

The preliminary findings of the new learning assessment measurement to be administered end of 2008 will allow some comparisons with the baseline data for the treated as well as for the control groups.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Number of teachers trained in the use of bilingual materials. Value (quantitative or Qualitative)

0 15,000 13,952


93%

Indicator 2 : Number of teachers trained in the use of multigrade methodology and materials. Value (quantitative or Qualitative)

0 3,800 1,513


93%

Indicator 3 : Number of classrooms refurbished and/or built in Project areas. Value (quantitative or Qualitative)

0 1,900 293


15%

Indicator 4 : Number of PEI's (Educational Strategic Plan) with CONEI-RED (Network councils) approval.

Value (quantitative or Qualitative)

0 300 0

Date achieved 01/01/2004 12/01/2007 12/31/2007 Comments (incl. % 0%

vi

achievement)

Indicator 5 : Percentage of municipal networks with at least eight coordination meetings per year.

Value (quantitative or Qualitative)

0 70 0


0%

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP

Actual Disbursements (USD millions)

1 06/23/2003 Satisfactory Satisfactory 0.00 2 12/22/2003 Satisfactory Satisfactory 0.00 3 06/08/2004 Satisfactory Satisfactory 0.53 4 11/10/2004 Satisfactory Satisfactory 2.38

5 04/29/2005 Moderately Satisfactory Moderately Unsatisfactory 4.01



8 06/07/2006 Moderately Satisfactory Moderately Satisfactory 14.67 9 12/27/2006 Moderately Satisfactory Moderately Satisfactory 19.41

10 05/18/2007 Moderately Satisfactory Moderately Satisfactory 22.15



13 12/21/2007 Moderately Unsatisfactory Unsatisfactory 26.49

H. Restructuring (if any) Not Applicable

vii

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design (this section is descriptive, taken from other docjuments, e.g., PAD/ISR, not evaluative)

1.1 Context at Appraisal (brief summary of country and sector background, rationale for Bank assistance)

At the time of project preparation (2003), 28 percent of Peru’s population lived in rural areas, characterized by poverty, cultural diversity, geographic dispersion and inaccessibility to public services, such as education and health. About 90 percent of the rural population lived in clusters of less than 500 inhabitants; 60 percent were poor, 37 percent were below the extreme (“food-only”) poverty line and a third spoke one of Peru’s 42 indigenous languages. Social indicators for these populations were also much below those for urban population: (i) rural women averaged only 4.9 years of education as compared to 9.1 years for urban women; and (ii) infant mortality rate was 58 per 1,000 life births in rural areas as compared to 32 per 1,000 life births in urban areas.

Although Peru had done an impressive job over the previous last two decades in achieving universal access to primary schooling, average schooling attainment and student learning were sharply lower in rural than in urban areas. While gross enrollment rates of about 93 percent were similar for rural and urban areas in (2003), on-time completion rates in rural primary schools were 20 percent as compared to 57 percent in urban areas. This was a result of high repetition and dropout rates in rural areas (dropout rates in rural areas were 10.1 percent, far higher than 2.1 percent in urban areas) as well as late entry into the system (23.4 percent of poor rural children entering first grade of primary school were over-aged as compared to 14.2 percent in urban areas; an estimated 63 percent of Quechua-speaking children attending primary school were over-aged). A 1996 national learning assessment of children in the fourth grade found that only 38.5 percent of rural school children mastered at least 9 out of 14 basic competencies, compared to 63.5 percent of urban school children. Findings of the 2001 learning assessment showed that children in multi-grade schools in rural areas performed well below those in multi-teacher schools in urban areas, and that Quechua- and Aymara-speaking children scored worst of all, performing in the bottom 1% of the distribution of reading comprehension scores. Issues. The Project attempted to address the following three key relevant issues underpinning the above-mentioned disparities in the educational sector: (a) low education access in rural areas; (b) low quality of rural schools; and (c) dysfunctional education management system. (a) At the time of project preparation (2003) Peru displayed low coverage in both pre-school and initial education as well as in secondary education in rural areas. Exposure to early childhood development programs (mainly through non-formal childcare and parent training) and to formal pre-school education has been shown worldwide to stimulate early brain development, avoid malnutrition and stunting, boost primary school readiness and subsequent learning attainment, especially among economically disadvantaged poor and

2

indigenous children, such as those living in rural areas. Despite the fact that one year of pre-school education was (and continues to be) mandatory in Peru, in 2003 only 56 percent of rural 3-5 year old children had access to pre-schooling, compared to 67 percent in urban areas. At the time, the Government of Peru (GOP) had not yet developed an overall strategy for mainstreaming pre-school access in rural areas or supporting the delivery of non-formal initial education programs to target population. The net enrollment rate in secondary education in rural areas was 28 percent in 2003 as compared to 57 percent in urban areas. Peru’s extremely low population density and poor roads and transportation in most rural areas made expansion of secondary education using traditional face-to-face instructional models prohibitively expensive and thus favored the implementation of distance-learning education models. According to the 2001 national learning evaluation, where secondary schools did exist in rural areas, quality was usually lower than that of urban schools. (b) The low quality of rural schools was linked to: (i) low effective time on task (hours of instruction) by teachers; (ii) ill-adapted curriculum and teaching methods; and (iii) poor teacher quality and motivation. About 90 percent of rural primary schools in the country were small multi-grade schools in isolated communities having difficulties in attracting and retaining well-prepared teachers because of the prevailing harsh physical and environmental living conditions. Teacher housing in these communities was often rudimentary, if it existed at all, prompting many teachers to leave their schools for distant towns on the weekend, running the risk of not arriving back on time for class, or having to leave before the end of the school-week, because of limited transportation. In addition, teachers had to leave their schools several times a month to collect their paycheck and needed teaching supplies for the school in the provincial capital city, attend training sessions and comply with administrative procedures, forcing them to leave classrooms unattended at least several days per month. These factors, combined with high student absenteeism linked to families’ need for children’s labor, contributed to very low effective hours of instruction during the school calendar year in rural areas – between an average of 200 to 250 hours per year as compared with the nationally mandated 900 hours during the school calendar year. Teachers in multi-grade rural schools, usually were not trained in multi-grade teaching methods, and thus found it difficult to adapt and implement the existing primary curriculum. In addition, students in rural areas lacked the self-paced instructional materials that were considered essential for effective teaching and learning in multi-grade settings. Finally, the centralized teacher deployment system in existence at the time of project preparation failed to ensure that teachers could speak the local language and be able to deliver bilingual education in the early grades of primary schools in rural areas. While the formal level of qualifications of Peru’s teaching force is high, with some 95 percent of teachers holding tertiary education degrees, their average teaching performance is considered quite low as compared with other countries of similar level of income. The following are key issues underpinning low teacher motivation and performance in the rural areas: (i) mismatch between supply and demand causing an

3

oversupply of teachers in urban areas and a deficit in the rural areas; (ii) unregulated low quality pre-service teacher preparation; (iii) low and unclear standards for entry into the teaching profession; (iv) low pay and inadequate incentives; (v) duality of treatment between appointed and contracted teachers; and (vi) poor teacher deployment and supervision. Peru is a country with very limited formal employment opportunities and also limited opportunities for secondary graduates to study at the tertiary level. Thus, despite unattractive pay, because of the unconditional, rigid stability attached to tenured status, the teaching profession became an opportunity to easily enter the job market, with full-life tenure, through an accessible training. As a result, there was a massive and growing oversupply of teachers. In 2001, some 270,000 teachers were employed, while another 80,000 had a title but no position. The pre-service teacher training system produced between 15,000 and 20,000 graduates a year against needs of about 4,000. As mentioned before, while on the aggregate there was an oversupply of teachers in urban areas for primary level, ironically there were at the same time shortages in specific categories (pre-school, multi-grade, secondary and bilingual education). There was an absence of an accreditation and quality certification system regulating the entire public and private pre-service teacher training institutions in the country (53 universities with faculties of education and 400 Higher Institutes of Pedagogy, 250 of which were private). This unregulated proliferation of, by and-large, low-quality training programs had produced a dramatic increase of badly trained teachers. In addition, the disparities and deficiencies in teacher preparation were not corrected at entry into the teaching profession through rigorous and sound certification procedures prior to appointment. Hiring of new teachers was not based on formal teaching standards and rigorous procedures. Notwithstanding that recruitment tests were introduced in 1998, these were not criterion-based and did not evaluate the pedagogical ability, methodological skills, interpersonal and team skills as well as the community orientation of the candidates. The average teacher salary in Peru at the lending phase was approximately 1.6 times of the gross domestic product per capita compared to a multiple of 2 to 4 times in similar countries. This was low pay, even after adjusting for average hours of work, which were less in Peru than in other Latin American countries, and less for teachers than for other professions in the country. As a result, about 57 percent of the teaching force had taken up a second job further eroding the already limited time on task teachers could dedicate to their schools. The salary structure existing at that time was fragmented and opaque, hardly designed to promote quality, generate motivation or produce results. Most important, the allocation for rural areas and hardship posts was not only insufficient, but also portable, instead of being tied to the job position. In addition, some teachers had permanent appointment status, while others worked under fixed term contracts. The fact that there was no relationship between teachers’ qualifications and employment status had been a constant source of teacher lack of motivation and discontent. Finally, the centralized teacher deployment system had not ensured an appropriate allocation of teachers based on their language skills and/or their previous teaching experience.

4

Accordingly, many inexperienced teachers were posted to rural schools, while others lacking language skills other than Spanish were posted to Quechua and Aymara, as well as other areas speaking other indigenous languages. (c) The difficulties in the education management system were mainly manifested by: (i) an inefficient administration; (ii) a deficient school supervision and support; and (iii) low accountability of school performance. The central Ministry of Education (MED) shared the responsibilities for nationwide provision of educational services (from pre-school to tertiary education) with 24 Regional Education Directorates (DRE), which varied considerably in terms of geographic size, school system development and student population, ranging from Lima with over 2.1 million students to Madre de Dios, with less than 29,000 students. A director headed each DRE. Between the DREs and the schools level, there were 268 sub-regional offices known as Local Educational Management Units (UGELs) entrusted with administrative oversight, some of which also had a budgetary authority. The MED was heavily centralized and inefficient. Top-down initiatives emanated regularly from the central level with little concern for prioritization, avoiding duplication and losing implementation effect in the cascade-type delivery by the time they reached the schools. At the DRE there was little autonomy for decision making as well as accountability for the performance of schools in their region. Data on school performance and student learning was not organized and monitored at the DRE level, and DREs were neither rewarded nor sanctioned for their progress in improving educational indicators over time. System-wide, there was a lack of focus on the school level for attaining results. For example, at the time of project preparation there was not yet in place a standardized system of student assessment to track learning outcomes. There were also clear problems at the UGEL level, which were marked by confusing geographic and functional demarcation of responsibilities, inadequate staffing and equipment for them to carry out their duties and low valued added to their schools under their responsibility. There was no systematic school-level development planning at the school level. Plans, where they existed, were not resourced, nor were they used as an instrument to hold schools accountable for implementing proposed improvements, especially in the learning dimension. Schools did not have budgetary autonomy, and could not even undertake minor purchases or repairs. Parents were invited to serve on Parents Association (Asociación de Padres de Familia, APAFAs) basically designed to collect money to benefit the school with additional inputs, such as inter-alia, infrastructure and fungible materials. Finally, communication and access problems made contact between rural schools, regional education offices and the MED sporadic to non-existent. In a case study carried out during 2000, rural schools reported less than one supervision visit per year. Government’s Strategy. Prior to the identification of this project, the GOP launched an effort to bring together all the key stakeholders from political, religious, civil society and government organizations in order to agree on key state policies, especially in the social sector. The result was the establishment of a social pact within the framework of a National Agreement setting, which established under its second objective the key policy to provide universal access to free quality public education. Specific sub-policies under

5

this major one included Early Childhood Development, Rural Education, Teacher Development and Strengthening the Quality and Relevance of Basic Education through greater school autonomy. Under these key policies, the GOP aimed at reducing the large disparities in schooling access and completion between rural and urban areas to achieve a substantial increase in the quality and efficiency of the education system overall. The GOP determined that a holistic approach was necessary to address the issues described above and designed the following mutually reinforcing strategies: (i) improvement in education coverage and quality in the rural areas where 28 percent of Peru’s population lived; (ii) increase in the quality of teacher preparation and in-service training through the introduction of teacher standards, teacher career path, continuous training and incentives mechanisms; (iii) overhaul of the education system management to increase accountability for results through decentralization, clarified governmental attributions and responsibilities and better measurement of outcomes so that actors at all levels could be held accountable for results; (iv) education for democracy and intercultural sensitivity through curriculum modifications and stronger emphasis in bilingual education; (v) indigenous peoples strategy to address one third of the national population speaking one of Peru’s 42 indigenous languages; and (vi) demand-driven interventions to complement the traditional supply-driven interventions. Accordingly, the GOP and the World Bank (Bank) agreed on a ten-year Adaptable Program Loan (APL), to be carried out in three phases lasting four (Phase I 2004-2007), three (Phase II 2008-2010) and three (Phase III 2011-2013) years respectively, to help design evaluate and implement these policies and institutional reforms and allocate the needed financial, human and physical investment (more detail on the other two phases of the proposed APL is provided in section 5.1(a) of this Implementation Completion and Results Report - ICR). Rationale for Bank’s Assistance. The rationale for the Bank’s assistance was argued on the basis of the Bank’s extensive experience in supporting projects in rural education, with strong multi-grade, bilingual and intercultural components. In addition, the Bank preparation team made the case concerning the Bank’s wide experience with decentralization programs where local communities played an important role in the setting and implementation of social policies. The experience gained by the Bank in Central America, Brazil and Colombia in these areas was seen as instrumental in assisting MED’s efforts in developing quality education in rural areas in a more decentralized and effective fashion. Finally, the Bank had already been working extensively with the GOP in the early implementation of the proposed reforms set for in the National Agreement. The proposed Project was tied to previous Bank-financed investments in the education sector in Peru, some closed and some under preparation while others were still ongoing investments at the time of Board approval : (i) the Programmatic Social Reform Loans (PSRL I, Loan 4615-PE and II Loan 4678-PE, under preparation) promoting education policy reform through the development of the new teacher career structure and the piloting of a rural teacher incentive program, as well as providing the first steps towards decentralization and community participation; (ii) the Primary Education Quality Project (Loan 3826-PE, closed) to support expansion of access and improvement of education

6

quality for the poor; (iii) the FONCODES II – Second Social Development and Compensation Fund Project(Loan 4068-PE, closed) to support infrastructure in rural areas; (iv) the National Program for Hidrological Basin Management and Soil Conservation (Programa Nacional de Manejo de Cuencas y Conservación de Suelos, PRONAMACHCS) (Loan 4130-PE, ongoing) to promote rural development; (v) the Basic Health and Nutrition Project (Loan 3701-PE, closed) and Health Reform Project (Loan 4527-PE, ongoing) to promote rural health; and (vi) the Rural Roads Project, under preparation.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The Rural Education Project (Project) known in Spanish as Proyecto de Educación en Áreas Rurales (PEAR) was intended to be the first phase of a ten-year Program aimed at improving the quality and equity of basic education through cost-effective investments and systemic reforms of teacher quality and education management. According to Section A.2 and Annex 1 of the Project Appraisal Document (PAD) dated April 30, 2003 and Schedule 2 to the Loan Agreement (LA) dated December 5, 2003, the objectives of the Project (the first phase of the long-term Program) were to: (a) expand access in priority rural areas to early childhood development and pre-school programs and secondary school; (b) improve primary education quality in rural areas through teacher professional development, provision of multi-grade and bilingual education materials and rehabilitation of infrastructure where needed; and (c) improve efficiency system-wide by introducing a major reform of teacher policy and system management. The following key outcome indicators (appearing in Section A.3 and Annex 1 to the PAD), were agreed with the GOP at the Loan approval stage. For the long-term ten-year Program, outside the scope of the Project, the agreed outcome indicators were: (a) primary completion rates aiming to reach 97 percent by 2013; (b) rural initial education coverage for 0-2 year-olds increased from 1 to 9.7 percent by 2013 and net pre-school coverage for 3-5 year-old children increased from 58.9 to 81 percent by 2013; (c) secondary school gross enrollment rate in rural areas increased from 37 to 55.5 percent and rural secondary school coverage in grade 5 for youngsters less than 16 years old increased from 8.8 to 16.6 by 2013; and (d) Peru’s performance on national standardized learning assessments improved. For Phase I (2004-2007), the Project being assessed in this ICR, the agreed key performance (outcome) indicators and targets were:

• Percentage of 11 year-old children enrolled in sixth grade or higher in rural areas increased from 27.6 to 28.2 percent;

• Percentage of 11 year-old children enrolled in grades 5 and lower will not increase beyond 67.1 percent;

• Rural 11 year-old children who do not go to school decreased from 5.3 to 4.6 percent;

• 375,000 children received bilingual materials and their teachers trained; • Rural pre-school coverage (0-2 years) increased from 1 to 4.7 percent;

7

• Rural pre-school coverage (3-5 years) increased from 58.9 to 64.4 percent; • Secondary school gross enrollment ratio in rural areas increased from 37 percent

to 45 percent; • Secondary school coverage at grade 1 for 12 or less year-old youngsters in rural

areas increased from 8.8 to 12.1 percent; and • Learning outcomes of children in target Project Areas1 showed improvement on

national assessment at the primary level. In addition, the following trigger indicators were agreed with GOP to graduate to Phase II of the APL (Section B.4 of the PAD):

• Twenty five percent of teachers working in the Project Area undergo performance evaluation;

• No less than 3,000 teachers in rural areas under incentive scheme; • About 3,040 teachers in rural areas trained in multi-grade methodology (which

represent 80 percent of the total universe in Project Area estimated in 3,800 teachers) and all rural schools in Project Area provided with quality learning materials;

• About 7,500 primary bilingual teachers in the country (which represent 50 percent of the total universe estimated in 15,000 teachers) trained in dual language instruction and redeployed in line with their acquired skills;

• Assessment of teacher training programs carried out satisfactorily; • Under the framework of the management capacity build-up at the school level,

about 250 Institutional School Councils (Consejos Educativos Institucionales - CONEIs) established in an equal number of rural schools and operating according to current regulations; and

• At least 200 networks operating with the networked Institutional School Councils under approved Educational Institutional Projects (Proyectos Educativos Institucionales –PEIs).

Annex 1 of the PAD also includes an extensive list of output indicators for each component agreed for Phase I as well as preliminary outcome indicators for Phases II and III of the APL and agreed triggers to graduate to Phase III. All these indicators are not listed in this section of the ICR. However, Section 3.2 and Annex 2 of the ICR account for the status on the achievement of the agreed triggers for the first phase as well as for the implementation performance of each Project Components including the status of some of their corresponding outputs at the time of the closing of the Loan.

1 According to Article I, Section 1.01(r) of the LA, “Project Areas” meant that originally the project comprised the Regions of Amazonas, Ancash, Ayacucho, Cajamarca, Cusco, Huancavelica, Loreto, Piura, Puno, San Martin and Ucayali, and any other Department that may be acceptable to the Bank in addition to, or in substitution of, the above Regions .

8

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification PDO and key outcome and output indicators were not formally revised during project implementation. As further explained in Sections 2.3 and 3.2 of the ICR, some of the key outcome and output indicators included in the PAD were slightly modified in the ISRs and are included in the ICR data sheet. These changes were implemented in order to reflect the new methodology used by the Statistics Unit in MED to estimate efficiency indicators.

1.4 Main Beneficiaries, (original and revised, briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project) Section C3 of the PAD stated that the Project would benefit rural children of all school ages through increased coverage and quality of education. With respect to the expanding access to be financed by the Project, the non-formal and formal pre-school education aimed at benefiting 30,000 children 0-2 years old in rural areas, including a large proportion of indigenous children and 38,000 additional children 3-5 years-old in pre-school programs of which 11,000 would represent increased coverage financed by the Project. In addition, the distance secondary education program was expected to benefit approximately 5,000 rural students in 116 schools. With respect to the quality interventions to be financed by the Project, the continuous teacher training was aimed at benefiting 1,500 teacher trainers in public sector non-university training institutions as well as 3,800 teachers in the Project area working in multi-grade schools and 15,000 bilingual teachers nationwide. In addition, 330,000 rural children in the Project Area and 375,000 indigenous children nationwide were expected to benefit from all the quality-related interventions (provision of learning materials including bilingual educational materials and improved teaching practices). Finally, about 1,900 classrooms in poor rural areas were expected to benefit from rehabilitation or substitution of infrastructure to be financed by the Project. With respect to the reform of teacher policy and education management, the increased school autonomy and the networks supported by the Project were expected to benefit about 3,180 schools by allowing teachers and local communities to carry out local planning and receiving closer and more frequent support from itinerant teacher support groups. Finally, the Project was expected to restructure the 24 DREs and to strengthen the statistical and student assessment systems.

1.5 Original Components (as approved) The following three components were included in the Project’s original design as fully explained in Section C.1 and Annex 2 to the PAD and in Schedule 2 to the LA (more

9

detail on the implementation of these components and their outputs is provided in Section 3.2 and Annex 2 to the ICR): Component A – Expanding Education Access for Rural Children (Total Cost US$10.54 million; US$6.97 million from the Loan). This component, representing about 11 percent of the total loan, aimed at financing access to quality: (i) initial (for 0-2 year-old children) and pre-school education (for 3-5 year-old children) costing about US$2.71 million; and (ii) secondary distance education costing about US$7.83 million. Activities under this component included, inter-alia: (a) carrying out baseline studies on rural families’ upbringing practices as well as other studies related to pre-school and indigenous education; (b) design of alternative service delivery models for both initial and secondary education; (c) preparation, publication and distribution of manuals and training modules for facilitators of the initial modality (Animadoras), teacher coordinators and parents; (d) production and dissemination of audiovisual educational materials; (e) selection and procurement of other educational materials as well as improvement and procurement of existing learning materials; (f) in-service teacher training; (g) creation and delivery of pedagogical support mechanisms; (h) development of community programs; and (i) the carrying out of several evaluation studies including assessments of alternative initial education, integral care delivery models and secondary distance education. Component B – Improving Quality in Rural Primary Schools (Total Cost US$63.58 million; US$32.05 million from the Loan). This core component, representing about 68 percent of the total loan, aimed at financing: (i) continuous teacher training systems costing about US$24.86 million; (ii) curricular adjustment and provision of educational and communications materials costing about US$9.09 million; and (iii) rehabilitation and equipping of rural schools costing about US$29.63 million. Activities under this component included, inter-alia: (a) a redefinition of the pre-service teacher training curriculum and adoption of new training and performance standards; (b) establishment of accreditation system for the Teacher Training Institutions (TTIs); (c) training for masters trainers at the TTIs; (d) provision of access to updated educational information and best national and international practices through the financing of competitive TTI’s innovation initiatives; (e) establishment of in-service teacher training modules focusing on multi-grade methodologies and bilingual education; (f) developing social marketing activities designed to promote teachers’ professional standing among the educational community and society at large; (g) evaluation of the revised pre-and in-service teacher training programs including a baseline study; (h) design, validation, acquisition and provision of educational and communicational materials to support multi-grade methodologies and bilingual education; (i) carrying out studies on systematization of knowledge, practices and histories of indigenous peoples for curriculum diversification as well as on ethnographic issues related to classroom work, children’s games and use of free time; (j) rehabilitation and equipping of approximately 1,900 classrooms and 1,500 complementary facilities in the Project Area; and (k) provision of preventive maintenance of the classrooms and complementary facilities rehabilitated under the Project.

10

Component C – Reform of Teacher Policy and Educational Management (Total Cost US$20.08 million; US$12.86 million from the Loan). This component, representing about 21 percent of the total loan, aimed at financing: (i) a Teacher Policy Reform costing about US$1.6 million; (ii) a Reform of Educational Management costing about US$6.94 million; (iii) a School Development Fund costing about US$1.76 million; (iv) a National Assessment System costing about US$4.61 million; (v) Strategic Analysis and Policy-oriented Research costing about US$2.82 million; and (vi) Project Management costing about US$1.83 million. Activities under this component included, inter-alia: (a) the design and implementation in the Project Areas of a teacher career development system; (b) development and dissemination of guidelines, operating manuals, software and other instruments to share information on the teacher career development system; (c) establishment of six Amauta Centers 2 ; (d) upgrading the teacher management information system as well as refurbishing of the statistical system to ensure timely collection, processing and analysis of educational data and validation of school networks; (e) carrying out of national and local workshops to involve teachers and other members of the educational community of the policy reform pursued by the Project; (f) establishing, training and monitoring Institutional School Councils and School Network Councils; (g) designing, implementing and disseminating school network institutional development plans, including participatory planning, management, monitoring and evaluation instruments; (h) establishment and evaluation of rural school networks; (i) developing and disseminating the concept of educational community involving a variety of stakeholders; (j) incorporating community initiatives into the operation of Institutional School Councils, School Networks Councils and the design of school network institutional development plans; (k) designing, establishing and operating of a school development fund, to finance the design and implementation of improvement programs developed and managed by schools and school networks in the Project Areas; (l) developing and administering in 2004 a series of national and census-based learning assessments in language, mathematics, communication skills, citizenship education and social skills for primary and secondary education students; (m) carrying out longitudinal studies, school effectiveness studies, teacher career development studies, decentralization and school autonomy studies, demand-driven incentives studies, classroom intercultural related studies and other related institutional research; and (n) the management of the Project by a Project Coordinating Unit (PCU) including the required annual audits and impact evaluation studies.

2 The “Amauta Centers” established by the Project were conceived as a pro-active and demand-driven teachers professional development centers aimed at: (i) planning, coordinating, monitoring and evaluating regional in-service teacher training activities; (ii) organizing regional in-service teacher training demand expressed by schools and school networks and help identify teacher training institutions able to address such demands; and (iii) providing technical assistance to schools, school networks and MED regional and district offices.

11

Table 1 shows the original Project costs and all sources of funding as they appear in Section C.1 of the PAD.

Table 1. Original expected costs and sources of financing at appraisal

Components Total indicative Costs

(US$M) (1)

Bank financing (US$M)

(2)

GOP counterpart

(US$M) (3)

% financed by the Bank

(2)/(1)

1. Expanding Education Access for Rural children

1.1 Access to quality initial and pre-school education

1.2 Secondary distance education 2. Improving the quality in

rural schools 2.1 Continuous teacher

development systems 2.2 Curricular adjustment and

provision of educational materials

2.3 Rehabilitation and equipping of rural schools.

3. Reform of teacher policy and

education management 3.1 Teacher policy studies 3.2 Reform of education

management 3.3 School development fund 3.4 National assessment system 3.5 Strategic analysis and policy-

oriented research 3.6 Project Management

2.71

7.83

24.86

9.09

29.63

1.60 6.94

1.76 4.61 2.82

1.83

1.45

5.52

15.74

3.45

12.86

1.25 5.26

0.07 2.75 2.26

1.37

1.26

2.31

9.12

5.64

16.77

0.35 1.68

1.69 1.86 0.56

0.46

53.5

70.5

63.3

37.9

43.4

78.1 75.8

0.04 59.6 80.1

74.9

Total project costs 93.68 51.98 41.7 55.5 Front-end-fee 0.52 0.52 0.00 100.0 Total financing required 94.20 52.50 41.7 55.7

1.6 Revised Components Components were not formally revised, though the GOP and the Bank had agreed in early 2007 to proceed with the restructuring of the Project. The restructuring would have been aimed to simplify the original project design, including the cancellation of five sub-components, and the extension of 18 months to the loan’s closing date. MEF decided, late in 2007 not to extend the Project. Likely reasons for this decision included poor project implementation performance, weak managerial and implementation capacity and unclear political commitment of MED, as well as an overall political mandate to reduce externally-funded loans. Thus, the original components were not revised during Project

12

implementation. Section 2.3 of the ICR describes in more detail the reasons for restructuring and the restructuring proposal. 1.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) No other significant changes to the original project design were made during project implementation. Still, there were certain actions that resulted in some changes in Project implementation strategy. In June 2005 MED redesigned the implementation strategy. This new strategy called for project activities to be phased in, targeting initially few specific places before launching their scaling up. This strategy arose in response to the fact that the various pilots that were expected to take place during the lending phase did not occur and had to be implemented at the beginning of the Project. The LA experienced two minor and simple administrative-type amendments: (a) the first, approved by the Bank on May 26, 2005 and countersigned by the GOP on June 17, 2005, allowing the Director General of Public Credit of the Ministry of Economy and Finance of the GOP to sign loan-related withdrawal applications; and (b) the second, approved by the Bank on August 30, 2005 and countersigned by the GOP on September 13, 2005, eliminating the FONCODES Special Account that was originally opened for the rehabilitation of school infrastructure in rural areas and allowing transfers to this institution to occur through the MED Special Account. The second amendment came about by the absorption of FONCODES into the Ministry of Women and Social Development during the implementation cycle, thus legally impeding FONCODES to execute project-related funds that were part of MED’s budget. During the mid-term review, the Bank and the GOP agreed that project restructuring was necessary. Although the restructuring did not occur, MED started implementing some of the changes in mid-2007. The purpose of the restructuring, which would have included an 18-month extension, would be to: (i) reduce the number of sub-components from 11 to 6; (ii) replace targets for outcome and output indicators with more modest, but achievable targets; and (iii) focus the geographical scope of project implementation. The proposed restructuring was to emphasize initial education, multi-grade and bilingual intercultural education in rural areas and the strengthening of in-service teacher training for the above-mentioned levels and modalities of education. Alongside, a request to cancel about US$12 million of the outstanding loan was going to be made with the extension. While actively participating in the early restructuring discussions, at the end, MEF did not support the extension based on their concerns with respect to: (i) the weak financial, managerial and physical performance record shown by MED since the Loan effectiveness; and (ii) the perception that MED would not be able to implement the Project in the absence of a solid administrative infrastructure. As a result, the GOP and the Bank decided not to proceed with the second phase of the APL scheme considered at appraisal.

13

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) This and the following sections in the ICR were written based on: (i) a thorough review of the Project files; and (ii) the findings of an ICR field mission carried out in May 2008 which including a visit to the Region of Cusco, which was one of the Regions that received the greatest amount of support from the Project. The Project was subjected to a formal Quality at Entry Assessment (QEA) carried out on July 15, 2003, about two months after Board approval. The QEA rated the preparation cycle as satisfactory. On the other hand, this ICR rates Quality at Entry as moderately unsatisfactory, based on current best quality at entry practices recommended in Bank guidelines, which were not in place at the time of Project appraisal. A summary of the ICR’s assessment on these topics follows, while section 5.1 (a) of this ICR provides more detail on this assessment. The strategic relevance and approach selected by the Bank team was the correct one based on the extensive sector work underpinning the Project design. Nonetheless, the great complexity associated to the holistic nature of the design imposed a series of challenges for the implementation, coordination and monitoring of activities that should be taken into account for further Bank operations of this type on behalf of efficiency, efficacy, impact and sustainability. The selection of an APL as the financial instrument to gradually achieve GOP’s education sector medium-term vision derived from the national consultation process was also a correct decision by the Bank team. The Project’s lending files attest to client orientation through effective policy dialogue with the GOP and an extensive consultation process ensuring in the process GOP’s ownership of the Project design. With the constant changes of Governments, the Bank team and the regional management rightly waited until the new Government provided clear signs of its full support of the Project, and thus, this explains the long time it took to prepare this operation. Preparation took approximately four and a half years, from late 1998 until May 2003, when the Project was approved by the Board. During this period, there was political turmoil, resulting in the re-reelection and subsequent resignation of Mr. Fujimori (2000), a Transition Government (November 2000-July 2001) that did not want to commit to medium-term initiatives, and the elected Government of Mr. Toledo (starting in July 2001). Once the Toledo Government showed interest and commitment, the preparation process on the Government side required that the Project be appraised under the National Public Investment System (Sistema Nacional de Inversión Pública, SNIP). Given the rigidities and requirements embodied in the SNIP, this process took more than a year to conclude.

14

There is evidence of effective partnership by the Bank team through the development of good working relationships with other donor agencies, in particular the Inter-American Development Bank (IDB) and the United Kingdom’s Department for International Development (DFID). The Bank team ensured compliance with Bank’s safeguard policies, in particular with the environmental and indigenous people guidelines. With respect to the risk assessments, the PAD did not mention any risk of newly trained and upgraded teacher demanding higher pay and the impact that this would have on the sustainability of the Project. The Bank team included lessons learned from the Social Development and Compensation Fund (FONCODES, Ln.4068-PE, closed) and the Primary Education Project (Ln. 3826-PE, closed). The ICR detects four quality-at-entry shortcomings associated to initial assumptions that were not met. First, although the approaches included in the Project design were well-tested and technically proven effective elsewhere, the Project design was overly ambitious covering too many fronts thus hindering implementation performance, especially in the context of the high centralization and managerial weaknesses of MED. Second, as a direct consequence of the complexity of the project design (in terms of components, sub-components and activities), the implementation strategy was not well defined, thus lacking a critical route that would have allowed the technical team to visualize priorities and to update them during the implementation cycle. Third, the original targets set at the time of appraisal resulted too ambitious given the weak capacity of the MED. Finally, notwithstanding the institutional capacity building assessment done by the Bank team during the lending cycle, the ICR detects additional shortcomings in this assessment (further explained in more detail in section 5.1(a) of the ICR) leading to the conclusion that implementation readiness conditions were not fully present at the time of Board approval. Weighting the above strengths and weaknesses in the preparation stage of the lending cycle, in particular the over ambitious design and the lack of implementation readiness at the time of Board approval, the ICR ranks the Quality at Entry of this Project as moderately unsatisfactory.

2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) This section provides the key factors and events through the project implementation cycle that contributed to its achievements and shortcomings. While there were no factors beyond government control there were three areas where events and decisions at the government level significantly jeopardized project implementation. While the National Agreement and the ample consultation process carried out during the preparation phase raised the need to target social programs, particularly in education and health, to rural and indigenous populations, frequent changes in high-level decision

15

makers and technical staff meant that priorities with respect to policies and strategies changed. The current government’s education policies are now focused on teacher development reforms through evaluation, strengthening the quality-at-entry of prospective teachers into the educational system and training. Moreover, the focus on the development of new pedagogical and management models appropriate to the rural context lost priority. Thus, decision related to rural and bilingual education as well as for community participation and network support took extremely long times to come to closure, if any, and the technical staff working on these activities did not feel supported in their own decision-making process. This Project was the first one in the social sectors to be subject to the SNIP, thus paying the price of working in the context of a highly rigid, still untested, assessment norms. The SNIP required that any changes implemented by the Project be previously approved by the Dirección General de Programación Multianual (DGPM). The proposed restructuring required an elaborate process, including supply-demand and cost-benefit analyses. The preparation of the required documents for the restructuring took about one year (starting after the mid-term review in December 2006). The length of this process can be attributed to the weak capacities of the MED team to undertake the Project analyses required by the SNIP, in light of a very small team carrying out a very difficult task with extremely complex requirements. Furthermore, organizational changes within MED essentially short circuit its project implementation capacity. As part of a national policy and law on merging of programs aimed at achieving greater efficiency in public sector management, a national directive was issued on August 30, 2007, ordering the merging of the PCU in the Administrative Office (Oficina General de Administración) of MED. Despite the repeated warnings made by the Bank supervision team with respect to the need to revise the administrative processes prior to proceeding with the restructuring that had been agreed in principle with the Bank3, MED enacted the merging without having in place the appropriate transferring mechanisms. The PCU legally ceased to exist as such, and the Project came to a complete stop. With this measure, the GOP failed to comply with articles 3.03 (a) (ii) and (b) of the LA with respect to maintaining a PCU, staffed by core professional staff in numbers and with the experience and qualifications acceptable to the Bank, throughout the entire implementation cycle. In addition to the above-mentioned areas at the government level, three other factors at the project level negatively affected implementation. First, administrative and technical readiness conditions were not met as agreed, resulting in implementation delays. Effectiveness conditions, including establishment of the steering committee, approval of the operations manual, and signing of the MED-FONCODES agreement took longer than originally expected, thus the very slow start up lasting practically until the end of 2005.

3 The Bank team discussed the implications of these changes with MED during the mission of June 2007 and followed-up with a workshop led by the Financial Management Specialists where Bank policies on fiduciary responsibilities were discussed.

16

In addition, project design was built on the commitment from MED to allocate adequate resources to develop and initiate validation of the initial, multi-grade and secondary education models. This commitment was not met, and piloting actually occurred during the first two years after effectiveness. Second, the design assumed that MED had the capacity to effectively implement the Project, given the satisfactory implementation record shown on the previous Bank-financed project. Thus, the previous implementation design was used, consisting of a small PCU that would manage project funds and carry out the planning and coordination of the Project. Technical staff would be stationed either in the technical directorates within MED or in the regions. These assumptions were not met, and the Project’s design resulted to be extremely complex given the current managerial, fiduciary and technical limitations shown by the centralized and weak managerial capacities of MED. To counter the difficulties in project management, MED established a cumbersome coordination structure4 which did not operate well and did not resolve the problems. Throughout the implementation phase, activities were not well articulated (specifically between the pedagogical-related activities and the institutional-related interventions), and role confusion was pervasive.5

Third, in the context of the decentralization process at the time, which attributed new responsibilities to the Regional Governments, the Project design assumed that Regional Education Directorates would take on a very active role in the implementation process. Thus, the Project was expected to strengthen technical and management skills in the Regions, which would have a fluid communication with MED, in order to implement centrally issued policies. Nonetheless, in practice, such coordination was either poor or inexistent. The modus operandi of MED was based on direct communication with Local Educational Management Units and prioritized logistics instead of technical aspects. This was evidenced during the field visit to Cusco in which several authorities expressed their uneasiness about how the implementation process had actually been conducted “in the field” following the path Lima - Canas (local level) with no coordination with Cusco (regional level).

4 An Executive Committee (Comité Directivo) with seven members was established in March 2004 by Ministerial Resolution overseeing and Administrative Committee for the Project (Comité de Gerencia) also established by Ministerial Resolution in October 2004 and including nine members. In addition, under the context of ensuring more participation of key areas of the MED and other important stakeholders outside the MED, a third layer of decision making was established – the Operational Committee. These three layers of decision-making significantly delayed project implementation.

5 This was notorious in the interaction of the central authorities of the MED with its more decentralized administrative units in the field (DREs and UGELs).

17

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The Project’s monitoring and evaluation scheme was not ready at the time of Board Approval. The baseline was provided in September 2006, more than three years after the approval of the loan and more than two years after the project launching and the effectiveness of the LA. Some output indicators, very few as compared to the original list included in the PAD, were first reported in the Implementation Status Results and Report seventh (June 6, 2006), while also few outcome indicators were first reported in the ninth ISR (December 12, 2006), one year before closing the loan. At the time the ISRs were introduced in the Bank as the official format to communicate supervision findings (much after the date of the Board approval), a directive was issued to limit the number of outcome indicators to no more than four or five. Therefore, the task team leader was encouraged to select “tracer” indicators to be reported in the ISRs, and thus, the number of outcome and output indicators reported in the ISRs is smaller to that in the PAD. The indicators not reported in the ISRs were continuously included in the quarterly Financial Management Reports (FMRs) sent to the Bank. According to the ISRs, the quality of the information on outcomes and outputs provided by the GOP was qualified as fair. Prior to the implementation of the Project, MED did not have a monitoring and evaluation system in place. Thus, the Project facilitated the opportunity to design, validate and implement an information platform that was expected to be up-scaled at the sector-level. Still, the Project’s effort in this area raised awareness among MED officers and staff about the importance of generating high-quality data for making informed outcome-oriented decisions. During the field visit to Cusco, the team observed that, while monitoring exercises were actively promoted by project staff from Lima, local actors indicated that there were several challenges around this issue. Among them, there are four that deserve attention: (i) emphasis of monitoring as a “checklist of activities” (inputs and outputs) rather than focusing on outcomes; (ii) lack of a minimum set of clear, understandable and timely indicators that could locally orient the monitoring process and the involvement of relevant actors; (iii) insufficient support from regional and local education staff associated to a limited capacity to comply with this role given the lack of experience and/or human resources; and (iv) focus of monitoring exercises as a control mechanism instead than as an opportunity to locally provide feedback and readjust targets. The Bank team insisted, both during project preparation and implementation, on independent evaluations of the pilots being tested in initial and secondary levels. Despite the insistence of the Bank supervision teams to ensure a results-based project, MED did not contract rigorous evaluation studies of these pilots to be conducted during the implementation cycle, beyond the excellent work conducted to establish a comprehensive

18

baseline data6. The Project was key for MED to administer the standardized learning assessment of 2004 and use the findings to fine-tune project activities. Notwithstanding the activities financed by the Project to strengthen the national learning assessment system, a decision was taken by MED not to conduct, during 2007, a new assessment to update the information collected in 20047. The reason for this decision was that MED gave priority to the implementation of the second census-based student learning assessment for second graders and bilingual fourth graders administered in December 20078. The results of a Project impact or outcome evaluation study and/or new learning assessment information to compare with the 2004 findings were not available at the time of preparation of this ICR. Therefore, it has not been possible to establish that the few outcome gains indicated in the data sheet of the ICR are a direct result of the interventions of the PEAR (more on this in Section 3.2 of the ICR).

2.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable) The implementation readiness condition with respect to the procurement of goods, services and civil works to be financed under the loan were not present at the time of Board Approval. The procurement capacity assessment undertaken during the lending cycle identified some procurement-related issues that were not appropriately addressed by the GOP prior to effectiveness. No approved final Project’s Operational Manual was available at this time, although a draft had been discussed with the Bank. There was a preliminary draft of a first-year procurement plan that was discussed during appraisal but was not included in the Board package. Finally, the fact that the PCU lacked appropriate capacity to carry out these activities following Bank guidelines and procedures at the time of loan effectiveness, contributed to implementation delays. This Project was subjected to three ex-post Procurement Reviews that were carried out in June 2005, April 2006 and November 2006. Some of the findings of these reviews were: (i) lack of formal status and inability to make decisions caused the PCU, in practice, to

6 This baseline study pioneers work done in the social sectors in Peru, in particular in the education sector. It includes very rich and useful information concerning, inter-alia: (i) net enrollment rates for different age groups; (ii) internal efficiency data with respect to completion rates, dropout rates, repetition rates in primary and secondary levels; (iii) enrollment data by educational levels and modalities; (iv) number of teachers and school principals by educational level and modality; (v) learning assessment outcomes; and (vi) other information related to school community participation. The above information was collected for a comparable control group as well as for the treatment group. Once the preliminary findings of the 2008 learning assessment are made available, comparisons between schools benefited by the Project with respect to schools in the control group could be made.

7 The assessment has been postponed for 2008. Funds have been already allocated under the national budget.

8 Most likely, the preliminary findings of these assessments would be made available in the second semester of 2008, after this ICR has been submitted to the Board.

19

rely heavily on approvals and procurement-related decision made at different levels of the MED (including the General Administrative Office, the Legal Department and the Secretariat General) generating, in turn, lengthy and heavy acquisition and selection processes (on average, twice as long as the time set for in the Operational Manual); (ii) the PCU lacked appropriate procurement-related capacity to follow through the Bank’s guidelines and procedures effectively and in a timely manner, a situation that worsened with the continuous turnover of key personnel at both, the PCU and the MED; (iii) duplication due to unnecessary reviews and approvals caused by confused roles, cumbersome coordination mechanisms and accountability channels making the existing administrative arrangements in place not efficient nor conducive to expeditious and timely decisions; and (iv) inappropriate filing system. These additional processes not only did not add value for ensuring more effective controls, as intended, but to a large extent, explain the reasons for major delays and inefficiencies present in the low implementation progress rate achieved in the first two years after loan effectiveness. The review of the FMRs submitted by GOP from the third quarter of 2004 to the third quarter of 2007 showed that: (i) overall, the FMRs prepared and furnished to the Bank complied with Section 4.02(a) to the LA but were, often incomplete and, by and large, submitted to the Bank with delay outside the 45 day period after the end of each subsequent quarter; (ii) weak planning, budgeting and accounting processes led to a debilitated financial management oversight of Project execution by the PCU; (iii) deviations with respect to expected physical and financial targets were not sufficiently explained; and (iv) the PCU did not follow-up appropriately the corrective measures agreed upon with the Bank. Audit reports were submitted to the Bank on time. Even though the auditors found that the MED complied with the terms of the agreement and the applicable laws and regulations, some exception to the compliances of the clauses were identified, including: (i) the Operations Manual was not updated on a timely basis, (ii) the Project suffered continuous delays in submittal of the Annual Operational Plan (POA) for the Bank’s approval, (iii) project progress reports and FMRs were also sent to the Bank after the agreed deadlines, (iv) transfers for an amount of S/.6,136,570 made to FONCODES in 2005 were not registered as an advance account (cuenta pendiente de rendición) in the Balance Sheet, requiring further monitoring by the Bank’s FM team. In addition, the Auditor’s Management Letter identified some deficiencies with respect to the internal control process: (i) procurement process documents were not adequately supported and did not keep a sequence; (ii) Annual Procurement Plans experienced several modifications without appropriate approval; (iii) the Operations Manual was never updated to reflect changes in the PCU; (iv) some consultants services were incorrectly registered under Category 5-Operational Costs and some training events were registered under Category 3-Consultant Services; and (v) cash advances that were not utilized by consultants were only reimbursed to the Project between 6 and 88 days after contracts were completed.

20

Given the above, in addition to supervision mission findings, the procurement and financial management were rated unsatisfactory (U) in the last ISR (13). The indigenous peoples plan was appropriately implemented, albeit at a slower pace as anticipated and was also subject to weak support from the decision-making levels in the MED. Although an evaluation was not made, a spot check showed that FONCODES’ environmental guidelines were used and, overall, there were no significant environmental impacts.

2.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable) Despite the decision taken by the GOP and the Bank not to proceed with the second phase of the proposed APL, the Borrower is technically, institutionally and financially sustaining some of the activities financed by the Project, although at a low pace. First, the validated pedagogical innovations in initial, multi-grade education, bilingual and intercultural education are being gradually scaled up by the MED, including the corresponding teacher training and the acquisition of learning materials. Second, the strengthened national assessment system will undertake a learning assessment measurement and produce the preliminary findings of the Student Census Learning Assessment for second-graders and bilingual fourth graders in the second semester of 2008. Third, the teacher policy reform is progressing, albeit at a slower pace than expected. The Peruvian Congress just approved the Laws corresponding to the establishment of the Teaching Professional Development Career (Carrera Magisterial) and the National Educational Assessment and Certification System (Sistema Nacional de Acreditación y Certificación Educativa, SINEACE).

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) At the lending stage. The Project design was consistent with the Country Assistance Strategy (CAS) for Peru (Report 24205-PE), discussed by the Board on August 19, 2002, covering the period until December 2006. The Project was aligned with the overall CAS objective of supporting the GOP’s strategy for poverty reduction focusing on competitiveness and employment generation, access to basic social services (health, education and culture) and creating a public administration that served the people. That CAS aimed at promoting the recovery of growth by investing in education, which was thought to be the key to raising productivity, especially for the poor, whose only asset was their human capital.

21

The Project was expected to contribute to macroeconomic stability by improving the efficiency of social spending, in this case, through investments in the education sector in rural and indigenous areas of greatest poverty, helping, as a consequence, to mitigate the negative impact of the economic slowdown on the poor. The long-term ten-year program, for which the Project constituted the first four-year phase, was thought to explicitly: (a) support the development of more efficient institutions in the education sector by strengthening accountability and streamlining regional administration; and (b) strengthen stakeholder voice in education by decentralizing managerial decision-making processes to the school-level councils involving parents and other key community members. Current GOP’s policy and Bank strategy. The current Country Partnership Strategy (CPS) for the Republic of Peru (Report 37913-PE) dated December 19, 2006 discussed the direction of future Bank support to help the new administration of President Alan Garcia achieve its goals of poverty reduction, decreased inequality and more effective and responsive governance. The CPS describes the proposed support corresponding to the Bank’s fiscal years 2007-11 grouped into a combination of activities, referred to as clusters that together aim to contribute to the achievement of the envisaged outcomes. The CPS includes six clusters, two of them directly related to the PDOs of the Project: (i) promoting and developing a new social contract in education, health and nutrition; and (ii) modernizing state institutions. Under the first cluster mentioned-above, Bank support would be geared to continue financing interventions aimed at improving the quality of education as a way to raise the basic standards in the educational levels, especially among the poorest segments of the population. It would also support the key policy changes needed to develop this results orientation in the social sectors (education included) and to support the decentralization of education and health programs to the municipalities. Under the second cluster, the focus would include, inter-alia, interventions to strengthen public sector management and assist in the process of decentralization, including the one envisaged for the MED. As per the above, the Project’s PDO currently continues to be highly relevant.

3.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) This section summarizes progress in terms of the PDO outcomes and output indicators. Annex 2 provides further details on the achievements of the Project’s outputs by component while Section 3.3. and Annex 3 present the efficiency and economic analysis. Assessment of PDO Outcome Achievements. Achievement of the PDOs are rated moderately unsatisfactory for the following reasons: The PAD included indicators for all three levels of schooling, initial, primary and secondary. There were three efficiency indicators for primary (see table below) which aimed at monitoring desertion as well as overage children, two key problems found in

22

rural areas. In addition, there was a specific indicator aimed at bilingual schools. This indicator was expected to be modified once the data on indigenous bi-lingual population was measured. Unfortunately, the 2005 census did not include the question on language spoken at home and the data for the 2007 census is still not available. The situation for secondary education is similar to that for primary, that is, there are two indicators aiming at measuring overall enrollment and overage children.9. All PDO indicators (see Table 2), where available, showed an improvement, often meeting the target or surpassing it. This is the case for: (i) percentage of 11 year-old children enrolled in sixth grade or higher in rural areas which increased by 0.6 percentage points; (ii) rural 11 year-old children who do not go to school which decreased by 0.7 percentage points, (iii) rural pre-school coverage (3-5 years) which increased by 5.5 percentage points; and (iv) secondary school coverage at grade 1 for 12 or less year-old youngsters in rural areas which increased by 3.3 percentage points. While it is possible that the Project did contribute to this improvement in these indicators, but it is impossible to determine attribution, particularly given that output indicators did not reach the intended targets (see section below).

Table 2

Achievement of PDO Indicators10

Indicator* Baseline Target Actual Percentage of 11 year-old children enrolled in sixth grade or higher in rural areas increases by 0.6 percentage points


37.66% 41.70% (rural) 50.21% (project area)

Percentage of 11 year-old children enrolled in grades 5 and lower will not increase

67.1% 67.1% (or less)

NA

Rural 11 year-old children who do not go to school decreases by 0.7 percentage points

7.41% 6.71% 3.87%

375,000 children received bilingual materials and their teachers trained

0 375,000 239,520

Rural pre-school coverage (0-2 years) increases 1% 4.7% NA Rural pre-school coverage (3-5 years) increases by 5.5 percentage points

48.09% 53.59% 54.79%

Secondary school gross enrollment ratio in rural areas increases by 8 percentage points

71,39% 79.39% 77.74%

9 The preliminary findings of the new learning assessment measurement to be administered during 2008 will allow some comparisons with the baseline data for the treated as well as for the control groups.

10 Note that baseline indicators are different from those in the PAD. The reason for this is that the baseline study recommended the use of different sources of data. The original baseline numbers were calculated using the national student census as denominator and population projections from INEI, based on the 1993 census. These projections, as compared to the data from the 2005 census, showed to be inaccurate. Thus, baseline data used as denominator the data from the latter census. Baseline data are for 2004, actual data for 2007, unless otherwise indicated.

23

Indicator* Baseline Target Actual Secondary school coverage at grade 1 for 12 or less year-old youngsters in rural areas increases by 3.3 percentage points


40.3% 41.70% (rural) 50.21% (PEAR districts)

Learning outcomes of children in target Project Areas11 showed improvement on national assessment at the primary level

2.3% (language) 1.3% (math)

increases NA

None of the triggers to graduate to the second phase (included in section 1.2 of the ICR) were complied by the GOP. Assessment of the intermediate output indicators. Table 3 below shows the output indicators for the various interventions under the project. As per the discussion that follows, outputs were only partially attained, and, thus, achievement of expected results of the Project’s components is rated as moderately unsatisfactory. The data shows that MED made some advances in project implementation and that, assuming an adequate management structure, outputs and results could have been achieved if the Loan’s closing date and project implementation would have been extended for some of the components. In particular, the bilingual education program showed significant progress, both in teacher training (71 percent of the initial target) as well as materials (95 percent). Also, the multigrade model validation was completed, and was ready to be brought to scale (39 percent of the target teachers were already trained). Although the national learning assessment shows as having achieved only one of two sample based tests in the Project area, the learning assessment unit carried out, instead, a census based reading-comprehension test in December 2006 and a second one in December 2007 which included, in addition to reading-comprehension, a math test for second graders. The Bank agreed with MED that these census-based tests could potentially have a much greater impact in school accountability and improvement. On the other hand, implementation of other activities fell significantly short of the original targets. In particular, initial and secondary education validation of pedagogical models, which were supposed to occur during the preparation phase, took a significant amount of time to implement and, thus, were never brought up to scale. As a result of the difficulties in FONCODES, which was responsible for rehabilitation of infrastructure, only 27 percent of the target was achieved. In the case of the third component, Reform of Teacher Policy and Education Management, some progress was made in training CONEI

11 According to Article I, Section 1.01(r) of the LA, “Project Areas” meant that originally the project comprised the Regions of Amazonas, Ancash, Ayacucho, Cajamarca, Cusco, Huancavelica, Loreto, Piura, Puno, San Martin and Ucayali, and any other Department that may be acceptable to the Bank in addition to, or in substitution of, the above Regions .

24

members, but this progress was not sufficient to ensure the development of the school development plans (PEI), the main instrument for school-level improvement.

Table 3. Other output indicators

Sub-Component

Output Unit PAD target Status as of December

2007

Achievement (%)

0-2 year-olds enrolled

Children 30,000 1,877 6

3-5 year-olds enrolled

Children 38,000 4,225 11

Initial Learning

materials distributed

Texts

200,000

109,500

55

Enrollment Students 5,000 879 18 Secondary Learning materials distributed

Distributed materials

50,000

28,018

35

In-service teacher training institutions restructured using new curricula

Institutions

21

0

0

Teachers trained at these institutions

Teachers

1,200

0

0

Teachers trained for initial education (0-2)

Teachers

1,000

362

36

Continuous in-service teacher training

Teachers trained for pre-school education (3-5)

Teachers

1,000

374

37

Teachers trained in multi-grade methodology

Teachers

3,800

1,513

40

Teachers trained for bilingual education

Teachers

15,000

13,952

93

Centros Amauta functioning

Centers

6

3

50

Curriculum and materials

Bilingual learning materials distributed

Sets of materials

1,800,00

1,712,140

95

25

Sub-

Component Output Unit PAD target Status as of

December 2007

Achievement (%)

Curriculum and materials

Validated models for multi-grade and bilingual education

Validated model

1

1

100

Infrastructure Classrooms rehabilitated or substituted

Classrooms

1,900

293

15

Teacher reform Teachers under the new incentive system

Teachers

3,000

1,160

39

School improvement fund

PEIs being financed under this fund

PEIs being implemented

100

0

0

Support networks

Networks 290 96 33

CONEIs established

CONEIs 3,000 1,231 41

Institutional Strengthening

CONEI members trained

Persons 10,000 7,647 76

National learning assessment

Learning assessment measurement in Project areas

Administration of a learning assessment test on a sample basis in the Project area

1

1

100

3.3 Efficiency (Net Present Value/Economic Rate of Return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and Financial Rate of Return) Data quality and limited implementation precluded a substantial economic analysis to be carried out. The Project may have had some impact on greater efficiency in children graduating from primary schools in rural areas. Still, the data was not sufficiently accurate to draw solid conclusions with respect to Project efficiency.

3.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency) Rating: Moderately Unsatisfactory Given the partial achievement of outcome and output indicators with respect to the targets originally established for the project, the relevance of the PDOs, and the Project efficiency, overall outcome is rated as moderately unsatisfactory.

26

3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) (a) Poverty Impacts, Gender Aspects, and Social Development Poverty impacts are likely not significant given that the Project’s overall outcome is rated as moderately unsatisfactory and with high risk to sustainability (see below) due to the under achievement of results. Still, part of the expected target group, particularly the indigenous-bilingual population did benefit from the Project. Given that the final evaluation has not been delivered, it is impossible to state whether the partial achievement of results could have a lasting impact on poverty, gender and overall social development. (b) Institutional Change/Strengthening (particularly with reference to impacts on longer-term capacity and institutional development) The Project had a low impact in improving the efficiency and effectiveness of the MED. Some processes were improved as a result of the Project, particularly in the pedagogical area. For example, MED established internal review procedures requiring the units responsible for bilingual education teacher training and basic education (the latter being the regulatory entity responsible for curriculum development and implementation) to coordinate activities so that they would be aligned with the curriculum and they would be articulated in the field. The Project also contributed to the improvement of the statistical system, particularly in the area of quality control and in the development of the GIS which has been critical for other interventions in MED, including the census-based student evaluations. Finally, the Project supported the student assessment system, thus ensuring that results become an integral part of the decision making process in MED. On the other hand, MED’s relationship with regional management units (DRE and UGEL) was haphazard and depended mostly on the good will of the parties involved. The project did not achieve its original objective of strengthening deconcentrated offices to provide support to the schools or the autonomy of schools. (c) Other Unintended Outcomes and Impacts (positive or negative) Not applicable.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes)

Not applicable.

27

4. Assessment of Risk to Development Outcome Rating: Substantial Risk There is a substantial risk to development outcomes. As per the above assessment of development outcomes above, it is not clear that the few achievements in outcomes will all be sustained. While bilingual education interventions are expected to continue, thanks mainly to indigenous stakeholder demands, financing may not be sufficient to ensure coverage of all targeted beneficiaries. Given the weakness shown by the regional offices (DRE and UGEL), interventions at the local level are not expected to be sustained in time and the validated models face significant risk of not being brought to scale. On the other hand, progress made under the student performance and statistical system units continue to be fully supported and funded and sustainability is highly likely. In addition, the initial education model is expected to be implemented under an upcoming IDB-financed project. The Bank is currently working with the Education sector, but under a different focus and thus, specific Project outcomes are not being directly supported. The current CPS includes continued involvement with the Education sector through a DPL program and investment loans (both for basic and tertiary education). The REACT DPL program is under way; the first loan has been signed and the second is under preparation. The DPL program includes support to the education sector in areas relevant to actions under the Project, but with a focus on the overall accountability of the primary education system. This support does not particularly focus on rural areas and the specific intervention models needed for that context. The Bank recently launched the preparation of the tertiary education loan, while the GOP has not yet requested the Bank to start preparing the basic education project.

5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately Unsatisfactory Preparation took about four years between identification and Board presentation. The political instability in the country caused by Fujimori’s re-reelection in July 2000 and resignation in September 2000, followed by a transitional government which had as its mandate ensuring open and transparent elections, and the election of Alejandro Toledo, who took office in July 2001, made it necessary for the Bank to wait about 2.5 years between the identification (August 1999) and the Project Concept Note (PCN) management review meeting (11-28-01), when GOP commitment could be confirmed.

28

As recorded by the Quality Assurance Group (QAG), this wait resulted in a prudent and correct decision by the Bank to ensure GOP’s ownership of the project design. The Bank was ready to submit the Project to the Board in June 2002, but MED was not able to proceed as it had to meet the requirements imposed by the National Public Investment System (the country’s appraisal system). Thus, approval by the Board occurred on May 29, 2003. As a result, preparation costs amounted to about US$1,117,516 (US$669,241 from three PHRD grants and the rest from the Bank’s operational budget). The lending cycle included a total of seven missions from identification to appraisal in April 2002. An eighth mission was carried out between appraisal and negotiations, in early April 2003, negotiations finally took place on April 21-22, 2003. The Bank provided effective project preparation support to the GOP through a highly qualified and experienced core Bank team. When specific technical issues required attention, the Bank team drew on senior expertise from across the Bank and from outside the Bank. There was only one Country Office-based task team leader assigned to this operation from the very beginning (identification) to Board approval thus providing a welcomed continuity in the lending cycle. As shown in Annex 4 of the ICR, a significant number of professionals were involved in the preparation cycle. Despite all this, as already mentioned in Section 2.1, the ICR rated the quality at entry of this Project as moderately unsatisfactory. What follows is a more detailed explanation on the reasons of this rating, highlighting the strengths as well as the shortfalls emerging from the assessment carried out on the Project’s lending cycle file along the dimensions appearing in bold. Strategic relevance and approach. The Project was underpinned by very good sector work highlighting the relevant issues in need to be urgently addressed in the education sector. An APL was agreed as the financial instrument for this operations based on the following rationale: (i) MED had a clear medium-term vision derived from the national consultation process of where it wanted the education sector to be ten years hence and was highly committed to achieve it; (ii) it allow MED to start “small” and gradually proceed to a scaling-up process based on prior assessment of ongoing experiences financed in the early stages; and (iii) it required a functional monitoring of agreed outcomes and outputs as well a accomplishments of agreed triggers as a disciplined way of proceeding with subsequent phases of the long-term operation. Thus, the first phase of the APL aimed at setting the right policies in pre and in-service teacher training and management of the education sector in a more decentralized fashion, allowing for learning from the development and piloting of project activities in early childhood development, increased access in pre-school education in rural areas, multi-grade teaching in rural areas, bilingual and intercultural education and untested secondary distance education. The second phase of the proposed APL aimed at fine-tuning activities and scale-up those that proved to be cost-effective. The third and final phase, aimed at consolidating the Program, after careful assessment of the experiences financed during the previous two phases.

29

Level of GOP’s ownership and client orientation. The country’s education and governance priorities at the time of the lending cycle were appropriately integrated in the Project design. With the changes of Governments between November 2000 and July 2001, the Bank team and the regional management rightly waited until the new Government provided clear signs of its full support of the Project. The preparation involved extensive consultations within the country. The Project design was largely underpinned on the results of the 2001 national consultation on education emerging from the implementation by the GOP of the social pact framed under the National Agreement. A 24 member National Commission was created to lead the process and over 300,000 individuals throughout the country expressed their opinions, demands and expectations on the following six major topics, which were included in the Project: tomorrow’s skills, school learning, extra-school learning, school management, education priorities and financing and education information and evaluation. The findings and recommendations of this massive consultation effort published as Las Voces del País (the Voices of the Country) and included a Proposal for a National Agreement on Education. The team demonstrated a rich understanding of the political and social context as well as of the issues and risks faced at the national level, but more particularly, in the rural and indigenous regions of the country. Partnerships. The team developed good working relationships with other donor agencies, in particular with the: (i) IDB, which co-financed a large chunk of the required civil works as well as the PCU12; (ii) DfID, which piloted activities to strengthen capacity for school management and provided the design and materials for the project, and (iii) GTZ, which participated the discussion on teacher development and provided technical assistance in the development of the Centros Amauta.. The above attest to a history of extensive and successful consultation among the various external-funding agencies and is a good example of the harmonization efforts sought later on by the 2004 Paris Declaration13. Outcome and output indicators. The stated developmental objectives were appropriate. However, due to the complexity of the Project’s design involving three components and 11 sub-components, there were too many outcome and output indicators included in Annex 1 of the PAD making it difficult to monitor. As it turned out during the implementation cycle, some of these indicators were not monitored in a timely manner. (see section 2.3 of the ICR concerning Monitoring and Evaluation).

12 This ICR does not assess the physical and financial performance of the IDB parallel loan for this Project, although it provides in some sections of the report some related information.

13 The 2005 Paris High-Level Forum on Aid Effectiveness approved a set of principles to improve the effectiveness of donor assistance. Agreement was reached by 60 partner countries, 30 donor countries, and 30 development agencies, including the World Bank to adhere, when possible to: (i) country ownership; (ii) harmonization of fiduciary practices; (iii) alignment with country-led development priorities; (iv) managing for development results; and (v) mutual accountability for the use of aid.

30

Technical, financial and economic aspects. While recognizing that the approaches included in the Project’s design were well-tested elsewhere and proven effective, the Project design was overly ambitious covering too many fronts – early and preschool education, primary education and secondary education in rural areas, bilingual and intercultural education, pre and in-service teacher reform, learning assessment, community participation and management efficiency. As became evident during the implementation cycle, it took a considerable amount of time to get implementation going and the implementation performance as recorded by the ISRs was at the end rated unsatisfactory. This observation is even more striking in a context of the high centralization as well as institutional and implementation capacity weaknesses of MED. Implementation arrangements. Despite the institutional capacity building assessment done by the Bank team during the lending cycle, the following shortcomings were identified, leading to the conclusion that implementation readiness conditions were not fully present at the time of Board approval and thus partially delayed by about one year project implementation after effectiveness and the first GOP and Bank disbursements to the Special Account: (a) The inclusion of effectiveness conditions resulted in delayed project launching. The LA included three conditions: (i) the project Steering Committee (Comité Directivo) with membership structure and responsibilities satisfactory to the Bank; (ii) a subsidiary agreement between FONCODES and MED, addressing pending matters with respect to the financial coordination between these two institutions including the mechanisms to consolidate the project accounts, financial reports, the Financial Management Reports, audit and other control functions; and (iii) the project Operational Manual14, although a preliminary draft existed at the time of Board approval. Current good practices in the Bank point to the fact that no conditions of effectiveness are desirable beyond the legal opinion of the country. Furthermore, nowadays, to complete an appraisal the Bank recommends that an Operational Manual, satisfactory to the Bank, is agreed with the Borrower; (b) The pedagogical models corresponding to the different pilots in initial education and secondary education were not ready. The assumption made during the Project’s design was that during the lending cycle the MED was going to pilot and validate these pedagogical initiatives, and then, at appraisal, discuss with the Bank the findings of their validation assessment and agree with the Bank on specific activities to fine tune and scale

14 The Project’s Operation Manual was supposed to include, inter-alia: (i) the Project’s institutional setup, organizations and functions of the PCU; (ii) disbursement, accounting, auditing and reporting procedures; (iii) procurement responsibilities, procedures and internal controls; (iv) monitoring and evaluation plans for the project; and (v) procedures for identification, preparation, approval, execution and supervision of infrastructure sub-projects to be financed under the loan, environmental assessment procedures and a list of non-eligible civil was not available works.

31

up these innovations during the implementation phase. MED did not carry out the agreed pilots so that, effectively, they became part of implementation. (c) Project management and implementation conditions were not set. (i) despite having carried out a procurement capacity assessment, the Board package also lacked a first-year procurement plan, which current procedures in the Bank require to completing the appraisal phase; (ii) no baseline data for the proposed outcome and output indicators was available and the corresponding monitoring and evaluation system was not in place. The Bank team argued that the available information to construct a baseline at the time of Board approval was not reliable, thus a specific tailored-made study was contracted producing the excellent baseline in 2006. Having an agreed baseline is nowadays considered as a recommended practice by Bank management to complete the appraisal phase; and (iii) according to the Project’s files, the PCU manager was not appointed prior to the Board approval. Compliance with Bank’s safeguard policies. A set of appropriate environmental guidelines was prepared for the design, evaluation and supervision of the project’s civil works. In addition an appropriate Indigenous Communities Strategy was prepared and integrated into the different Projects components. Risk assessments. The Project design did not take into account the risk that the Government would have a significant change in policy towards teacher policy in terms of implementing the teacher career reform. Unfortunately, soon after board approval, a massive teacher strike resulted in the GOP deciding to postpone the approval of the teacher reforms. As a result, rural teacher incentives were not implemented institutionally (they remained as a pilot), and retention of these teachers became a sustainability risk. Lessons learned. The Bank team included lessons learned from the FONCODES Project (Loan 4068-PE, closed) and the Primary Education Project (Loan 3826-PE, also closed) as part of this Project’s design. In sum, although the Project was prepared according to Bank standards of that time, the Bank should have aimed at a much simpler project design and ultimately evaluated whether to go with the Project, given that there were critical unresolved issues that were not met during project preparation. Furthermore, weighting the relevant achievements and shortfalls of the Bank team at the lending stage in accordance with today's good standard practices in the Bank, in particular the over-ambitious design and the lack of implementation readiness conditions described above, ICR rates the Bank performance at the lending stage as moderately unsatisfactory.

32

(b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Unsatisfactory As previously mentioned, the Project had only one Task Team Leader (TTL) from identification to the closing of the Loan, thus providing continuity to the entire lending and implementation. The fact that the TTL was stationed in Lima, Peru also provided an effective and timely supervision of this Project, as well as just-in-time support to the client. After the launching of the Project in January 2004, PEAR was subjected to ten field supervision missions, including a Mid-term Review conducted at the end of November 2006, three ex-post Procurement Review missions (June 2005, May 2006 and November 2006) as well as quarterly Financial Management Reports (from July 2004 up to September 2007). In addition, there were 13 ISRs archived in the Project Portal. There was an average of between two to three supervision missions per year during the four-year implementation cycle after the Project launching. The fact that there are 13 ISRs seems to indicate that these were updated, on average, at least three times per year. Supervision missions included an adequate skill-mix to carry out their envisaged terms of reference. By and large, the reporting in the ISRs was realistic. The only exception is with the Moderately Satisfactory (MS) ratings for the PDO in the ISRs number 10, 11 and 12 corresponding to 2007, a rating that was not consistent with the story reflected in the corresponding Aide-Memoires. Furthermore, the slim quantitative data presented in these ISRs suggests that the Project was far away from reaching the proposed PDOs. The findings of each supervision mission were conveyed to Bank management clearly and in a straightforward fashion, including the accomplishments as well as the implementation shortcomings and potential risks assessed by the Bank team. The Bank supervision teams followed-up on commitments made by the GOP on previous missions and the findings reported accordingly in the ISRs. The Bank was supportive to ensure a prompt and effective implementation process, facilitating an induction workshop for key stakeholders at the MED on August 30, 2003, just after Board approval, and a launching workshop in January 2004, about four months before Loan effectiveness. The team supervision missions consistently voiced their concern regarding MED’s refusal to follow up on GOP’s policy on decentralization with respect to the publicly-financed social services, including education. Decisions were taken at the higher echelon of MED without consultation with the Regional and/or Local Management Unit (DRE and UGEL). There were continuous discussions during the various Bank supervision missions to have a focus on results, including the carrying out of an impact evaluation study and other related assessment and monitoring studies by MED. Still, the only comprehensive evaluation was the baseline study carried out in 2006. One of the reasons for the absence of evaluations was the fact that model validations took much longer than expected.

33

The Bank team worked closely with counterparts towards the restructuring and extension of the Project. During more than one year, the Bank met periodically with MED teams, and subsequently with MEF to discuss both the content and the process for restructuring. An agreement was reached with MED, whereby the Project would have been simplified significantly. Unfortunately the negotiations with MEF were not successful and the Project closed as per the specified date in the LA. Despite the effort to provide technical assistance as well as to carry out overall supervision responsibilities for the Project, it is evident that the Bank did not address sufficiently the implementation challenges faced by the Government. In particular, given the complex design, the Bank should have negotiated more emphatically with the Government on the restructuring process, precluding the implementation of the proposed changes made by MED until the Project was fully restructured, as recommended during the mid-term evaluation. Weighting the above, Bank supervision performance is considered as moderately unsatisfactory. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory Based on the above lending and supervision observations and ratings, the ICR rates the overall Bank performance as moderately unsatisfactory.

5.2 Borrower Performance (a) Government Performance Rating: Moderately Unsatisfactory Rating at the lending stage: Moderately Unsatisfactory As mentioned above, preparation took about four years and preparation costs were also high (as compared to average preparation costs in the education sector in LAC). Project preparation activities were mainly financed by three Japan Policy and Human Resources Development (PHRD) grants (TF025935 which was Bank executed, TF025902 and TF026867, which were both GOP executed). The first two PHRD grants were approved on February 16, 1999, started to be utilized about a year later (February 9, 2002) and closed two and half years later (June 16, 2002). The third PHRD grant was approved on June 29, 2001, started to be utilized 6 months later and closed on April 17, 2003. The documented expenditures in consultants (mostly for rural education, early childhood education, secondary distance education, use of new educational technologies, infrastructure, monitoring and evaluation) and project-related studies against these three grants amounted to about US$669,241.

34

The time and cost overruns during the lending cycle were a reflection of the volatile political situation in Peru, which resulted in two longish interruptions in project preparation. The political uncertainty caused by the resignation of President Fujimori in September 2000, the subsequent appointment of an interim government by the Peruvian Congress and the election of President Toledo that followed in July 2001 considerably slowed down the pace of the lending process, in particular the formal review of the PCN by Bank Management that took place 27 months after the identification mission. The second interruption occurred between appraisal and negotiations (April 2002 and April 2003). This delay was caused by recent approval of the SNIP, which introduced extensive project appraisal requirements, demanding about 12 months for MED to achieve. The SNIP does not allow the GOP to negotiate a loan before the project is declared viable. These political events also brought many changes in the MED, in particular in the offices of the Minister and of the Vice-Minister for Pedagogical Management and all of its Directors (for Initial, Primary and Secondary in rural areas and Bilingual and Intercultural Education), which constituted the key technical counterparts during the lending cycle. Just in the lending cycle (from identification in August 1999 to Board approval in May 2003), the Bank team had to deal with six different Ministers of Education, whose average tenure, except the first one15, was about one year16. The arrival of new teams at the MED’s and the frequent turnover of key high-level management had a significant negative impact on the early stages of project preparation as each new team needed to assimilate the project’s proposed ethos and the Bank’s procedures in implementing its loans. Ironically, when some of these new teams were ready to be utilized at their full capacity to complete the project preparation, new resignations took place in a framework of constant turnover of key political and technical personnel at the MED. Although during the early stages of project preparation the Bank team interacted with a highly qualified technical counterpart that was involved in the implementation of the previous Bank-financed project in education (Primary Education Quality, Loan 3826 –

15 The lending cycle started under the Ministerial tenure of Domingo Palermo Cabrejos (1996 – 1999), which was also heading the MED during the satisfactory implementation of the previous Bank-financed project in education (Primary Education Project; Ln. 3826-PE). It was under the leadership of this Minister that the highly qualified technical counterpart cadre in the MED initiated the Project design through an effective policy dialogue with the Bank team. This technical group was also familiar with the Bank’s financial and procurement procedures. Unfortunately, with the arrival of new Ministers (see next footnote), this technical group was gradually dismantled and replaced with staff, that not only resisted to buy into the proposed project design, but they were also not familiar with Bank procedures.

16 After the departure of Minister Palermo, the MED was headed by the following 5 Ministers during the entire lending cycle: (a) Felipe Ignacio García Escudero (November 1999- July 2000); (b) Federico Salas Guevara (July 2000- November 2000); (c) Marcial Rubio Correa (November 2000- July 2001); (d) Nicolás Lynch Gamero (July 2001-June 2002); and (e) Gerardo Ayzanoa del Carpio (June 2002-May 2003).

35

PE), this team was gradually replaced during the volatile political years of 2000 and 2001 with other MED staff not necessarily familiar with Bank operations and not at the same skill competency than the previous one. For example, the Project files attest to the lack of capacity within the MED to effectively link the pedagogical dimension with the technical aspects of the secondary distance education. MED teams responsible for the various components tended to work in isolation, precluding the needed articulation among project components. The Project’s files also reflect a concern raised by the Bank team that detected the lack of sufficient skilled personnel at the MED to undertake project implementation activities. Rating at the implementation stage: Moderately Unsatisfactory The same changing phenomena observed in MED during the lending cycle was mirrored during the implementation phase. Since Board approval until the closing of the Loan, four Ministers headed the MED, with an average tenure of about 15 months17. Each change meant changes along the hierarchical chain of command thus considerably slowing down project implementation. Appointments of higher-level technical staff associated to the Project’s implementation were mostly based on political, rather than skill and experience criteria. There was also a radical change of policy of the current administration with respect to its support to the decentralization effort and the bilingual and intercultural education, limiting political support to the Project during the last year of the implementation cycle. In addition, the requirements for inclusion of the Project in the national budget were changed in 2003, having an impact on the initial disbursements and counterpart allocations to the project. Finally, there were instances during the project implementation cycle that GOP did not comply with the Covenants in the Loan Agreement. In particular, Covenant 3.08(b) concerning the timely furnishing to the Bank on progress reports on the execution of the Project was often complied with significant delay. In addition, during the last phase of the Project’s implementation, the Covenant requiring that the Borrower maintain an adequately staffed PCU, was not met. Rating: For all of the above reasons, the ICR rates the Borrower performance at implementation as moderately unsatisfactory. Overall Rating of Borrower’s Performance is rated moderately unsatisfactory. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory MED, through is mainline Directorates and the PCU became the implementing agency of this Project. Along the changes of Ministers of Education, the PCU also experienced a

17 Gerardo Ayzanoa del Carpio (June 2002- May 2003), Carlos Malpica Faustor (May 2003-February 2004), Javier Sota Nadal (February 2004-July 2006) and José Antonio Chang Escobedo (July 2006 – present).

36

change of coordinators. A total of five PCU coordinators18, with an average tenure of about one year, attempted, quite unsuccessfully, to manage Project activities from Board approval in May 2003 to the closing of the Loan in December 31, 2007. The same issues described above concerning the arrival of new teams at the MED, was reflected in the PCU. Besides, it took some time after loan effectiveness to have the PCU fully staffed, not necessarily with qualified staff. The PCU, had difficulties in establishing a strategic vision in their undertaking of their managerial and executive tasks. They lacked, by and large, leadership and clarity as to how to make policies happen and displayed a weak implementation capacity. The lack of managerial strategy was, for example, manifested and perceived in the field where regional authorities and school members complained that Project consultants arrived to carry out their own tasks without prior coordination. The Bank supervision reports consistently attested to the insufficient or non-existent internal coordination of the PCU. Despite the heavy and cumbersome coordination mechanism put in place in MED, effective accountability mechanisms were still lacking. The ex-post procurement assessment findings and recommendations took a considerable amount of time to be appropriately addressed by MED and the PCU. Likewise with the financial supervision recommendations, which were not timely followed up by the MED and the PCU, to the point that the last ISRs rated the fiduciary aspects of the Project as unsatisfactory. For the above reasons, the ICR rates the implementing agency performance of moderately unsatisfactory. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Unsatisfactory Weighing the ratings of the Borrower at the lending and implementation phases, with the ratings of the implementing agency, the ICR rates the overall Borrower performance as moderately unsatisfactory.

6. Lessons Learned (both project-specific and of wide general application) (a) Assessment of sector policy, commitment and capacities is critical for the

selection of the appropriate lending instrument. This Project was designed as the

18 The first one lasting from 2002 to 2003; the second from 2003 to 2005; the third from 2005 to 2006; the fourth from 2006 to 2007; and the last one just recently appointed in 2007.

37

first phase of an Adaptable Program Loan (APL). APLs support a long term vision and depend on the compliance of agreed “triggers” linked to the medium term sector policies. This instrument is relevant when institutions are strong and have a track record of consistency for a period of time that goes beyond that expected for the project itself. While this Project was designed during a period when the Ministry of Education had shown continuity in its policies, the frequent changes in Ministers and the subsequent changes in vision were not foreseen. Thus, this type of lending instrument was not appropriate for such a highly uncertain context where the continuity of the sector policy framework was not in place. In such cases, selecting a Sector Investment Loan might have been more effective.

(b) Ensuring that sector data is reliable and timely facilitates sustaining an efficient

monitoring and evaluation system. High quality information at the start of a project is a prerequisite for designing a sound monitoring platform that allows monitoring targets throughout the process and make adjustments, if necessary. Additionally, baseline data is key for conducting an impact evaluation to assess change overtime. One of the main challenges faced by this Project was the lack of reliable data during the design. Thus, the Ministry had to use inaccurate demographic projections and home survey data, for lack of adequate administrative institutional data. The baseline data for the Project was made available only in mid-2006 providing evidence that some of the estimated baseline data, at the time of appraisal, was not accurate. Unfortunately, the Ministry did not show sufficient interest nor the capacity to undertake an impact evaluation study beyond the 2004 National Learning Assessment and the 2007 Student Census, so that the outcomes could not be measured.

(c) Validating and evaluating pilot experiences prior to scaling-up is critical when

models have not been previously tested. Piloting allows the researcher to test procedures and assumptions that can later on be readjusted, if necessary, to scale up. This Project was originally conceived as an opportunity to finance the design and implementation of a series of pilots to test pedagogical and school management models in rural areas which had been implemented successfully in other countries. Still, given that the Peruvian context was different, piloting was deemed necessary, to assess the design, prior to scaling up. In practice, the delayed launching of these pilots let to a rush towards scaling up prior to the completion of the evaluation cycle. Thus, several issues remained unanswered including the interconnection among pilot activities, as well as the identification of conditions for their generalization and the definition of the evaluation criteria for the models. All this worked against the successful implementation and sustainability of the Project.

(d) While the use of institutional administrative and fiduciary institutions is

desirable, capacity needs to be assessed prior to its use for project implementation. The potential for sustainability is greater when institutional administrative systems are used. However, if capacity is weak, the potential for project failure is very large. This was the case for this Project, where the policy was implemented de facto without appropriate assessment and with no strengthening plan. Thus, streamlining

38

into sectoral administrative systems needs to be phased, ensuring adequate capacity for implementation and accountability.

(e) Political commitment, support and ownership are key conditions for success.

Political support is linked to decision-making about human and financial resources allocation to ensure adequate implementation of a project. In the case of this Project, the frequent changes in ministers (there were six different Ministers, whose average tenure, was about one year) and the consequent changes of Directors and Project Managers, resulted in reduced ownership of the project and changes in priorities. As a result, the higher levels of the Ministry did not prioritize the Project and human resource allocation weakened with time. Even though the Bank team discussed the Project with all new incoming teams, the vision of improving rural education was lost for other priorities brought by the new administrations. Thus, political risk is critical for Project implementation.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The Ministries of Education and Finance submitted comments to the draft ICR. While both agree that Project implementation did not lead to the desired results, they do not fully agree on the underlying causes. Both agree that design was much too complex. But, the Ministry of Education also argued that the context changed significantly between preparation and implementation. Furthermore, MED suggested that the evaluation should have been carried out against the new targets proposed under the proposed restructured Project. Given Bank evaluation guidelines, this was not possible, as the restructuring process did not officially occur. On the other hand, MEF argued that, once problems in Project design were found, implementation should have stopped in those areas to be restructured, and redesign be carried out under new rigorous diagnostic studies. While this is desirable technically, it runs counter to the agreed timing for Project implementation. (b) Cofinanciers Not applicable (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) Not applicable

39

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

1. Expanding Education Access for Rural children 1.1 Access to quality initial and preschool education 1.2 Secondary distance education

2.71 7.83

3.11 2.87

114 37

2. Improving the quality in rural schools 2.1 Continuous teacher development systems 2.2 Curricular adjustment and provision of educational materials 2.3 Rehabilitation and equipping of rural schools.

24.86

9.09

29.63

14.81

5.28

14.70

60

58

50

3. Reform of teacher policy and education management 3.1 Teacher policy studies 3.2 Reform of education management 3.3 School development fund 3.4 National assessment system 3.5 Strategic analysis and policy-oriented research 3.6 Project Management

1.60 6.94 1.76 4.61

2.82 1.83

0.88 7.55 0.02 3.94

0.90 2.69

57 109 1

85

32 147

Total Baseline Cost 93.68 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total project costs 93.68 56.75 61 Front-end-fee 0.52 0.52 100.00 Front-end fee IBRD 0.00 0.00 0.00

Total 94.20 57.27 61

40

(b) Financing

Source of Funds Type of Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

IBRD 52.5 28.98 55

GOP Counterpart 28.84 17.63 61

IDB Parallel 12.86 10.66 83

Total 94.2 57.27 61

41

Annex 2. Outputs by Component This section describes the outputs of each of the three components including all sources of financing. The assessment is primarily based on the quarterly FMRs, the November 2006 mid-term review report and the note that was prepared by the Bank in mid-2007 justifying the possibility of extending the closing of the loan by 18 months and restructuring (significantly simplifying) the project design. Two cost tables are presented to substantiate the story provided in the ICR: (a) the documented expenditure by component and by source of financing at appraisal and at the time of closing the Loan; and (b) the documented expenditure by category and by source of financing at appraisal and at the time of closing the Loan. Component A – Expanding Education Access for Rural Children. According to Table 6 below, this component estimated to cost about US$10.56 million (US$6.99 million from the Loan and US$3.56 million from GOP), ended up documenting expenditures amounting to US$5.98 million (US$3.22 million from the Loan and US$2.76 million from GOP), 57 percent of the total estimated at appraisal. This financial shortfall is explained by the limited expansion in access for rural children achieved during project implementation as compared to the original targets set at appraisal for reasons described below. Component A was implemented in 57 districts in 7 of the original 11 Regions included in the Project Area (Amazonas, Ayacucho, Cusco, Huancavelica, Piura, Puno and San Martin). It financed the piloting and validation of three models for initial education for 0-2 year-old children and the strengthening of another two existing preschool education models for 3-5 year-old children (the Centros de Educación Inicial – CEIs -and the Programas No Escolarizados de Educación Inicial - PRONOEI). Alongside the validation of these pedagogical models, the project also financed the alignment of in-service teacher training, the provision of appropriate learning materials for children, parents and teachers and the training of parents. After validations of these models, they were gradually scaled up benefiting about 1,875 children in the age-group 0-2 years-old, including their families. This Component also financed the strengthening of 59 CEIs and 141 PRONOEIs benefiting 4,225 children in the 3-5 year-old age-cohort. These accomplishments fell significantly short with respect to the original targets set for in the PAD of benefiting 30,000 children in the 0-2 age-cohort and another 11,000 children in the 3-5 year-old range. Assumptions made at the preparation phase with respect to the provision of initial education models in rural areas, which did not match reality during implementation, partially explain this shortfall19. This caused the need for

19 The model required more on-site support that originally planned in the design, which in turn required the hiring of more qualified consultants in the MED’s technical team. The assumptions made on the expected demand of parents willing to enroll into the program did not materialize either. Lack of local capacity to implement the delivery models also contributed to the delays.

42

additional fine-tuning of the delivery model, thus further delaying implementation and therefore, limiting the expected financial and physical progress of this activity under Component A. The above efforts entailed the training of about 362 staff and facilitators in all the initial and preschool services financed by the Project as well as the provision of about 109,500 sets of learning materials and training modules (against 200,000 sets included as a target in the PAD). The Project also financed the replacement of 10 classrooms in 5 CEIs. Component A also financed the launching and validation of the secondary distance education model (Educación Secundaria con Metodología a Distancia – ESMED) benefiting 875 students (against the original target of 5,000 set in the PAD). This component financed the acquisition and distribution of about 28,018 textbooks including other learning materials and educational videos (against the original target of 50,000 set in the PAD). It also financed the training of 40 teachers and 20 school principals involved in the delivery of ESMED. Rating component A. This Component fell short of its envisaged physical and financial targets, and is thus rated moderately unsatisfactory. IDB is currently preparing an initial education project which is expected to bring to scale the validated models under this project. However, the SNIP appraisal period is resulting very extensive and political commitment appears doubtful. The secondary education model is not expected to continue as it has not been validated. Component B – Improving Quality in Rural Primary Schools. According to Table 4 below, this core component estimated to cost about US$63.59 million (US$32.07 million from the Loan and US$31.52 million from GOP), ended up documenting expenditures amounting to US$34.79 million (US$15.78 million from the Loan and US$19.01 million from GOP), 55 percent of the total estimated at appraisal. This financial shortfall is also explained by the deficit in achieving the original targets set at appraisal (fully described below). Component B financed the establishment of in-service teacher training modules focusing on multi-grade methodologies and bilingual and intercultural education including a review of the curriculum, training, development and acquisition of learning materials and rehabilitation and construction of classrooms. The bilingual and intercultural activities were mainstreamed into other related activities (like multi-grade, teacher training, development and procurement of learning materials) during Project implementation. Accordingly: (i) a multi-grade teaching model was validated and provided to 10,711 children in 222 schools including the training of 448 teachers; (ii) a bilingual intercultural pedagogical strategy was defined, validated and applied benefiting 239,250 children including the training of 1,976 initial and 11,976 primary bilingual intercultural teachers; (iii) a total of 374 Spanish speaking teachers in initial education and 1,513 in primary education were trained; (iv) a total of 1.7 million working books in 15 ethnic languages were distributed, not necessarily on a timely fashion, benefiting about 94,580 children; (v) a total of 293 classrooms were rehabilitated; and (vi) school and other type of

43

furniture was provided to 717 classrooms and 196 local administrative offices in the education sector. These achievements fell short of the original targets included in the PAD, namely: (a) continuous teacher training to be provided to 3,800 teachers in the Project area working in multi-grade schools and 15,000 bilingual teachers nationwide; (b) 330,000 rural children in the Project Area and 375,000 indigenous children nationwide to benefit from all the quality-related interventions (provision of learning materials including bilingual educational materials and improved teaching practices); (c) a total of 1.8 million working books to be distributed to multi-grade and bilingual schools; and (d) equipping, rehabilitating and substituting of 1,900 classrooms and 1,500 complementary facilities in the Project Area. This component also did not achieve the original design objective and required adjustments during the implementation cycle, in particular with respect to the multi-grade strategy. In addition, despite the multiple recommendations made by the Bank supervision missions, MED was not able to connect and enrich the ongoing multi-grade and bilingual interventions financed by the Project with lessons learned from other similar ongoing experiences in the country provided by some non-governmental organizations (such as TAREA, APRENDES, a USAID financed project, CEPROSI, ADEAS Qullana, etc) and elsewhere in the region, like “Escuela Nueva” in Colombia and Guatemala. With respect to the infrastructure sub-component, delays in implementation were caused both by exogenous and endogenous problems. Changes in national legislation made it impossible for the Bank to deposit the funds directly to FONCODES forcing an amendment that would allow MED to make the transfers to FONCODES. In addition, there were problems with change in the unit costs due to rapid devaluation of the dollar, climatologically conditions, like rain, partly explain the construction delays. Finally, FONCODES was severely weakened when it was transferred to MIMDES and had difficulties meeting its side of the contract. Rating component B. Despite this Component fell short of its envisaged physical and financial targets, it is being rated moderately satisfactory because it achieved the validation of a multi-grade 20 and bilingual bicultural pedagogical strategies and corresponding in-service teacher training as well as provision of appropriate learning materials, which are currently being technically, institutionally and financially sustained and expanded by the GOP with their own resources, albeit to a lesser extent than would have occurred under the Project. Further to this, the Project contributed to the curriculum development and implementation of the bilingual education program at the initial level. Previously, initial education children received their education in Spanish regardless of their mother tongue or the language of schooling they would receive at the primary level.

20 Still with possibilities to improve by incorporating good practices and lessons learned from other multi-grade approaches utilized in other countries.

44

Component C – Reform of Teacher Policy and Educational Management. This component was originally estimated to cost about US$19.53 million (US$ 12.96 million from the Loan and US$ 6.57 million from GOP), ended up documenting expenditures amounting to US$15.98 million (US$9.46 million from the Loan and US$6.52 million from GOP), 82 percent of the total estimated at appraisal. With respect to the Teacher Policy Reform, this Component financed: (i) the development of a plan to strengthen 16 pre-service teacher-training institutions that will be implemented during 2008; (ii) the assessment and implementation of revised curriculums for pre-service teacher development in the specialties of primary education, bilingual and intercultural education, art and initial education; and (iii) the implementation of 3 out of 6 Centros Amauta. The teacher career development system envisaged in the original design was not implemented during the project cycle as the forthcoming National Educational Assessment and Certification System (Sistema Nacional de Acreditación y Certificación Educativa, SINEACE) is soon to be launched. With respect to the national assessment system, this Component financed activities of the Unidad de Medición de la Calidad Educativa and the Unidad de Estadística Educativa. The former implemented several activities under the Project: (i) the learning measurement undertaken in 2004 and (ii) the Student Census Learning Assessment for second graders and bilingual fourth graders administered in December 2007, whose preliminary findings will be made available in the second semester of 2008. Additionally, the Unit implemented several other research, dissemination and training-related activities such as: (i) Dissemination of the 2004 learning assessment through workshops at local level and training, (ii) dissemination of the school self-assessment kit, (iii) longitudinal study to track learning outcomes in a cohort, (iv) teacher research context based on the 2004 learning assessment results, (v) implementation of the Communication learning assessment for native language-speaking communities Shipibo-Conibo and Quechua Ayacucho-Chanka, (vi) implementation of the SERCE-LLECE evaluation, (vii) pre-pilot study for the 2007 learning assessment (that included Initial level), and (viii) a qualitative study on school efficacy. In addition, the Unidad de Estadística Educativa worked on the expansion of geo-referenced educational models, the design of spatial accessibility protocols and indicators, the reconfiguration of Rural School Networks and the database that provided input for the School Linguistic Map in coordination with the Dirección Nacional de Educación Bilingüe, Intercultural y Rural. The Unit also coordinated the study on Demand and Supply of Secondary Education in Rural Areas that generated updated information on this subject. With respect to the establishment, training and monitoring of School Councils and School Network Councils, this Component benefited 96 educational networks and established 1,231 CONEIs aimed at empowering the communities and democratize the decision making at the school level. This training program was designed and implemented using the model validated by DFID. This effort included the training of 7,647 School Council and School Network Council members. However, the tangible benefits and impact these councils and networks had in improving community participation and social control of

45

the schools are negligible, to the point that these activities were not any longer incorporated in the restructuring proposal. In addition, the targets achieved are below the ones set at appraisal - 3,000 CONEIs to be established and 10,000 people to be trained. The Component also financed activities to strengthen nine lower level education administrative units (DRE y UGEL), with no visible improvement on the management effectiveness and efficiency of the MED. The School Development Fund to finance the design and implementation of improvement programs developed and managed by schools and school networks in the Project Areas was never implemented Finally, the subcomponent on Strategic Analysis and Policy-oriented Research financed the implementation of the baseline study under the supervision of the PCU’s Monitoring and Evaluation Unit. This component gets a mixed rating. The strengthening of the national assessment system is rated satisfactory and is considered by the ICR as one of the best achievements of this project. It is however unfortunate for the purposes of the ICR assessment that the MED decided to postpone the new learning assessment to 2008 instead of the original date of 2007 thus preventing gathering key data to measure if there were learning improvements in the Project Area. The teacher policy reform is rated as moderately satisfactory, as the conducive environment to implement the proposed improvements was set but the actual implementation of these recommendations just barely got off the ground at the time of closing the loan. The efforts to improve community participation in the education sector through the School Councils and School Council Networks is rated unsatisfactory because, despite the limited achievements with respect to the original targets, their impact in improving community participation and social control is neither visible nor measurable. Finally, the School Development Fund is rated highly unsatisfactory because it never was implemented. Weighting the above, the implementation of this Component is rated as moderately satisfactory.

Table 4: Documented expenditures by component and by source at appraisal and closing of the Credit (in US$ millions)

At the time of appraisal At the time of Credit closing Percentages Component

IBRD (1)

GOP (2)

Total (3)

IBRD (4)

GOP (5)

Total (6)

(4)/(1)

(5)/(2)

(6)/(3)

Component A. 6.99 3.56 10.56 3.22 2.76 5.98 46 78 57 Component B. 32.07 31.52 63.59 15.78 19.01 34.29 49 60 55 Component C. 12.91 6.62 19.53 9.46 6.52 15.98 73 98 82 Total project costs

51.98 41.70 93.68 28.46 28.29 56.75 55 68 61

Front-end fee 0.52 0.00 0.52 0.52 0.00 0.52 100 NA 100 Total 52.50 41.70 94.20 28.98 28.29 57.27 55 68 61 According to Table 4, the total project cost estimated at appraisal was US$94.2 million equivalent, corresponding US$52.5 million equivalent from the Bank (55 percent of the total) and US$41.7 million equivalent from the GOP (44 percent of the total). At the time of the closing of the Loan, the total documented expenditure amounted to US$57.27

46

million equivalent (61 percent of the appraised total), corresponding US$328.98 million equivalent from the Bank (55 percent of the appraised total) and US$328.29 million equivalent from the GOP (68 percent of the appraised total).

Table 5: Documented expenditures by category and by source at appraisal and closing of the Loan (in US$ millions)

Category of Expenditure

At the time of appraisal At the time of Loan closing Percentages

IBRD (1)

GOP (2)

Total (3)

IBRD (4)

GOP (5)

Total (6)

(4)/(1)

(5)/(2)

(6)/(3)

Infrastructure 6.80 2.91 9.71 2.41 10.76 13.17 35 370 136 Goods 6.60 1.65 8.25 1.97 0.48 2.45 30 29 30 Consultants 14.67 3.66 18.34 14.82 3.54 18.36 101 97 100 Training 12.50 7.34 19.84 5.02 2.94 7.96 40 40 40 Operating costs

9.40 26.13 35.5 4.22 10.57 14.79 45 40 42

Unallocated 2.00 2.00 0.00 0.000 Front-end fee 0.52 0.52 0.52 0.525 100 100 Total 52.50 41.70 94.20 28.98 28.29 57.27 55 68 61 According to Table 5 above, the project underperformed with respect to the expected, acquisition of goods and training set at appraisal. It did, however, reach the consultant financial target. The above is an indication of that training was carried out, but using mainly consultants as opposed to the original design that proposed more workshops and follow-up strategies.

47

Annex 3. Economic and Financial Analysis (including assumptions in the analysis) Estimation of economic rates of return (ERR) was not possible. Increase in the average number of school years in the target areas could not be calculated with the available survey data. Improvement in the educational outcomes could not be calculated as the 2007 student assessment was not carried out. Without this information, it was not possible to estimate increase in beneficiary productivity and future salaries. As a formal ERR could not be estimated, actual costs were compared with the outcomes of the project for the primary education interventions. In the case of initial education, there was no available data that would allow for the comparison before and after the project. On the other hand, the analysis for the secondary education interventions was not relevant since the secondary distance education program was carried out as a pilot only in X schools. Costs The total Project investments were US$ 61.05 million (US$ 30.56 million Bank-financed, and US$ 30.49 million financed by the GOP and the IDB), representing 64.8 percent of the amount estimated at appraisal. Investments linked to primary education expenditures amounted to US$ 54.67 million. Benefits The primary school enrollment increased during the implementation period for both the rural areas and the whole national territory (with the increase being higher in rural areas). From 2003 to 2007, the net coverage rate of primary education either increased or remained with no significant changes every year (see Table 6).

Table 6 Net coverage rate for primary education1

(confidence intervals)

Year National Rural 90.47 89.38 2003

( 89.47 91.48 ) ( 87.79 90.96 ) 89.82 88.17 2004

( 88.78 90.87 ) ( 86.37 89.97 ) 90.66 91.77 2005

( 89.58 91.75 ) ( 90.36 93.17 ) 92.52 91.51 2006

( 91.62 93.42 ) ( 90.00 93.03 ) 92.66 93.09 2007

( 91.95 93.37 ) ( 92.03 94.16 ) Source: National Household Surveys 2003-2004

48

An analysis of the distribution of the primary school students by age shows that there was an increase of the percentage of children that attend their corresponding grade. This increase has been much higher in the districts were PEAR was carried out than in the national average.

Table 7 Percentage of enrollment at the appropriate age

2004 2005 2006 Variation 2004-2006

(%) Grade

National PEAR districts National PEAR

districts National PEAR districts National PEAR

districts 1 69.29 63.18 69.49 62.90 71.57 65.48 3.28 3.65 2 57.11 45.96 56.24 48.22 59.28 48.78 3.80 6.15 3 50.87 36.49 51.34 38.80 52.11 41.72 2.44 14.33 4 48.58 34.69 48.80 34.82 50.32 38.68 3.58 11.50 5 45.91 32.26 46.91 34.28 48.11 34.10 4.80 5.71 6 45.94 33.67 45.97 32.37 48.12 36.09 4.74 7.20

1-6 53.10 41.58 53.22 42.54 54.95 44.57 3.48 7.21 Source: School Census 2004-2006 The fact that more children are attending school at the appropriate age for grade level shows that students are completing the primary school faster. This means that students remain less years in the school. This reduction of the number of years that children enroll in primary school, lead to savings for the government and the families, since the government will have to finance less average years of education per student and the families will have to spend less money sending their children to school during a shorter period. Unfortunately, the exact amount of the savings cannot be calculated because the short analysis period.

49

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/ Specialty

Lending Livia Benavides SeniorSocial Sector Specialist LCSHH Task team leader Keisgner Alfaro Senior Procurement Specialist LCOPR Procurement Fabiola Altimari Legal Specialist Consultant Legal issues

Ana Maria Arteaga Staff Assistance LCSHD Mission logistical support

Barbara Bruns Institutional Reform Specialist HDNED Education reforms Virginia Cachay Cost specialist Consultant Cost tables Jorge Cavero Procurement Specialist Consultant Procurement Daniel Cotlear Sector Leader LCSHD Bank management Ernesto Cuadra Senior Education Specialist ECSHD Peer reviewer Santiago Cueto Education Specialist Consultant Education Francoise Delannoy Education Specialist Consultant Teacher reform

David Harding Education Specialist LCSHD Pedagogy, teacher training

Antonio Gomes Pereira Education Specialist Consultant Teacher training

Jorge Gutierrez Technical Education Specialist WBIHD World Links Peru program

Gabriela Falconi Program Assistant LCSHD Drafting of PAD Veronica Jarrin Language Program Assistant LCSHD Operation support

Peter Knight Information Technology Specialist Consultant Educational technology

Luis Enrique Lopez Rural Education Specialist Consultant Bilingual education Ramiro Lopez Rural Education Specialist Consultant Education networks

Patricia McKenzie Financial Management Specialist LCOFM Financial management

Isabella Micali Drossos Senior Legal Counsel LEGLA Negotiations

Michael Moore Distance Education Specialist Consultant Secondary distance education

Xiomara Morel Senior Finance Officer LOAG1 Disbursements Ruth Moya Bilingual education specialist Consultant Bilingual education Paud Murphy Senior Education specialist AFTH1 PCN Peer reviewer Julie B. Nannucci Language Program Assistant LCSHD Operation support

Cristina Pareja Education Management Specialist Consultant Institutional strengthening

Carlos Rojas Senior Education Specialist LCSHD Rural education

Ana Karina Rozas Junior Professional LCSHD Financial management

Jaime Saavedra Education Economist Consultant Economic analysis Eleanor Schreiber Senior Operations Officer LCSHD Education policy

50

Luis Secco Infrastructure Specialist Consultant Civil works and equipment

Isabel Segovia Education Consultant Aide Memoire coordinator

Paul Sisk Financial Management Specialist LCOFM Financial management

Maria Teresa Tato Education Specialist Consultant External peer reviewer

Cecilia Thorne Education Materials Specialist Consultant Learning materials David Varela Legal Counsel LEGLA Loan agreement

Nelly Vergara Project Assistant LCSHD Mission logistical support

Luisa Yesquen Language Program Assistant LCSHD Operation support

Alfonso Zarzar Social Sector Specialist LCSES Gender and indigenous issues

Alberto Zuñiga Infrastructure Specialist Consultant Civil works and equipment

Supervision/ICR Livia Benavides Senior Social Sector Specialist LCSHD Task team leader Keisgner Alfaro Senior Procurement Specialist LCOPR Procurement Patricia Alvarez Operations Officer LCSHD Implementation Erika Bazan Operational Assistant LCSHD Logistical support

Erik Bloom Human Develop. Economist LCSHD Monitoring and evaluation

Daniel Cotlear Sector Leader LCSHD Bank management

Nocolas Drossos Financial Management Specialist Consultant Financial management

Antonio Gomes Pereira Education Specialist Consultant Teacher training

Nelly Ikeda Financial Management Specialist LCOFM Financial management

Luis Enrique Lopez Rural Education Specialist Consultant Bilingual education

Patricia McKenzie Financial Management Specialist LCOFM Financial management

Aldo Ortiz Implementation Specialist Consultant Institutional arrangements

Carmen Osorio Junior Professional Associate LCSHD Operations support

Cristina Pareja Education Management Specialist Consultant Institutional strengthening

Carlos Rojas Senior Education Specialis LCSHD Rural education

Ana Karina Rozas Junior Professional Associate LCSHD Financial management

Luis Secco Infrastructure Specialist Consultant Civil works and equipmentg

Luis Schwarz Financial Management Specialist LCOFM Financial management

Sergei Soares Education Economist LCSHD Monitoring and evaluation

Cynthia Vainstein Summer Intern LCSHD Overall supervision support

Silvana Vargas Monitoring-evaluation specialist Consultant Monitoring and

51

evaluation

Eduardo Velez Bustillo Sector Manager LCSHD Bank management Evelyn Villatoro Procurement Specialist LCOPR Procurement Luisa Yesquen Language Program Assistant LCSHD Operation support

Alfonso Zarzar Social Sector Specialist LCSES Gender and indigenous issues

(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)

Stage of Project Cycle No. of staff weeks USD Thousands (including

travel and consultant costs)Lending

FY98 0.00 FY99 1.71 FY00 19 74.69 FY01 8 17.92 FY02 41 264.64 FY03 10 45.36 FY04 0.00 FY05 0.00 FY06 0.00 FY07 0.00 FY08 0.00

Total: 78 404.32 Supervision/ICR

FY98 0.00 FY99 0.00 FY00 0.17 FY01 0.00 FY02 0.00 FY03 12 41.54 FY04 25 94.78 FY05 24 80.64 FY06 26 89.95 FY07 39 138.74 FY08 22 46.91

Total: 148 492.73

52

Annex 5. Beneficiary Survey Results (if any) Not applicable

53

Annex 6. Stakeholder Workshop Report and Results (if any) Not applicable

54

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR The Ministry of Education sent its Project Completion Report to the Bank on April 8, 2008. Additionally, the Ministry of Education submitted comments to the draft ICR and updated data on June 19, 2008. The Ministry of Finance submitted separate comments on June 20, 2008. Both sets of comments have been included in this final version of the Bank’s ICR. A copy of the Project Completion Report and of the comments are available in the Project file.

55

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable

56

Annex 9. List of Supporting Documents Bank preparation documents Peru Country Assistance Strategy (CAS), discussed by the Board July 22, 1997 (Report 97-161). Preparation Mission Terms of Reference, Aide-Memoire, and Back-to-Office Report. The World

Bank. August 11, 1999. Preparation Mission Terms of Reference, Aide-Memoire, and Back-to-Office Report. The World

Bank. November 5, 1999. Identification Mission Terms of Reference, Aide-Memoire, and Back-to-Office Report. The World

Bank. October 1, 2001. Pre-appraisal Mission Terms of Reference, Aide-Memoire, and Back-to-Office Report. The

World Bank. March 5, 2002. Appraisal Mission Terms of Reference, Aide-Memoire, and Back-to-Office Report. The World

Bank. May 08, 2002. Pre-negotiations Mission Terms of Reference, Aide-Memoire, and Back-to-Office Report. The

World Bank. May 7, 2003. Project Concept Document (PCD), The World Bank, October 19, 2001. Project Appraisal Document (PAD), Report No 23843-PE. The World Bank, April 30,2003. Agreed Minutes of Negotiations. The World Bank. April 25, 2003. Bank project implementation documents Loan Agreement 7176-PE. The World Bank. Conformed copy December5, 2003. Amendment to the Loan Agreements. September 13, 2005 (Cancellation of the Special Account) Statement of Mission Objectives, Aide-Memoires, Back-to-Office Reports, Management Letters,

Project Status Reports (PSRs) and Implementation Status and Results (ISRs) of all the Supervision Mission.s

Bank and Borrower other project implementation-related documents Project Implementation Plan (2003 – 2005) Quarterly FMRs Annual FM audits Ex-post Procurement Audits Annual Project Implementation Reports (2004, 2005, 2006) PEAR Final Evaluation (April, 2007). Operations Manual Comments from MED to Draft ICR (June 19, 2008) Comments from MEF to Draft ICR (June 20, 2008)

Public Disclosure Authorized IMPLEMENTATION...

Documents

Transcript of Public Disclosure Authorized IMPLEMENTATION...