Document of The World Bank...Document of The World Bank Report No: ICR00001840 IMPLEMENTATION...

Document of

The World Bank

Report No: ICR00001840

IMPLEMENTATION COMPLETION AND RESULTS REPORT

ON A

LOAN 7317-CL

IN THE AMOUNT OF US$25.13 MILLION

TO THE

REPUBLIC OF CHILE

FOR A

TERTIARY EDUCATION FINANCE FOR RESULTS PROJECT

IN SUPPORT OF THE FIRST PHASE OF THE TERTIARY EDUCATION FINANCE

FOR RESULTS PROGRAM

June 24, 2011

Human Development Sector Manager Unit

Bolivia, Chile, Ecuador, Peru and Venezuela Country Management Unit

Latin America and the Caribbean Regional Office

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

Currency Unit = Chilean Pesos

472= US$1

FISCAL YEAR

January 1 – December 31

ABBREVIATIONS AND ACRONYMS

AFD Direct Public Support (Aporte Fiscal Directo)

AFI Indirect public support

CAE State Guaranteed Student Loan Program (Crédito con aval del Estado)

CASEN Survey of National Socioeconomic Characterization (Encuesta de

Caracterización Socioeconómica Nacional)

CAN National Accreditation Commission

CFT Technical Training Centers

CPS Country Partnership Strategy

CRUCH Consejo de Rectores de Universidades Chilenas

DIVESUP Higher Education Division in the Ministry of Education (División de

Educación Superior)

FIAC Academic Innovation Fund

GoC Government of Chile

IBRD International Bank for Reconstruction and Development

ICR Implementation Completion Report

IPDP Indigenous People Development Plan

ICU Institutional Coordination Units

ISR Implementation Status Report

KPI Key Project Indicators

MECESUP Higher Education Improvement Project

Mece1 Higher Education Improvement Project (1998 - 2005)



M&E Monitoring & Evaluation

OECD Organization for Economic Co-operation and Development.

PAD Project Appraisal Document

PDO Project Development Objectives

PIU Project Implementation Unit

PSU University Admission Exams

SCT-Chile Academic Credit Transfer System of Chile

SIES Tertiary Education Information System

TEI Tertiary Education Institution

UCH University of Chile (Universidad de Chile)

UFRO Universidad de la Frontera

Vice President: Pamela Cox

Acting Country Director: Laura Frigenti

Sector Manager: Chingboon Lee

Project Team Leader: Michael Crawford

ICR Author: Roberta Malee Bassett

CHILE

TERTIARY EDUCATION FINANCE FOR RESULTS PROJECT

(First Phase Adaptable Program Lending)

CONTENTS

Data Sheet

A. Basic Information

B. Key Dates

C. Ratings Summary

D. Sector and Theme Codes

E. Bank Staff

F. Results Framework Analysis

G. Ratings of Project Performance in ISRs

H. Restructuring

I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 1

1.1 Context at Appraisal ............................................................................................. 1

1.2 Original Project Development Objectives (PDO) and Key Project Indicators

(KPI). .......................................................................................................................... 2

1.3 Revised PDO ......................................................................................................... 3

1.4 Main Beneficiaries ................................................................................................ 3

1.5 Original Components ............................................................................................ 4

1.6 Revised components ............................................................................................. 5

1.7 Other significant changes:..................................................................................... 5

2. Key Factors Affecting Implementation and Outcomes .............................................. 5 2.1 Project Preparation, Design and Quality at Entry ................................................. 5

2.2 Implementation ..................................................................................................... 6

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization ...... 8

2.4 Safeguard and Fiduciary Compliance ................................................................... 9

2.5 Post-completion Operation/Next Phase .............................................................. 10

3. Assessment of Outcomes .......................................................................................... 11

3.1 Relevance of Objectives, Design and Implementation ....................................... 11

3.2 Achievement of Project Development Objectives (PDO) .................................. 11

3.3 Efficiency ............................................................................................................ 14

3.4 Justification of Overall Outcome Rating ............................................................ 15

3.5 Overarching Themes, Other Outcomes and Impacts .......................................... 15

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops ... 17

4. Assessment of Risk to Development Outcome ......................................................... 17 5. Assessment of Bank and Borrower Performance ..................................................... 18

5.1 Bank Performance ............................................................................................... 18

5.2 Borrower Performance ........................................................................................ 19

6. Lessons Learned ....................................................................................................... 20

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 21 Annex 1. Project Costs and Financing .......................................................................... 22 Annex 2. Outputs by Component ................................................................................. 23 Annex 3. Economic and Financial Analysis ................................................................. 30 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 36

Annex 5. Beneficiary Survey Results ........................................................................... 38

Annex 6. Stakeholder Workshop Report and Results ................................................... 39 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 40

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 51 Annex 9. List of Supporting Documents ...................................................................... 52 MAP

i

A. Basic Information

Country: Chile Project Name:

Tertiary Education

Finance for Results

Project- First Phase

APL

Project ID: P088498 L/C/TF Number(s): IBRD-73170

ICR Date: 06/24/2011 ICR Type: Core ICR

Lending Instrument: APL Borrower: GOVERNMENT

Original Total

Commitment: USD 25.1M Disbursed Amount: USD 25.1M

Revised Amount: USD 25.1M

Environmental Category: C

Implementing Agencies:

Ministry of Education

Cofinanciers and Other External Partners:

B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 09/17/2004 Effectiveness: 11/15/2005 11/15/2005

Appraisal: 02/24/2005 Restructuring(s):

Approval: 07/05/2005 Mid-term Review: 01/19/2008 01/19/2009

Closing: 06/30/2009 12/31/2010

C. Ratings Summary

C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: Low or Negligible

Bank Performance: Moderately Satisfactory

Borrower Performance: Highly Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR)

Bank Ratings Borrower Ratings

Quality at Entry: Moderately

Satisfactory Government: Highly Satisfactory

Quality of Supervision: Moderately

Satisfactory

Implementing

Agency/Agencies: Highly Satisfactory

Overall Bank

Performance:

Moderately

Satisfactory Overall Borrower

Performance: Highly Satisfactory

ii

C.3 Quality at Entry and Implementation Performance Indicators

Implementation

Performance Indicators

QAG Assessments

(if any) Rating

Potential Problem Project

at any time (Yes/No): No

Quality at Entry

(QEA): None

Problem Project at any

time (Yes/No): No

Quality of

Supervision (QSA): None

DO rating before

Closing/Inactive status: Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Central government administration 10 10

Tertiary education 90 90

Theme Code (as % of total Bank financing)

Administrative and civil service reform 20 20

Education for the knowledge economy 40 40

Managing for development results 40 40

E. Bank Staff

Positions At ICR At Approval

Vice President: Pamela Cox Pamela Cox

Country Director: Acting CD: Laura Frigenti Evangeline Javier

Sector Manager: Chingboon Lee Eduardo Velez Bustillo

Project Team Leader: Michael F. Crawford Lauritz B. Holm-Nielsen /

Kristian Thorn

ICR Team Leader: Michael F. Crawford

ICR Primary Author: Roberta Malee Bassett

F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) The project development objective is to increase the effectiveness of public funding for

tertiary education by enhancing coherence, responsiveness, equity and quality in the

system, through stronger accountability for performance.

Revised Project Development Objectives (as approved by original approving authority)

There were no revisions made to the PDO during the implementation of the project.

iii

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target

Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Number of students graduating from CFTs (Technical Training Centers) and IPs

(Professional Institutes) transferring to universities.

Value

quantitative or

Qualitative)

0 1000 2432

Date achieved 01/01/2004 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target exceeded. Baseline revised in 2005 to 72 students transferring to CRUCH

universities. Final value corresponds to data available for all universities.

Indicator 2 : Socioeconomic profile of university first year students-- Q1/2.

Value

quantitative or

Qualitative)

Q1/2 16.1% Q1/2 19% Q1/2 24.7%

Date achieved 01/01/2000 12/31/2007 12/31/2009

Comments

(incl. %

achievement)

Target exceeded. Baseline revised in 2003 to 23.8%. Target not revised.

Indicator 3 : Socioeconomic profile of university first year students--Q3.

Value

quantitative or

Qualitative)

16.3% 19% 15.9%

Date achieved 01/01/2000 12/31/2007 12/31/2009

Comments

(incl. %

achievement)

Baseline revised in 2003 to19.8%. Target not revised.

Indicator 4 : Socioeconomic profile of university first year students-- Q4/5.

Value

quantitative or

Qualitative)

67.6% 62% 59.3%

Date achieved 01/01/2000 12/31/2007 12/31/2009

Comments

(incl. %

achievement)

Baseline revised in 2003to 56.4%. Target not revised.

Indicator 5 : Socioeconomic profile of CFT (Technical Training Centers) first year students--

Q1/2.

Value

quantitative or

Qualitative)

28.6% 33% 25.4%

Date achieved 01/01/2000 12/31/2007 12/31/2009

Comments

(incl. %

achievement)

Baseline revised in 2003to 42.5%. Target not revised.

iv


Q3.

Value

quantitative or

Qualitative)

27% 30% 28.8%

Date achieved 01/01/2000 12/31/2007 12/31/2009

Comments

(incl. %

achievement)

Baseline revised in 2003 to 26.5%.Target not revised.


Q4/5.

Value

quantitative or

Qualitative)

44.4% 37% 45.8%

Date achieved 01/01/2000 12/31/2007 12/31/2009

Comments

(incl. %

achievement)

Target exceeded. Baseline revised in 2003 to 31%. Target not revised.

Indicator 8 : First year drop-out rate in undergraduate education.

Value

quantitative or

Qualitative)

Available March 2006. 22.2%

Date achieved 03/01/2006 12/31/2009

Comments

(incl. %

achievement)

Baseline and targets not defined in PAD. In 2005, baseline of 19% established

for Cruch universities. Values achieved are: Cruch: 22.2%; Private Universities:

28.5%; Professional Institutes: 38%, and CFTs: 36%.

Indicator 9 : Percentage of indicators in performance agreements with a scoring at or above

the agreed target by institution.

Value

quantitative or

Qualitative)

0% 90% 52.5%

Date achieved 01/01/2005 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Indicator counts targets achieved or exceeded. When progress against targets is

measured, the average value achieved is 84% (University of Bío Bío: 98%,

University of La Frontera: 92%, University of Tarapaca: 80%, and University of

Chile: 67%).

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target

Values (from

approval

documents)

Formally

Revised

Target Values

Actual Value

Achieved at

Completion or

Target Years

Indicator 1 : Perception of DIVESUP policy instruments by university leaders.

Value

(quantitative

or Qualitative)

Survey in 2005 End of project

survey.

Survey to be

conducted.

Date achieved 12/31/2005 12/31/2008 12/31/2010

v

Comments

(incl. %

achievement)

Survey to establish baseline was conducted in 2008. Follow-up survey expected

in 2011.

Indicator 2 : Number of studies on Tertiary Education conducted by date.

Value

(quantitative

or Qualitative)

0 4 7

Date achieved 05/19/2005 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target exceeded. 7 studies were completed. Refer to Annex 2 for a list of these.

Indicator 3 : Percentage of undergraduate tertiary enrollees studying in accredited programs

and programs in the process of being accredited.

Value

(quantitative

or Qualitative)

31% 37% 51%

Date achieved 12/31/2003 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target exceeded.

Indicator 4 : Percentage of masters degree tertiary enrollees studying in accredited programs

and in process of being accredited.

Value

(quantitative

or Qualitative)

12% 20% 35%

Date achieved 12/31/2002 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target exceeded.

Indicator 5 : Percentage of doctoral degree tertiary enrollees studying in accredited programs

and in process of being accredited.

Value

(quantitative

or Qualitative)

22% 60% 85%

Date achieved 12/31/2002 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target exceeded.

Indicator 6 : Number of sub-projects supported at the tertiary level (cumulative).

Value

(quantitative

or Qualitative)

30 120 371

Date achieved 01/01/2004 12/31/2008 12/31/2008

Comments

(incl. %

achievement)

Target exceeded. Target set at 40 subprojects per year in 2006, 2007 and 2008.

Actual was 94 subprojects in 2006, 158 in 2007 and 119 in 2008.

Indicator 7 : Number of sub- projects targeted at degree structure reform (cumulative).

Value 0 81 153

vi

(quantitative

or Qualitative)

Date achieved 01/01/2004 12/31/2008 12/31/2008

Comments

(incl. %

achievement)

Target exceeded. Target set at 27 subprojects per year in 2006, 2007 and 2008.

Actual was 25 subprojects in 2006, 64 in 2007 and 64 in 2008.

Indicator 8 : Enrollment in preparatory programs.

Value

(quantitative

or Qualitative)

0 3,900 35,713

Date achieved 01/01/2005 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target exceeded. Baseline revised in 2005to 307 students. Of the value achieved,

75% of students were enrolled in Mecesup-supported preparatory programs.

Remainder were enrolled at independently established programs.

Indicator 9 : Relation between average PSU score for admission in teaching programs

supported and those not supported.

Value

(quantitative

or Qualitative)

No information available

for PAD.

5% increase from

the baseline. 4.1%

Date achieved 01/01/2005 12/31/2008 12/31/2009

Comments

(incl. %

achievement)

Target partially achieved. Baseline set in 2004 as 0.4%. Target achieved in 2006

(7.4%) and 2007 (5.7%), and not in 2008 (5.0%) and 2009 (4.1%).

Indicator 10 : Number of enrolled doctoral students in Chile.

Value

(quantitative

or Qualitative)

1,930 2,500 4,055

Date achieved 07/01/2003 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target exceeded. Baseline revised in 2005 to 2,334. These value achieved

includes doctoral students from private universities, of which there were 501 in

2010.

Indicator 11 : Number of university academic staff enrolled in a PhD program in Chile or

overseas by field.

Value

(quantitative

or Qualitative)

2,861 3,100 893

Date achieved 01/01/2004 01/01/2008 01/01/2010

Comments

(incl. %

achievement)

Baseline revised to 860. Universities are hiring new staff with PhDs rather than

sponsoring the PhDs of existing staff.

Indicator 12 : Number of universities covered by a performance agreement.

Value

(quantitative

or Qualitative)

0 4 4

Date achieved 01/01/2005 12/31/2008 12/31/2010

Comments

(incl. %

achievement)

Target achieved. Non-MECESUP funded performance agreements have spread

to humanities programs at Universidad de la Frontera and Universidad de Chile.

Additional agreements to be negotiated in 2011 with the universities of

vii

Valparaíso, Talca, and Magallanes.

Indicator 13 : Percent of base funding (AFD) for institutions with a performance agreement

allocated through the agreement.

Value

(quantitative

or Qualitative)

0 13% 2.4%

Date achieved 05/19/2005 12/31/2008 12/31/2009

Comments

(incl. %

achievement)

Indicator calculated as MECESUP funds (numerator) over the sum of

MECESUP and Direct Fiscal Grant (AFD) funds. The AFD has grown more than

expected because it is tied to enrollment, which has grown exponentially.

G. Ratings of Project Performance in ISRs

No. Date ISR

Archived DO IP

Actual

Disbursements

(USD millions)

1 12/20/2005 Satisfactory Satisfactory 0.06











H. Restructuring (if any)

The Project was not restructured during implementation.

viii

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design

1.1 Context at Appraisal

Country Background. Chile enjoys a reputation for high economic growth rates, but its ability to

grow and remain competitive has been uneven. Since 1998, growth rates have been only half of

the seven point one percent (7.1%) achieved from 1984-97. Total Factor Productivity, arguably

the most important measure of the quality and sustainability of growth, has dropped from an

annual 2.8% in that period to only 0.4% annually from 1998-2005. Returning to high levels of

economic growth is a critical policy concern for Chile.1 To do so, Chile needs to diversify and

raise productivity. Because the latter requires building the skills and knowledge necessary to

succeed in a highly competitive global environment, improving the quality of and expanding

access to tertiary education is at the core of the Government‘s strategy.

Sector Background: the Chilean tertiary education system. After two decades of

underinvestment accompanied by a rise in unregulated private provision, a consensus emerged to

invest in the rehabilitation of Chile‘s tertiary education institutions. The Government of Chile

(GoC) and the World Bank partnered to create the Program to Increase Quality and Equity in

Tertiary Education (Programa para el Mejoramiento de la Calidad y Equidad de la Educación

Superior), or MECESUP, as it was known. The 1998-2005 MECESUP project (hereafter Mece1)

invested in infrastructure and academic renovation in universities using a competitive fund with

merit-based allocation. It piloted a quality assurance system through the creation of accreditation

councils and sought to raise the critical mass of human capital available for Chile‘s expanding

tertiary enrollment by investing in domestic and foreign PhD training for university staff.

Ultimately, Mece1 became synonymous with serious higher education reform in Chile, and the

Project‘s ICR rated the achievement of its development objective as highly satisfactory.

After 2005, and building on the successes of Mece1, the principal policy goals for tertiary

education expanded to include introducing performance-based funding and continued expansion

of access and improvement of quality. It was hoped that these mechanisms could continue

progress against the system‘s principal challenges, which included:

Lack of coherence and articulation (i.e., no credit transfer system) between tertiary

education institutions (TEIs) and across tertiary education levels.

Considerable variation of quality of tertiary education and poor responsiveness to student

and labor market needs.

Inequities stemming from low participation rates and high dropout rates for students from

lower socioeconomic quintiles.

Low availability of advanced human capital, especially of PhDs to staff the expanding

system.

Insufficient accountability in the use of public funding and continued reliance on historical

criteria rather than on merit for funding allocation.

Lack of reliable data on the tertiary education sector.

1 Consejo Nacional de la Innovación para la Competitividad (CNIC), 2010.

2

Country Partnership Strategy (CPS) and rational for Bank Assistance. Mece1 showed the

value of the Bank as a catalyst and broker for key sector reform. In particular, the establishment

of a competitive funding mechanism for quality improvement at the program level based on

transparent selection criteria was an important innovation in Chilean tertiary education. Mece2

was a continuation of previous Bank operations in Chile and capitalized on their many lessons.

Having successfully supported ‗management for results‘ in selected sub-projects, the Bank was

in a key position to assist in the adoption of new performance-based financing mechanisms that

moved funding to the institutional level.

Higher level objectives to which the Project contributed

The Project formed part of the GoC‘s efforts to place Chile in a favorable position to realize the

benefits of the emerging knowledge economy. The goal was that the educational sector would

provide the skills and the knowledge necessary to sustain Chile‘s economic and social

development and enhance its national innovation capacity. To this end, the proposed Project, in

conjunction with the Science for the Knowledge Economy Project (P077282) and the Life-long

Learning and Training Project (P068271), formed a coherent set of policies and investments in

skills, knowledge and equity.

CPS-goals supported by the Project

The proposed Project was firmly rooted in the three pillars of intervention in the 2007-2011

Country Partnership Strategy with Chile. First, Mece2 added to the CPS goal of sustaining

overall economic growth and social progress in Chile. Second, it contributed to heightening

inclusion by improving tertiary learning opportunities for students from poor and/or indigenous

backgrounds. Finally, it supported the modernization of the state by (i) building capacity for

strategic policy formulation; (ii) strengthening sector monitoring and evaluation; and (iii)

increasing accountability for the use of public funding.

1.2 Original Project Development Objectives (PDO) and Key Project Indicators (KPI).

The Mece2 PDO is to increase the effectiveness of public funding for tertiary education by

enhancing coherence, responsiveness, equity and quality in the system, through stronger

accountability for performance.

Mece2 was conceived as a two-phase Adjustable Program Loan (APL). The Program

Development Objectives for the Phase 1 APL mirrored the Project Development Objectives,

although they were phrased in different terms. The Table below shows how Program and Project

objectives overlapped and how Key Project Indicators (KPI) corresponded to them. The PAD did

not include definitions of each objective, nor correspondence between objectives and indicators.

The below table was created by the ICR author.

3

Table 1. Program Objectives, Project Development Objectives, and Key Project Indicators

Program Objective for APL

Phase 1

PDO for APL Phase 1 Project:

Increase the effectiveness of

public funding by enhancing:

Key Project Indicators (KPI)

Increase GoC planning and

oversight in the tertiary

education sector

Coherence

Equity

Socio-economic profile of first

year students in tertiary

education

A. Improve policies and strategies

for tertiary education

Coherence

B. Consolidate a national system

for quality assurance

Quality

C. Improve national information

system for tertiary education

Accountability

Coherence

Responsiveness

D. Competitively fund academic

innovation

Coherence

Quality

Equity

Responsiveness

- First year dropout rate in

undergraduate education.

- Number of students from

CFT‘s and IPs transferring to

universities

E. Pilot performance-based

funding contracts

Accountability, Coherence,

Quality, Equity,

Responsiveness

Percent of indicators in

performance agreements met or

exceeded

The Project was to accomplish this through three main mechanisms:

New legislation, policy studies and initiatives for tertiary education.

A competitively-allocated small grants program (referred to as the Academic Innovation

Fund or FIAC for its Spanish acronym).

Four large results-based financing agreements (referred to as Performance Agreements).

1.3 Revised PDO

There were no revisions to the Project‘s Development Objectives during implementation.

1.4 Main Beneficiaries

The primary target groups benefitted mostly from the FIAC and the Performance Agreements.

Tertiary education institutions (TEIs) benefitted from increased investments in their degree

programs, better access to funds, more qualified staff, enhanced physical and online

infrastructure, more quality assurance, and greater ability to conduct research, carry out

special projects and perform monitoring and evaluation.

Academics and staff in TEIs benefitted from more investments in their human capital, an

increased ability to conduct research with better equipment, and more exchange

opportunities with academics in other universities both within and outside the country.

Current and future students of TEIs, particularly those that would not have attended—or

been able to stay—in tertiary education otherwise. This group benefitted from increased

quality of education, more remedial classes, shorter degree programs, more coherent

curricula, new and better PhD programs and teachers, more online course material, and

more exchange opportunities with other universities both within and outside the country.

4

The group also benefit from increased capacity in the Ministry of Education to manage

student financial aid.

Employers and society as a whole benefitted from the increased quality of tertiary education

graduates and production of knowledge throughout the country.

The secondary target group benefitted mostly from a strengthened institutional framework for

tertiary education.

Society as a whole, including think tanks, students, families, and Government, benefitted

from more and better quality information regarding tertiary education.

Ministry of Education benefitted from increases in the capacity provided by the Project

Implementation Unit (PIU), the investments to set up the quality assurance system, and the

strengthened student financial aid system.

1.5 Original Components

As noted in the PAD, the Project had two components: (i) strengthening institutional framework

for tertiary education and (ii) sub-projects to enhance coherence, efficiency, equity and quality.

Below each component are listed the subcomponents and their causal linkages to the PDO. These

linkages are the interpretation of the ICR author as they are not made explicitly in the PAD.

Project Component 1 (US$ 8.00 million)

Strengthened the capacity of the Ministry of Education and relevant agencies to develop policies,

strategies, and management skills to achieve the PDO.

Improvement of sector oversight and regulation. Partially targeted coherence by

strengthening the regulatory framework and the supervision mechanisms for the tertiary

education sector. Partially targeted equity by supporting the establishment of adequate

structures in the Ministry of Education for the administration of a new law on student

financial assistance. Loan disbursement for this subcomponent was conditioned on the

passing of the law.

Strengthening of policies and strategies for tertiary education. Targeted coherence by

financing studies of priority issues for post-secondary and university education.

Consolidation of national system for quality assurance. Targeted quality by supporting the

operation of the accreditation commissions for undergraduate and graduate tertiary

education. Loan disbursement for this subcomponent was conditioned on the passing of the

quality assurance law.

Establishment of a national observatory for tertiary education (later called the Sistema de

Información de Education Superior or SIES for its Spanish acronym). Targeted cohesion by

increasing the information available for decision-making. Targeted responsiveness by

increasing monitoring, evaluation and accountability. The quality assurance law made it

mandatory for TEIs to collect and report data to the Government.

Project management. Supported the operation of the team in charge of directing the

activities of component 1 and the two sub-components of component 2.

Project Component 2 (US$ 83.25 million)

Supported strategically targeted funds aimed at stimulating innovation and institutional

performance and increasing accountability for results. Funds were disbursed directly to

accredited TEIs who reported data to the SIES.

5

Innovative sub-projects for tertiary programs (FIAC). Targeted coherence, responsiveness,

equity and quality by stimulating innovation and strengthening tertiary programs.

Competitively allocated grants allowed TEIs to tactically address priorities in specific areas

of need. The latter centered on improving academic instruction and management quality,

specifically focusing on advancing human capital, academic innovation, improvement of

academic management, and curriculum reform.

Subprojects to stimulate institutional effectiveness in critical areas (Performance

Agreements). Targeted increased accountability and responsiveness for the use of public

subsidies in TEIs through four pilot performance-based financing agreements. Agreements

provided TEIs an incentive to implement plans of action in areas of high national priority, as

financing depended on their performance measured against agreed upon targets. Targeted

equity and quality by furthering, among others, remediation programs for poorly prepared

first-year students and higher quality pedagogy programs.

1.6 Revised components

The components were not revised during the course of the project.

1.7 Other significant changes:

The original closing date was extended from June 30, 2009, to December 31, 2010, in order to

finish implementing the subprojects from the FIAC and allow for discussions regarding a new

project (Mece3), particularly in light of the presidential elections in January 2010.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry

Soundness of the background analysis

The background analysis supporting the Project was sound and the rationale for the Bank‘s

intervention was unambiguous. The strategic relevance and approach of Mece2 was clear within

Chile‘s specific country context, and the technical aspects of the lending program evidenced

rigorous and thorough analysis. Lessons learned from the earlier Mece1 were incorporated into

Mece2‘s design. These included:

Identification of performance-based funding as the key means of overcoming the historical

inertia linked to stagnant quality.

Recognition that the small grant program (FIAC) under Mece1 did not sufficiently

strengthen core institutional management of TEIs nor build capacity within them.

Establishment of Performance Contracts sought to overcome this limitation.

Realization that funding physical infrastructure had declining returns after initial critical

investments were made. Greater returns would be had from focusing on ―soft‖ inputs like

curriculum quality and institutional management. Physical infrastructure was eliminated as

an eligible expenditure category in the Mece2 project.

Poor quality, partial information and data hampered the ability to make sound policy.

Without a legal mandate, institutions had no incentive to provide accurate data. This led to

the inclusion of a clause in the 2006 Quality Assurance Law that required provision of data

to the Ministry. Passing this legislation was a condition for Mece2 disbursement.

6

Assessment of the project design

The solidity of project design rests on the correct proportion of ―pilots‖ of performance contracts

(four) with the continuation and relative reduction of smaller competitive grants under the FIAC.

The mix demonstrated appropriate realism about the degree of complexity achievable and the

pace with which new reforms could be tested. The inclusion of a covenant for legislation to

consolidate the quality assurance system (by legally creating the new National Accreditation

Commission (Comisión Nacional de Acreditación—CNA) was also highly appropriate and

proved very successful. Project design also provided mechanisms to allow Mece2‘s staff to

facilitate high-level analytical work. This led, inter alia, to a series of high-profile studies by the

World Bank and the OECD on instrumental issues in tertiary education policy.

Project design could have improved in the articulation of the PDO with the selected indicators,

and the realism of the Project timeline.

The APL Program Objectives are phrased in such a way that they do not easily align with

the statement of the PDO. Moreover, the Key Performance Indicators are not clearly linked

to the specific Program/PDO objectives; a degree of inference is required.

The Project timeline did not sufficiently take into account the Borrower‘s regular cycle for

budgetary approval in Congress, where resources awarded one fiscal year can only be

disbursed the following year.

Adequacy of GoC‘s commitment, stakeholder involvement, and/or participatory processes

Government commitment during the preparation process was sound and continuous. Stakeholder

participation benefitted from the excellent reputation that the Mece1 project had and the deep

links it had forged with universities.

Assessment of risks and mitigation measures

Assessment of the risk to continued substandard information was accurate and more than

adequately remedied by the 2006 legislation requiring tertiary institutions to comply with

Ministry of Education data requests. Resistance to transparency, credit transfer systems, and

changes to the legal and regulatory framework were deemed mostly moderate and proved to be

so; none presented an impassable obstacle to implementation, although some resistance was

detected. Risks of organized resistance to the proposed changes, insufficient scale of the pilots,

or procurement transparency were also judged moderate and no significant problems

materialized around these risks.

2.2 Implementation

The Project was successfully implemented due to several enabling factors:

Strong commitment on behalf of the GoC facilitated the enabling environment for Mece2. In

2005 the GoC passed the Tertiary Education Student Financing Law and extensively financed

and promoted the expansion of the student aid system. This improved equity throughout the

country and provided TEIs with additional funds with which to upgrade their quality. In 2006 the

GoC shepherded the Quality Assurance Law through Congress and widely encouraged

institutional accreditation thereafter.

7

Competent and stable leadership and staff in the Project Implementation Unit. The PIU

conducted frequent formal and informal stakeholder consultations, and developed solid

relationships with stakeholders and partners. It maintained very fluid communications with TEIs,

quickly becoming a conduit between the Government and them. The PIU also made up for weak

institutional capacity in the Ministry of Education‘s Division of Higher Education, particularly in

the initial years of the project. In addition, the PIU:

Was open to suggestions for improvement of supervision and monitoring.

Was diligent about compliance with Bank requests for information.

Played an active role in the overall policy dialogue for tertiary education, enabling high

quality work and maintaining Mece‘s reputation as one of the premiere agencies for tertiary

education in the region.

Facilitated World Bank analytical work on relevant topics not directly included in Mece2

(State-Guaranteed Student Loan Program, International Scholarship Program, inter alia).

This, along with the implementation of the information system subcomponent, made the

PIU a center for the best policy information available on Chilean tertiary education.

Fluid dialogue with the Ministry of Education included constant feedback on technical issues for

the tertiary education sector as a whole, and not just for Mece2 itself. This active technical

cooperation led to a robust program of Analytic and Advisory Activities.

Some of the events that impacted implementation included:

Initial implementation delays largely due to an unrealistic schedule for the start-up of

activities. This was true both in the Project Implementation Unit and in each TEI with a

FIAC subproject and/or a Performance Agreement.

A 1.5 year project extension. The original closing date was extended from June 30, 2009 to

Dec. 31, 2010 in order to finish implementing the subprojects from the FIAC, achieve

substantial progress in the four pilot Performance Agreements signed, and allow for

discussions regarding a new project (Mece3), particularly in light of the presidential

elections (January 2010).

The Government‘s strict ex-ante fiduciary procedures and high personal penalties for

infractions sometimes led to an ―overly cautious‖ approach to execution of procurement.

When doubt arose about whether national or Bank rules might be in force for a given

transaction, implementers often chose to comply with both sets of procedures. This caused

delays and proved difficult to overcome.

The assessment of implementation should be considered against the backdrop of the rapid

expansion of Chilean tertiary education over the course of Mece2. As figure 1 shows, enrollment

increased by 51% between 2005 and 2010, from 646,000 students to 973,000. According to

Chile‘s Survey of National Socioeconomic Characterization (CASEN for its Spanish-language

acronym), 25% of students enrolled in 2009 hailed from the bottom two income quintiles and

43% from the bottom three quintiles. During this period of massive expansion, Mece2 played a

key role in helping TEIs increase equity while simultaneously improving the quality of their

education. Mece2 did this directly by supporting specific TEI-driven initiatives and indirectly by

prompting others TEIs to take on similar efforts. As this ―spillover‖ effect rippled through the

tertiary education system it further leveraged Mece2‘s investment.

8

Figure 1. Tertiary enrollment in Chile between 1998 and 2010

Source: SIES (http://www.mineduc.cl/index6.php?id_seccion=3224&id_portal=46&id_contenido=13035)

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

At Preparation, the main source of information about tertiary education was the voluntary

contribution of data organized through the Council of Rectors of Chilean Universities (CRUCH

in its Spanish language acronym). These institutions, however, had complex political and

economic reasons for wanting to withhold or selectively provide data. As a result, when the

project began, there was no reliable information on even basic descriptive statistics such as

overall enrollment. The situation changed dramatically during implementation, thanks to the

2006 Quality Assurance Law. The Law created the National Observatory for Tertiary Education

(SIES) and made it a legal requirement for institutions to provide accurate data to the

Government. In addition, only TEIs who reported data were eligible for Mece2 funds. By 2009

99% of institutions were complying. The SIES, financed by subcomponent 1.4, vastly increased

the amount and quality of data available throughout the tertiary education system. This would

have been considered impossible in 2005.

The SIES tracks a variety of statistics related to tertiary education and labor market outcomes for

graduates by institution, including duration of study across degree programs, drop-out rates, and

overall enrollment by institution and by degree program. Additionally, labor market outcomes,

including employment rates and expected salary levels at graduation and over time are also

available. Data improvement throughout the life of the Project was incremental and continuous.

As new information became available, it became increasingly apparent that data originally used

for indicator baselines had been poor and unreliable. Today, Chile has one of the best, if not the

best, tertiary education information systems in Latin America. This statistical compendium

allows for straightforward analysis of the impact of changes in policies on high-level metrics.

This is an important step forward for education policy, as regulators and educators can base

future policy on reliable and rigorous information.

http://www.mineduc.cl/index6.php?id_seccion=3224&id_portal=46&id_contenido=13035

9

M&E design

Although the quality of information improved dramatically over the life of the Project, the

PAD did not sufficiently recognize the poor quality of data available at the time of

Preparation. Baselines were in need of constant review and revision as better data became

available. Wider and more accurate figures routinely replaced what came to be seen as the

―best estimates‖ that were included in the PAD.

Indicators did not always tie directly to Project activities. In some cases, the chain of

causality from Project activity to indicator was diffuse or weak.

No indicators tracked outcomes expected in the PAD‘s Economic and Financial Analysis.

M&E Implementation

Data for indicators was frequently collected and continuously updated. To enable better

tracking of performance and progress, the PIU revisited indicator baselines as data improved.

The PIU also broadened the number of indicators tracked to include the outcomes generated

by Mece2 in private universities, technical training centers and professional institutes. This

was important to capture because Mece2 had ―spillover effects‖: TEIs not supported by the

Project copied some of the developments and innovations implemented in TEIs that were

supported.

At the request of the Task Team, the PIU developed quantitative performance metrics across

eight qualitative dimensions of FIAC projects. These metrics improved vetting of outcomes,

and allowed better comparisons of methods and results across projects.

An International Advisory Committee was established with renowned experts to monitor

and evaluate the progress of Mece2 and give advice with regards to its strategic direction. In

mid-2010, members of this Committee conducted the Midterm Review of the four

Performance Agreements and an evaluation of the four triggers that conditioned the second

phase of the Adaptable Program Loan. These assessments were both positive, resulting in

congratulatory remarks and feedback.

M&E Utilization

The data tracked by the SIES is used continuously in policy decisions by the Ministry of

Education and the GoC as a whole. Think tanks draw on the SIES regularly.

Employers, students, and families use more and better quality information. The labor market

observatory has become particularly important for employment and career decisions made

by students and their families.

The SIES serves as the basis for analytical studies on tertiary education, including several of

the reports produced by the World Bank.

2.4 Safeguard and Fiduciary Compliance

As part of the implementation of the IPDP, the PIU sought to gather the best possible

information regarding the effectiveness of existing scholarships and financial aid for indigenous

Chileans in tertiary education. Policies and benefits specifically for indigenous students already

existed in Chile prior to Mece2, but there was very little information regarding their

effectiveness. The Task Team encouraged more accurate measurement and data collection on the

situation of indigenous students early on in the project. This was particularly important given

there was no empirical base for additional indigenous student support mechanisms and no data

regarding the impact of existing policies.

10

As better data became available, the focus of the IPDP became ensuring indigenous Chileans had

access to new remedial opportunities made available under the Project. Initial data indicates that

indigenous students are benefitting from these programs in amounts greater than their proportion

of the overall tertiary student population, with positive impacts on retention and graduation rates.

Further analysis is on-going, and the next project is expected to mainstream the results achieved

for indigenous students. ISR ratings of the implementation of the IPDP were satisfactory.

With regards to fiduciary compliance, the Project encountered some difficulties during

implementation. There was insufficient coordination between the TTL, procurement specialists

and consultants designated by the procurement specialists to carry out fiduciary supervision. This

resulted in a failure to agree on a common approach for supervision and on a realistic set of goals

for mitigating deficiencies during the last two years of implementation. Procurement planning

and compliance moved towards being an end in itself and not a means of appropriately and

efficiently procuring goods and services. The task team failed to develop clear criteria to guide

the TEIs regarding which items to include in their procurement plans. TEIs thus felt obliged to

include all transactions regardless of their size, resulting in unwieldy and ineffective

procurement planning. By 2010 the procurement plan had 2300 entries with an average

transaction size under $10,000.

2.5 Post-completion Operation/Next Phase

Although the four triggering conditions that allowed Mece2 to transition to the second phase of

the APL were met, the GoC has requested a new project. The APL2 is no longer an adequate

instrument for the GoC because of Chile‘s changed policy context brought about by the election

of a new administration in January 2010. Building on the PDO of Mece2, the GoC may seek to

fine-tune the new PDO to better fit the current context. In addition, because the size of the

Mece3 loan ($100 million) is four times that of Mece2‘s ($25 million), the GoC may seek to

broaden the PDO. Mece3 will extend the reach of Mece2 by extending performance-based

funding agreements and specifically and proactively addressing quality and relevance in

pedagogy programs, professional institutes and technical training centers, among others.

Transition arrangements

Since project closing the Government has continued to finance the activities of the PIU, and will

continue to do so until Mece3 starts to operate in January 2012. More importantly, the Ministry

of Education recently absorbed the PIU, renaming it the Department of Institutional Funding and

giving it formal responsibilities for all public funding of TEIs. This type of arrangement is a

positive step forward because it ensures continuity and permanence for Mece2 activities.

Performance indicators

M&E will continue until it is revised during the new project. Because the SIES has become the

information backbone of the tertiary education system, hundreds of indicators are continuing to

be tracked, collected and analyzed.

Follow-up by the Bank

The Bank is expected to deliver Mece3 in time for the GoC to include it in its annual budget.

11

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation

In 2010, Chile became a full member of the OECD. OECD member countries have the world‘s

most complex economies, structures of production, and need for highly skilled human capital.

As Chile seeks to achieve mean income levels and other economic and social indicators of its

OECD peers, the Project objectives are arguably more pertinent than ever. The Program

Objective of supporting Chile‘s development into a knowledge-based economy by strengthening

its tertiary education system therefore continues to be highly relevant.

The areas of the PDO continue to be a focal point in the 2011-2016 Country Partnership Strategy.

The CPS outlines a general vision of the country improving competitiveness and achieving

sustainable growth, while reducing inequities and poverty. Stakeholders from civil society and

the private sector identified that a key factor in this vision was improving education quality.

The goals articulated in the CPS directly build upon successes of the Mece2 design. The project

design emphasized performance-based agreements, whereby future disbursements were

contingent upon visible progress against agreed upon metrics. This is a significant departure

from previous disbursement mechanisms that were simply tied to historical allocations and

lacked any incentive or performance aspect. The CPS calls for continued transition from

historical allocations to performance-based agreements. Linking funding allocation to a feedback

loop where disbursements are dependent on progress toward goals and priorities set by the

Ministry of Education further empowers the GoC to articulate clear policy goals across TEIs.

The implementation of the Project is highly relevant to the CPS. The CPS envisions well-

managed TEIs coalescing around goals, and modulating their approaches based on information

as it accrues. Though significant weaknesses remained from the design of the Mece2 PDO and

its related key performance indicators, the Project as implemented has lead to observable and

significant sector-wide improvements. The Project allowed the management of TEIs to become

more professionalized; moreover, through the SIES and through studies funded as part of the

lending program, the Ministry of Education has increased the relevance and coherence of tertiary

education policy across institutions. The data and information systems central to Mece2 have

allowed for a feedback loop, whereby forward decisions are based on outcomes and data trends

visible from prior decisions. In this respect, the Project as implemented has resulted in significant

capacity building. The increased transparency of results and trends from M&E has consolidated

more power with the central management of institutions whereas before each individual faculty

within an institution directed its own policy.

3.2 Achievement of Project Development Objectives (PDO)

Mece2 increased the effectiveness of public funding for tertiary education by enhancing

coherence, responsiveness, quality and equity in the system, through stronger accountability for

performance. Below listed are Mece2‘s specific contributions to each PDO.

Coherence

Led directly to the conversion of the PIU into the Ministry of Education‘s Department of

Institutional Funding within the Division of Higher Education. The mandate of the

Department is to rationalize and optimize the direct public funding of tertiary education

12

institutions. Chile has never had such an explicit mechanism to ensure the coherence of

funding policies. The creation of the Department marks a significant departure from the

―historical‖ basis of funding, and facilitates a central goal of the Project.

Promoted the design and implementation of a credit transfer system (SCT-Chile: www.sct-

chile.cl). It is now possible to transfer credits earned in technical tertiary education toward a

university degree within a single institution, and implementation has begun to enable

transfers between institutions as well. Lack of ―pathways‖ from technical to university

education was a perennial problem before, and this is a major accomplishment. That Mece2

vastly exceeded the student transfer indicator testifies to the success of this initiative.

PDO Indicator # 1 Baseline Target End value

Number of students graduating from CFTs

and IPs transferring to universities 72 (revised) 1000 2,432

Promoted the creation of 18 Institutional Research Units and 22 Teaching and Learning

units in TEIs. These units have increased the coherence of institutional strategies at their

home institutions.

Implemented four performance agreements and succeeded in transforming institutional

culture at the pilot institutions, showing a clear way forward in tertiary education. By

linking future funding to realization of the Ministry of Education‘s policy goals, the

coherence of national tertiary education will continue to improve.

Intermediate Outcome Indicator # 12 Baseline Target End value

Number of universities covered by a

performance agreement 0 4 4

Developed seven major policy studies. These continue to be used to strengthen and guide

tertiary education policy. Studies examined causes of first-year drop-out; university leaders'

perception of Divesup policy instruments; employers' and graduates' perceptions of the

quality of higher education graduates; the situation of advanced human capital for education,

research and innovation; the current state of the higher education sector; the newly

implemented foreign scholarship program; and the situation of indigenous students. These

studies are of high interest to National Educational policy and demand for them exceeded

the goals initially set by Mece2.


Number of studies on Tertiary Education

conducted by date 0 4 7

Responsiveness

Set a new standard for responsiveness in education financing through four performance

agreements with Cruch universities. These resulted in a high attainment of project goals,

with the majority of agreed upon metrics reached.

Targeted the competitively allocated FIAC to tactically address the needs of institutions in

specific areas, in alignment with the priorities of national education policy. These areas

centered around improving academic instruction and management quality, specifically

focusing on advancing human capital, academic innovation, improvement of academic

management, and curriculum reform. The competitively allocated funds were responsive to

the needs of both the institutions as well as the Ministry of Education. Since no single

13

institution could receive more than 18% of FIAC funds in any single round, funds were

equitably distributed regionally and among TEIs. The target for subprojects was exceeded.


Number of sub-projects supported at the

tertiary level (cumulative) 30 120 371

The PIU, at the suggestion of the Bank, developed a quantitative assessment system for

measuring the quality of output from FIAC subprojects. The 371 subprojects were reviewed

at mid-term and completion on 8 dimensions of quality. Individual grades were given for

each dimension, and discussed with subproject teams. Almost 80% of subprojects were

rated at or above satisfactory. Special ―probationary‖ attention was given to low-scoring

projects, and unsatisfactory projects could be terminated less improvements were made.

Number of projects % of total Grade

26 7% excellent

265 71% good / satisfactory

53 14% marginally satisfactory

27 7% unsatisfactory

Quality

Financed the implementation and expansion of a quality assurance system such that by the

end of the Project accreditation was accepted as the norm for all TEIs. In 2006 Congress

passed the Quality Assurance Law and the National Accreditation Commission became the

principal agency for quality assurance in Chile. Accreditation targets were exceeded.

Intermediate Outcome Indicator # 3, 4 and 5 Baseline Target End value

Percentage of tertiary students in accredited programs and programs in the process of being accredited.

Undergraduate 31% 37% 51%

Master‘s 12% 20% 35%

PhDs 22% 60% 85%

Promoted an increase in the quality of pedagogy programs, the number of doctoral programs

and the percentage of faculty with doctoral degrees. The latter increased to 40.9%, from a

baseline of 33%. Directly through Mece2 funds, 69 faculty members are getting PhDs, 15

are pursuing post-doctorates, and 170 have benefitted from short specialization programs

abroad. Additionally, 347 students have PhD scholarships and 5 have graduated to join the

ranks of faculties. Today, there are 4,055 PhD students in Chile. These developments have

raised the quality of tertiary instruction across the board, and will continue to do so, as

institutions work to retain their accreditation.


Number of enrolled doctoral students in Chile 2,334 (revised) 2,500 4,055

Promoted increases in retention rates and a 2.5 month average decrease in time-to-degree.

Gave each of the four TEI performance agreement pilots a chance to define what improved

quality meant, and receive funding to meet agreed indicators. The fact that the majority of

agreed indicators were met argues that the Agreements were significant promoters of quality.

14

Equity

During the course of the project, the number of Chilean tertiary students from the bottom two

income quintiles (Q1 and Q2) rose from 125,000 to 219,000. This equals 29% of all new

students for the period. Since by definition Q1 and Q2 represent 40% of the Chilean population

(each quintile is a fifth of the population), the enrollment rate is converging on the proportion of

Chilean families at this income level. This represents massive progress against earlier times

when students from upper quintiles were enrolling at ten times the rate of poorer students.

These new Q1 and Q2 students were almost exclusively ―first-generation‖ and many were from

poor quality secondary schools. The objective of increasing equity in tertiary education was met

through funding projects to strengthen students‘ basic competencies and increase teacher quality.

The number of students in remediation programs reached over 35,713, surpassing the target

many times over. Chilean TEIs now take responsibility for making sure admitted students

succeed in ways they did not prior to the Project.


Enrollment in preparatory programs 307 (revised) 3,900 35,713

The number of teachers with post-graduate degrees increased significantly over the span of

Mece2, along with the number of students enrolled in post-graduate education. Over time, this

should increase the quality of faculties and thus should promote greater achievement across

student backgrounds and learning styles. Projects to strengthen basic competencies were tailored

to reduce the impact of poor quality secondary education instruction, and increase achievement

of tertiary enrollees with more limited basic skills. The massive increase in Q1 and Q2

enrollment rates changed the composition of first year students: the latter contained an increasing

proportion of students with lower chances of succeeding and higher expected drop out. The first-

year drop-out rate, which remained stable around 21% for Cruch universities and decreased by 2-

3% in private universities, IPs and CFTs, is a testament to the efficacy of these program

enhancements. Overall, the proportion of students from the lowest socioeconomic quintiles in

tertiary education increased from 21% in 2003, to 23% in 2006, to 25% in 2009.

3.3 Efficiency

The value of the Project has been assessed through two distinct NPV calculations, each focusing

on an outcome associated with Mece2. The approach is conceptually similar, if not the same, as

that originally used in the Project Appraisal Document (PAD). As mentioned before, however,

none of the indicators defined in the PAD tracked the outcomes expected in the PAD‘s economic

and financial analysis. The data for the below analysis comes directly from SIES.

Estimates indicate Mece2 has an NPV USD (2005) of $677 million. The funds associated with

the Project had an NPV USD (2005) of $80 million, suggesting that conservatively, the total

leverage of Mece2 was approximately 8.5. This conclusion is explained in detail in Annex 3:

Mece2 has increased tertiary education quality, leading to an increased premium for wage

earners with tertiary education. Thirty percent of this benefit was assumed to be directly

attributable to the Project and its value is estimated at NPV USD (2005) $342 million.

15

Mece2 has led to shorter-duration academic programs and quicker time-to-degree, resulting

in time savings and increased career earnings for graduates of tertiary education. This

benefit is estimated at NPV USD (2005) of $335 million.2

As per the PAD, Mece2 is also expected to result in additional value by increasing

graduation rates; this effect (and its value) cannot yet be observed. Because Mece2 targeted

its efforts to increase retention rates on first year tertiary education students, beneficiaries

have not yet graduated.

3.4 Justification of Overall Outcome Rating

Rating: Satisfactory

The Project is rated satisfactory. Through stronger accountability for performance, important

strides have been made in increasing the coherence, responsiveness, equity, and quality in the

system. Major successes associated with the Project include:

The legislation on quality assurance and student financial aid, and the subsequent major

increases in equity, enrollment, and accreditation across Chile.

The robust and continuously updated Tertiary Education Information System (SIES).

The performance agreements piloted with four universities, including proof-of-concept for a

scale-up of results-based financing.

The increases in advanced human capital and research capacity, instrumental for a

knowledge economy.

3.5 Overarching Themes, Other Outcomes and Impacts

(a) Poverty Impacts, Gender Aspects, and Social Development Due to inadequate preparation during secondary education, a notable group of undergraduates

enrolling in tertiary education today lack basic skills and require remedial work prior to

embarking on degree programs. Historically, in the absence of remedial or basic competencies

opportunities, these students faced high drop-out rates and/or slow educational progress.

A number of elements of the Mece2 program directly addressed this source of inequity. Several

institutions receiving funds through the performance agreements established or expanded

remedial classes. One of the loan triggers targeted a minimum enrollment of 2,000 students in

remedial courses, and by the end of 2008 had attained actual enrollments of 9,724 students. By

2010, 35,713 students had enrolled in such classes; while 75% of them were in programs

supported directly by Mece2, the remainder was in independently created programs. Indeed,

Mece2‘s initial push for remediation courses created a powerful spillover effect. The expanded

remedial education likely accounts for a portion of the reduction in drop-out rates in the lower

socioeconomic quintile undergraduates.

2 The granularity of the data was complete only to the university level. Analyzing just the subset of

CRUCH universities instead of all the universities (Cruch & private) would have been preferable,

however, as most of the funds associated with Mece2 accrued to CRUCH universities. This may make the

NPV more conservative, as CRUCH degree durations are expected to have decreased more than the

university-wide average.

16

Additionally, the competitively allocated FIAC targeted projects to increase the effectiveness of

the remedial curriculum, and included funds directed at increasing educator quality through

improved pedagogy curriculum and increased numbers of pedagogy graduates. Over time,

increased rigor in teacher education is expected to increase teacher quality. Ultimately this is

expected to drive significant social development as higher quality teachers return to their

communities and increase the quality of other educational institutions.

In terms of gender, female undergraduate enrollment in tertiary education has increased from

48% to 51% of total enrollment between 2005 and 2010. In addition, females have higher

retention rates than males in their first year of study (72% versus 66%), and have an average time

to degree that is two semesters shorter than males (10.5 semesters versus 11.6 semesters). These

changes may lead to increased female participation in the labor force.

(b) Institutional Change/Strengthening

Mece2 has driven significant change in tertiary education through strengthening information

infrastructure, and enhancing policy direction and accountability in management.

The SIES has allowed for significant institutional strengthening by creating a policy feedback

loop based on outcomes. In particular, the quantity of funds allocated through performance

agreements are dependent on measurable progress towards goals agreed to by the institutions and

the Ministry of Education. More transparent outcomes have increased accountability in the

system, and with results closely tracked and measured for improvement, institutions have an

incentive to develop strong coordinated policies to improve performance. This has had the effect

of increasing the centralization of policy and power with institutional management.

Importantly, Mece2‘s Project Implementation Unit has been fully institutionalized in the

Ministry of Education as the entity responsible all funding to TEIs that is not student-related.

Initial conversations with the GoC suggest that more of the direct transfer funding for CRUCH

universities will evolve to performance based funding. Additionally, if educational funding

continues to be allocated through performance agreements, as is expected in the upcoming

Mece3 project, the increased accountability of the funding mechanism will allow the Ministry of

Education to coordinate national educational policy more closely with the newly strengthened

institutional management. Previously, with power decentralized across various faculties, this

coordination would not have been possible.

(c) Other Unintended Outcomes and Impacts (positive or negative) Mece2‘s reputation allowed it to facilitate analytical reports on tertiary education quality beyond

the number expected. The World Bank and the OECD collaborated on two major studies—one

on the tertiary education system and one on Chile‘s Foreign Scholarship Program (Becas Chile).

The recommendations of both reports were well received and key elements adopted.

The success of the GoC in implementing projects to bolster the quality of tertiary education has

resulted in significant South-South knowledge transfer. Chile has shared its experience and

expertise in implementing or advising funding programs in Tunisia, Bangladesh, India, Ghana,

Mozambique, Lesotho, Montenegro, and Indonesia.

17

Finally, Mece2‘s increased focus on tertiary education corresponded with an increased

commitment by the GoC to improving equity in tertiary education, manifest most visibly in the

new State-Guaranteed Student Loan Program (CAE). The financial commitment to CAE has

dwarfed the scale of the commitment to Mece2 activities, with ongoing annual disbursements at

a scale similar to the entire lifetime disbursements of the Mece2 project.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

No beneficiary survey and/or stakeholder workshops were conducted for the ICR.

4. Assessment of Risk to Development Outcome

Rating: Negligible to low

Despite the March 2010 change in administration (from President Bachelet to President Piñera),

the Government has focused time and effort on improving quality and equity in tertiary

education. Going forward, the GoC will emphasize incentive schemes for students, educators,

and institutions to improve management and results based on standards, and will promote equity

through scholarships and loans. The GoC is also developing training programs for educators and

better information systems for graduates to have access to more granular labor market data.

Many of these goals will be addressed through the Mece3 project. The risk to development

outcomes is thus negligible: the Government has made specific policy decisions to further and

continue the objectives of Mece2, and has requested preparation of Mece3 to extend their reach.

Enhance quality and relevance of tertiary education

This will likely be one of the development objectives of the new Mece3 project. The latter‘s size,

$100 million, is a reflection of the Government‘s commitment to this issue. The new project will

extend the reach of Mece2 by specifically and proactively addressing quality and relevance in

technical education (professional institutes and technical training centers).

Ensure access to tertiary education for all talented individuals

Equity in Chile has been significantly enhanced through programs complimentary to Mece2. The

CAE program gives academically qualified, poor students loans to attend tertiary education.

CAE has grown exponentially and now has over 216,000 borrowers (equivalent to almost one

fourth of Chile‘s undergraduate students) and 63% of them come from the lowest two income

quintiles. The Government has taken steps to improve implementation and increase repayment

rates among borrowers, and the program is highly likely to continue.

Improve internal efficiency and strengthening accountability for results

The Division of Higher Education in the Ministry of Education continues to upgrade the quality

of its human capital and improve its internal efficiency. Several new PhDs have been brought on

board, including one in charge of managing the newly created the Center for Higher Education

Studies within the Ministry. This center will further improve internal efficiency by providing

opportune evidence and information for policy decisions.

In terms of accountability for results, the Government has requested that all the resources in the

new project be disbursed using competitively-allocated, performance-based contracts. In addition,

18

the Government wants to change the way base funding is allocated to public universities and the

President and Minister of Education have already discussed a proposal to move towards a

performance-based system for an additional 5% of base funding.

5. Assessment of Bank and Borrower Performance

5.1 Bank Performance

(a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Satisfactory

The PAD provided high quality guidance and presented a strong case for investment. The

strategic relevance and approach of Mece2 was clear within the specific country context, and the

technical, financial and economic aspects of the lending program evidenced rigorous and

thorough analysis. Moderate shortcomings included:

The indicator structure was overly defined and had limited regard for sufficient data

availability and quality.

The link between APL Program Objectives, PDO and key performance indicators was

unclear. The APL Program Objectives were phrased in such a way that they did not easily

align with the statement of the PDO. Moreover, the Key Performance Indicators were not

clearly linked to the specific Program/PDO objectives.

No indicators tracked outcomes expected by the PAD‘s Economic and Financial Analysis.

The timeline for the Project did not sufficiently take into account the Borrower‘s regular

cycle for budgetary approval in Congress.

Project implementation arrangements significantly increased transaction costs. Since the

majority of items in the Procurement Plan were non-procurable, fiduciary responsibilities

would have been better carried out through Financial Management rather than through

procurement mechanisms. Insufficient coordination on this front created duplicative work.

(b) Quality of Supervision Rating: Moderately Satisfactory

The Bank was particularly strong with regards to its focus on development impact, its technical

supervision, and its role in ensuring and encouraging adequate transition arrangements after

project closing. In addition to timely compliance with normal supervision responsibilities, the

Bank suggested new ways of measuring outcomes of FIAC subprojects. The Bank Task Team

maintained active communication with the PIU. The TTL was regularly invited to address

important tertiary education events, and his opinion was routinely sought by the Chilean press on

related issues. Fluid dialogue with the Ministry of Education included constant feedback on

technical issues for the tertiary education sector as a whole, and not just for Mece2 itself. This

active technical cooperation led to a robust program of Analytic and Advisory Activities.

Supervision of safeguards aspects was active and vigilant, assessing the Project regularly.

For a satisfactory rating, procurement difficulties would have been addressed sooner in the life of

the Project. Insufficient coordination between the TTL and the procurement specialists resulted

in a failure to agree on a common approach for supervision and on a realistic set of goals for

19

mitigating deficiencies during the last two years of implementation. Procurement planning and

compliance thus moved towards being an end in itself and not a means of appropriately and

efficiently procuring goods and services.

(c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory

Overall Bank performance in ensuring quality at entry and in the quality of its supervision was

moderately satisfactory. The PAD provided effective guidance and presented a strong case for

investment. It would have benefitted from a more flexible indicator structure, a better selection

of indicators, a more streamlined procurement plan, and a more realistic timeline for

implementation. Supervision was very strong on technical aspects and on post-closure transition

arrangements, but would have benefitted from more in-the-field coordination between the core

task team and the fiduciary team.

5.2 Borrower Performance

(a) Government Performance Rating: Highly Satisfactory

The GoC showed strong commitment to achieving project development objectives, facilitating

the enabling environment for Mece2, utilizing the data generated through it in its decision-

making, and ensuring adequate transition arrangements for it after closing. The GoC:

Passed the 2006 Quality Assurance Law and widely encouraged institutional accreditation.

Passed the 2005 Tertiary Education Student Financing Law and extensively financed and

promoted the expansion of the student aid system.

Commissioned several additional tertiary education policy studies, including a WB-OECD

report on the BioBio Region, and a WB report on the state-guaranteed student loan program.

Increased the breath of the tertiary education information system and actively used its data.

Integrated the PIU into the Ministry of Education and renamed it the Department of

Institutional Funding, making it fully responsible for all public funding for TEIs.

Transitioned Mece2 into a new project where resources will be conditioned on performance.

Financed the time gap between the Mece2 and Mece3 projects.

Is currently discussing how a portion of traditional funding for public universities can be

linked to performance.

(b) Implementing Agency or Agencies Performance Rating: Highly Satisfactory

The Project Implementation Unit (PIU) showed strong commitment to achieving project

development objectives, conducted frequent formal and informal stakeholder consultations, put

in place adequate monitoring and evaluation arrangements, and developed solid relationships

with stakeholders and partners. More concretely, the PIU:

Improved the indicator structure originally set out in the PAD, re-setting baselines when

new information was available and increasing the number of indicators tracked to more

adequately inform decision-making.

20

Facilitated additional work requested by the Government to the World Bank.

Compensated for weak institutional capacity in the Ministry of Education‘s Division of

Higher Education, particularly in the initial years of the project. In later years, and more

recently, it became a conduit for extra capacity.

Maintained fluid communications with Government and TEIs, quickly becoming a conduit

between the two. Proof of this is the PIU‘s recent absorption into the Ministry of Education,

and its new additional responsibilities regarding all public funding for TEIs.

(c) Justification of Rating for Overall Borrower Performance Rating: Highly Satisfactory

The Borrower showed strong commitment to achieving project development objectives: it

facilitated the enabling environment for Mece2, put in place adequate monitoring and evaluation

arrangements, conducted frequent formal and informal stakeholder consultations, and ensured

adequate transition arrangements for Mece2 after closing. Even with the initial project

implementation delays due to the Borrower‘s regular cycle for budgetary approval in Congress,

the Borrower‘s performance can be assessed as highly satisfactory.

6. Lessons Learned

Mece1 and Mece2 both provided valuable learning opportunities regarding tertiary education

reforms, leaving lessons that could prove useful for projects both in Chile and in other country

and regional contexts. The primary take-away from Mece2 is the importance of having a well-

defined results framework from the beginning and ensuring the indicators in the framework are

explicitly related to the PDO. In addition, a key lesson is taking advantage of the opportunities

afforded by coincidental timing—in this case, supportive legislation being enacted at just the

moment when it could be leveraged to achieve program objectives. Finally, the following

summarize specific lessons gleaned through the course of project implementation and assessment.

Competent and stable leadership and team in the PIU were a key component of Project success.

The PIU already had institutional experience with the Mece1 project, which facilitated

implementation of Mece2. More importantly, however, the team was composed of competent

and well-prepared professionals committed to Project goals. Little turnover allowed for

continuity and good leadership facilitated Project coordination, execution and credibility.

Rigorous results scoring in FIAC subprojects increased comparability and performance tracking.

To understand which FIAC subprojects functioned well and which did not, performance metrics

across eight qualitative dimensions were developed for projects that did not easily lend

themselves to explicitly quantitative assessments. These metrics allowed for better vetting of

outcomes, and also served to compare methods and results. Without a rigorous approach, the

default tendency was to assert all subprojects had worked well or poorly, with no differentiation.

Realistic timelines for implementation require taking into account the Borrower‘s cycle for

budgetary approval in Congress and the internal approval processes for the various legal

documents needed throughout the life of the Project. For both Performance Agreements and

FIAC, resources awarded in one fiscal year could only be disbursed the following year, initially

21

triggering some implementation delays. In addition, because the time required for internal

approval processes is highly variable, adequate flexibility should be built into Project timelines.

Execution of performance-based funding requires experienced staff to mediate between the

universities implementing the agreements and the Ministry of Education. As implemented, the

performance agreements depended heavily on negotiators hired specifically to mediate between

the TEI Presidents and the Ministry of Education. Without suitable negotiators, national policy

goals would be difficult to reconcile with the reality of the institutional environments where they

were meant to be implemented. The highly qualified negotiators helped the agreements gain

traction. Generally these were former Presidents or academic Vice-Presidents of universities

themselves, familiar with and well regarded in the academic community. Their past experience

gave them credibility with both parties to the negotiations, and allowed the negotiations of

performance metrics to be productive and grounded in the reality of the situation of each

institution. Further, these negotiators were instrumental in developing specific goals for each

university and the detailed indicators used to measure progress in achieving them.

7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agencies

The Borrower noted this ICR accurately represented the primary activities completed by the

Project, the results obtained and the impact made. Specific comments were incorporated

throughout the text.

The Borrower‘s own Implementation Completion Report is attached in Annex 7.

(b) Cofinanciers Not applicable.

(c) Other partners and stakeholders Not applicable.

22

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent):

Components Appraisal Estimate

(USD millions)

Actual/Latest

Estimate (USD

millions)

Percentage of

Appraisal

Strengthening institutional

framework for tertiary education 8.00 8.00 100%

Sub-projects to enhance

coherence, efficiency, equity and

quality

83.25 83.25 100%

Unallocated 0.75 0.75 100%

Total Baseline Cost 91.25 91.25 100%

Physical Contingencies

0.00

0.00

100%

Price Contingencies

0.00

0.00 100%

Total Project Costs 91.25 91.25 100%

Front-end fee PPF 0.00 0.00 100%

Front-end fee IBRD 0.13 0.13 100%

Total Financing Required 92.13 92.13 100%

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal

Estimate

(USD

millions)

Actual/Latest

Estimate

(USD

millions)

Percentage of

Appraisal

Borrower 75.00 75.00 100%

International Bank for

Reconstruction and Development 25.13 25.13 100%

23

Annex 2. Outputs by Component

The project goal of increasing the effectiveness of public financing for tertiary education was

accomplished by way of two separate components with multiple sub-parts. Component one

focused on strengthening the institutional framework for tertiary education at the policy level

through government capacity building, while component two comprised sub-projects, either

competitively allocated or based on performance agreements, that were implemented across a

wide number of tertiary institutions and across a number of focus areas.

Component One: Strengthening the Institutional Framework for Tertiary Education

Subcomponent 1.1 - Improvement of Sector Oversight and Regulation

To increase sector oversight, funds were used to create an office within the Ministry of

Education to focus on tertiary education policy. In part, personnel hired for this office provided

support for the implementation of a 2005 law on student financial assistance. The law gave rise

to the state-guaranteed student loan program for tertiary education, which is now providing

funding for over one hundred thousand new students each year. Additionally, funds were used to

hire staff to design and implement the performance agreements. Conceptualization and program

design for the competitively allocated funds was also financed by this subcomponent.

Subcomponent 1.2 - Strengthening of Policies and Strategies for Tertiary Education

The Government of Chile commissioned a number of studies to strengthen tertiary education

strategies and policies. The goal of four studies was exceeded by three. The studies focused on

critical policy areas specified by the Ministry of Education. Their translated titles are:

Study to determine causes of dropout in the first year of college.

University leaders' Perception of the Policy Instruments of the Division of Higher

Education in the Ministry of Education.

Employers' and Graduates' Perceptions Regarding the Quality of Higher Education

Graduates.

Survey and Evaluation of Advanced Human Capital for Education, Research and

Innovation in Chile.

Reviews of National Policies for Education: Tertiary Education in Chile, co-authored by

the OECD and The World Bank, 2009.

Human Capital Formation Abroad: A Review of Chile‘s Higher Education Scholarships

Programme, co-author by OECD and the World Bank Review, June 2010.

Description and Evaluation of Educational Policies Regarding Access to Benefits of

Indigenous Students in Chile.

Subcomponent 1.3. - Consolidation of the National System for Quality Assurance

In 2006, law 20,129 was passed, creating the National Accreditation Commission, which has

coordinated accreditation of TEIs, and set quality and process standards across tertiary education.

Project funds, subsequent to the passage of the law, were used to fund infrastructure, consultant

services, and operational expenditures involved in the establishment of the commission. Funds

have been used to understand elements common to the highest level international quality

assurance programs. Further, Mece2 facilitated study of common elements in provision of

24

quality tertiary instruction across Chile. The number of accredited institutions has doubled from

the end of 2005 to 2010, rising from 41 to 87.

Subcomponent 1.4 - Establishment of a National Observatory for Tertiary Education

The law that established the national quality assurance commission also created a system for

information in higher education. Participation in the information system was made compulsory

for accredited institutions. Institutions participating in the performance agreements and the

competitively allocated FIAC were required to share indicator information with the observatory.

The system collects information at the level of enrolled students, and then aggregates it for

public access.

The information system has allowed for unprecedented transparency into tertiary education, with

information on historical enrollment, graduation rates, faculty characteristics, program durations,

as well as information regarding labor market outcomes by study area. Labor market outcomes

include current wage data, historical wage trajectory data, unemployment percentages, as well as

the underlying cost of study. Aggregated data is publicly available through a user-friendly

website.

Subcomponent 1.5 - Project Management

Project management funds facilitated all operations associated with performance agreements and

the FIAC, which are described further below. These funds expanded the Ministry of Education‘s

capacity to coordinate with representatives from tertiary institutions, allowing them to better

facilitate compliance with program requirements. Funds were used to design and execute

evaluation metrics for projects in the competitively allocated FIAC. Additionally, funds were

used to hire appropriate representatives for negotiation of the performance agreement indicators

and objectives, as well as international experts for evaluation of program success.

Component Two: Sub-Projects for Enhancing Coherence, Efficiency, Equity and Quality

Subcomponent 2.1 - Competitively Allocated Academic Innovation Fund (FIAC)

The FIAC funds were allocated to 371 distinct projects in five critical policy areas. FIAC funds

were matched at 30% of the funding level with the resources of the implementing institutions

through their own internal resources. The projects were rated across eight factors, including

integration with the university‘s strategic plan, implementation of the project, meeting project

deadlines, accomplishment of hoped-for results, achievement of project indicators, quality of

management, replicability of the project, and commitment of the institution to the project. In

general, projects scored highly, with most projects rated satisfactory. The lagging grade on the

majority of projects was the achievement of the agreed upon indicators. General descriptions of

project areas are related below, along with enumerations of example projects.

Development of Academic Personnel for Research and Management

Approximately 20% of the FIAC funds were allocated to projects to increase the number and

quality of personnel with research and/or management capabilities. A variety of projects were

undertaken, with 62 distinct projects implemented across an array of universities. Examples of

projects intended to develop research capabilities include exchange programs for faculty at

25

Pontificia Universidad Católica de Chile, as well as the recruitment of international speakers and

researchers. At the same university, to make research more effective across the region, the

agronomy and biology faculties instituted a program to make research equipment more

accessible to other nearby researchers. At Universidad Santiago de Chile, a biotech initiative was

launched at the post-graduate level. At Universidad de Concepción, resources were made

available to increase the number of faculty members with graduate degrees and to increase

English speaking skills of faculty members. Additionally, the oceanography program was

expanded, with funds used to procure an exploration boat with advanced research equipment.

The program was awarded a seven year accreditation. The Universidad de Valparaiso

strengthened its post-graduate research capacity by recruiting highly qualified individuals to the

university and implementing performance standards for them, resulting in the accreditation of its

graduate degrees. Its neuroscience PhD program is now internationally recognized after

investments were made in major scientific equipment.

Examples of projects focused on increasing management capacity included one in which the

Pontificia Universidad Católica de Chile developed a centralized clearinghouse for faculty to

design and propose FIAC projects, ensuring that the best developed ideas were promoted on

behalf of the whole university. Further, university management ensured that the proposed

projects were coordinated to increase the university‘s capabilities where they were most needed

first. At Pontificia Universidad Católica de Valparaíso, an information management system for

libraries was implemented, and the institutional analysis unit received funding to purchase

software and computers necessary to fulfill its mission.

Development of National Post-graduate Programs

Approximately 35% of the disbursed funds through the FIAC went to create or further develop

post-graduate programs, including fostering collaborative efforts between researchers across

institutions. More than eighty individual projects were undertaken to further develop post-

graduate programs. Examples include strengthening PhD programs at Pontificia Universidad

Católica de Chile through introduction of student scholarships. At the same university, a holistic

approach to postdoctoral research has cultivated a collaborative approach to research in

agronomy and biology, with coordination across both fields. At Pontificia Universidad Católica

de Valparaíso, two joint PhD programs were developed with Universidad Técnica Federico

Santa María. The two coordinated programs were able to attract professors and students, and

gain maximal leverage from investments in equipment. At Universidad Santiago de Chile, the

university partnered with Universidad de Chile to create a joint Microbiology PhD program.

Additionally, a PhD program in history was created. Universidad de Tarapacá created a PhD

program in anthropology. The Universidad de Concepción developed an ecology PhD program

with the Universidad Austral.

Improvement of Learning Outcomes

Approximately 13% of the FIAC funds were disbursed to institutions to improve the learning

outcomes of students through better infrastructure, greater focus on teacher ability, increased

remedial education resources, and closer tracking of student successes and failures. More than

seventy distinct projects were funded across a wide array of universities. For example, the

26

Universidad de Chile increased focus on learning outcomes by linking professor evaluations to

teaching ability in addition to the traditional focal area of research. At Universidad Técnica

Federico Santa María, a Center for Integrated Learning in the Sciences was created to help

students in basic skills and competencies, including bolstering introductory level science skills,

but also teaching time management, study habits, and general planning. Additionally, a system to

track student academic performance was implemented. Teacher accountability was emphasized,

and awards for teacher quality were introduced. At Universidad Católica de Temuco, a teacher

excellence project was implemented to define and publicize what makes a good teacher.

Additionally, an information tracking system for students was implemented to closely evaluate

competences at the start of their university careers. The Universidad de Talca created an alumni

career center to better understand graduates‘ labor market outcomes, increase brand recognition

for the university, and cultivate relationships with potential employers to create more internship

opportunities for its students.

Curriculum Modernization Based on Learning Outcomes and Competencies

Approximately 22% of the FIAC funds were allocated to 119 projects intended to upgrade

curriculum based on learning outcomes. For example, at Universidad de Chile, general education

courses were implemented to increase interactions within the student body for students across a

variety of study areas. Additionally, the university partnered with other local universities to

allow Philosophy students to attend classes at the other institutions. Critically, a program was

undertaken among several universities to increase the transferability of credits across institutions.

Universidad Católica de Temuco instituted an action plan for improvement of undergraduate

education. A Center for Improvement in Teaching was launched in 2008 to define competencies,

and drive curriculum redesign. A program for insertion into university life was launched to help

students successfully bridge the gap between their secondary and tertiary educations.

Improving the Offerings of Higher Technical Education

Approximately 10% of the total funds were allocated to improving the course offerings of

technical education centers. Thirty-five different projects were undertaken, but successes were

limited in this focus area. A number of operational hurdles limited participation, and no funds

were disbursed in several rounds of the allocation competition. High turnover in faculty and

procurement challenges were cited as the main drivers for limited accomplishments. However,

some notable successes included a transformation of curriculum at DUOC to focus on

competencies. Additionally, the number of e-learning offerings was increased at DUOC, and

additional programs in high employment areas were developed. At Universidad Católica del

Norte, a technical degree for ―Tourism Technician‖ was created, a field with high levels of post-

graduate employment.

Subcomponent 2.2 - Performance Agreements

Four performance agreements were established at four different universities. The goals, metrics

and outputs associated with each performance agreement varied by institution. In general, across

the institutions, the performance agreements succeeded in coalescing distinct faculty segments

around common goals as articulated and agreed upon by central management figures. Previously,

the Ministry of Education was unable to engage in policy discussions with a central figure

27

capable of directing change; at the four universities with performance agreements, that situation

has changed.

University of La Frontera

Resources of $2.3M were allocated to the performance agreement with University of La Frontera,

with an additional $1.0M in funding provided by the university. These resources were targeted

toward increasing the capability of management at the university, increasing the quality of

instruction, and increasing the scientific accomplishments of the university in the area of

bioresources and agricultural resources.

In the third year of the agreement, the university had succeeded in meeting a number of the

promised indicators to improve educational quality. A strategy was formulated and implemented

to expand basic core competencies. Remedial education courses were developed for 74% of core

subjects, but the number of students who failed to pass their courses stubbornly stuck at a high

61%. A professional development center was established at the university to track professional

certifications and competencies. An employability test for graduates fell short of the goal of 87%,

but did increase to 81%. The Division of Quality Assurance and Institutional Development was

redesigned, and computer systems and web platforms were put in place to assist in implementing

new policies

The university met a number of the promised indicators for increased scientific output. A new

scientific laboratory was constructed, and new academics with doctoral degrees were hired onto

the staff. Among Chilean universities, La Fontera was ranked seventh in the research and

technological development of bioresources.3 Seventeen patents were requested, significantly

exceeding the goal of seven. Increases in scholarly work led to 90 new publications in ISI,

exceeding the goal of 60.

The university also committed to increasing credit recovery from graduates. The university

exceeded the promised level of ~$2.4M with ~$2.6M in recoveries. To achieve this, La Frontera

developed and implemented the first system among Chilean universities to allow for electronic

payment of tuition, fees, and debts.

University of Bío-Bío

Resources of $2.3M were allocated to the performance agreement with the University of Bío-Bío,

with an additional $0.95M in funds provided by the university. These resources were used to

improve the level of professional and academic success of the students in the university.

The university was successful in reaching a number of the promised indicators to increase

retention and academic quality. The university implemented a number of new programs to

welcome matriculating students, and increase retention of those students. These included an

introduction program for new students, a survey to better characterize the core academic

competencies of incoming students, and a tutoring program for students in mathematics and

3 According to a recent Elsevier Scopus overall productivity review for 2004-2010, Universidad de la

Frontera is now ranked 9 in Chile.

28

engineering. Additionally, leveling programs to ensure basic competencies in incoming students

were developed based on survey results. First-year drop-out rates fell from 17% to 3.5%, and

third year drop-outs have fallen from 40% to 34%. Concern for meeting the metrics of the

performance agreement was cited as a major reason for the decline in drop-out rates, as faculty

focused more time and attention on student retention than they otherwise would have.

The university succeeded in reducing the real duration of its academic programs, with four year

programs declining from a real duration of 6.56 years to 6.25 years, five year programs declining

from a real duration of 7.11 years to 6.75 years, and six year programs declining from a real

duration of 8.5 years to 7.7 years. This represents significant progress toward reducing the time

and cost of a university education.

A program to manage alumni data was implemented to allow the university to maintain linkages

with graduates. The university succeeded in increasing the quality of the integration of its

graduates into the workforce, with an employer satisfaction survey increasing to 95.3% from

94.9%. The number of graduates attaining authority positions increased from 16.7% to 21%, but

failed to reach the goal of 25%. Student satisfaction with course quality actually declined slightly,

from 82.9% to 81.5%, though this decline can be explained by the inclusion of more programs

with lower initial satisfaction levels than were included in the baseline.

The university developed a plan to standardize and manage the accreditation process. The

university succeeded in expanding its accredited program offering, increasing them from 25.7%

to 65%, with the proportion of students in accredited courses increasing to 58.7%, well above the

target goal of 55%. The university created a quality assurance program that, through the

repetition and standardization of processes, has significantly decreased the timeline to earn

accreditation for a program, from one year and four months to just nine months.

University of Chile

Resources of $3.4M were allocated to the performance agreement with the University of Chile

with an additional $0.44M in funds provided by the university. These resources were used to

reorganize central agencies to streamline the management processes of the university, to increase

central management and execution capacity across campuses, and to implement an information

system to support efficient management of the university.

The university management reorganization was composed of three parts; evaluation, re-design,

and implementation. Of the thirty-six organizational units on campus, all were evaluated in the

context of increasing their operational effectiveness and increasing coordination with

management. Half of those organizational units have been re-designed, with some initial

progress in implementing the management redesign.

Limited progress was made in increasing central management and execution capacity across

campuses. A new administrator was hired for the South Campus as part of the expansion of

central management, but no additional progress was made.

29

The University succeeded in implementing a new integrated information system that allows easy

access to critical information, including classroom facilities, library archives, human resources

information, teacher evaluation results and research projects in progress at the university. On the

academic side, the information system allows for student access to medical records, course

materials, as well as online access to the university application. The information system as

implemented will increase transparency into outcomes, serving to create baseline data against

which the progress of future policy revisions can be considered.

Finally, the University recently began the implementation of a new, non-Mece2 funded, 5-year

humanities performance agreement designed with the Mece2 pilot model.

University of Tarapaca

Resources of $2.2M were allocated to the performance agreement with the University of the

Tarapaca with an additional $0.64M in funds provided by the university. These resources were

used to bolster the management capabilities of the university, to position the university as an

agent of academic integration with Peru and Bolivia, and to promote research in areas closely

linked to desert life: the indigenous Chinchorro culture, the properties of endogenous desert corn,

and the problems of arsenic and borum in the land and water.

To bolster the management capabilities of the university, a strategic and operational control

system was designed and configured by the third year of the performance agreement. The system

allows the university to better evaluate both the academic and general management within the

university through collection and analysis of data. The university community has previewed the

system, which is currently being implemented.

The university has successfully coordinated with institutions in Peru and Bolivia to increase

integration with them, and has used that integration process to bolster academic quality within

the university. The university‘s brand is now recognized by 65% of the twenty-six targeted

institutions in Peru and Bolivia, exceeding the goal of 30% acceptance. The indexed publications

from the university have increased by 43%, falling just short of the goal of a 44% increase, and

the count of faculty with doctorates has increased by 9%, just short of the target of 11%. Finally,

student mobility (exchanges and internships) between universities in Peru, Bolivia, and Chile has

increased significantly, with (approximately 9 times the baseline number of transfer credits).

30

Annex 3. Economic and Financial Analysis

The value of the Project has been assessed through two distinct NPV calculations, each focusing

on an outcome associated with Mece2. The approach is conceptually similar, if not the same, as

that originally used in the Project Appraisal Document (PAD). Estimates indicate Mece2 has an

NPV USD (2005) of $677 million. The funds associated with the Project had an NPV USD

(2005) of $80 million, suggesting that conservatively, the total leverage of Mece2 was

approximately 8.5. This conclusion is summarized below and explained in turn:

Mece2 has led to shorter-duration academic programs and quicker time-to-degree,

resulting in time savings and increased career earnings for graduates of tertiary education.

This benefit is estimated at NPV USD (2005) of $335 million.

Mece2 has increased tertiary education quality, leading to an increased premium for wage

earners with tertiary education. Thirty percent of this benefit was assumed to be directly

attributable to the Project and its value is estimated at NPV USD (2005) $342 million.

As per the PAD, Mece2 is also expected to result in additional value by increasing

graduation rates; this effect (and its value) cannot yet be observed. Because Mece2

targeted its efforts to increase retention rates on first year tertiary education students,

beneficiaries have not yet graduated.

Increased Earnings from Shortened Education Duration

A significant focus area of the project was shortening the duration of tertiary education in Chile.

Historically, university study required an investment of approximately seven years. By reducing

time-to-completion, students spend less money and less time in acquiring their education.

Earnings increase because the avoided study time can be productively allocated to work. The

NPV of the project was calculated in part by examining the value of these incremental earnings.

Data collected through the SIES (funded by subcomponent 1.4) was used to complete this

analysis. Specifically, an analysis of time-to-degree across a fixed set of study-areas was

conducted for 2007 and 2009.4 The result was a shortening of 0.4 semesters (2.5 months) in the

weighted average of program durations, which means increased earnings for 2008-2017

graduates of US$692 million. 5 Though general funding for tertiary education increased in Chile

over the Mece2 project, no funds targeted this particular goal, suggesting that the most likely

explanation for the majority of these increased earnings is the impact of the Project. Figure 2

illustrates the shortening of degrees between 2007 and 2009.

4 Study areas account for most high enrollment degree programs, including: Architecture, Accounting,

Law, Emergency Medicine, Commercial Engineering, Construction Engineering, Journalist, Psychologist,

and Social Worker, as well as 43 others. 5 The number of 2009 graduates by program was used to create a weighted average duration of a

university education. If 2007 graduation numbers had been used in addition to those from 2009, student

program selection bias across years may have altered the outcome.

31

Figure 2. Weighted Average of Education Duration for a Sample Set of Study Areas

Figure 3. Program Length in Semesters, 2007 vs. 2009

As can be seen from figure 3, the reduction in program length was not evenly distributed among

all programs. The line follows y=x; points above the line represent programs that have

lengthened marginally (11), and points below the line indicate programs that have shortened (41).

Because the change in program duration was not consistent, a weighted average of the reduction

in program duration by number of 2009 graduates in each program was used. This was done in

order for the weighted average to reflect the true shortening of the programs as experience by

graduates in 2009.

The reduction in study length between 2007 and 2009 allows students to work and earn for an

incremental 2.5 months on average. An average monthly salary and an employment rate were

13

13.1

13.2

13.3

13.4

13.5

13.6

13.7

2007 2009

Sem

este

rs

Year of Graduation

y=x

Electrical

Engineer

Acting

Law

32

calculated for 2009 university graduates, again from the SIES data. The salary for the 2.5

incremental months of work was crossed with actual (as available) and projected university

graduates from 2008 until 2017, and discounted to 2005 values, as was done for the NPV

calculation in the PAD. Though benefits would continue to accrue to graduates after 2017,

uncertainty around projections would increase significantly. Restricting the outlook to a single

decade span means the estimate of program NPV is more conservative than it might otherwise be.

A number of assumptions were made in calculating the NPV associated with this aspect of the

project:

The size of each year‘s graduate cohort is based on the average growth rate in CRUCH

graduates from 2002 to 2009, which was 5.2% (slightly higher than the projected rate of

4.6% used in the PAD).

The employment rate was assumed to be 81.8 percent, which was the weighted average

employment of graduates from all university programs in 2009, and was marginally

lower than the assumption from the PAD.

The weighted average salary earned by 2009 graduates was based on SIES data, and was

determined to be Ch$622,694.

First year wages were assumed to grow at an annual rate of 2.5%, (the per capita growth

of the economy), as assumed by the PAD.

The discount rate is 6%, as in the PAD, and the exchange rate is taken as Ch$ 550,

marginally lower than that of the PAD.

Table 2 shows these assumptions result in NPV US $2005 of $669 million in increased earnings

for 2008-2017 graduates. Not more than half of this increase, or $335 million, is likely

attributable to the project.

Table 2. Earnings from Decreased Education Duration

Year CRUCH

Graduates

NPV

(USD 2005 million)

2008 32,024 61.4

2009 33,966 63.0

2010* 35,732 64.1

2011* 37,590 65.2

2012* 39,545 66.3

2013* 41,601 67.5

2014* 43,765 68.6

2015* 46,040 69.8

2016* 48,434 71.0

2017* 50,953 72.3

Total 669.2

Percentage attributable to Mece2 50%

Amount attributable to Mece2 334.6 Note: considers graduates from Consejo de Rectores universities, assumes new graduates growing at 5.2% (average

2002 to 2009), initial mean wage of M$1,132, employment rate of 81.8%, US$ 1.0 = Ch$550. * = projected

33

The NPV of this calculation compares favorably with the one estimated in the PAD; the realized

NPV of 669.2 (millions $2005) is well in line with the PAD estimate of 615 (millions $2005) for

a reduction in degree duration of 2.5 months. Exchange rate effects slightly raised the realized

NPV, but not by more than 10%. The disbursed funds of Mece2 had an NPV of 80 (millions

$2005); if half of the shortening of program length is attributable to the project, leverage of 4.2X

from the funds has been realized. In general, tertiary education funding has increased

significantly since program implementation, as a new student loan program has resulted in

significant increases in enrollment as well as increases in funds available to universities to

bolster quality. However, given the focused nature of Mece2 and the more diffuse nature of those

other funds, attributing half of the present value of the improvement to Mece2 appears to be

conservative.

The granularity of the data was complete only to the university level. Analyzing just the subset

of CRUCH universities instead of all the universities (Cruch & private) would have been

preferable, however, as most of the funds associated with Mece2 accrued to CRUCH universities.

This may make the NPV more conservative, as CRUCH degree durations would be expected to

have decreased more than the university-wide average.

Increased earnings from Higher Tertiary Wage Premium

A significant focus area of Mece2 was increasing the quality of tertiary education. Improved

quality is expected to increase the relevance of tertiary education for labor market outcomes. As

the quality of tertiary education increases, the wage premium for having completed tertiary

education is expected to rise, with employers willing to pay increasing premiums for graduates

with well developed and highly relevant skills. The NPV of the project was calculated in part by

examining the value of these incremental earnings.

Data collected through the SIES was used to complete this analysis. Since Mece2‘s funds went

almost entirely to universities (>90%), the wage premium driven by this program was calculated

by comparing earnings of university graduates to earnings of graduates from technical training

centers and professional schools. Specifically, the analysis used information from students who

had graduated between 2005 and 2008 and compared the growth in salaries earned in the first

year after graduation. The compensation data indicated that technical and professional school

graduates saw their incomes appreciate by 4% from 2005 to 2008. University graduates enjoyed

an increase in compensation of 5.5%, or an incremental salary growth of 1.5 percentage points

by 2009.

The value of the incremental 1.5 percentage points in wage premium from 2005 translates into

1.4% of the nominal 2009 salary. A number of assumptions were made to assess the impact of

this premium on the earnings of university graduates over the decade from 2008 to 2017.

Because the graduates are expected to benefit from the wage premium over their entire career

[30 years], the assumptions calculate the present value of the wage premium in the year of

graduation; future year premiums are discounted. The rest of the assumptions are detailed below.

The same assumptions used for calculating increased wages from shortened study-spans are used

here, as well as a few supplementary assumptions:

34

In addition to the 2.5% annual growth rate applied to first year wages, through the 30

year employment span, a 1.5% annual lifecycle earnings increase was projected.

As in the PAD, a discount rate of 6% is applied. This rate is used to normalize present

value earnings from 2008-2017 back to USD 2005 and to calculate the 30-year present

value of the wage increase for each graduation cohort.

Table 3 shows these assumptions result in NPV US $2005 of $1,026 million in increased

earnings for 2008-2017 graduates. Not more than a third of this increase, or $342 million, is

likely attributable to the project.

Table 3. Increased Wage Premiums Associated with Tertiary Education Quality

Year CRUCH

Graduates

NPV

(USD 2005 million)

2008 32,024 94.2

2009 33,966 96.6

2010* 35,732 98.2

2011* 37,590 100.0

2012* 39,545 101.7

2013* 41,601 103.4

2014* 43,765 105.2

2015* 46,040 107.0

2016* 48,434 108.9

2017* 50,953 110.7

Total 1,025.9

Percentage attributable to Mece2 33%

Amount attributable to Mece2 342

Note: considers graduates from Consejo de Rectores universities, assumes new graduates growing at 5.2% (average

2002 to 2009), initial mean wage of M$1,132, employment rate of 81.8%, US$ 1.0 = Ch$550, assumes 2.5%

continued initial salary growth, with incremental 1.5% lifecycle growth. * = projected

This estimate of $1,025 million 2005 USD compares favorably with the PAD estimate of an

NPV of ~$828 million 2005 USD (extrapolated for 1.4% growth from 1.5% and 0.5% growth

calculations). Some input assumptions changed slightly; undergraduate enrollment and

graduation have increased more rapidly than projected, and salary and exchange rates have

varied. Each of these factors appears to account for half of the difference between the initial

project and the current calculation.

As was the case with the value of the shortened education duration, other factors are likely to

have contributed to the realization of the project goals in increasing wage premiums and

educational quality. Endogenous factors include the student loan program cited above, which has

dramatically increased university enrollment and funds available to universities. Additionally,

exogenous factors, including Chile‘s shift to a knowledge economy, are also expected to increase

the premium associated with tertiary education. For these reasons, attributing a third of the

benefit to Mece2 appears to be a conservative approach to assessing the total value of the project.

35

Increased Earnings from Higher Graduation Rates

A significant focus area of Mece2 was increasing resources available for remedial tertiary

education and other programs intended to decrease tertiary drop out. Historical graduation rates

in Chile have varied between fifty and seventy-five percent, though reliable data in this area is

limited. Graduates of tertiary education enjoy a significant wage premium over those with only

partial completion of tertiary education. Increasing the graduation rate would thus have the effect

of moving more wage-earners into a higher income group. These incremental wages would

translate into a significant source of NPV for the project.

The long duration of Chilean tertiary education is a limiting factor in calculating the realized

value of Mece2. With programs commonly lasting seven or more years, students who

matriculated in the year Mece2 began will not graduate until 2013 at the earliest. If the effects of

Mece2 were delayed to any extent by the disbursement schedule, changes to graduation rates

may well not be observed until 2014-2015. As more information accrues, it will be possible to

further assess the contribution of a lowered attrition rate to the NPV of the Mece2 project, but at

present, any increase in graduation resulting from the project cannot yet be observed. Per the

calculations in the PAD, as the benefits of a higher salary are enjoyed by a wage earner over

their entire lifetime, and the premiums for complete vs. incomplete tertiary education are

significant, even very marginal changes to graduation rates would lead to realized NPV of

several multiples of the Mece2 investment.

36

Annex 4. Bank Lending and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/

Specialty

Lending

Antonio Leonardo Blasco Sr Financial Management

Specialist LCSFM Fin. Management

Andreas Blom Sr Education Econ. SASED Education Econ.

Max Brennan Consultant LCSHE Consultant

Maria E. Castro-Munoz Sr Social Scientist LCSSO Social Scientist

Maria Lucy Giraldo Senior Procurement Specialist LCSPT Senior Procurement

Specialist

Lauritz B. Holm-Nielsen Consultant LCSHE Consultant

Jette Samuel Jeppesen Consultant LCSHE Consultant

Marta Elena Molares-Halberg Lead Counsel LEGES Lead Counsel

Kristian Thorn Consultant ECSHD Consultant

Morag N. Van Praag Senior Finance Officer CTRDM Finance

Martha P. Vargas Program Assistant LCSDE Assistant

Supervision/ICR

Diego Ambasz Operations Officer LCSHE Operations

Antonio Leonardo Blasco Sr Financial Management

Specialist LCSFM

Financial

Management

Alejandro Caballero Sr Education Spec. LCSHE Education Spec.

Maria E. Colchao Senior Program Assistant LCSHE Assistant

Maria Lucy Giraldo Senior Procurement Specialist LCSPT Procurement

Ana Maria Grofsmacht Procurement Analyst LCSPT Procurement

Jesko S. Hentschel Sector Manager ECSH4 Sector Manager

Daniela Marotta Country Economist MNSPR Country Economist

Omar Arias Sector Leader LCSHD Sector Leader

Chingboon Lee Sector Manager LCSHE Sector Manager

Janet Entwistle Senior Operation Officer LCSHE Operations

Livia Benavides Lead Operations Officer LCC6C Operations

Luz Maria Meyer Junior Professional Associate LCSFM Junior Professional

Associate

Michael Crawford Sr. Education Specialist LCSHE Team Leader

Maria Paulina Mogollon E T Consultant LCSHE Consultant

Marta Elena Molares- Halberg Lead Counsel LEGES Lead Counsel

Luis M. Schwarz Senior Finance Officer CTRFC Finance

Alejandro Roger Solanot Sr Financial Management LCSFM Financial

37

Specialist Management

Kristian Thorn Consultant ECSHD Consultant

Morag N. Van Praag Senior Finance Officer CTRDM Finance

Martha P. Vargas Program Assistant LCSDE Assistant

Lourdes Linares Sr. Financial Management

Specialist LCSFM

Financial

Management

Mariana Montiel Sr. Counsel LEGLA Sr. Counsel

Cidalia Brocca Finance Analyst CTRDM Disbursement

Patricia de la Fuente Hoyes Sr. Finance Officer CTRFC Disbursements

Christopher Sharp Consultant LCSHE Consultant

Maria Elena Paz Gutzalenko Program Assistant LCSHE Assistant

(b) Staff Time and Cost

Stage of Project Cycle

Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including

travel and consultant costs)

Lending

FY05 25 145

FY06 4 30

Total: 29 175

Supervision/ICR

FY06 16 50

FY07 15 90

FY08 9 58

FY09 33 176

FY10 44 169

FY11 39 149

Total: 156 692

38

Annex 5. Beneficiary Survey Results

(Non applicable)

39

Annex 6. Stakeholder Workshop Report and Results

(Non applicable)

40

Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR

IMPLEMENTATION COMPLETION AND RESULTS REPORT

LOAN (IBRD-73170) TO THE REPUBLIC OF CHILE: TERTIARY EDUCATION

FINANCE FOR RESULTS PROJECT- FIRST PHASE APL - MECESUP2 PROGRAM

1. Project Development Objective and Implementation

1.1 Project Development Objectives (PDO)

The program, known locally as Mece2 (Higher Education Improvement) Program, fulfilled its

original objective of supporting Chile's development into a knowledge-based economy by

strengthening its tertiary education system. To this end, the funding system for tertiary education

was revamped in order to: i) enhance the quality and relevance of tertiary education; ii) ensure

access to all talented individuals; iii) improve internal efficiency and strengthen accountability

for results and iv) support national innovation system by increasing the availability of advanced

human capital. Beyond the program, the goal was accomplished regarding the support of the

Chilean Government in completing the transition from the current incremental funding system

for quality tertiary education to a results and performance-based system applied to an agreed

proportion of base funding for universities.

1.2 Project Implementation

In general terms, Project implementation was highly satisfactory with no great setbacks. It

sustained the continuity of the main objectives set in the Mece2 Program (Loan 4404-CH) with

essentially the same organizational structure, projected its intentions more intensely towards

results-based management and public accountability and introduced Performance Agreements on

a trial basis.

Implementation was conducted with resources from Loan 7317-CH, as well as resources from

the Chilean Government.

At the Tertiary Education Institution (TEI) level, Institutional Coordination Units (ICU) were

used. Each ICU had an upstream role of helping TEIs prepare Performance Agreements and

sub-project proposals and a downstream role focused on implementation, fiduciary management

and M&E of sub-project and Performance Agreements activities.

In 2010 the new Government and the Administration of the Division of Higher Education

(Divesup) agreed to institutionalize the Mece2 Program in a Department of Institutional Funding

which manages all resources allocated to tertiary education (direct public support (AFD), indirect

public support (AFI), traditional funding for institutional development, Mece2 and other

ministerial funding), excluding student financial assistance. Thus, after 15 years of successful

operation, the Program has acquired the necessary organizational, administrative and financial

stability to sustain the quality enhancement of Chilean tertiary education.

The Project incorporated modifications to funding instruments: the Competitive Fund for

Projects pertaining to the Mece2 Program turned into an Academic Innovation Fund (FIAC),

now focused on strategic axes that mediate structural weaknesses in the tertiary education system,

41

highlighting results-based management and seeking the highest degree of replication and impact.

To this end, 5 competitions were held in the 2006-2008 period and USD 115 million was granted

to 371 TEI sub-projects. This award includes Technical Training Centres and both public and

private universities of Council of Rectors of Chilean Universities (CRUCH), as well as

accredited autonomous universities (in the area of pedagogical or Ph.D. programs). Additionally,

Performance Agreements were incorporated as a new instrument on a trial basis, which enabled

the successful exploration of negotiated funding means between the Ministry of Education and

the Universities to achieve results, performance or notable outcomes on a national and

institutional level.

With regard to quality assurance, the implementation of voluntary accreditation processes (based

on self-assessment and external verification) in the Chilean university system, fostered during

the Mece2 Program, eventually brought about Law No. 20.129 in 2006. This Law currently

directs the processes for institutions and undergraduate and postgraduate degree programs via the

National Commission for Accreditation (CNA).

Additionally, the new web portal www.mece2.com was implemented, incorporating modern

online database technologies. The aim is to ensure adequate public diffusion and transparency of

the information and results generated in the Project.

2. Performance Assessment

The Project accomplished its development objectives and successfully demonstrates an improved

means of supporting tertiary education in Chile. The foregoing may be corroborated by the

compliance with triggers which measured the progress towards the PDO: i) employers‘

perception of graduate quality according to a survey; ii) number of graduates from Technical

Training Centres and Professional Institutions transferred to universities; iii) socioeconomic

profile of first year tertiary education students; iv) first year drop-out rate at institutions; and v)

percentage of competitively awarded or performance-based tertiary education public funding.

2.1 Studies

Studies pertinent to decision-making and impact on the Project and on TEIs were conducted,

including the following: Determination of Causes of Student Drop-out Rate in First Year of

University (2008); University Leaders‘ Perception of the Division of Higher Education‘s Policy

Instruments (2008); Employer and Graduate Perception of Tertiary Education Graduate Quality

(2008); Review of Chilean Tertiary Education, commissioned to OECD and the World Bank

(2009); State of Development of Advanced Human Capital for Education, Research and

Innovation in Chile, commissioned to National Academy of Sciences USA (2008).

2.2. Outcomes and Impact.

The Project enabled the foundations to be laid for:

The development and strengthening of an incremental funding system based on

Performance Agreements;

The initial implementation on a national scale of a new curricular design for tertiary

education based on learning outcomes and demonstrable competencies in tune with

international standards;

http://www.mece2.com/

42

The definition and initial implementation of an academic credit transfer system (SCT-

Chile, compatible with the European credit transfer system) which would allow national

and international student mobility and progress towards the articulation of the different

tertiary education levels;

The consolidation of improving tertiary education quality and its continuous innovation,

in accordance with TEI strategic planning and CNA regulations (CNA, Law no. 20.129).

All participating Institutions in the Mece2 Program strengthened their capacities as a result of the

actions implemented by the Project, particularly with the experimental Performance Agreements

described below.

2.2.1 World Bank-Ministry of Education.

Project implementation enabled the identification of weaknesses and shortcomings in both the

Ministry of Education and the World Bank, which in turn allowed the study of options to make

internal changes so as to guarantee greater success in FIAC and Performance Agreements sub-

projects. For instance, studies are being conducted regarding ways to accelerate administrative

management approval cycles, facilitate reprogramming, have support systems available (for

example, specialized area-specific databases), orientate towards outcomes (and not only towards

activities and processes) and expedite procurement and hiring procedures, which is being

planned for inclusion in the new loan to be negotiated in 2011 with the World Bank.

2.2.2 Ministry of Education-DIVESUP-MECE2.

Divesup, Ministry of Education and pertinent agencies‘ capacities have been strengthened to

develop policies, strategies and management skills which enable the accomplishment of agreed

PDO, namely:

Creation of a Research and Performance Agreement Unit which has provided Divesup

with support for Performance Agreement design, negotiation and monitoring, as well as

the preparation of terms of reference, tender and monitoring of studies , statistics

preparation, triggers and analysis of results and impacts generated in sub-projects from

FIAC, Performance Agreements and the Project.

Generation of information for decision-making in both Divesup and TEIs on the basis of

studies conducted and/or contracted by the Project to support specific areas pertaining to

tertiary education and to move towards a more ambitious organization, modernization

and allocation of resources.

Support of the quality assurance institutionality and accreditation established in Law No.

20.129, 2006 which currently directs institutional processes for undergraduate and

postgraduate degree programmes through National Commission for Accreditation.

Implementation of a Tertiary Education Observatory since August 2007 (today known as

the National Information Service for Tertiary Education (SIES)).

2.2.3 Tertiary Education Institutions (TEIs).

The Mece2 Program has had a huge impact on institutions. For instance:

43

Generation of capacities and enhancement of management of recipient TEIs. As a

prerequisite of applying to the Mece2 Program Competitive Fund, strategic institutional

plans and participation in program and institutional accreditation were required, as

developed in the Mece2 Program through FIAC and Performance Agreements, so as to

consider resources for the design and implementation of 18 Institutional Research Units

which support institutional decision-making based on evidence and 22 Teaching-

Learning Units to achieve teacher training and strengthen student learning and their

demonstrable competencies. These institutional research capacities did not exist before

the start of the Mece2 Program, implying new opportunities for duly informed decision-

making and policy making.

These new capacities represent potent leverage management mechanisms in the

university system. Their introduction has meant, to a larger or lesser extent, substantial

and fundamental changes in attitude and practices in many TEIs.

The existence of legal accreditation requests and mechanisms of tertiary education in

Chile has enabled the installation and formalisation of quality assurance at an institutional

and program level, which in turn increases the amount of information available to the

public regarding educational alternatives and a satisfactory international presence.

The generation of institutional auto-evaluation skills, including analysis, strategic

diagnosis and accreditation results.

Strengthening of academic innovation in recognising academic credits (SCT-Chile), of

national and international student mobility and consolidation, competency-based

curricular innovation for lifelong learning, support of academic management and

strengthening of technical and vocational training.

In universities with experimental Performance Agreements, management has been

improved since more information is now available. Thanks to Performance Agreements,

there is currently a greater capacity for analysis and evaluation of information which

allows information to be compared to international experiences, as well as to decision-

making regarding their implementation and monitoring.

It has been shown that Performance Agreements open up a space to strengthen leadership

within institutions, which is essential for their implementation. They thus fall into a

virtuous circle in which they act collectively and not as a group of independent units.

The application of Performance Agreements ―at the highest level of the organisation‖ in

TEIs, without the need for the creation of ad-hoc intermediary structures, has allowed

internal bureaucratic barriers to come down and instil a results-based management style

at management level, not as an inconsistent fact, but as an intrinsic part of proactive

management.

Performance Agreement implementation, complementing the FIAC competitive fund, is

allowing the Ministry of Education to discover resource allocation alternatives which

include intense components of results-based management and public accountability. This

44

learning shall be decisive when considering a future redesign of funding mechanisms of

Chilean tertiary education.

Performance Agreements represent an alternative for base funding, streamlining public

interest with university autonomy.

Finally, FIAC and Performance Agreement sub-projects have strengthened TEIs

undergraduate, graduate and management areas.

3. Project Management Assessment

3.1 General Coordination

A full-time, seasoned General Coordinator with knowledge pertinent to large-scale and high-

impact project implementation has been in charge under the direct leadership of the Head of

Divesup. The Head of Divesup, the General Coordinator and the Director of FIAC composed

the Project Coordination Committee. It is important to highlight that Dr. Ricardo Reich has

permanently been in charge of the Project has greatly supported its continued vision and

implementation. Finally, the institutionalization in 2010 of the Mece2 Program must be

highlighted. It became part of the Department of Institutional Funding of Divesup, also directed

by Dr. Reich.

The General Coordination, with the collaboration of the Program Administration Unit and the

Research and Performance Agreement Unit, has been able to keep abreast of the global

management of the Project and the World Bank fiduciary procedures, monitor implementation

and offer technical assistance to beneficiaries. The Project staff is renowned for their

consistency over time with no significant changes for over a decade (since its creation in 1999),

in both the General Coordination and the management of its subdivisions and work teams, and in

its procurement units and financial management units, maintaining ongoing close coordination of

the Project work and integration of the information. Today they all compose the Department of

Institutional Funding, dependent on Divesup.

3.2 Internal controls

Throughout the implementation of the Project, there were no changes in the internal control

policies of issues regarding sub-project and Performance Agreement monitoring, funding,

accounting or procurements, with only the necessary adjustments to adapt to World Bank

requirements, Controller General's Office and internal and Government administration. As for

Financial Management, they complied in both form and substance, and by the deadlines set,

which was acceptable to the World Bank on every occasion. Likewise, the Controller‘s audits

and World Bank missions did not report fiduciary breaches in the Loan.

3.3. Innovations in Monitoring and Evaluation (M&E)

As a means of strengthening M&E, certain innovations were highlighted: i) Creation and

maintenance of an online monitoring platform; ii) Evaluation and qualification of sub-projects;

and iii) Sub-project and Performance Agreement Mid-Term Evaluations have been essential for

determining implementation progress, the feasibility of completing initiatives within the agreed

terms and the implementation of application modifications.

45

3.4 Mid-Term Evaluations and Implementation Evaluation

Mid-Term Evaluations, project implementation and funding instrument (FIAC and Performance

Agreements) evaluation reports issued in collaboration with the World Bank, the International

Advisory Committee and external consultants, were always positive with greatly appreciated

recommendations and suggestions to enhance application.

3.5 Risky Projects and Adopted Measures

During Project application, no sub-projects or Performance Agreements suffered important risks;

only a few temporary resource transfer suspensions occurred as TEIs took necessary measures to

correct delays in progress and to face some management problems, which were all fully resolved

in each case. These measures were generally a result of strategic implementation after the Mid-

Term Evaluation.

3.6 Use of Loan

The loan from the World Bank and the contribution from the Chilean Government were fully

disbursed by the completion date (31/12/2010), when the proposed objectives and goals had been

met. Disbursements were made with no setbacks, given that there were no expenditure eligibility

problems in the Statement of Expenditure.

3.7 Audits

The Project fulfilled all audit requirements (intermediate or semesterly and final or annual), in

form, contents, substance and term which was acceptable to the World Bank in each case. It is

important to mention that the information required and figures calculated always coincided with

World Bank records and Controller audit observations were fully resolved.

3.8 Workshops and Seminars

A series of communicative activities and themed workshops were held in which experts, TEIs

and Ministry of Education authorities, and sub-project and Performance Agreement directors

took part so as to tackle relevant sector-based topics, achieve sub-project synergy in similar areas

and diffuse accomplishments, lessons learnt and best implementation practices; the aim was to

encourage their replicability and use in future debates on tertiary education policy. In total, 14

were held on Performance Agreements and 54 on FIAC.

3.9 Compliance with Safeguard Regulations

The Project incorporated a World Bank safeguard policy related to increasing tertiary education

quality in underrepresented regions, seeking ways of improving access to university education

for underprepared school leavers and non-university tertiary education students. To this end, a

Development Plan for Indigenous Peoples had to be drawn up with three main causes of action:

i) the design of analysis mechanisms to identify and promote indigenous students participation in

tertiary education; ii) incentives so that universities from chosen areas offer remedial learning

opportunities to indigenous students; and iii) studies on ways to reinforce grants for talented

individuals of indigenous origin.

With that in mind, the Project conducted the study ―Description and Evaluation of the

Educational Conditions and Access to Benefits of Indigenous Students in Chile‖, whose

objective was to make a statistical description of the socioeconomic and educational conditions,

46

and the access to benefits of secondary and tertiary education indigenous students in Chile. The

study is theoretically relevant not because it is recent, but also because indigenous people‘s

education conditions were vague to analysts: no clear and accurate data was available on the

subject. The census-like nature of the research, in addition to the comparatively good results of

the benefits policies in Chile, facilitates the improvement of the validity of the results and

contributes to the international debate on the effects of grants and benefits in the facilitation of

access to tertiary education. In practical terms, this study represents a substantial contribution to

public policy since it presents analytical elements regarding the impact of State-provided benefits

to compensate for the lack of access to tertiary education, which will enable them to be adapted,

thus increasing their efficiency and equity.

3.10 Lessons learnt in Performance Agreements

Performance Agreements are a valuable instrument of institutional change, which with sufficient

focus, greater participation from the Ministry of Education, sufficient adaptability and

simplification, may effectively contribute to bringing about a substantial improvement in tertiary

education in Chile. To this end, simplification of their focus and process towards a national

perspective is recommended.

3.10.1 The negotiators.

The negotiators in both the Performance Agreement design and implementation phase have

proven to be a key factor in the successful development of the new instrument. The Negotiation

Panel is also important. As part of the Panel, the General Coordination of Mece2 Program, in

addition to the Research Units and Performance Agreements, has played a fundamental role in

coordinating the design, monitoring and control efforts.

3.10.2 The negotiation.

For an effective negotiation, procedures must be systematically applied. The most important

appear to be: i) attaining effectiveness to establish agreements on ―notable outcomes‖ to be

reached with the Performance Agreement, ii) acquiring mutual Negotiator-TEI trust during the

negotiation, iii) having a monitoring system which enables enforcement of commitments and

programs, iv) integrating and organizing experiences of negotiators and Project technical team

for the different Performance Agreements. Likewise, Performance Agreement implementation

has enabled the conclusion that in a possible future round, strategies to simplify negotiation

processes and the Performance Agreements themselves must be sought, what and how any type

of support can be linked to which results and performance, the amount of institutional support

the government is prepared to provide must be defined and how much is needed to incentivize

TEIs to achieve mutually desired changes.

3.10.3 International Advisory Committee.

The universities with Performance Agreements have stated that the International Advisory

Committee‘s transparent visits, questions, observations, suggestions and reports have been useful

and have supported their leadership. According to participating universities, Committee member

peers were important counterparts who shared additional knowledge and experience.

47

3.10.4 Management Skills.

A successful Performance Agreements implies the parties involved committing to a change in

administrative structures. The limitations which compose and subsist in the system must be

resolved: University – Ministry of Education – World Bank.

3.10.5 Procurement procedures.

World Bank procurement procedures are not entirely appropriate for Performance Agreement

implementation given that they unnecessarily slow down the acquisition of goods and services.

Time is key to completion in a Performance Agreement. With regard to Public Market (the

national purchasing system of the Republic of Chile), adjustments must also be introduced to

adapt them appropriately to a Performance Agreement.

3.10.6 Information and communication.

The implementation of actions so that all Chilean universities have access to information of

Performance Agreement outcomes and limitations, in addition to their replicability in other

institutional contexts to support university management, data collection and analysis, research or

curricular redesign.

In a possible future round of Performance Agreements their current heterogeneous nature ought

to be maintained, although if possible, the operating mechanism should generally be simpler and

the Ministry of Education‘s focus via the Performance Agreement should be: i) university

management and management training; ii) fostering of research; iii) Supporting student

exchanges and mobility; and iv) long-term sustainability of these activities.

All these recommendations and learning experiences have been considered in the design of their

escalation and mentioned in the new Performance-based Agreement Program 2011-2014.

3.11 Lessons learnt in FIAC

The implementation of FIAC over more than a decade has resulted in several learning

experiences regarding the procurement of incremental resources by competitive mechanisms.

Below are some of the most important.

3.11.1 Definition of priorities

All FIAC competitions have been held with a clear definition of set by the Ministry of Education

and the Project. Thus, attention has been on structural problems in the Chilean tertiary education

system. The clarity of objectives, in addition to significant incentives, has provoked a positive

reaction from TEIs to administer change.

3.11.2 Stability and flexibility.

FIAC is a resource allocation instrument for academic innovation and quality improvement in

student learning, providing stability and flexibility. The fact that the main axes of intervention

have remained in place for several years has enabled a greater number of TEIs to tackle these

priorities more widely and deeply. On the other hand, flexibility has been decisive for

incorporating innovations and making adjustments to priorities in accordance with a reality

changing with progress, the experiences acquired and new challenges.

48

3.11.3 Selectivity.

Given that it is a competitive fund, the selection of the best sub-projects has been reflected in

greater efficiency in allocating resources (only the best sub-projects are supported), which also

implies selectivity in TEIs. They know that their sub-projects shall be selected based on their

quality; consequently, many have started to be concerned with selecting proposals that are worth

developing, writing and submitting to each Competition. Besides, since FIAC defines a

maximum threshold of resource requests to be presented (corresponding to 18% of the total

amount of resources to be allocated), TEIs of a certain size which have great capacities to

prepare sub-projects, are forced to prioritize and select proposals internally and strategically

before presenting them to FIAC.

3.11.4 Evaluation of Impact.

The process to evaluate impact through specific sub-projects has begun in order to measure

impacts as a means of feedback and ongoing development of improvement systems.

3.11.5 Seal of rigorousness and transparency.

The rigorousness and transparency with which the competitions have been held have contributed

greatly to the credibility and seriousness of FIAC in the eyes of participants.

3.11.6 Implementation.

In the sub-project implementation process, a seal has been imprinted on the style of work. In

that way, all TEIs are aware of the rules and procedures employed for the submission of

proposals, their evaluation, selection and subsequent implementation. The various stages during

sub-project monitoring ensure that the rules are followed and known by all, and there is an

interest in achieving good results.

3.11.7 Principles of the Competitive Fund.

The total amount of lessons learnt shows that the success of an operation of this type lays in

rigorous compliance of the principles of a competitive fund; for instance, equal opportunities to

access funding resources, a transparent convocation process, evaluation and selection of

proposals, and the involvement on all levels of renowned professionals with impeccable

performance.

3.11.8 Equity and Quality.

Equity has been handled in different areas. Firstly, the 18% threshold enables smaller

institutions to allocate sub-projects. With regard to allocated areas, even though basic sciences

and engineering have stood out in their capacity of sub-project procurement, the support received

by the areas of education, arts, and humanities has been significant. Furthermore, it should be

noted that given the sub-projects implemented from Arica to Magallanes, there has been an

enhancement in the quality of training for students all over the country. On the other hand, the

concern, reviews, feedback and ongoing signs from FIAC and the project towards TEIs, with

regard to the quality of activities conducted in the sub-projects, have contributed to institutional

improvement about what results-based management and public accountability are.

Using thresholds for FIAC resource allocation has proven to be a powerful focalization

technique to specify a series of important public educational and social objectives related to

49

equity. The use of competitive funding instruments must carefully consider the public policies in

force in their design, in addition to the potential effect of individual threshold conditions (such as

relative development in regions, academic, scientific and technological production inventories)

on their expected results.

3.11.9 Administrative aspects.

Although it is considered that recipient TEIs do have professionals and a basic organisation that

supports sub-project directors in administrative aspects (through ICUs), sub-project

administrative responsibility is a challenge. The prerequisites and reviews of purchasing

processes by the World Bank, of local instances (Public Market) and within the same institutions,

in addition to a growing Controller audit, imply that great dedication is required in administrative

matters when conducting a sub-project. There are many procedures, accounting instances and

reports which must be prepared during the life of a sub-project. Most are necessary and

important. However, it should be possible to simplify processes and notify of requirements and

regulations beforehand so as to avoid an overemphasis on procedures and reports, and above all,

to focus on enhancing ICU institutional capacities. Good sub-project implementation and use of

allocated resources depend on their capacity to coordinate efforts and support any necessary

administrative processes. FIAC has noticed that when this institutional role is complied with,

better sub-project outcomes are achieved.

3.11.10 Networked Projects

TEI work has brought important benefits and challenges. Many networked sub-projects stand

out in both PhD and undergraduate programmes and also in the area of management. It has

strengthened links between colleagues at different institutions and has socialised and shared

experiences and ways to tackle common challenges. Also, it has allowed the achievement of

results and impacts that if they had not formed part of collaborative networked assignment, they

would not have been feasible. Some examples are the SCT-Chile system, the Qualifications

Framework and the institutional research networks, among others.

4. World Bank Performance Assessment

The Project deems its partnership with the World Bank as satisfactory. It particularly appreciates

the exceptional quality of its professionals and technical consultancy to the Project, as well as the

friendliness of their staff in their interpersonal relationships. Less favourable aspects include

important errors in the legal framework of the loan agreement, several changes of Project

manager and often inaccurate signs in procurement management according to World Bank

regulations, which in addition to structural faults in the Chilean fiduciary system have

unnecessarily complicated subproject and Performance Agreement implementation. The lessons

learnt in these areas are reflected in the fiduciary arrangements that have been pre-agreed with

the World Bank for the next phase of the project.

5. Arrangements for Future Implementation

By the termination date of Loan Agreement 7317-CH, the triggers and objective of the Project

were met, along with 100% disbursement on USD $25,130,000 loan and it used 100% of the

contribution of the Government of Chile USD $ 67,000,000. However, given that the sub-

50

projects are awarded by FIAC on a regular basis throughout the year and their term may also be

extended for reasons beyond TEI management, when the Loan expired, there were sub-projects

in progress to which the Chilean Government will contribute during 2011, 2012, and 2013 with a

sum additional to the one provided in the Loan Agreement about USD $ 32.436 billion to

complete it successfully and achieve its objectives, using the same sub-project implementation

strategies awarded with World Bank funds.

On the other hand, as there is concern regarding the sustainability of the 4 initiatives

implemented in the pilot Performance Agreement competition, which involved a commitment of

institutional resources beyond what was received to achieve and maintain effectiveness,

continuity actions will be implemented through total complementary contributions to the order of

USD $ 2 million, whose terms shall be set out in 2011 Competition Regulations with funding

from the Chilean Government.

Finally, as a way of sustaining the reforms initiated in the Mece2 program and strengthening

acquired institutional capacities, the new Government has decided to scale the pilot Performance

Agreement instrument with fiscal resources, while negotiations and the signing of a new loan

agreement with the World Bank for the 2012-2015 period is being finalized. To this end, in 2011

one or more Performance Agreement competitions will be convened in Teacher Training, Tuning

of Curricula, Internationalization of National PhD programs, Quality of Learning and

Management (FIAC2) and Technical and Vocational Training, which will be allocated in

accordance with the provisions of the Loan Agreement No. 7317-CH. As part of new

Performance Agreement orientation, it has been decided that FIAC will be updated to a second

version (FIAC2), introducing mechanisms for pre-selection of proposals, elements of negotiation

and a greater focus towards achieving outstanding results and performance.

51

Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders

(Not applicable)

52

Annex 9. List of Supporting Documents

1. Aide Memoire, Mid Term Review, January 2009

2. Avances de Implementación, Consejo de Rectores de las Universidades Chilenas,

Ministerio de Educación, Antofagasta, 2009

3. CAE, Chile‘s State guaranteed Student Loan Program Report, March 2011

4. CASEN-Survey 2009, 2006, 2003,and 2000

5. Convenios de Desempeño, Lecciones Aprendidas

6. CPS, Country Partnership Strategy Progress Report, 2010, 2011

7. Implementation Status Report or the Tertiary Education Finance for Results Project- First

Phase APL (P088498), January 2011, June 2010

8. Información del Comité Internacional MECESUP2, sobre convenios de desempeño y

triggers, July 2010

9. Informe del Comité Internacional MECESUP 2 (Enero 2009)

10. Mid-Term Review report for the Tertiary Education Finance for Results Project- First

Phase APL (P088498), June 25, 2009.

11. Programa Mecesup2. BIRF 7317-CH, Fondo de Innovación Académica, Lecciones

Aprendidas

12. Project Appraisal Document for the Tertiary Education Finance for Results Project- First

Phase APL (P088498), May 19, 2005.

13. Reich Albertz, Ricardo, Convenios de Desempeño 2011-2014, Formación de Profesores,

April 2011

14. Reich Albertz, Ricardo, ―Bases y Desafíos de la Aplicación de Convenios de Desempeño

en la Educación Superior de Chile, Documento de Posición‖

15. www.futurolaboral.cl/

16. www.sies.cl

http://www.sies.cl/

Atta

cam

a D

esert

An

de

s M

ou

nt a

i ns

An

de

s

Mo

un

t ai n

s

Nevado OjosNevado Ojosdel Saladodel Salado(6,880 m)(6,880 m)VallenarVallenar

OvalleOvalle

ChillánChillán

VictoriaVictoria

OsornoOsorno

PuertoPuertoQuelliónQuellión

Puerto AisénPuerto Aisén

ManatialesManatiales

PuertoPuertoYungayYungay

CalamaCalama

SANTIAGOSANTIAGO

XIIXII

XIXI

X

IXIX

XIVXIV

VIIIVIII

VIIVII

VIVIRMRM

V

IVIV

IIIIII

IIII

I

XVXV

Vallenar

Ovalle

Chillán

Victoria

Osorno

PuertoQuellión

Puerto Aisén

Manatiales

PuertoYungay

Calama

Tocopilla

Iquique

Antofagasta

Copiapo

La Serena

Valparaíso

Concepcíon

Temuco

Puerto Montt

Valvidia

Coihaique

PuntaArenas

Talca

Rancagua

Arica

SANTIAGO

XII

XI

X

IX

XIV

VIII

VII

VIRM

V

IV

III

II

I

XV

ARGENTINA

BOLIVIA

PARAGUAY

PERU

URU

GUAY

Cape Horn

Strait ofMagellan

PACIFIC OCEAN

ATLANTICOCEAN

To San Sebastián

To Río Gallegos

To Paso

Río Mayo

To Nahuel Huapí

To BardasBlancas

To Villa Nueva

To Rodeo

To Fiambalá

To Salta

To Uyuni

To Corque

To La PazTo

Tacna

To Tolar

Grande

To Las Lajas

Atta

cam

a D

esert

An

de

s M

ou

nt a

i ns

An

de

s

Mo

un

t ai n

s

Nevado Ojosdel Salado(6,880 m)

85°W 80°W 75°W 70°W 65°W 60°W

40°S

35°S

30°S

25°S

50°S

45°S

40°S

35°S

30°S

25°S

20°S

80°W 75°W 70°W 65°W 60°W 55°W 50°W

CHILE

TARAPACÁ

ANTOFAGASTA

ATACAMA

COQUIMBO

VALPARAÍSO

REGIÓN METROPOLITANADE SANTIAGO

LIBERTADOR GENERALBERNADO O’HIGGINS

MAULE

BIO BÍO

LA ARAUCANÍA

LOS LAGOS

AISÉN DEL GENERAL CARLOSIBÁÑEZ DEL CAMPO

MAGALLANES Y DE LAANTÁRTICA CHILENA

LOS RÌOS*

ARICA-PARINACOTA*

I.

II.

III.

IV.

V.

RM.

VI.

VII.

VIII.

IX.

X.

XI.

XII.

XIV.

XV.

REGIONS OF CHILE

* Regions approved by cabinet and parliament, will become effective on October, 2007.

This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other informationshown on this map do not imply, on the part of The World BankGroup, any judgment on the legal status of any territory, or anyendorsement or acceptance of such boundaries.

0 200 400

0 200 400 Miles

600 Kilometers

IBRD 33386R

JUNE 2007

SELECTED CITIES AND TOWNS

REGION CAPITALS

NATIONAL CAPITAL

RIVERS

MAIN ROADS

RAILROADS

REGION BOUNDARIES

INTERNATIONAL BOUNDARIES

FALKLAND ISLANDS (ISLAS MALVINAS)A DISPUTE CONCERNING SOVEREIGNTY OVER THE

ISLANDS EXISTS BETWEEN ARGENTINA WHICH CLAIMSTHIS SOVEREIGNTY AND THE U.K. WHICH ADMINISTERS

THE ISLANDS.

CHILE

Document of The World Bank...Document of The World Bank Report No: ICR00001840 IMPLEMENTATION...

Documents

Transcript of Document of The World Bank...Document of The World Bank Report No: ICR00001840 IMPLEMENTATION...