Document of The World Bank · CURRENCY EQUIVALENTS (Exchange Rate Effective June 11, 2014) Currency...

Document of The World Bank

Report No: ICR2936

IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-7458)

ON A

LOAN

IN THE AMOUNT OF US$11.7 MILLION

TO THE

REPUBLIC OF CHILE

FOR THE

INSTITUTIONAL STRENGTHENING OF THE MINISTRY OF PUBLIC WORKS

TECHNICAL ASSISTANT PROJECT

July 28, 2014

Sustainable Development Department Transport Unit Latin America and the Caribbean Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective June 11, 2014)

Currency Unit = Chilean Pesos (CLP) 1.00 CLP = US$ 0.001803

US$ 1.00 = CLP 554.48

FISCAL YEAR

ABBREVIATIONS AND ACRONYMS BP Bank Policy IFC International Finance Corporation M&E Monitoring and Evaluation IMF International Monetary Fund CPS Country Partnership Strategy IT Information Technology DPL Development Policy Loan MOF Ministry of Finance FDI Foreign Direct Investment MOPW Ministry of Public Works FM Financial Management NCB National Competitive Bidding GDP Gross Domestic Product OP Operational Procedure GIS Geographic Information System PAD Project Appraisal Document GNP Gross National Product PIU Project Imp1ementation Unit HH Herfindahl-Hirschman Index PDO Project Development Objective IADB Inter-American Development Bank R&D Research and Development

IBRD International Bank for Reconstruction and Development SSS Sole-Sourced Selection

ICB International Competitive Bidding TAL Technical Assistance Loan IDA International Development Association

Vice President: Jorge Familiar Calderon Country Director: Livia M. Benavides Sector Manager: Aurelio Menendez Project Team Leader: Daniel Alberto Benitez ICR Team Leader: Daniel Alberto Benitez

ii

CHILE Institutional Strengthening of the Ministry of Public Works

CONTENTS

Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph

1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 8 3. Assessment of Outcomes .......................................................................................... 13 4. Assessment of Risk to Development Outcome ......................................................... 18 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 22 Annex 1. Project Costs and Financing .......................................................................... 23 Annex 2. Outputs by Component ................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................. 34 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 35 Annex 5. Beneficiary Survey Results ........................................................................... 37 Annex 6. Stakeholder Workshop Report and Results ................................................... 38 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 39 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 47 Annex 9. List of Supporting Documents ...................................................................... 48

MAP

iii

A. Basic Information

Country: Chile Project Name: Ministry of Public Works Institutional Strengthening TAL

Project ID: P102931 L/C/TF Number(s): IBRD-74580 ICR Date: 09/19/2013 ICR Type: Core ICR Lending Instrument: TAL Borrower: REPUBLIC OF CHILE Original Total Commitment:

USD 11.70M Disbursed Amount: USD 6.84M

Revised Amount: USD 11.70M Environmental Category: C Implementing Agencies: Ministry of Public Works Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual Date(s) Concept Review: 08/10/2006 Effectiveness: 02/27/2008 02/27/2008 Appraisal: 03/13/2007 Restructuring(s): Approval: 06/14/2007 Mid-term Review: 06/08/2011 08/08/2011 Closing: 01/31/2013 01/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Negligible Bank Performance: Moderately Satisfactory

Borrower Performance: Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Moderately Satisfactory Government: Satisfactory

Quality of Supervision: Moderately Satisfactory Implementing Agency/Agencies: Satisfactory

Overall Bank Performance:

Moderately Satisfactory

Overall Borrower Performance: Satisfactory

iv

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators QAG Assessments

(if any) Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

None

Problem Project at any time (Yes/No):

Yes Quality of Supervision (QSA):

None

DO rating before Closing/Inactive status:

Moderately Satisfactory

D. Sector and Theme Codes

Original Actual Sector Code (as % of total Bank financing) Central government administration 100 100

Theme Code (as % of total Bank financing) Administrative and civil service reform 29 29 Legal institutions for a market economy 14 14 Managing for development results 14 14 Other public sector governance 14 14 Regulation and competition policy 29 29 E. Bank Staff

Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Livia M. Benavides Axel van Trotsenburg Sector Manager: Aurelio Menendez Jose Luis Irigoyen Project Team Leader: Daniel Alberto Benitez Jordan Schwartz ICR Team Leader: Daniel Alberto Benitez ICR Primary Author: Mirtha Pokorny F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project Development Objective is to assist the Government in achieving a sustainable and integrated approach to the planning of infrastructure; levels of service standards for infrastructure as well as model contracts and procedures that promote competition and assign risks appropriately; an integrated project management process; a

v

regulatory framework for all of MOPW’s sectors of responsibility that increases consumer confidence in infrastructure service provision; and the legal and organizational separation of the areas of planning, contracting, execution and regulation of infrastructure provision. Some of the original PDOs and intermediate indicators were found difficult or impossible to measure. Thus in 2009 the Borrower and the Bank agreed to a revised set of indicators. (See Section 1.3 of the ICR) The new indicators were reflected in the Aide Memoire dated November 20, 2009 and, although they were never formally changed, they became the framework on which the project was monitored. The information below is based on the agreed revised indicators. a) PDO Indicator(s)

Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 New planning methodology for infrastructure functioning and incorporated in the

investment decision of the MOPW Share of projects approved based on integrated planning methodology

Projects approved based on

the new planning methodology

25% n.a. 93%

Date achieved 2008 2014

Comments (incl. %

achievement)

Fully achieved. This component, one of the main linchpins of the project, has helped to successfully establish a new paradigm in MOPW’s planning, introducing a new methodology that integrates regional needs and captures the synergies of different projects within a development program. Several consulting contracts contributed to develop the methodology and apply it to different programs (e.g. City Ports, Meterx, City Plans, and Systems of Territorial Information). By 2011 thirteen Regional Infrastructure Plans and Hydrological Management Plans were developed and approved. By 2013 the new methodology was fully mainstreamed and 99% of the investments included in the 2014 budget come from regional plans developed with the new methodology.

vi

Baseline Value Original Target

Values (from approval documents)

Formally Revised Target Values



Indicator 2

Greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer responsive service targets

More competition in biddings to be reached by (i) improvements in bidders’ registry (reglamento de obras y consultorías) and (ii) improved standard bidding documents

Public works and concession contracts

using the new standard bidding

documents

80% of new works and services

contracts based on standard bidding

documents

See description below

Date achieved 2008 2009 2014

Comments (incl. %

achievement)

Partially Achieved. MOPW went through a lengthy and extensive revision of the regulations governing public works and consulting services and made proposals that were internally approved in MPOW to revise the contractors and consultants registries and bidding documents. The process was slowed down by the Contraloría office, and is still under revision.

Baseline Value


approval documents)

Formally Revised

Target Values



Indicator 3

Greater responsiveness to consumer concerns related to the levels of service; and greater oversight of performance by public sector service providers as well as

concessionaires

Number of concession contracts under the jurisdiction of the MOPW that incorporate (explicitly) service standards

Public works and

concession contracts using explicit service standards

3 contracts (2010), 2 contracts

(2011) and 2 contracts (2012)



Comments (incl. %

achievement)

Fully achieved. By the end of 2013 MOPW had entered the following number of contracts with explicit service standards: (a) five contracts of airport concessions; (b) five concession contracts of interurban roads with partial definition of service standards; (c) 111 contracts of performance-based contracts and ad-measure contracts, of which contracts by service levels comprise 13,371.53 km or 71.7% of the country’s paved network; (d) concession contracts for the three hospitals (Sotero del Rio, Santiago Occidente and the Geriatric Hospital).

vii

Baseline Value


approval documents)

Formally Revised

Target Values



Indicator 4

Reduction in variance between values established at contract signature and final values at contract execution

Reduction in the variance between costs forecasted when contract is signed and actual cost of works

Variance of cost when contract is signed and

actual cost of works in 2011 15% (as revised)

More than 32%


Comments (incl. %

achievement)

Fully Achieved. In 2010 MOPW completed the base line built with contracts implemented in 2005, 2006 and 2007. By 2011 projects managed under the integrated management system and completed that year had a cost of 32% below the base line cost. Projects completed in 2012 had a reduction in cost of 14% with respect to the projects completed in 2011.

Baseline Value


approval documents)

Formally Revised

Target Values



Indicator 5

Normative and organizational structure that provides clear separation and definition of roles and responsibilities for planning, contracting, works execution

and oversight Regulatory oversight system in place to be used by entity independent of planning and

execution divisions of MOPW

Chile does not have a regulatory oversight system

suitable for an entity independent of the planning and execution divisions of

MOPW

System in place

See description

below


Comments (incl. %

achievement)

Partially Achieved. MOPW developed and implemented oversight systems for 20 interurban road contracts, six urban road contracts and two public transport corridors. They also completed and implemented the necessary forms for contract oversight of concessioned airports (e.g. information screens, escalators, bathrooms, etc.), and for contract oversight of rural potable water. In all cases complaints forms have been completed and being used for remedial actions. Several web platforms were created for the oversight efforts and their effectiveness were confirmed during the reconstruction period after the 2010 earthquake.

b) Intermediate Outcome Indicator(s)

viii

Baseline Value


approval documents)

Formally Revised

Target Values



Component 1 Strategic and tactical plans approved – number of Investment Plans (National and Regional) elaborated

No investment plan and regional plans exist Not defined

16 Investment Plans (1 Master

Plan and 15 Regional

Investment Plans)

16 Investment Plans implemented


Comments (incl. %

achievement)

Fully Achieved. A Master Plan and 15 Regional Investment Plans were developed using the new integrated planning methodology. Two regional plans were elaborated using a more demanding service standard (Arica – Parinacota and Los Ríos)

Baseline Value Original Target

Values (from approval

documents)

Formally Revised

Target Values



Component 2 Identify, define and adopt service quality standards in a wide set of work typologies under the responsibility of the MOPW

No model for quality

standards per type of work exists.

n.a

9 type of works with identified services and

standards being elaborated.

Adopted target: 13 type of works with identified services

and standards being elaborated – see

description below Date achieved 2008 2009 2014

Comments (incl. %

achievement)

Substantially Achieved. By 2013 MOPW had completed the identification services and relevant quality attributes as well as standards of service for the following types: (a) airports under concession; (b) public buildings; (c) roads under concession; (d) maintenance of paved roads not under concession; (e) rain waters; (f) hospitals under concession; (g) identification of users and services of small fisheries. MOPW is well advanced in the definition of standards for (h) projects of lake, river and maritime connectivity for isolated communities; (i) rural potable water; (j) border passes under concession (Paso de los Libertadores); (k) urban renewal projects (Proyecto Marga Mar); (l) public assets building; (m) port infrastructure of border coasts; and (n) irrigation works.

ix

Baseline Value


approval documents)

Formally Revised

Target Values

Actual Value Achieved at Completion or Target Years

Component 3 Reduction in the variance of construction time between the time envisaged when contract is signed and actual construction time

Variance of time when

contract is signed and actual construction time in 2011

15% (as revised)

19%

Date achieved 2011 2013

Comments (incl. %

achievement)

Fully Achieved. The base line was completed in 2011 using contracts completed in 2005, 2006 and 2007. Projects completed in 2011 under the integrated management system showed a marginal (1%) difference in completion time when compared to the base line. Projects completed in 2012 showed a difference of 19% with respect to projects completed in 2011.

Baseline Value


approval documents)

Formally Revised Target

Values



Component 4 Develop and implement a system for the evaluation of users’ satisfaction

No model for the evaluation of users’ satisfaction exists.

Develop a model for the evaluation

of users’ satisfaction for 8 types of works



Comments (incl. %

achievement)

Substantially Achieved. Efforts to develop and implement systems for evaluating users’ satisfaction started fully in 2009 with the system developed and tested through surveys in 12 interurban concessioned roads, three sections of interurban roads under the administration of the Regional Directorates of Roads, and four highways. By 2013 MOPW has completed the models for evaluating users’ satisfaction and the base line for comparisons for six different types of service: (a) Interurban concessioned roads, (b) Interurban not concessioned roads, (c) concessioned urban highways, (d) connectivity of isolated areas’ (d) concessioned airports of the primary network, and (e) Non-concessioned airports of the primary network.

x

G. Ratings of Project Performance in ISRs

No. Date ISR Archived DO IP Actual

Disbursements (USD millions)

1 09/11/2007 Satisfactory Satisfactory 0.00 2 04/28/2008 Satisfactory Satisfactory 0.00 3 10/30/2008 Satisfactory Satisfactory 0.00 4 05/06/2009 Satisfactory Satisfactory 0.00

5 11/11/2009 Moderately Unsatisfactory Moderately

Unsatisfactory 0.00


Unsatisfactory 0.00


Unsatisfactory 1.19

8 05/21/2011 Moderately Unsatisfactory Moderately Satisfactory 1.58

9 06/29/2011 Moderately Satisfactory Moderately Satisfactory 1.58 10 01/30/2012 Moderately Satisfactory Moderately Satisfactory 2.36 11 11/10/2012 Moderately Satisfactory Moderately Satisfactory 3.97 12 06/26/2013 Moderately Satisfactory Moderately Satisfactory 3.97 13 01/04/2014 Moderately Satisfactory Moderately Satisfactory 6.81

H. Restructuring (if any) By mid-2012 the Borrower requested and the Bank agreed to a one- year extension of the loan with a new closing date of January 31 2014. The objective of the extension was to allow further disbursements on long-term consultancies financed under the loan and also included a few changes in the procurement rules and the implementation of Anti-corruption Guidelines as described below. The Procurement and Consultant Guidelines were updated to reflect the versions dated January 2011, which allows the use of framework agreements, known as “Convenios Marcos” in Chile, for the procurement of goods, works and non-consultant services. In addition, the Bank agreed to raise the maximum limits for using ChileCompra for consultant services, from US$100,000 to US$300,000. In addition, the definition of “Convenios Marco” was amended to include non-consultant services. Finally the Anti-corruption Guidelines dated October 15, 2006 and revised in January 2011 was to apply to the Project to be consistent with the updated version of the Procurement and Consultant Guidelines.

xi

Disbursement Profile

xii

1 Project Context, Development Objectives and Design

1.1 Context at Appraisal

In the early 1990s, the Government of Chile took stock of its infrastructure needs and estimated its infrastructure deficit at US$11 billion with additional losses due to lack of competitiveness of US$1.7 billion annually.1 In determining how to respond to this need for massive investments and significant improvements in service delivery, the Government also recognized that the public sector did not have the resources to carry out the major works required, nor could it commit its budgetary resources exclusively to infrastructure investments given the country's social investment needs. To help face this challenge, the Government first called on the private sector to take part in the building, maintenance and operation of major road works. In the years since its inception, the concession program has expanded both in terms of sectors and design. Chile had at the time of appraisal (and still has) what may be the world's most extensive concession program encompassing roads, ports, railroads, bridges, tunnels, reservoirs, jails and canals. The Ministry of Public Works' (MOPW) concessions portfolio-including roads and highways, bridges, airports, dams, canals, prisons, and other public edifices-has risen in real terms from about 450 billion pesos in 1995 to over 1 trillion pesos in 2005 . More strikingly, the share of private sector participation in investments had increased from 9 percent to 65 percent over the course of the 1995-2005 decade. In addition to the growing concessions program, at the time previous to project appraisal the Government of Chile had announced a significant expansion in the public financing of infrastructure. The intent was to provide Chile's regions and export products with greater access to markets and to further improve the levels of basic services for Chilean consumers. The administration recognized that both its growing public investment and concession program would require greater attention in terms of planning, contract design, project management and oversight in order for them to maximize their potential as contributors to higher growth levels. The Chilean administration had identified specific constraints to the operational and institutional structure of the MOPW that it believed had to be addressed in order for Chile to reach new level of efficiency and accountability in infrastructure investment. Those constraints included: Planning:

1 Sectors covered by Ministry of Public Works include transport assets such as inter-urban roads, urban roads, highways, bridges, and airports; water resource infrastructure such as dams, storm drainage systems, and canals; rural infrastructure such as small water systems, ferry landings, and fishing ports; and public buildings such as jails, hospitals, and schools. Of transport, water resource and public building contracts, only ports are concessioned and monitored by a separate agency. There are separate regulatory agencies for privatized utilities such as water supply, telecommunications and energy

1

• Lack of medium and long-term planning with an integrated geographic perspective oriented to the competitiveness of Chile's productive sectors.

• Low level of interaction with other relevant stakeholders in infrastructure sector and a lack of societal or consumer voice in project planning.

• Investment programs which are contract-driven rather than integrated along a supply chain, leading to coordination difficulties and inefficient expenditures.

Standards, Contracts and Competition:

• Lack of explicit service standards embedded in concessions or public works due to an historic focus on input-driven rather than performance-based contracts.

• Uneven levels of competition during bidding processes for either concessions or public works exacerbated by lack of information correlating outcomes to contract size, structure, design elements or bidding procedures

Project Management:

• Low levels of accountability related to project outcomes due to an atomized approach to project management with different individuals responsible for each stage of project development.

• Poor availability of information about project costs, contract management or design and construction issues.

Oversight and Regulation:

• Institutional inability to provide effective oversight of contracts and concessions according to agree upon levels of service, let alone cost elements.

• Lack of ability to consider consumer concerns or even to assure public access to information in the oversight of services.

Organizational Reform:

• An organizational structure that does not provide for clear separation of roles and responsibilities among the four major areas of activities-planning, contract design, project management and regulation.

Measures to address these challenges made up the core components of the Institutional Strengthening Program of the Ministry of Public Works. The Government of Chile requested that this reform initiative be supported with a Bank Development Policy Loan (DPL) and, the subject of this report, a Technical Assistance Loan (TAL). 2 The DPL3 intended to achieve explicit short term outputs and defined a subsequent series of tasks that would require several years to complete and would be implemented under the TAL. Under the DPL the following tasks were undertaken: (i) completion of competitiveness

2 http://www.mop.cl/modernizacion/Paginas/Inicio.aspx 3 Institutional Strengthening of the Ministry of Public Works Development Policy Loan, 2007. Report No: 39306-CL

2

plans for key clusters; (ii) development of sample performance standards for high and low volume roads, for both concessions and public works contracts, and (b) review of bidding processes and recommendations on new procedures; (iii) completion of design for regulatory accounting system; and (iv) determination of standards to be regulated for roads and one other sector. Bank's involvement in this initiative was sought to leverage the Bank's experience in logistics and territorial planning, in the design and execution of public-private contracts, in project management and in the development of regulatory frameworks for infrastructure. In addition, the Bank offered a comparative advantage in its ability to draw upon global knowledge, sustain a policy dialogue, and exploit synergies with other operations and technical support to Chile including, but not limited to, the Infrastructure for Territorial Development Project, the TranSantiago Urban Transport DPL and TAL, and the fee-for-service technical assistance on the valuation of contingent liabilities in infrastructure - see Annex 9 for a complete list of projects. Finally, both the Ministry of Finance and the MOPW were attracted by the synergies that could be gained from having both a DPL and a TAL to finance the key components of Institutional Strengthening Program. The thematic and substantive linkages of the two projects were expected to provide an effective framework for monitoring and evaluating the implementation of the reform program.

1.2 Original Project Development Objectives (PDO) and Key Indicators The Project Development Objective was to assist the Government in achieving a sustainable and integrated approach to the planning of infrastructure; levels of service standards for infrastructure as well as model contracts and procedures that promote competition and assign risks appropriately; an integrated project management process; a regulatory framework for all of MOPW's sectors of responsibility that increases consumer confidence in infrastructure service provision; and the legal and organizational separation of the areas of planning, contracting, execution and regulation of infrastructure provision.

Output and Outcome Indicators for monitoring the progress of the objectives centered on: (a) increased return of infrastructure projects due to the new planning methodology; (b) greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer responsive service targets; (c) greater responsiveness to consumer concerns related to the levels of service; and greater oversight of performance by public sector service providers as well as concessionaires; (d) reduction in variance between values established at contract signature and final values at contract execution; and (e) normative and organizational structure that provides clear separation and definition of roles and responsibilities for planning, contracting, works execution and oversight. The methodology for calculating the baseline and subsequent indicators would be updated to reflect the availability and reliability of data as well as the most appropriate approach for monitoring of change. The baseline for each indicator was to be determined during the first four months of the project after effectiveness. The project would also

3

monitor progress towards the expected outcomes through the medium-term indicators for each of the components as seen in the next paragraph.

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification

The PDO remained unchanged during the life of the project. With respect to the key indicators, they were found to be difficult to measure and, in some cases, impossible to establish a strong causality between project and outcome. In May 2009 the PIU and the Bank agreed on a revised set of indicators which were included in the Aide Memoire dated November 20, 2009. Although the changes, summarized in the table below, were never formalized, the new agreed indicators were the ones used by MOPW and this ICR to assess the achievements under the project.4 ORIGINAL REVISED (a) PDO Indicators (a) PDO Indicators Increased return of infrastructure projects due to the new planning methodology

New planning methodology for infrastructure functioning and incorporated in the investments decisions of the MOPW

Greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer responsive service targets

Unchanged

Greater responsiveness to consumer concerns related to the levels of service; and greater oversight of performance by public sector service providers as well as concessionaires

Unchanged

Reduction in variance between values established at contract signature and final values at contract execution Unchanged

Normative and organizational structure that provides clear separation and definition of roles and responsibilities for planning, contracting, works execution and oversight

Unchanged

(b) Intermediate Indicators b) Intermediate Indicators Strategic and tactical plans approved (Component 1) Unchanged Share of works contracts and concessions based on measurable, output-oriented standards and greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer-responsive service targets. (Component 2)

Identify, define and adopt service quality standards in a wide set of works typologies under the responsibility of the MOPW.

Reduction in variance between unit values for Reduction in the variance of construction time

4 The five PDO Indicators are respectively measured by: i) Share of projects approved based on integrated planning methodology; ii) More competition in biddings to be reached by (i) improvements in bidders’ registry (reglamento de obras y consultorias) and (ii) improved standard bidding documents; iii) Number of concession contracts under the jurisdiction of the MOPW that incorporate (explicitly) service standards; iv) Reduction in the variance of the difference between costs forecasted when contract is signed and actual cost of works; and v) Regulatory oversight system in place to be used by entity independent of planning and execution divisions of MOPW. Note: the Result Framework in the PAD and the revised Result Framework do not provide a methodological description on how these five indicators should be measured.

4

construction/investment established at contract signature and final values at project execution. (Component 3)

between the time envisaged when contract is signed and actual construction time

Greater responsiveness to consumer concerns related to levels of service; and greater oversight of performance by public sector service providers as well as concessionaires. (Component 4)

Develop and implement a system for the evaluation of users’ satisfaction

A legal and organizational framework in place that provides for clear separation of roles and responsibilities. (Component 5)

Unchanged

1.4 Main Beneficiaries The immediate beneficiary of the project would be the MOPW and its ability to respond more effectively to the needs of the users of public infrastructure. The main beneficiary of the project is the Chilean population through the design and implementation of projects focusing on regional development and equity, a more efficient use of private capital in providing public services, and a strong focus on user satisfaction. Ultimately, those benefits are expected to stimulate economic activities and spread the benefits of the project to the entire population in Chile. In addition, improvement of efficiency of public works management would save public expenditures, which leads to benefits to all taxpayers.

1.5 Original Components (as approved) The operation had five components, four of which address the core aspects of MOPW's business activities: planning; setting standards for contracts, and conducting bids; managing projects; and regulating or overseeing ("la fiscalización de") both concessions and public works. A fifth component (Organizational Reform) was to bring coherence to the program through the development of a legal and organizational structure that defined the expected new roles and responsibilities of the MOPW. Each corresponding component as described in the Loan Agreement is presented below. Section II of the PAD provides a more detailed description of the project. Part I: Integrated Planning 1. Design of a Long-Term Planning Methodology for MOPW's Infrastructure Development:

a) Design of a methodology for the planning process of infrastructure investments at the strategic level (national level and long-term planning) as well as at the tactical level (sub- national level and medium-term planning).

b) Design of an environmental evaluation methodology to be included in the planning process and provision of training on such methodology.

c) Design of a strategy of transition towards implementation of the planning methodology referred to in (a) above, and for the monitoring of such planning process.

5

2. Implementation of a Long-Term Planning Methodology:

a) Development of components of a strategic national infrastructure plan using the methodology developed under Part I.1.(a) above.

b) Design and implementation of an information system, database and geographic information system (GIS).

c) Design of policies for long-term planning in the areas of land use, resettlement, transport and water resources management.

d) Implementation of the transition strategy referred to in Part I.1.(c) above, including implementation of an internal and external communication plan, and dissemination of the various infrastructure plans developed under Part I.2.(a) above through workshops, courses and seminars.

3. Carrying out of studies and training related to the development of a sustainable and integrated approach to the planning of infrastructure, as proposed by the Borrower and agreed to by the Bank. Part II: Standards, Contracts, Competitiveness 1. Establishment of Service Standards:

a) Establishment of a methodology for setting, evaluating and updating MOPW's standards of service.

b) Development of a standard-of-service manual that assigns standards according to type of contract, sub-sector, cost or cost categories and demand levels.

c) Training of MOPW's and other selected agencies' staff in the use of service standards.

d) Strengthening the capacity of the MOPW's staff to apply the service standards referred to in Part II.1.(a) above, to the design of infrastructure projects.

2. Contracts and Competitive Processes:

a) Analysis of MOPW's current contracts and bid processes as well as those used in international best practices.

b) Development of a database of MOPW's public works contracts and concessions. c) Assessment of the variables of contract design and bidding processes against their

ability to generate competition and investment results. d) Design of model contracts for each primary type of modality of contracting of

infrastructure works, and each sub-sector, based on international best practices. e) Analysis of appropriate risk allocations in contracts by modality and sub-sector.

3. Carrying out of studies and training related to the development of service standards for infrastructure as well as model contracts and procedures that promote competition and assign risks appropriately, as proposed by the Borrower and agreed to by the Bank.

6

Part III: Project Management 1. Design of an Integrated Project Management Model:

a) Detailed design of the MOPW's project management model. b) Establishment of information and software tools to accompany the project

management model designed under (a) above. 2. Implementation of a Project Management Model: Carrying out of communications and capacity building plans to accompany the implementation of the project management model developed under Part III.1 (a) above. 3. Carrying out of studies and training related to the development of an integrated management process in MOPW, as proposed by the Borrower and agreed to by the Bank. Part IV: Strengthening of Regulation of Infrastructure Works 1. Development of roles and regulations for the oversights of public works contracts and concessions, including:

a) An operations manual to lay out the procedures for such oversight; and b) Economic and financial models, and their accompanying regulatory accounting

manuals, for public-private partnerships for selected infrastructure sectors. 2. Carrying out of studies and training related to the development of a regulatory framework for MOPW's sectors of responsibility that increases user confidence in infrastructure service provision, as proposed by the Borrower and agreed to by the Bank. Part V: Organizational Reform 1. Norms and Regulations:

a) Provision of technical assistance for the redrafting of the bill for the establishment of oversight mechanisms for public contracts and concessions.

b) Provision of technical assistance for drafting or redrafting, as the case may be, of other laws and regulations needed for implementation of the Program.

2. Institutional Development:

a) Definition of alternative arrangements for MOPW's reform in accordance with the proposed legal changes referred to in Part V.1 above, including:

b) Design of changes to the institutional structure of the MOPW consistent with any legal or regulatory reforms implemented under the Program.

c) Design of the work scope and institutional arrangements that would integrate the management of water resources.

d) Establishment of: (i) a cadaster or registry of existing MOPW norms; (ii) procedures that regulate the creation, evaluation and approval of new norms; (iii) procedures for populating, maintaining, disseminating and updating information systems; and (iv) a web-based database of the new norms and procedures.

7

e) Development of a human resource strategy based on performance for MOPW. f) Development of an information technology strategy to support the quality and

efficiency of MOPW's business processes. 3. Carrying out of studies and training related to the development of an organizational

separation of the planning, contracting, execution and regulatory functions of infrastructure provision, as proposed by the Borrower and agreed to by the Bank.

1.6 Revised Components The project components remained unchanged throughout the life of the project.

1.7 Other significant changes By mid-2012 the Borrower requested and the Bank agreed to a one- year extension of the loan with a new closing date of January 31st, 2014. The objective of the extension was to allow further disbursements on long-term consultancies financed under the loan and to include changes in the procurement rules and the implementation of Anti-Corruption Guidelines as described below. The Procurement and Consultant Guidelines were updated to reflect the versions dated January 2011, which allows the use of framework agreements, known as “Convenios Marcos” in Chile, for the procurement of goods, works and non-consultant services. In addition, the Bank agreed to raise the maximum limits for using ChileCompra for consultant services, from US$100,000 to US$300,000. In addition, the definition of “Convenios Marco” was amended to include non-consultant services. Finally the Anti-Corruption Guidelines dated October 15, 2006 and revised in January 2011 was to apply to the Project to be consistent with the updated version of the Procurement and Consultant Guidelines.

2 Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design and Quality at Entry The ICR finds the quality at entry as moderately satisfactory, since: Project design was in line with the main pillars of the Bank’s strategy for Chile. By supporting the implementation of the Government's Institutional Strengthening Program, the TAL was in line with the goals and strategic approach set out in the joint IBRD-IFC Country Partnership Strategy (Report No 38691-CL). The objective of the CPS was to support Chile's efforts to converge with OECD income levels and living standards by (i) accelerating growth and (ii) building a more equitable society. The Bank Group strategy was based on the maintenance of close engagement with the authorities, which

8

in recent had yielded important results in support of key Government programs and through flexible, timely support that helped to advance Chile's development vision A strong policy dialogue and analytical underpinning contributed to a sound project design. The objectives and actions of the project were the result of more than a year of work at MOPW and from consultations with relevant external stakeholders. The effort was coordinated by a multidisciplinary team composed of MOPW staff with specific responsibilities for each technical area of reform. That team identified the main shortcomings of the MOPW and developed the proposals for their improvement. The Bank supported the work of these teams with technical and regulatory advice since the early stages of the reform’s inception. The result was a very comprehensive and detailed set of objectives and tasks that provided an excellent blueprint for a complex institutional reform. The choice of lending instruments contributed to project continuity. The TAL was presented to the Bank’s Board of Directors together with the DPL. The Ministry of Finance and the MOPW, as already mentioned, were attracted by the synergies that could be gained from having both loans to finance several of their key initiatives. First, because the thematic and substantive linkage of the two projects was expected to provide an effective framework for monitoring and evaluating the implementation of the Ministry’s reform program for a period long enough that exceeded one administration. Second, by having both a DPL and TAL, the political profile of the Bank-financed projects increased and made possible a contingent role of the Bank as an honest broker between the Ministries of Finance and Public Works when the reform suffered from delays during implementation. The quality at entry would have deserved a satisfactory rating if it had included a more adequate results framework. Some of the originally proposed PDO and intermediate indicators were difficult to measure and unrealistic in terms of timing. As shown in Section 1.3 preceding and discussed in more detail in section 2.3 below, some of the indicators had to be revised to make them more precise, measurable and in some cases, achievable within the time frame of the project.

2.2 Implementation The project was approved by the Bank's Board on June 14th, 2007 and it became effective on February 27, 2008. Implementation progressed steadily throughout the life of the project, albeit at different pace during different periods. In particular, internal resistance to the modernization program, changes in government administrations and staff directly involved with project implementation, at times lack of adequate funding, and poor disbursement rates that jeopardized Bank involvement, contributed to slower

9

project implementation.5 However, in spite of its rocky beginnings, these problems were eventually overcome and the modernization program, although a work still in progress, has succeeded in bringing profound changes to MOPW’s view of its role in a world in which users have a stronger voice and increasing demands. The following paragraphs summarize the most salient aspects of the implementation process. Resistance to change. The launching of the project by its original champion (minister Bitar during the first Bachelet administration) was immediately faced with strong opposition from MOPW’s staff. Unions interpreted the proposed reforms as implying privatizations and the redundancy of some services. Senior management within MOPW realized that the reform principles were not properly communicated and slowed down implementation. The impasse allowed further work with the unions to convey the message that the objective of the reform was to improve the efficiency of public spending and that no redundancies were being considered. Even when the issue of potential redundancies was put to rest, there was (and, with lesser strength still is) some institutional inertia to change the vision from a provider of hard infrastructure to a provider of high quality infrastructure services in those remaining few areas that are still not fully mainstreamed. Team leaders have been key in involving MOPW senior and junior staff (both in Santiago, where the headquarters of the MOPW are located and in all the country’s Regions) in the modernization process. They have fostered implementation of the reform by organizing trainings, strengthening the dialogue and building coordination stages among areas that had traditionally worked somewhat in isolation in order to bring everybody on board of the institutional changes. Changes in administrations and staff. During the first two years of project implementation the high rate of turnover of PIU’s coordinators considerably slowed down progress in the modernization program. Since appraisal to late 2009 the project had four coordinators, an average of one every six months. Subsequently, the upcoming elections and expected change of the administration in March 2010 coupled with a change in priorities to focus on a stimulus package to increase economic activity, contributed to further delays. The elections brought about a change in the coalition in power (the new one being from the center-right) after more than 20 years of a center-left coalition and the new government was faced immediately with the devastating earthquake that hit Chile in February, 2010. After the first few months of the new administration, the Modernization Program received a strong endorsement. The tools developed and implemented by the Program were tested, fully utilized and proved effective in the emergency reconstruction projects that followed the February earthquake, a fact that contributed to the renewed strong Government support of the program. The MOPW undertook several actions to increase the implementation pace of the modernization efforts, including providing a stronger institutional support with its transfer from the Directorate General of Public Works to the Undersecretary of Public Works and the appointment of a new highly effective project coordinator.

5 It should be noted that progress on project implementation was fairly independent from the rate of disbursements during the first half of execution, since in Chile loan funds are not additional to the Ministry’s budget.

10

Disbursements and Project funding. The first two years of the Modernization Program were implemented without Bank funding, with the first disbursement of loan funds taking place in mid-2010. The poor disbursement rates and the slowing down of project implementation during 2009 caused the revision of the PDO rating to MU. Supervision missions identified the problem as a worrisome lack of communication with the MOF with respect to how budget allocation mechanisms work in Chile when funds from a Bank loan are available. Bank funds are not additional to budget allocations and the latter were many times assigned to areas of the program that were not eligible for Bank financing. The dismal disbursement rates were the subject of the mid-term review and the subsequent meetings involving the Bank’s country director in which the Bank team agreed with the PIU and MOF that if changes were not made before June 2011 the Project would be canceled. The agreement included to: (1) appoint a new Project Coordinator, with knowledge of the Modernization Program and credibility within the organization; (2) request at least one disbursement before June 2011; (3) define the new administration’s prioritization areas and launch an activity (consultancy, workshop) in at least one of these areas; and (4) provide a stronger commitment to the Modernization Program. Authorities in the Ministry of Public Works, confirming the importance that was attributed to the continued presence of the Bank for the success of the Modernization Program complied extensively and fast with the agreements made. The actions undertaken included: (1) a new and highly effective Project Coordinator, a long term qualified official in the Ministry was appointed; (2) a new request for retroactive disbursement was presented to the Bank; and the Minister gave a stronger institutional support to the Modernization Program by transferring the management from the Directorate of Public Works to the Undersecretary of Public Works. In practice, this implies giving more visibility and power to the PIU. Moreover, the Strategic Directorate that was created as a high level coordinating unit to monitor the progress of the Modernization Program started to meet again once a month, after 15 months of discontinuity. Several articles about the Modernization Program were published in many newspapers as a result of interviews with the Undersecretary, providing clear support to the Modernization Program. Disbursement issues were satisfactorily addressed to a great extent through the already discussed loan restructuring. By the beginning of 2013 disbursements reached 34 percent of loan funds and accelerated considerably during the year. By the loan closing date US$8.24 million were disbursed (70 percent of the loan) and the remaining US$3.45 million were canceled. Lower than expected budget allocations also contributed to slow the implementation of the Program. An estimated front-loaded cost structure resulted inadequate to finance a program in which the initial stages were devoted to communicating visions and goals, seeking consensus, discussing strategies and alternative methodologies, seeking information from different international experiences and subsequently develop time consuming terms of reference for complex studies and/or technical assistances. Since the initial allocations were not fully utilized, and the MOF relied on past execution rates to

11

define subsequent allocations, funding was decreasing when the needs increased. Budget allocations during 2009 and 2010 were at a level that jeopardized the continuation of the program. Again, these issues were resolved once discussions were held to restructure the project and place it in the right track.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization

As already discussed, the result framework of the TAL included targets that were not measurable or lacked a clear distinction between end-of program outcomes monitorable at the time of completion and outcomes that would materialize in the longer term as the reform process consolidates. In addition, some of the indicators for the medium term outcomes were hard to measure, not very realistic, and misleading.6 These drawbacks of the M&E framework were identified very early during the project execution and by 2009 a revised set of more realistic and measurable indicators were prepared by MOPW and agreed with the Bank. The revised set of medium term outcomes helped to narrow down and provided useful milestones for each policy area objectives. The fine tuning of the indicators to ensure their alignment with the expected outcomes, the availability of data for their periodic monitoring, and the establishment of baseline values was carried out as part of the supervision activities of the TAL. The process of revision of the M&E and its implementation was useful to strengthen the dialogue with the counterpart and identify new opportunities for support. Both the interim semiannual reports prepared by the MOPW and the supervision missions were particularly useful to collect data relating to the project implementation. The missions provided the framework for an in depth follow-up of the project components’ progress and served to highlight pending issues to the authorities. In addition the supervision missions gave the Bank and the MOPW the opportunity to deepen the policy dialogue and identify areas of joint cooperation that were not identified during project preparation. Finally, a field visit conducted during project supervision served to corroborate the process of the reform implementation in the regions.

6 One PDO indicator was the increase in the rate of return (of an investment project) due to a better planning methodology. Even when some positive correlation may occur, the causality between the new planning methodology and the rate of return could be very difficult of being proven. Moreover, the indicator does not provide guidance if the rate or return should be measured ex ante or ex post (once the project is operative). For instance, some typical issues in project preparation such as the underestimation of cost and overestimation of demand can be avoided by the implementation of the new planning methodology. However, the rate of return may decrease during the project preparation if cost is higher and demand is lower. A second indicator was defined to capture the increase in market competition due to a new contract and process standardization measured by a Herfindahl index (HHI). While the contract and process standardization can lead to a more transparent and simple procurement process, the capacity to affect the market share of public contractors can be negligible or impossible to capture. Indeed, the entry of new contractors to the market of public works, if occurs, can take some time and be affected by other variables.

12

2.4 Safeguard and Fiduciary Compliance Limitations imposed by the Bank’s procurement rules contributed to the initial substantial lag in disbursements. The first disbursement took place in October 2010, more than two and a half years after the loan became effective. Initially, one of the main reasons for disbursement delays was the role played by Contraloría (auditing body of Chile) who added at least 6 months to the procurement process of each consulting work. Eventually, Contraloría agreed to work on a standard form for Bank-financed consultancies. Subsequently, and as the pace of implementation improved, the limitations were imposed by the Bank: i.e. matters related to anticorruption clauses in contracts and the limit of US$ 100,000 for contracts using the ChileCompra system.7 The Borrower repeatedly stressed the importance of revising the procurement procedures and the positive impact that it would have on disbursements, as several contracts could not be financed by the loan due to that particular limitation. On December 12, 2012 the Loan Agreement was revised to extend the original closing date to January 31st, 2014 and to update the Procurement and Consultant Guidelines to reflect the versions dated January 2011, which allows the use of framework agreements, known as “Convenios Marcos” in Chile, for the procurement of goods, works and non-consultant services. In addition, the Bank agreed to raise the maximum limits for using ChileCompra for consultant services, from US$100,000 to US$300,000. In addition, the definition of “Convenios Marco” was amended to include non-consultant services, and the Anti-corruption Guidelines dated October 15th, 2006 and revised in January 2011 was to apply to the Project to be consistent with the updated version of the Procurement and Consultant Guidelines. MOPW had adequate financial management systems, software, and trained staff. No systemic problems were identified during implementation. The rating of the project, however, remained as MS (moderately satisfactory) mainly because of the high rotation of the financial management staff. This issue impacted the preparation of financial reports, which presented problems and needed some corrections to be accepted. Contraloría, the National Audit Office, also made comments requesting some changes and improvements in the presentation of the information. Finally, the correction of some issues took time and a few issues were repeated in other reports.

2.5 Post-completion Operation/Next Phase The vehicle used for the implementation of the Institutional Strengthening Program of the MOPW (Programa de Modernización del MOP) ends at the end of June 2014. The long-

7 The hiring of consultant firms with estimated cost less than US$100,000 was to be generally carried out using national procedures which include advertisement and processing through the Chilecompra Portal: such procedures - that may include open competition are not consistent with the short listing procedures as specified in the Bank's guidelines but in Chile such procedures ensure transparency and quality service and therefore, the Bank allowed the direct contracting of the firms thus selected.

13

term sustainability of the achievements seems to be guaranteed by the fact that the program was a mean to improve the functioning of the different units of MOPW. That is, each unit involved in the program was the final recipients of the technical assistance and they are advancing independently with cementing their new role. The MOPW, however, is interested in continuing with institutional strengthening in two areas where the Bank might be requested to provide support in the near future: i) contract management in infrastructure concessions; and ii) institutional set up of the water sector. The first topic refers to the need to improve the oversight function of the concession unit and it was identified as a priority by the recently appointed management of MOPW. On the water sector, some progress was achieved during the program but there are other reforms that required further support and the timing of the project was not suitable for their implementation.

3 Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation The main objective of the TAL remains extremely relevant to Chile’s current agenda. The modernization program of MOPW remains very much in line with the government’s focus on (i) investment decisions based on programs fostering equity, territorial integration and regional development; (ii) a more equitable distribution of public-private risks and better oversight of the provision of services; and (iii) the shift towards a performance based infrastructure management as measured by users’ satisfaction. The recent reelection of President Bachelet who’s administration was the original champion of the modernization process, provides further enhancement to the relevance of the objectives which have been already mainstreamed in the operations of MOPW. Chile’s continued efforts to converge with OECD countries performance provide a favorable support for the reform. Finally, MOPW has an undisputed role in the Chilean economy as a vehicle for implementation of countercyclical policies. 8 The need to improve the efficiency and performance of the MOPW becomes even more important in the light of its increasing role as growth and employment generator. The objective of the loan also remains relevant for the achievement of the Bank strategic objectives, which emphasized the need to continue to focus on the modernization of the state through a close engagement with the Chilean authorities. Likewise the project objectives are fully consistent with those set in the Country Partnership Strategy (CPS) endorsed by the Board in May 2007 and in February 20109, contributing directly to objectives of supporting the efficiency of infrastructure and public services in addition to decentralization and regional development.

8 Chile has been cited as an example of how to implement adequate countercyclical economic policies. See for instance the article titled “Stimulating” published by The Economist on February 19, 2009 (available at http://www.economist.com/world/americas/displaystory.cfm?story_id=13145570) 9 World Bank 2011. Chile Country Partnership Strategy 2011-2016. Report No. Report No. 57989-CL

14

3.2 Achievement of Project Development Objectives (PDO) PDO Indicator 1: “New planning methodology for infrastructure functioning and fully incorporated in the investment decision of the MOPW” The project fully achieved all the intermediate institutional targets leading to MOPS’s full adoption of a novel integrated planning. The new methodology derived from a strategic long-term view of regional development and was brought down to the tactical selections of project with an integrated geographic perspective oriented to the competitiveness of Chile's productive sectors and the equitable and sustainable access to goods and services to all the country’s territories. The new methodology, now being used in 99% of the MOPW’s newly approved projects, includes a high level of interaction with other relevant stakeholders in the infrastructure sector and a societal or consumer voice in project planning. The integration of the investment programs along a supply chain is expected to substantially decreased coordination difficulties and inefficient expenditures. All MOPW’s projects included in Chile’s 2014 budget come form strategic investment programs evaluated using the new methodology. In 2010, with the new administration, the Bank was requested to pay increased emphasis on water resource management under the integrated planning activities. A large consultancy launched in 2011 produced a manual being successfully used for the planning and management of rainwater in large urban centers, thus contributing to the objective of improved planning in a one of MOPW’s critical priority areas. PDO Indicator 2: “Greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer responsive service targets”. This objective was partially achieved. The objective was fully achieved with respect to the incorporation of standard of services in concessions and civil work contracts. Progress in ensuring greater competition in the bidding process centered in a series of studies and proposed revisions to reduce the excessive restrictions that limited the inclusion of participants in the registry of consultants and construction companies, to increase the precisions of the bidding documents and to reduce discretionary decisions. The proposed regulations for public works incorporating the new proposals were presented to the Contraloría in 2012. Contraloría raised several comments on the document but stop short of its approval. Clearly this is the most contentious area were many strong interests are at stake and decisive political support is necessary for any meaningful change. Nevertheless, even without the actual formal revision of the Regulation of Public Works (Reglamento de Obras Públicas), some progress was made in the standard bidding documents, such as the handling of the two- envelope system, the improvement of the polynomial adjustment method and the dropping of the publication of MOPWs project budget. Further progress and full approval of a new Reglamento is a challenge left to the new Bachelet administration.

15

PDO Indicator 3: “Greater responsiveness to consumer concerns related to the levels of service; and greater oversight of performance by public sector service providers as well as concessionaires” This objective was fully achieved. This was probably one of the more complicated objectives in terms of resistance to a complete change in the conceptual and methodological model to implement a service- oriented approach to public works. After a process of identifying service levels and standards for the design, construction and exploitation of each type of works, these are being introduced as explicit obligations in an increasing number of concession contracts for airports, roads, hospitals, etc., actually surpassing the projections of the intermediate targets. By the end of 2013 MOPW had entered more than 120 contracts with explicit service standards which, in addition to several contracts in hospital and airport concessions, include performance-based and ad-measurement contracts comprising 13,371.53 km or 71.7% of the country’s road network. In addition MOPW successfully developed and implemented through surveys a system for the evaluation of user’s satisfaction in at least four types of services and is continuing with the task of developing the systems for other services under their responsibilities PDO Indicator 4: “Reduction in variance between values for construction/investment established at contract signature and final values at project execution.” The objective was fully achieved. Reductions in variance between costs for construction/investment established at contract signature and final values at project execution were confirmed by comparing contracts carried out before and after the full institutionalization of the new project management approach. The creation of Project Management Offices at the central and regional levels in 2010 and the full adoption of the “project manager” figure increased accountability and reduced atomized decisions. The new organization is being supported by the development of a solid digital platform to provide rapid information on project costs, contract management and/or design and construction issues. The effectiveness of the new project management system was fully proved during the reconstruction efforts following the 2010 earthquake, a fact that was instrumental in securing the new administration support for the project. Objective 5: “Normative and organizational structure that provides clear separation and definition of roles and responsibilities for planning, contracting, works execution and oversight” Partially achieved. The Concession Law, one of the targets in this component, was approved and made public as Law 20.410 on January 1st, 2010. However, the main laws changing MOPW’s organization (the Superintendence Law and the Organic Law of MOPW) were not pursued and were not considered a Government priority. Instead, MOPW is implementing a strong oversight effort through contract oversight systems that are already in place for concessioned airports (e.g. information screens, escalators, etc.) and for contract oversight of rural potable water.

16

3.3 Efficiency The project should be considered as highly cost effective since costs should be exceedingly compensated by improved capacity reflected in fiscal savings and efficiency gains. The MOPW manages a budget for 2014 of about US$ 3.3 billion of which about US$ 2.4 billion are allocated to investments. The cost of the project, at a total of US$ 29.15 million, represents less than 1% of the budget of one year, yet it was sufficient to introduce deep and permanent improvements in efficiency and service levels. The already confirmed reduction in the costs and execution times of contracts that are being derived from the adoption of an “integrated management system” would exceedingly compensate for the costs of the project. Additional gains, probably more sizable but more difficult to quantify, should be derived from the selection of projects that focus on equity and the social benefits that were previously ignored.

3.4 Justification of Overall Outcome Rating Rating: Satisfactory The ICR considers the overall outcome of the project as satisfactory since (a) project objectives remain highly relevant to the development of public infrastructure in Chile and the government’s pursue of growth with equitable regional development; (b) the project demonstrated satisfactory achievements of its DOs, especially in regard to full institutionalization of integrated planning, efficient performance based management of projects and services, a more substantive oversight of the provision of services, and the inclusion of user satisfaction as a key measure of performance; and (c) the project passed the test of approval of administrations covering a wide range of the country’s political spectrum and is fully embedded in the modus-operandi of MOPW. Moreover, the project was instrumental in bringing about the modernization of MOPW. Without the project, and thus the Bank involvement, it is unlikely that the Modernization Program would have been kept in the agenda and in the country’s budget. Even when the financial contribution of the loan was at times minimal, MOPW insisted that the continuous presence of the project was vital for the continuation of the reform program. The satisfactory rating, however, does not coincide with the last ISR’s rating of Moderate Satisfactory, which unfortunately gives inordinate weight to the poor disbursement performance of the project rather than to the achievements of the development objectives and was not revised when disbursements picked up.

3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development

The operation built the foundations for generating an indirect impact on poverty alleviation by inducing greater efficiency, effectiveness and accountability in the utilization of public resources. It also broadened participation in public infrastructure investment, and fostered transparency and accountability in infrastructure service delivery (principally by enhancing oversight, and access to information). In addition, the

17

project promoted instruments such as strategic environmental assessments that have already fostered (and will continue to foster) the integration of environmental aspects into the planning process. All the above, will help Chile to achieve OECD standards for public works projects that contribute to sustainable development. (b) Institutional Change/Strengthening The operation’s institutional impact was and will be substantial. It established a new institutional paradigm for the MOPW. This includes a new planning methodology designed to provide an integrated strategic vision for all types of infrastructures. The operation introduced the profile of the project manager which was key to create an integrated project management approach that covers the full life cycle of projects. It also designed a Project Management Office (PMO) to improve the management and oversight of the portfolio of projects under the responsibility of the MOPW. The operation further developed an integrated technological platform that is building a database for monitoring the performance of concessions and public works. (c) Other Unintended Outcomes and Impacts (positive or negative) One interesting outcome that was not included in project design was the establishment of the Public Works Institute in the MOPW. The reform process required a substantial amount of training on planning, project management, contract standards, concessions, information technology, etc. As it was found that many of the professional being sought in academia for providing training were members of the MOPW, the government decided to create the in-house training Institute, a move that is resulting in the continuous upgrading of the professional skills in the Ministry. Finally, the relevance of this project for the Bank agenda should be highlighted as Chile is an example for the Bank in the area of infrastructure service provision. The Bank has constantly relied on Chile to illustrate to other countries in different regions how infrastructure reforms should be conducted, in particular in the area of attracting private sector financing and operators through public private partnerships. World Bank staff and public officials from East Asia and South East Asia have conducted study tours to learn from the Chilean experience. Thus, the Bank will benefit significantly from the MOPW modernization process as it will provide valuable lessons for other countries that request the assistance of the Bank to follow a similar path of reforms.

3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops

N.A.

4 Assessment of Risk to Development Outcome

Rating: Negligible

18

The sustainability of the achievements of the TAL operation is deemed substantial, given that most of the objectives have been fully institutionalized and became an integral part of MOPW’s modus operandi. While to continue to move forward and deepen the modernization efforts, particularly for the still incomplete legal and regulatory changes, would clearly require the presence of a reform-minded management team and strong political support progress made to date is most likely to endure changes in leadership. There are several factors that mitigate the risk associated with potential changes in leadership. In addition to the strong ownership of the new MOPW’s vision within the institution, risks are reduced by: i) the already strong support from different administrations; (ii) the users pressures for reform resulting from improved performance; iii) the heightened role of public consultation organisms; and iv) the strong ownership within MOPW’ of its strategic plans. Finally, the risk of cultural resistance and vested interests within the institution opposing project implementation, probably the most tangible one during the early years of project implementation, has been largely contained. A well-established organizational culture and self-interest inertia originally resisted institutional change within the MOPW. Mitigation measures aimed at fostering stakeholders’ participation and promoting change management served to counter-balance this resistance to change. The gradual integration of new profiles (such as the project manager) and the progressive definition of new institutional bodies for supervision and coordination, also served to mitigate this risk. Addressing the risks imposed by interests outside the institution to move forward with the legal and regulatory changes still necessary to fully complete the modernization program will require further work, particularly to strengthen the communication channels between the executive and the legislative branches of government to gather support for the modernization of the Ministry’s legal structure.

5 Assessment of Bank and Borrower Performance

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: moderately satisfactory The rating for Bank performance during the preparation phase is moderately satisfactory. The justification for this rating is as follows: acknowledging the complexity of the reform, the Bank assigned a highly qualified team with extensive experience in the relevant policy areas. The team was able to maintain a close collaboration with the Minister and his team from the beginning, and responded promptly and with high levels of flexibility to their requests for assistance in certain topics (particularly the drafting of the Concessions and Superintendence of Public Works Law) at the outset of the reform process. Experts from different parts of the world participated actively in preparation

19

missions and workshops, providing ad-hoc advice directly to the Minister and his cabinet on specific issues. The team’s appropriate skill mix helped deliver a well-designed project and provided value added to the Borrower. The Bank’s performance assisting the Borrower in identifying and appraising the project to ensure consistency with the CAS and with government’s priorities, and in a manner that would ensure attainment of expected outcomes, was also very valuable. Performance at entry could have been satisfactory if design of the M&E framework had included more relevant and measurable indicators. (b) Quality of Supervision Rating: moderately satisfactory Traditional supervisory activities to monitor progress toward mid-term outcomes were carried out and missions traveled to the field more than twice a year, with additional regular Bank support provided directly by the Project team. Beyond the focus on regular supervision matters, there was a strong interest on the part of the Bank in assessing impact of project activities and, in doing so, maintaining central government focus in the reform agenda. To ensure a feasible impact analysis, objective indicators, and the availability of the required data to monitor their evolution throughout project implementation, the M&E framework for the project was revised with Chilean counterparts. In addition the supervision team was very proactive and successfully intervened to keep the project alive when the Government support faltered. Supervision missions did play the productive envisaged role of “honest broker” between MOPW and MOF, and all of the MOPW officials interviewed for this ICR coincided in that the presence of the Bank was instrumental in saving the project when the political support was at its lowest. The less than satisfactory rating is due to the following: (i) the revision of the result framework was never formalized. Although a problem of form more than substance, the change would have improved the rather confusing presentations of the result framework in the ISRs, which mixed the original and revised indicators; and (ii) the supervision team relied heavily on disbursements rather than on achievements to evaluate project performance. (c) Justification of Rating for Overall Bank Performance Rating: moderately satisfactory. Based on the above, the ICR considers the overall Bank performance as moderately satisfactory.

5.2 Borrower Performance (a) Government Performance Rating: satisfactory Government performance is rated satisfactory due to the commitment and involvement in MOPW’s reform initiative, as evidenced by: (i) the active participation of the Ministry of Finance from project inception. The positive commitment of the Ministry of Finance, given the structure of power in Chile, is a sine qua non condition for the successful

20

implementation of any institutional reform in Chile; (ii) the explicit support from the highest levels of government to the reform program in public speeches and press releases; and (iii) the valuable contributions and participation from staff of other public agencies in multiple initiatives of the reform program (in particular, involvement of multiple agencies in the seminars carried out by MOPW under the Integrated Planning policy area). On the other hand, the poor budget execution of the reform program during its first two years prompted MOF to reduce budget allocations at a time in which the program was starting to take off. These shortages of funds contributed to slowing down the implementation of the project. (b) Implementing Agency or Agencies Performance Rating: satisfactory Implementing agency performance is rated as satisfactory. From the beginning of project implementation, MOPW has assigned a qualified team to lead the initiatives under each policy area. These teams have worked hard in the production of terms of reference, and in the organization of trainings and other events; strengthening the dialogue and building coordination stages among Ministry areas that had traditionally worked in relative isolation. The Project Coordinating Unit has complied with all of its fiduciary responsibilities in a satisfactory manner, and played an important role in providing relevant information to assess the attainment of outputs established for the ICR. (c) Justification of Rating for Overall Borrower Performance Rating: satisfactory Based on the above, the ICR considers the Borrower performance satisfactory.

6 Lessons Learned

• An intensive early period of communication and participation as part of the process of problem recognition and problem definition is fundamental for ensuring support to institutional changes. MOPW underestimated the need for launching an effective process of communication for gathering staff backing for the modernization program. Even once the initial concerns for job stability were addressed, the acceptance of new conceptual and behavioral norms in a big and powerful institution proved a difficult task that was eventually overcome by moving from a “diffusion of communications” to a more participatory approach for setting common goals.

• The endorsement and strong ownership of the leaders of an institution are crucial to ensure the continuity and success of major reforms. After Minister Bitrán, who was the original reform champion, left office in January 2008, immediate support by his successor, Minister Bitar, was decisive to providing continuity to the modernization program. This endorsement was critical at a time

21

when support to the reform was waning within the MOPW due to weaknesses in the internal communication of the reform’s principles and objectives among ministry’s stakeholders. The priority of the program was further and definitely confirmed in 2010 when the administration changed hands, thus ensuring its continuity.

• The cost curve of institutional reform should take into account the lead- time necessary for consensus building and task preparation. The cost curve estimated for the TAL was front- loaded. During the initial years, however, the needs focused on internal deliberations on visions, strategies and goals. These required the help of individual consultants to define future assistances and develop complex Terms of Reference for more resource consuming consultancies. Thus, actual need for funds increased with time (rather than decrease as projected). This discrepancy between actual and expected costs was at times a source of project underfunding.

• The Bank should evaluate more closely the procurement thresholds or consider more flexible procurement rules for countries with a proven record of controlling corruption. The procurement limits originally established for the use of ChileCompra, were at the center of the poor disbursement record of the project. Its revision and the use of limits more according to the procurement record of Chile (which probably should have taken place as soon as the problem was identified), proved instrumental in turning around disbursement s performance.

7 Comments on Issues Raised by Borrower/Implementing Agencies/Partners

(a) Borrower/implementing agencies No comment was provided – see Annex 7. (b) Cofinanciers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society)

22

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate (USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

1. Integrated Planning 7.59 4.12 -45.7% 2. Standards, Contracts, and

Competition 4.85

1.22 -74.9%

3. Integrated Project Management 4.00

6.28 +57.1%

4. Regulatory Strengthening (fiscalización) 6.23 1.74 -72.1%

5. Organizational Reform 6.50 9.13 +40.5% 6. Administrative costs and

contingencies 6.19 6.66 +7.5%

Total Project Cost 35.36 29.15 -17.6%

(b) Financing

Source of Funds Appraisal Estimate

(USD millions)

Actual (USD

millions)

Percentage of Appraisal

Borrower 23.66 20.80 -12.1% International Bank for Reconstruction and Development 11.70 8.35 -28.6%

23

Annex 2. Outputs by Component This annex lists the main goals achieved in each component. For a more detailed description including intermediate taken steps, studies, and training, etc. please consult the Modernization Program progress reports that MOPW prepared semiannually for the World Bank. COMPONENT 1: INTEGRATED PLANNING (Appraisal estimated cost: US$ 7.59 million; actual cost: US$ 4.12 million ) INFRASTRUCTURE INTEGRATED PLANNIG: Implementation of a full planning process for infrastructure services, which support the decision making process for investment by the Ministry of Public Works aligned with the strategic criteria and the vision of the country. A planning process which also strengthens competitiveness, which enhances the productive development of each territory, and ensures its inhabitants equal access to goods and services in a sustainable and integrated manner.

OBJECTIVES ACHIEVEMENTS

• Prospective Development o Development of Country Vision

2020 and its relation to infrastructure

o Design of policies for long-term planning concerning: land use, relocation of settlements, transportation, water resource management and sustainable territorial development.

o Production cluster’s analysis and its impact on infrastructure

o Elaboration of studies and delivery of training related to the development of a sustainable and integrated approach to infrastructure planning.

• Planning Model and Process o Installation of participatory

Initiatives o Design of methodologies to

promote a participatory planning process for infrastructure investments

• Transversal Support Systems o Development and

implementation of an information system and a database for a Territorial Information System “Sistema de

• Set up of the Int

Document of The World Bank · CURRENCY EQUIVALENTS (Exchange Rate Effective June 11, 2014) Currency...

Documents

Transcript of Document of The World Bank · CURRENCY EQUIVALENTS (Exchange Rate Effective June 11, 2014) Currency...