The World Bankdocuments.worldbank.org/curated/en/416971488305832727/pdf/ICR00003580... · the world...

i

Document of The World Bank

Report No: ICR00003580

IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-12516)

ON A

PROPOSED GRANT

IN THE AMOUNT OF US$5.0 MILLION

UNDER THE MULTI DONOR TRUST FUND FOR KHYBER PAKHTUNKHWA, FEDERALLY ADMINISTERED TRIBAL

AREAS AND BALOCHISTAN

TO THE

ISLAMIC REPUBLIC OF PAKISTAN

FOR A

BALOCHISTAN DISASTER MANAGEMENT PROJECT

December 7, 2016

Disaster Risk Management/Climate Global Practice for Social, Urban, Rural and Resilience South Asia Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

ii

CURRENCY EQUIVALENTS

(Exchange Rate Effective November 30, 2016)

Currency Unit PKR 1.00 = US$ 0.0095 US$ 1.00 = PKR 104.83

FISCAL YEAR

2017

ABBREVIATIONS AND ACRONYMS

ADB Asian Development Bank BDMP Balochistan Disaster Management Project CBDRM Community-Based Disaster Risk Management CERC Contingency Emergency Response Component CPS Country Partnership Strategy DMIS Disaster Management Information System DRM Disaster Risk Management FATA Federally Administered Tribal Areas GoB Government of Balochistan GSP Governance Support Project KP Khyber Pakhtunkhwa MDTF Multi Donor Trust Fund MTR Mid-term Review NDMA National Disaster Management Authority PCNA Post Crisis Needs Assessment PDMA Provincial Disaster Management Authority PEOC Provincial Emergency Operations Center PGEBP Promoting Girls Education in Balochistan Project PMU Project Management Unit R3T Rapid Rescue and Response Team SOPs Standard Operating Procedures UNDP United Nations Development Program WFP World Food Program

iii

Senior Global Practice Director: Ede Jorge Ijjasz-Vasquez

Practice Manager: Bernice K. Van Bronkhorst

Project Team Leader: Haris Khan

Implementation Completion and Results Report Team Leader:

Maha Ahmed

iv

PAKISTAN Balochistan Disaster Management Project

CONTENTS

Data Sheet A. Basic InformationB. Key DatesC. Ratings SummaryD. Sector and Theme CodesE. Bank StaffF. Results Framework AnalysisG. Ratings of Project Performance in ISRsH. RestructuringI. Disbursement Graph

1. Project Context, Development Objectives, and Design .............................................. 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 5 3. Assessment of Outcomes .......................................................................................... 12 4. Assessment of Risk to Development Outcome ......................................................... 27 5. Assessment of Bank and Borrower Performance ..................................................... 29 6. Lessons Learned ....................................................................................................... 32 7. Comments on Issues Raised by Borrower, Implementing Agencies, Partners ......... 33 Annex 1. Project Costs and Financing .......................................................................... 35 Annex 2. Outputs According to Component ................................................................ 36 Annex 3. Grant Preparation and Implementation Support and Supervision Processes 40 Annex 4. Summary of Borrower’s Implementation Completion and Results Report and Comments on Draft Report ........................................................................................... 41 Annex 5. List of Supporting Documents ...................................................................... 45

v

A. Basic Information

Country: Pakistan Project Name: Balochistan Disaster Management Project

Project ID: P127253 L/C/TF Number(s): TF-12516

ICR Date: 01/05/2017 ICR Type: Core ICR

Lending Instrument: SIL Grantee: GOVERNMENT OF PAKISTAN

Original Total Commitment:

USD 5.00M Disbursed Amount: USD 5.00M

Revised Amount: USD 5.00M

Environmental Category: C

Implementing Agencies: Provincial Disaster Management Authority (PDMA) Balochistan

Cofinanciers and Other External Partners: B. Key Dates

Process Date Process Original Date Revised / Actual

Date(s)

Concept Review: 01/26/2012 Effectiveness: 07/19/2012

Appraisal: Restructuring(s):

Approval: 06/27/2012 Mid-term Review: 03/17/2014 05/06/2014

Closing: 06/30/2015 06/12/2016 C. Ratings Summary C.1 Performance Rating by ICR

Outcomes: Satisfactory

Risk to Development Outcome: High

Bank Performance: Satisfactory

Grantee Performance: Moderately Satisfactory

C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings

Quality at Entry: Satisfactory Government: Moderately Satisfactory

Quality of Supervision: Satisfactory Implementing Agency/Agencies:

Moderately Satisfactory

Overall Bank Performance:

Satisfactory Overall Borrower Performance:

Moderately Satisfactory

vi

C.3 Quality at Entry and Implementation Performance Indicators Implementation

Performance Indicators

QAG Assessments (if any)

Rating

Potential Problem Project at any time (Yes/No):

No Quality at Entry (QEA):

High

Problem Project at any time (Yes/No):

No Quality of Supervision (QSA):

High

DO rating before Closing/Inactive status:

Satisfactory

D. Sector and Theme Codes

Original Actual

Sector Code (as % of total Bank financing)

Sub-National Government 100 100

Theme Code (as % of total Bank financing)

Natural disaster management 100 100 E. Bank Staff

Positions At ICR At Approval

Vice President: Annette Dixon Annette Dixon

Country Director: Patchamuthu Illangovan Patchamuthu Illangovan

Practice Manager/Manager:

Bernice K. Van Bronkhorst Bernice K. Van Bronkhorst

Project Team Leader: Haris Khan Haris Khan

ICR Team Leader: Maha Ahmed

ICR Primary Author: Maha Ahmed F. Results Framework Analysis

Project Development Objectives (from Project Appraisal Document) To strengthen the capacity of PDMA Balochistan to prepare for and respond to natural disasters. Revised Project Development Objectives (as approved by original approving authority) Not applicable

vii

(a) PDO Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : Improved disaster response mechanism at PDMA Balochistan.

Value quantitative or Qualitative)

0

SOPs for disaster response prepared and operationalized Number of drills and simulation exercises completed to assess operational coordination, preparedness and emergency protocols.

NA

SOPs were prepared, tested and formally notified by GoB. 2 drills conducted based on lessons learned and international best practice.

Date achieved 07/19/2012 07/19/2012 06/30/2014

Comments (incl. % achievement)

The SOPs were based on international best practices and tested during the first simulation drill. The drills enabled PDMA Balochistan to test the full spectrum of disaster response including coordination, communication and mobilization of relief goods.

Indicator 2 : Increased awareness of the hazard and risk environment in Quetta


0

DMIS established and populated with data collected during the hazard and risk assessment

NA

Hazard risk assessment completed for Quetta city and geo-referenced data of various sectors captured in DMIS to serve as baseline.

Date achieved 07/19/2012 07/19/2012 03/16/2015


Achieved. This data is already being used by the PDMA to better prepare for cyclical flash floods. DMIS linked to organisations at national level to ensure data sharing.

viii

Indicator 3 : Enhanced capacity at PDMA Balochistan for implementing CBDRM initiatives


0

15 master trainers trained 4 training sessions for responders in selected union councils in Quetta 10,000 instruction materials on CBDRM available for dissemination to communities.

NA

20 master trainers trained 5 training sessions organized for responders in selected union councils 20,000 instruction materials distributed across 12 districts

Date achieved 07/19/2012 07/19/2012 04/30/2014


Curriculum for training sessions developed based on national and international best practices. Technical specialists trained staff of GoB, local NGOs and staff of public schools. CBDRM material adopted as national standard by NDMA.

(b) Intermediate Outcome Indicator(s)

Indicator Baseline Value

Original Target Values (from

approval documents)

Formally Revised

Target Values

Actual Value Achieved at

Completion or Target Years

Indicator 1 : DMIS Database established

Value (quantitative or Qualitative)

0

DMIS database established

NA

DMIS established and data generated from hazard and risk assessment of Quetta collated in DMIS. Geo-referenced data of various sectors collated in DMIS to serve as baseline.

Date achieved 07/19/2012 07/19/2012 06/30/2014

ix


Indicator 2 : Achieved. Enhanced communications systems in place at the Provincial Emergency Operations Centre


0

Emergency communications equipment operationalized and integrated into response regime.

NA

Improved communications systems in place.

Date achieved 07/19/2012 07/19/2012 03/16/2015


Achieved. Communications system includes short messaging, service alert system, call management system, fleet tracking system, communication SOPs and directory listings.

Indicator 3 : Achieved. Balochistan DRM plan 2008 reviewed and updated


0

Draft Strategic updates agreed on after discussion with stakeholders

NA

Balochistan DRM plan 2014-19 prepared for presentation and approval at next Provincial Disaster Management Commission, which Chief Minister chairs.

Date achieved 07/19/2012 07/19/2012 04/30/2014 Comments (incl. % achievement)

Achieved.

Indicator 4 : PDMA capacity assessment undertaken and enhancement plan prepared


0

Capacity enhancement plan discussed with stakeholders and finalised

NA

Capacity assessment completed and capacity enhancement plan implemented through

x

procurement of various equipment as indicated in plan.


Achieved

Indicator 5 : SOPs for PDMA on disaster response prepared


0

Proposed systems tested and plans revised

NA

SOPs developed, tested during the first drill, finalized and formally notified by the Government


Achieved

Indicator 6 : Number of drills and simulation exercises completed to assess operational coordination, preparedness and emergency protocols


0

2

NA

2

Date achieved 01/31/2013 01/31/2013 06/30/2015


Achieved. The drills, based on international lessons learned and best practices aimed at testing the coordination and response capacity of PDMA Balochistan.

Indicator 7 : Number of exposure visits by PDMA Balocistan staff undertaken and knowledge products produced


0

3

NA

4

Date achieved 01/31/2013 02/28/2013 06/30/2015


Two in Pakistan organized by the National Institute of Disaster Management and two organized in Thailand by the Asian Disaster Preparedness Centre

Indicator 8 : Scoping exercise completed and methodology for hazard and risk assessment finalised

xi


0

Methodology for hazard and risk assessment finalised

NA

Methodology finalized, hazard and risk assessment for Quetta completed and data entered into DMIS to serve as baseline.


Achieved.

Indicator 9 : Data collected and collated in DMIS repository


0

Collection in CMIS of data generated through hazard and risk assessment of Quetta completed

NA

Data generated from hazard and risk assessment of Quetta collated in newly established DMIS. Geo-referenced data of various sectors collated in DMIS to serve as baseline.


Achieved.

Indicator 10 :

Information collected during hazard and risk assessment on sources of risk available to all departments and agencies responsible for development planning in provincial capital


0

Results from hazard and risk assessment published and disseminated to departments and agencies responsible for

NA

Hazard and risk assessment of Quetta completed, baseline data collated in DMIS will be updated through national risk assessment sharing platform.

xii

development planning

Date achieved 01/31/2013 02/28/2013 12/31/2014


Achieved. DMIS is linked to SUPARCO, Geological Survey of Pakistan and other national agencies to ensure coordination and regular updation.

Indicator 11 : Provincial CBDRM Programme developed


0

Provincial CBDRM programme agreed upon with stakeholders and finalised

NA

Provincial CBDRM plan developed and available at PDMA, along with training modules.

Date achieved 01/31/2013 02/28/2013 06/30/2014


Plan formally notified and published on PDMA website. Roll-out of plan has been initiated with support from UN agencies.

Indicator 12 : Number of training sessions conducted by master trainers in selected union councils in Quetta


0

5

NA

5

Date achieved 01/31/2013 02/28/2013 12/31/2014


Achieved. The union councils were selected in 12 districts where PGEBP was being implemented. The schools supported through PGEBP were used as a platform to launch the CBDRM Plan.

Indicator 13 : Logistics plan developed for emergency stockpiles in selected districts


0

Final checklists for emergency stockpile equipment developed

NA

Logistics plan developed; checklist for emergency stockpiles part of plan


Achieved

Indicator 14 : Number of pilot stockpiles established in Quetta based on logistics plan

xiii


0

2

2

Date achieved 01/31/2013 02/28/2013 03/31/2016


Achieved. These stockpiles are based on standard lists and contain relief equipment and goods.

Indicator 15 : Instruction materials on CBDRM available for dissemination to communities


0

10,000

NA

20,000

Date achieved 01/31/2013 02/28/2013 03/31/2016


Achieved. These materials were distributed through the MDTF-supported PGEBP across 12 districts in Balochistan. They have been adopted as the national standard by the National Disaster Management Authority post the October 2015 earthquake.

G. Ratings of Project Performance in ISRs

No. Date ISR Archived

DO IP Actual

Disbursements (USD millions)

1 11/04/2012 Satisfactory Satisfactory 0.00 2 05/21/2013 Moderately Satisfactory Moderately Satisfactory 0.35 3 01/29/2014 Satisfactory Satisfactory 1.87 4 06/22/2014 Satisfactory Satisfactory 2.97 5 12/22/2014 Satisfactory Moderately Satisfactory 2.97 6 06/15/2015 Satisfactory Satisfactory 4.71 7 12/22/2015 Satisfactory Satisfactory 5.00

H. Restructuring (if any) Not Applicable

xiv

I. Disbursement Profile

1

1. Project Context, Development Objectives and Design (this section is descriptive, taken from other documents, e.g., PAD/ISR, not evaluative)

1.1 Context at Appraisal (brief summary of country and sector background, rationale for Bank assistance) 1. Located in the southwest of Pakistan, the province of Balochistan covers 347,190 square kilometers and has an estimated population of approximately 13.16 million. It is one the most underdeveloped regions of Pakistan, with approximately 44 percent of the population living below the poverty line. The region is geographically isolated and has a sparsely distributed, dominantly rural population; population density is approximately 38 people per square kilometer. Access to health and sanitation is poor; 67 percent of the population has no access to sanitation, and 37 percent has no formal source of clean drinking water, relying mainly on ponds, canals, and streams. Balochistan is also prone to natural disasters such as earthquakes and floods. It suffered significantly from the 2000-02 drought, which caused gross domestic product (GDP) growth rates to fall from 4 percent to 2.5 percent. In 2007, Cyclone Yemyin severely affected large areas of the province, resulting in more than 400 deaths and internally displacing more than 150,000 people. According to a damage and needs assessment that the World Bank and the Asian Development Bank (ADB) conducted, total estimated losses in Balochistan amounted to US$394 million, or approximately 4 percent of provincial GDP. More recently, the 2010-11 floods have had a major effect, with damages and losses of US$620 million, or 6 percent of provincial GDP. Balochistan is particularly prone to earthquakes. In September 2013, earthquakes of 7.7 and 7.2 magnitudes on the Richter scale hit Awaran and adjacent districts in the southern part of the province. It is estimated that 399 people were killed and 599 injured. Balochistan has experienced several of high-magnitudes earthquakes, including in Mashkel (7.8 magnitude, April 2013), Dalbandin (7.2 magnitude, January 2011), Ziarat (6.4 magnitude, October 2008), and Harnai (7.1 magnitude, February 1997). Balochistan lies at the junction of the Indian and Eurasian tectonic plates and on several active fault lines. These disasters have damaged large-scale infrastructure and housing and interrupted vital government services. 2. Two main factors compound the difficulty of preparing for and responding to disasters in Balochistan. First, low population density combined with sparsely populated settlements spread across difficult terrain hinder access to those affected by disasters. This increases the complexity and cost of hazard mapping, estimating risk exposure, and the cost of relieve provision, particularly in the immediate aftermath of a disaster. Given that Balochistan’s socioeconomic development lags behind that of other provinces and that a large section of the population is near or below the poverty line, disasters can push these vulnerable people deeper into poverty. 3. Second, government capacity to address challenges in the aftermath of natural disasters remains weak in terms of institutional structures and organizational capacity. Over the last decade, at the national level, the government of Pakistan has worked to change its approach to addressing natural disasters gradually—managing risk before a disaster occurs instead of responding after the fact. The promulgation of the National Disaster Management Ordinance of 2006, which established the National Disaster Risk Management Framework, reflects this more-proactive approach to disaster risk management (DRM). The National Disaster Management (NDM) Act, promulgated in 2010 established the National Disaster Management Authority (NDMA) and provides for the establishment of Provincial and District Disaster Management Authorities (PDMAs and DDMAs). This is in line with the basic tenets of the 18th Amendment, promulgated in 2010, giving greater responsibility to the provinces to prepare for and respond to disasters.

2

4. Although the NDM Act (2010) led to the creation of PDMAs, significant challenges limit the effectiveness of these entities. First, operationalization of the PDMAs according to the NDM Act was not straightforward and required commitment of the provincial government. Such commitment may not have been readily forthcoming in an environment in which the government’s focus traditionally has been to respond to disasters using the limited resources available to provincial governments. Second, the PDMAs did not have an overarching strategic plan or guidelines to govern disaster management and response mechanisms. Such plans are essential to guide government interventions to prepare for and mitigate the risks associated with disasters, manage citizens’ expectations, and clarify roles and delineate responsibilities within the provincial government as well as between the NDMA and PDMAs. Third, specifically in the case of Balochistan, the PDMA was established with a total of five staff members, which was insufficient to undertake the day-to-day management of the organization. 5. The government of Balochistan (GoB) approached the World Bank in 2010 to help strengthen PDMA Balochistan and build its capacity to prepare for and respond to natural disasters. The World Bank responded to this request through the Balochistan Disaster Management Project (BDMP), budgeted at US$5 million and funded through the Khyber Pakhtunkhwa (KP), Federally Administered Tribal Areas (FATA), and Balochistan Multi Donor Trust Fund (MDTF), established in 2010. At the time of its inception, the BDMP was aligned with Pillar 4 of the MDTF, (building capacity and strengthening institutions). With the finalization of the MDTF Engagement Strategy in 2011, the BDMP was aligned with Pillar 2 (policy reform and governance). 6. BDMP was closely aligned with the Country Partnership Strategy (CPS) 2010, particularly after the back-to-back floods of 2010-11, which underscored the importance of the disaster risk management agenda and the need for institutionalization of improved response and coordination mechanisms at the federal and provincial levels. Previously, this issue had been addressed in a generic manner, under the overall umbrella of reducing vulnerability through strengthening the existing safety net system for vulnerable populations and developing emergency health service delivery during disasters. BDMP being the first stand-alone engagement in the country, dedicated to DRM, was included as part of efforts at the provincial level. Furthermore, the Bank’s CPS 2015 incorporated disaster risk management beyond emergency response by including activities related to increasing resilience through understanding disaster risk and strengthening DRM and early warning systems.

1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 7. The objective of the project was to strengthen the capacity of PDMA Balochistan to prepare for and respond to natural disasters. This involved the strengthening of institutional capacity, systems and processes that will enable PDMA Balochistan to more effectively prepare for and respond to natural disasters. Increased preparedness would also be achieved through improved understanding of hazards vulnerabilities as well as a more-robust disaster response infrastructure. 8. This is the same as the objective stated in the Grant Agreement, on which the ICR is based. 9. The project had three PDO indicators with six criteria used to evaluate and assess performance. These include: 10. The key PDO indicators include:

Improved disaster response mechanism at PDMA Balochistan

3

Increased awareness of the hazard and risk environment in Quetta1 Enhanced capacity at PDMA Balochistan for implementing community-based DRM (CBDRM)

initiatives

1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/ justification 11. Neither the PDO nor the key indicators were revised during implementation.

1.4 Main Beneficiaries, (original and revised, briefly describe the "primary target group" identified in the PAD and as captured in the PDO, as well as any other individuals and organizations expected to benefit from the project) 12. The project was aimed at building the capacity of PDMA Balochistan to prepare for and respond to natural disasters and had two main beneficiaries. The primary beneficiary was the GoB, which directly benefited from the intervention through increased awareness of disaster risk management at all levels of government. By strengthening the capacity of the PDMA, which is a new organization established to manage natural disasters, the project contributed to making the DRM effort more effective and efficient in Balochistan while creating greater visibility and legitimacy of the GoB. Secondarily, the project also benefited the people of Balochistan residing in the risk-prone areas of the province, who would benefit from improved service delivery, especially those living in Quetta and surrounding areas.

1.5 Original Components (as approved) 13. The project had four components. The first sought to strengthen the capacity of the PDMA to respond to natural disasters, and the second and third components focused more on improving preparedness to manage natural disasters. The fourth component was built into project design to ensure quick reallocation of resources in any event, which would require a quick response. Component 1: Institutional Strengthening of PDMA Balochistan (US$2.5 million) 14. This project component strengthened the institutional, technical and operational capacity of PDMA Balochistan to respond to natural disasters, which is a key element of the PDO. The component was divided into the following subcomponents. 15. Subcomponent 1a: Development of Institutional DRM Capacity for PDMA Balochistan. Under this subcomponent, the existing DRM institutional structure would be assessed in relation to the arrangements envisioned under the NDM Act 2010. This included analysis of existing operational capacity while identifying gaps and bottlenecks. Technical assistance was provided for the development of standard operating procedures (SOPs) to guide response operations of the agency after a natural disaster. 16. Subcomponent 1b: Strengthening of the Emergency Response System. Under this subcomponent, an effective response system was developed by strengthening the Provincial Emergency Operations Center (PEOC). Activities financed included a disaster management information system (DMIS) and the establishment of a rapid rescue and response team (R3T) within the PDMA. Fully equipping the PEOC

1 Quetta city is the capital of Balochistan province.

4

implied that more-effective communications and data analysis to manage disaster response and systems would be synchronized with national systems to promote information and data sharing. 17. Subcomponent 1c: Project Implementation. This subcomponent financed the costs associated with project coordination, implementation, monitoring, and evaluation. The expenditures financed under this subcomponent included training, consultant services (including for audits), equipment, operating costs (including project staff allowances for civil servants working on the project2), and expenditures necessary to perform other activities associated with project implementation, such as procurement-related complaint handling, third-party supervision, and establishment of a project-related grievance redress mechanism. Incremental operating costs under Component 4 would also be financed under this subcomponent. The project aimed to use internal resources and systems at PDMA Balochistan, and no independent project management unit (PMU) would be established. It was envisaged that this structure could be augmented with supplementary capacity support in critical areas as required, which could entail in-kind contributions from counterparts in terms of counterpart staff time and office spaces. Component 2: Hazard and Risk Assessment (US$1.0 million) 18. This component sought to build the capacity of the PDMA to better prepare for disasters by enhancing the general understanding of risks and vulnerabilities that the Province is exposed to. BDMP would develop a methodology for hazard and risk assessments, based on consultations with relevant actors and agencies and would be piloted in the provincial capital of Quetta. Understanding risks and hazard vulnerabilities is essential in i) mapping out geographical areas that can potentially be affected by natural disaster; and, ii) the number of people that could be impacted by a natural disaster. An improved understanding of the hazard and risk environment would facilitate the development of a robust disaster management and response infrastructure. Component 3: Development and Piloting of CBDRM Program (US$1.5 million) 19. This component aimed at developing a disaster risk management framework to strengthen capacity at the community level to prepare for and respond to disasters. Ensuring community involvement in disaster mitigation and response is essential for a comprehensive and holistic response in the immediate aftermath of a crisis because the community members are the most affected by but also the ‘first responders’ to a disaster. The CBDRM plan had three main aims: build and foster capacity at the community level to respond to disasters, create awareness amongst community members of how to use informal coping mechanisms in an efficient and effective manner, and create a minimum level of standards of CBDRM training. With this plan, the PDMA was able to ensure that any training delivered in the province followed the minimum requirements outlined in the program, enhancing uniformity of trainings across communities. Component 4: Contingent Emergency Response Component (US$0) 20. This zero-sum component was built into the overall design of the project to ensure flexibility in reallocating3 resources in the event that government priorities changed or evolved or the occurrence of a disaster, which could warrant immediate response. This component gave the GoB the flexibility to

2 Such project staff allowances will be financed in accordance with applicable government rules and regulations and, unless the Bank agrees otherwise, only for a period of 24 months after effectiveness of the Grant Agreement. 3 Such a reallocation would not constitute a formal Project restructuring, as permitted under the particular arrangements available for contingent emergency response components (ref. Including Contingent Emergency Response Components in Standard Investment Projects, Guidance Note to Staff, April 2009, footnote 6).

5

recategorize resources to partially cover emergency response and recovery costs. This component could also be used to channel additional funds should they become available as a result of an emergency.

1.6 Revised Components 21. Components were not revised during implementation of the project.

1.7 Other significant changes (in design, scope and scale, implementation arrangements and schedule, and funding allocations) 22. The project closing date was extended by 12 months, from June 30, 2015 to June 12, 2016. The MDTF Steering Committee extended the overall trust fund three times, to June 30, 2016. While project activities could have been completed by the original closing date, the BDMP closing date was extended till June 12, 2016 to undertake additional training activities that helped strengthen the project’s contribution to the PDO, utilize unused resources to prepare for a potential phase II of the project and finalize outstanding payments against contracts that had been delayed4.

2. Key Factors Affecting Implementation and Outcomes

2.1 Project Preparation, Design, and Quality at Entry (including whether lessons of earlier operations were taken into account, risks and their mitigations identified, and adequacy of participatory processes, as applicable) 23. The overall rating for project preparation, design and quality at entry was found satisfactory. 24. BDMP, like other projects prepared through the MDTF, was prepared under the OP/BP 8.0 – Rapid Response to Crises and Emergencies. Its design, therefore, was a response to emergency circumstances (caused mainly by floods and earthquakes) and took into account the significant risks that such circumstances entailed. Project Design 25. The design of the Project was driven by lessons learned from the World Bank’s response to the 2005 Pakistan earthquake and in line with the shift in the Government’s approach towards disaster management, from an ex post response mainly focusing on the implementation of coping mechanisms towards more ex ante risk management. BDMP was initiated as the first dedicated DRM project in Pakistan, aimed at building the capacity of the PDMA, which is the relevant institutionally mandated organization to plan for and manage disasters. At the time the project became effective, the creation of the PDMA had been notified by the Government, but with extremely limited resources the authority was not yet operational. 26. The province of Balochistan presented a unique environment to pilot this approach for a number of reasons. It had devastating floods in 2010, and the government’s response in supporting the population to manage the ensuing crisis was not very effective. Recognizing this, the government notified PDMA Balochistan, but with a skeletal structure and only three full-time staff members, who did not have the

4 These contracts were for procurement of equipment to assemble mobile units in Quetta. The equipment was being imported from abroad, which caused delays.

6

capacity to respond to the emerging needs. At the same time, resources were made available through the KP, FATA and Balochistan MDTF, which was established to respond to the Post Crisis Needs Assessment (PCNA). The PCNA framework provided mechanisms for crisis management in KP, FATA and Balochistan and the MDTF was established to ensure speedy availability of resources to the two provinces and FATA to build peace and sustainable development, including disaster risk management. 27. The choice of supporting PDMA Balochistan through BDMP was driven mainly by the fact that at the time this authority was notified, there were only two other PDMAs notified in the country including in KP and in FATA. Given the protracted and ongoing crisis in KP and FATA, bilateral donors were supporting the provincial/regional disaster management authorities in these areas, whereas the support that PDMA Balochistan received was negligible. Other provinces had multiple mechanisms for disaster management, such as surveillance facilities in line departments and flood monitoring systems, but in Balochistan, these alternative mechanisms were virtually nonexistent, and the PDMA was considered the sole entity responsible for disaster management. In addition, although PDMA Balochistan was not fully operational in performing its mandate, it was the first disaster management authority in Pakistan to formulate province-level policy documentation on how to manage and respond to disasters. Also, PDMA Balochistan had demonstrated its commitment to disaster response through the coordination of relief work, mainly in flood affected areas, although these initiatives remained unplanned, with a lack of emphasis on overall planning and preparedness. 28. BDMP built on key lessons learned from previous experiences of similar operations implemented internationally and within Pakistan. First, it was recognized that, to ensure development of sustainable capacity, it was essential to support line functions and not create a separate PMU, which usually lasts only for the duration of funds. Therefore, the project’s essence was to support ‘whole-of-authority’ reform that would be sustained after the lifetime of the project. Second, it was acknowledged that any spatial planning for disaster management or response should be based on a thorough understanding of prevalent risks, specifically hazard and risk assessments. Third, previous experiences in the field in Pakistan underscored the necessity of flexibility to respond to crises in a timely manner; to address this concern, the project design included a CERC. Fourth, the 2005 earthquake in Pakistan demonstrated the need to build capacity amongst communities pertaining to awareness and preparedness, which led to the inclusion of the community-based disaster risk management (CBDRM) measures in the design. 29. The PDO of the project was simple, focused on establishing the capacity for the promotion of the DRM agenda in Balochistan. Based on lessons learned, the design incorporated four factors essential to the PDO, including capacity building, institutional strengthening, responsiveness and preparedness.. It was further recognized that strengthening public sector institutions within a low capacity environment is a slow engagement, and requires a phased approach with incremental capacity building. The project was designed in a way that it front-loaded the acquisition of critical goods and equipment, followed by development of operational and administrative systems and decision-making tools. The project then focused on introducing more sophisticated and innovative technologies into the functioning of PDMA. Project Preparation 30. PDMA Balochistan was given the responsibility for project implementation, which required overseeing the project, preparing budgets, devising procurement and implementation plans, and ensuring vertical and horizontal coordination. The Director General (DG) PDMA was the Project Director and assigned the overall responsibility for project management, but given the multiple demands on his time, and the frequent turnover, a project coordinator was to be hired for day-to-day management of the project. This post was, however, not filled during the lifetime of the project, because as implementation began, it was decided that the DG PDMA could effectively manage the task of project director as well. Procurement

7

and financial management functions were separated by appointment of dedicated Accounts Officer for financial management and Assistant Director (Administration) retaining procurement responsibilities. This segregation of roles enhanced transparency. The existing organizational structure of PDMA, as seen in Figure 1 below, lacked any senior-level positions to manage important functional areas such as disaster recovery and rehabilitation, procurement, finance, information technology, internal audit, and logistics. A capacity assessment was conducted, and an enhancement plan was developed for PDMA Balochistan under the BDMP. The discrepancies in the original organizational structure were amended and a more-efficient function-based structure proposed. This will be discussed in detail in Section 3 of this report. Figure 1: Organizational Structure of Provincial Disaster Management Authority Balochistan – Before Implementation of the Balochistan Disaster Management Project5

31. To guide implementation, a MTDF Operating Manual was adopted that set out the roles and responsibilities of program stakeholders; policies, processes, and procedures for program administration; results monitoring; and fiduciary safeguards. To help increase implementation capacity, technical experts were hired to assist in implementation and augment capacity as required. Several firms were hired to support the PDMA by undertaking key assessments that would contribute to and inform capacity building and institutional strengthening within the PDMA, including a capacity assessment, implementation of an enhancement plan, development of a DRM plan and a logistics plan for the province, completion of an external audit, installation of an office management information system, hazard and risk assessment of Quetta city, and training and deployment of a R3T, among other project activities. PDMA staff, with the assistance of experts hired under the project, were responsible for managing these firms. Risks and Mitigation

5 AD refers to Assistant Director

8

32. The overall risk rating for the project was high, based on a number of factors. First, the volatile and uncertain environment of Balochistan posed a number of risks that, if manifested, could limit the ability of World Bank teams to supervise the project adequately. Second, although the PDMA had been notified, DRM was a relatively new field for the provincial GoB, which might mean that the PDMA would face institutional limitations in assuming their responsibilities. Third, there was a general lack of familiarity with World Bank procedures, particularly fiduciary arrangements, that caused initial delays in implementation. Fourth, the hazard and risk assessment was identified as a high risk activity for a number of reasons. The assessment was based on a methodology being implemented in the province for the first time, in the context of weak state capacity. The completion of the comprehensive assessment required coordination and involvement of a number of technical public sector organizations and it was difficult to mobilize the relevant stakeholders in a timely manner or as required. Most importantly, the data gathered through hazard and risk assessment was considered sensitive from a security viewpoint, making it difficult for GoB to find quality consultants to undertake the analysis6. 33. The task team managed these risks using a number of measures, including developing a communications campaign aimed at clarifying roles and responsibilities of the PDMA, ensuring availability of technical assistance through dedicated individual consultants and consultant firms to circumvent operational challenges in a timely manner, and limiting the scope of the hazard and risk assessment to the city of Quetta and making it dual purpose: to collect and collate hazard and risk exposure data for Quetta city and to develop a methodology for such assessments that could be readily replicated. Keeping in view the security risks prevalent in Balochistan, the task team invited PDMA and other GoB officials to undertake reverse missions in Islamabad to review progress and provide implementation support.

2.2 Implementation (including any project changes/restructuring, mid-term review, Project at Risk status, and actions taken, as applicable) 34. The overall rating for implementation is satisfactory. 35. The project’s implementation arrangements were aimed at ensuring institutionalization of capacity-building initiatives within government structures in a sustainable manner. As mentioned, no dedicated PMU was created for project implementation; line functions of the PDMA were strengthened instead. Institutionally, PDMA Balochistan was attached to the Senior Member, Board of Revenue, who is also the Home Minister in Balochistan. The support of the Home Minister ensured strong government control and empowered the PDMA to undertake its work without encumbrances. In the sensitive context of Balochistan, where the prevailing security situation impedes access of international organizations, the support of the Home Minister was instrumental in ensuring commitment to the PDMA’s activities, particularly those related to data collection and surveillance. 36. Despite strong ownership, significant delays in initiating project activities were experienced after the project was declared effective, which were beyond the control of the project or the task team. First, the lack of capacity and awareness of procedures delayed the opening of the designated account, restricting payments and initiation of necessary procurement activities. Second, high staff turnover led to inconsistency in project planning and implementation. The World Bank raised this concern with the GoB multiple times,

6 A hazard and risk assessment was planned in the aftermath of the 2005 earthquake, but it could not be conducted because the firm hired to undertake the study was foreign and it was not acceptable to the government of Pakistan that such an entity should collect data on topographical features (hazard database) or the built environment (exposure database). These lessons were incorporated into the design of the hazard and risk assessment planned and completed through BDMP.

9

but it was not effectively addressed. Third, various natural disasters, including the Masakhel and Awaran earthquakes and floods in Balochistan, kept PDMA staff occupied with emergency relief and less focused on the project. 37. These bottlenecks were managed effectively through multiple initiatives. Critical human resource capacity was incrementally added at the PDMA, which facilitated specific activities such as preparation of Terms of Reference (TORs) for essential procurement and updating DRM plans and policies. The procurement plan was revised to reflect new cost estimates and phase out activities, prioritizing quick-disbursing activities that would be undertaken on a fast-track basis. These quick-disbursing interventions were identified based on their contribution to the PDO. 38. Formal meetings were held semi-annually between the Bank and the implementing agency to review program implementation but the task team was regularly in contact with the PDMA to ensure smooth implementation. These meetings helped track project performance vis-à-vis the results framework and to agree on planned actions and how to proceed. Adverse security cause mission interruptions and in such cases, reverse missions were organized to ensure continued supervision and engagement with the project staff. A third-party monitoring and validation firm was engaged to monitor progress and ensure that the required outcomes were achieved. The firm also reviewed the adequacy of the financial management system and proposed actions that PDMA Balochistan needed to take in quarterly reports. A grievance redressal mechanism was established at PDMA Balochistan to address any concerns or complaints of beneficiaries and to serve as a feedback mechanism for project interventions. 39. During the Mid-Term Review (MTR), it was recognized that the earthquakes that occurred in 2013 provided impetus to develop and utilize much needed capacity within the PDMA on a fast-track basis. BDMP’s support in establishing the PEOC and emergency stockpiles of relief goods enabled the PDMA to communicate and coordinate relief work with the affected areas and ensure delivery of basic amenities to the affected populations. Specific steps were also taken to ensure that component-wise progress remained on track. The procurement plan was revised to reflect the recommendations of the capacity enhancement and CBDRM plans. The PDMA also updated the existing DRM plan, developed in 2008 for the province of Balochistan, and developed response SOPs to streamline roles and responsibilities. The CBDRM plan was piloted in collaboration with the MDTF-supported Promoting Girls Education in Balochistan Project (PGEBP). BDMP used the schools that the PGEBP supported not just as a venue, but also as a mechanism to access communities with a view to providing CBDRM training and distributing relevant materials. Instruction materials on CBDRM activities were distributed to an estimated 20,000 community members, which reaffirmed the project’s contribution to the MDTF higher-level objective of enhancing state-citizen trust.

2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 40. The overall rating for the M&E framework is satisfactory. M&E Design 41. The project results framework consisted of three PDO results measured by six indicators, four intermediate results and 15 output indicators. The M&E arrangements and the project results framework were reviewed at the MTR and found to be satisfactory. The results framework consisted mainly of process indicators, such as preparation of plans and SOPs and creation of databases. This is normal practice for governance reform and institutional strengthening projects prepared under Operational Policy 8.0 guidelines in fragile environments and those affected by conflict and violence. A deliberate effort was made to establish a coherent and strong results chain, with clear evaluation criteria identified for each indicator.

10

This helped link these process achievements to project outcomes and outline a clear project structure, which guided implementation modalities and sequencing of activities. M&E Implementation 42. The project design included appropriate monitoring and evaluation (M&E) mechanisms to track and review progress of the project against the agreed-upon results framework. As part of the overall institutional strengthening initiative of the PDMA, a dedicated M&E specialist was hired to monitor progress toward achievement of results, and the PDMA submitted detailed progress reports monthly, quarterly, and annually, in line with the requirements of the project results framework. 43. In preparation for the MTR, a quick review was conducted to assess whether the project was being implemented in a conflict-sensitive manner. This was essential in the context of Balochistan, where frozen conflict7 can influence the government’s ability to prepare for and respond to disasters. Project activities were mapped against PCNA objectives, and the analysis concluded that the project was conflict sensitive and addressed the key drivers of conflict. 44. A firm was hired for third-party validation (TPV), not just to provide an independent review of project achievements, but also to provide an additional layer of supervision based in Balochistan, especially in instances in which the task team was restricted from visiting Balochistan. (A brief summary of the findings is presented in Section 3). M&E Utilization 45. The M&E Framework of the project was aimed at building capacity within the PDMA to collect and collate knowledge to improve responsiveness. In this regard, the project created an extensive DMIS, which supported the PDMA in developing better strategies for disaster mitigation and ensuring faster response and greater capacity, although no mechanism has been established to ensure that this DMIS will be updated regularly. 46. Although the M&E system was developed to address the immediate needs of the project, no mechanism was developed to strengthen the long-term systematic monitoring of institutional transformation, indicating a lack of foresight regarding sustainability and transition. The data gathered through the existing monitoring arrangements helped track progress towards the PDO.

2.4 Safeguard and Fiduciary Compliance (focusing on issues and their resolution, as applicable) Safeguards 47. The overall performance rating of the environmental and social safeguards is rated satisfactory. 48. The project was categorized as Environment Category C in accordance with Operational Policy 4.01, and none of the policies related to social safeguards were triggered. There was concern at the time of preparation that environmental and social safeguards may be triggered through Component 4, and it was agreed that an environmental and social screening and assessment framework would be prepared, submitted

7 Frozen conflict refers to a situation where socio-economic dynamics remain uncertain and there is a threat of conflict.

11

to the World Bank for review, and disclosed before any disbursements were made under Component 4, but the CERC was not triggered during the implementation lifetime of the BDMP. Fiduciary Compliance 49. The overall risk rating for financial management and procurement was high at the time of appraisal. 50. Design of Fiduciary Arrangements: An assessment was conducted to review existing financial management arrangements, and key weaknesses included lack of detailed policies, non-segregation of financial management and procurement duties, lack of management reporting, weak record management, and weak inventory management. A similar assessment of the PDMA procurement staff was also conducted that showed that PDMA Balochistan had limited experience in procurement according to World Bank guidelines and procurements were being made according to the Balochistan Purchase Manual, which outlines the Government’s own procurement procedures. 51. Implementation: It was agreed to hire dedicated financial management and procurement staff within the PDMA. Although a procurement specialist was hired in a timely manner, hiring of the financial management specialist was delayed by 6 months, with the position filled in December 2013. The fiduciary arrangements of the project were rated as satisfactory until June 2015, at which time the risk rating was changed to moderate. With the ad hoc extensions of the MDTF, and consequently the project, the financial management and procurement positions were vacated in December 2015, which delayed the submission of the final Interim Financial Report, although the task team communicated with the PMU to ensure that this was resolved quickly. These positions are essential to ensure timely audit and updates of the procurement plan, but because the project was extended only until June 2016, it was agreed not to fill these positions for a 6-month period, and the project director performed the necessary activities related to the procurement plan and external audit.

52. No major problems were identified in the overall procurement arrangements of the project, and the project maintained a satisfactory procurement rating throughout its lifetime and even at time of project closure. Weaknesses in overall contract management identified at the time of MTR including timelines of contracts, quality of deliverables and capacity gaps. A fiduciary review was carried out after the MTR to develop an action plan to address these concerns. As a result, the roles and responsibilities of the procurement and M&E specialists were clarified and enhanced to ensure improved contract management. In addition, simplified guidelines for contract management were adopted by the project.

2.5 Post-completion Operation/Next Phase (including transition arrangement to post-completion operation of investments financed by present operation, Operation & Maintenance arrangements, sustaining reforms and institutional capacity, and next phase/follow-up operation, if applicable) 53. Given the satisfactory performance of the project and donor interest in expanding the MDTF for a second round of funding until 2020, it was expected that BDMP would be expanded and extended. This was discussed extensively during a consultation workshop held in Balochistan with key government officials in October 2014. The government expressed a keen desire to extend the project and expand its activities, specifically focusing on operationalization of the disaster management plan, full implementation of the capacity enhancement plan developed for the PDMA, strengthening of the district disaster management authorities in selected districts, and installation of an early flood warning system. The cost was estimated at US$10 million.

12

54. The BDMP was not extended or expanded and closed as planned in June 2016 for two main reasons. First, the expected financing envelope under the MDTF was reduced, which necessitated reprioritization of resources. Second, change in senior management within the World Bank and GoB resulted in changing: i) strategy for work methods (away from small catalytic interventions towards larger operations with the requisite scale to generate tangible benefits); and, ii) priorities (from capacity building towards resilience through water resource management).

55. While a second phase of the BDMP did not come though, discussions with the PDMA show that achievements made through the BDMP have been institutionalized and expanded on to the extent possible (to be discussed in Section 3 in more detail).

3. Assessment of Outcomes

3.1 Relevance of Objectives, Design and Implementation (to current country and global priorities, and Bank assistance strategy) Relevance of Design - High 56. The project design, development objectives, and outcomes of the BDMP remain relevant for a number of reasons. First, the BDMP is the first dedicated DRM project that the government and the World Bank implemented in Pakistan. Prior to the implementation of this project, disaster response was concentrated on ex-post coping measures and it was only in 2008, with the earthquake response, that there was a recognition of ex-ante mechanisms for disaster management. The BDMP was established through MDTF funding and was Pakistan’s first project dedicated to institutionalizing disaster management capacity within the government. 57. The project contributed to the World Bank CPSs for 2010-15 and 2015-19. CPS 2010 was focused on reducing vulnerability by strengthening the existing safety net system for vulnerable populations and developing emergency health service delivery during disasters under Pillar 2 of the CPS regarding improving human development and social protection, but in response to back-to-back floods in 2010 and 2011, the effect of disasters on the poverty rate, and funding large cash transfers to affected populations in the aftermath of floods, the Bank reviewed its CPS, which resulted in additional attention to its disaster risk management agenda and updated outcomes to incorporate better response and coordination at the federal and provincial levels. BDMP, the first stand-alone operation dedicated to DRM in the country, was fully aligned to these revised objectives and contributed towards the agenda at the provincial level. The Bank’s CPS 2015 incorporated disaster risk management beyond emergency response by including activities related to increasing resilience through understanding disaster risk and strengthening DRM and early-warning systems under Outcome 3.3 (increased resilience to disasters in targeted regions) of inclusion Pillar and climate change adaptation and mitigation cross-cutting theme. 58. The results framework clearly captured the project’s theory of change. As with most institutional strengthening/capacity building projects such as BDMP, prepared under the OP 8.0 guidelines, the results consisted mainly of process indicators. Clear evaluation criteria were included in the framework to ensure that these adequately captured progress towards project objectives. Relevance of Objectives - High 59. The objective of the project was to strengthen the capacity of the PDMA to prepare for and respond to natural disasters. The achievement of this objective was ensured through three key broad categories of

13

activities including capacity building, institutional strengthening and improving responsiveness of the PDMA. Figure 2 summarises the project’s theory of change. Figure 2: Balochistan Disaster Management Project: Theory of Change

60. Capacity building was ensured through the development of a capacity enhancement plan, training (including four national and international exposure visits) of PDMA staff in line with recommendations of this plan as well as community members, and undertaking simulation exercises, which were aimed at enhancing the PDMA’s ability to coordinate across multiple stakeholders and test all the relevant protocols. 61. The plans and SOPs that have been developed with BDMP support are formally notified, which has contributed to strengthening the institutional frameworks related to DRM as an agenda in Balochistan. These plans also outline the role and responsibilities of the PDMA, how the PDMA should operate, including day-to-day management and medium- to long-term strategic outlook regarding DRM. Preparedness and response capacity was enhanced through building stockpiles of emergency relief supplies, improved availability of data relevant to disasters, particularly through the hazard and risk assessment, and development of emergency response SOPs, which brought together multiple stakeholders and built PDMA capacity for coordination and data collection. In addition, the DMIS established to collate the data collected through the assessment enables the PDMA to inform DRM strategies and identify exposure to risks and shocks. 62. The activities undertaken through BDMP have contributed towards project objective by enabling the GoB, through the PDMA, to i) better respond to emergencies; and, ii) be better prepared for risks, which in turn ensures better service for communities. The project adopted a phased approach to develop the PDMA into a robust institutional of disaster management. Training and capacity building has increased the skills and competence of the PDMA staff to understand, manage and address issues related to DRM. In addition, the various plans and SOPs developed through the project have provided an institutional framework to

DMIS established and updated using data from multiple sources

Capacity Building Institutional Strengthening Responsiveness

Capacity development plan developed and implemented – PDMA operationalised

Logistics Plan developed and finalised

Stockpiles established at PDMA

CBDRM Plan developed with training modules and being utilized by multiple

SOPs for PDMA Balochistan developed and adopted

PEOC up and running

Training completed for master trainers

DRM Plan for Balochistan finalised

Simulation exercise completed to test protocols

Exposure visits for PDMA staff

Rapid Rescue and Response Team trained and deployed

Strengthened PDMA Balochistan

14

make DRM in Balochistan more systematic and coordinated. By ensuring that these plans are notified, the project has helped establish standards and mechanisms in preparing for and responding to disasters. The systems and capacity developed through the project for data collection and collation have enabled the PDMA to better understand risks and exposure to hazards and integrate this in the strategic planning for disaster preparedness and response. Relevance of Implementation - High 63. The Bank’s implementation arrangements remained relevant and responsive to the evolving political scenario and security situation in Balochistan. The project was not restructured during its lifetime except the time extension sought as part of the extension of the overall MDTF as well as for completion of on-going activities. 64. In addition to meeting all indicators in the results framework, project implementation has contributed towards lesson-learning accountability. One, the analytical exercises and dialogue with the government undertaken for the preparation of BDMP and the lessons learned through the implementation of the project have led to greater awareness of disaster management as a theme for development more widely in Pakistan; two, the effect of the BDMP in developing PDMA Balochistan into a robust institution for DRM has had a powerful demonstration effect on other PDMAs in developing their capacity; and three, the technical knowledge and dialogue with the governments has enabled the World Bank to develop similar operations in Punjab and Sindh. 65. Even-though the project was not extended or expanded, it allowed PDMA Balochistan to emerge as a robust institution of disaster management. BDMP activities are concentrated in the district of Quetta but benefit adjoining districts where DRM capacity is virtually nonexistent. This area is particularly prone to flash floods and droughts. The PDMA is now equipped to predict the occurrence of these disasters, ascertain losses, and respond quickly and effectively8. The PDMA has also been able to provide relief through deployment of human resources and equipment in the aftermath of terrorist attacks.

3.2 Achievement of Project Development Objectives (including brief discussion of causal linkages between outputs and outcomes, with details on outputs in Annex 2) 66. The overall achievement of the PDO assessed by the indicators in the Results Framework is rated as high. Based on a thorough review of the evaluation criteria, of the three PDO indicators (including i) improved disaster response mechanism at PDMA Balochistan; ii) Increased awareness of the hazard and risk environment in Quetta; and, iii) enhanced capacity at PDMA Balochistan for implementing community-based DRM (CBDRM) initiatives), all were achieved and the project exceeded its targets for one, related to the CBDRM Plan. Of the four intermediate indicators and 15 outputs, the project achieved all the targets outlined in the Results Framework, exceeding 2 (please see Table 1 below). Details of achievements vis-à-vis the Results Framework are provided in Annex 2. Table 1: Details of Project Achievements

Achievement against Target

8 As is expected with natural disasters, such prediction related to the occurrence as well as magnitude of losses may not be 100 percent accurate.

15

PDO (3)9 IO (15)10 Surpassed – HS (96%+) 1 2 Achieved – S (85 – 95%) 2 13 Partially achieved – MS (65 – 84%) Not achieved – MU (41 – 64%) Not achieved – U, HU (0 – 40%) Percentage (%) achieved (HS or S) 3/3=100% 15/15=100% Highly Satisfactory - High

Note: HS=Highly Satisfactory; S=Satisfactory; MS=Moderately Satisfactory; MU=Moderately Unsatisfactory; U=Unsatisfactory and HU=Highly Unsatisfactory Assessment of the achievement of the PDO 67. PDO Indicator 1: Improved disaster response mechanism at PDMA Balochistan. This indictor was measured using two criteria including: i) SOPs for disaster response prepared and operationalized; and, ii) number of drills and simulation exercises completed to assess operational coordination, preparedness and emergency protocols. The SOPs for emergency response were prepared and tested during the first simulation drill, further refined based on the results of the drill, formally notified and institutionalized and adopted by the GoB and PDMA Balochistan. The project conducted two extensive drills aimed at testing the coordination and response capacity of the PDMA Balochistan. Various scenarios were constructed including for floods and disasters and the PDMA led the exercise to coordinate response across relevant line departments. Such drills were useful also because they enabled the PDMA to test its communications equipment and its ability to mobilise relief equipment across the province. 68. PDO Indicator 2: Increased awareness of hazard and risk environment in Quetta. The evaluation criteria for this indicator was related to the establishment of the Disaster Management Information System (DMIS) and populating it with data collected during the hazard and risk assessment. The hazard and risk assessment was completed for Quetta city and the data generated from this report was collated into the DMIS, which was established through the project. This geo-referenced data of various sectors was captured in the DMIS to serve as a baseline.

69. PDO Indicator 3: Enhanced capacity at PDMA Balochistan for implementing CBDRM initiatives. This indicator had three evaluation criteria, including: i) Number of master trainers trained; ii) number of training sessions for repsonders in selected union councils in Quetta; and, iii) instruction materials on CBDRM available for dissemination to communities. The project surpassed its targets for this indicator, training 20 master trainers against a target of 15, organizing 5 training sessions against a target of 4 and distributing 20,000 instruction materials against a target of 10,000. The outreach for the last evaluation criterion was high because these materials were distributed through the schools supported by the MDTF supported Promoting Girls Education in Balochistan Project (PGEBP) across 12 districts.

9 The PDO indicator surpassed is ‘enhanced capacity at PDMA Balochistan for implementing CBDRM initiatives’ while those achieved include ‘improved disaster response mechanism at PDMA Balochistan’ and ‘increased awareness of the hazard and risk environment in Quetta’.

10 The two intermediary results that surpassed their targets include ‘number of exposure visits by PDMA Balochistan staff undertaken and knowledge products produced’ and ‘instruction materials on CBDRM available for dissemination to communities’. The targets for all other indicators were achieved.

16

70. The project achieved its overall objective of strengthening the capacity of the PDMA to prepare for and respond to natural disasters through multiple activities implemented in a phased and flexible sequencing. Project activities were categorized across the three main components of the PDO including capacity building, institutional strengthening and responsiveness. Preparedness remained a crosscutting theme across all the interventions BDMP undertook. Summary of key project achievements: The following outlines five achievements of the project, which have been instrumental in ensuring the PDO was met: Community Based Disaster Risk Management (CBDRM) First, the project developed an extensive CBDRM plan, which was a critical innovation for a province like Balochistan, mainly because the community is the first affected by a crisis but is also the first responder. The province has a scattered population, which makes service delivery expensive, making it difficult for the government to reach affected populations fully and effectively. CBDRM initiatives were to capacitate and sensitise the PDMA in the area of CBDRM and the role of communities in disaster response. It did emphasise capacity building of communities on CBDRM. The Plan, which is already under implementation, is published on the website of PDMA Balochistan, and the salient features of the plan are available through the mobile phone application that the PDMA has developed through the BDMP. The U.N. Development Program (UNDP) and World Food Program (WFP) are also using the CBDRM plan as the basis for piloting community-based disaster management techniques in the districts of Jaffarabad, Naseerabad, and Ziarat. Institutional Strengthening 71. Second, the project supported the development of the DRM Plan 2014-19, which is the first attempt to operationalize DRM policy at the provincial level. UNDP had developed a plan for disaster management in Balochistan in 2008, but this was before the notification of the PDMA and was not adopted as standard practice for disaster management in the province. The DRM plan developed with BDMP, which has been approved by the PDMA Balochista, is a comprehensive document, which provides a strategic roadmap for the sector covering a number of aspects while taking into account the specific circumstances of Balochistan. A number of features were proposed in the plan, including establishment of a provincial disaster management fund, strengthening of the PDMA (specifically the PEOC), awareness campaigns, capacity building and institutional strengthening of the district disaster management functions, and establishment of a provincial institute of disaster management.

The implementation of the DRM plan, was not an objective of BDMP. Nevertheless, it has been initiated using PDMA’s own resources and through support from agencies such as UNDP and WFP. Capacity Building and Responsiveness 72. Third, the project provided six trainings for all levels of PDMA staff (93 participants), from top management to the lowest cadre of workers, including consultants and civil servants, summarized in Table 2. The trainings have been useful in building the capacity of PDMA staff in the areas of disaster risk management, first aid, remote sensing and GIS technologies, project management and basic computer literacy. Exposure visits were organized to learn from best practices across the country. The skills and learning acquired through these initiatives have contributed towards operationalizing the PDMA to undertake its day-to-day tasks. Anecdotal evidence based on discussions with PDMA staff shows that the

17

skills acquired through the trainings are in fact being used extensively for the PDMA to perform its functions. However, no systematic evaluation has been undertaken to establish whether the PDMA is benefiting from the ‘new’ set of skills acquired by its staff. Also, with the closure of the project, some staff who had been employed in key positions have moved to other jobs, and the PDMA may not be using the capacity built through the project anymore. Nevertheless, approximately 50 mid- to low-level staff trained through the project have remained with the PDMA Balochistan, ensuring continuity of day-to-day operations. 73. These trainings were organized by multiple institutes including the National Institute of Disaster Management, Pakistan Space and Upper Atmosphere Research Commission (SUPARCO) and local universities. The curriculum for these trainings was developed by BDMP specifically for PDMA staff. Table 2: Summary of Trainings that the Balochistan Disaster Management Project Has Conducted Name of Training Recipient Duration Location Participants

Disaster Response/First Aid Mid to lower staff of PDMA Balochistan March 17 ‐ 19, 2015 Quetta 55

Disaster Risk Management Organised by National Institute of Disaster

Management for key staff at the PDMA

Balochistan involved in development of

Disaster Risk Management Policy

March 9 ‐ 11, 2015 Karachi 5

National Exposure Visit (Group 1

and 2)

Organised by National Institute of Disaster

Management for senior staff and consultants

of PDMA Balochistan. Institutes visited include

PDMA KPK, NIDM, Rescu 1122 KPK, Tarbela

Dam (managed by WAPDA), NDMA, PDMA

Punjab, Rawal Dam (managed by SDO),

Mangla Dam, Pakistan Meteorological

Department and SUPARCO. The training was

aimed at improving the planning and

monitoring capacity of staff at PDMA

Balochistan.

April 1 ‐ 7, 2015 and

April 12 ‐ 18, 2015

Islamabad 17

Remote Sensing and GIS Orgaised by SUPARCO at the National Centre

for Remote Sensing and Geo‐Informatics

(NCRG) Karachi, the course was aimed at

providing training on concepts related to GIS

techniques and applications and

operationalisation of remote sensing and GIS

in different disciplines.

April 13 ‐ 24, 2015 Karachi 2

Strategic Project Management Organised by the Institute of Business

Management (IBA), Karachi, the course was

aimed at training professionals to better

understand strategic aspects of project

management. The course was aimed at senior

management of the PDMA involved in

strategic planning and project management.

March 2 ‐ 4, 2015 Karachi 2

Computer Skills Organised in collaboration with the

Balochistan University of Information

Technology, Engineering and Management

Sciences (BUITEMS), the course was aimed at

lower level staff of the PDMA involved mainly

in day‐to‐day management and operations.

The training sought to develop basic computer

skills of this cadre of staff within the PDMA.

March 16 ‐ April 19,

2015

Quetta 12

74. In addition to the trainings mentioned above, BDMP facilitated two training sessions for PDMA staff organized under the auspices of the Asian Disaster Preparedness Center in Thailand. Each lasted 5

18

days, one focusing on DRM and hazard and early warning systems (8 participants) and the other on disaster risk reduction and policy (9 participants). The COMSATS Islamabad organized a 5-day project management training for five key PDMA staff members to develop the office management information system. 75. BDMP also organized a one-day gender sensitization training through COMSATS, Islamabad for 12 PDMA staff members. The training covered basic concepts of gender, social constructs of gender and its implications, women’s participation in the development of a society, third gender and human rights and Millennium Development Indicators on women, education health and empowerment. 76. Fourth, the project developed a detailed capacity enhancement plan based on detailed consultation and analysis with relevant stakeholders. The plan was designed to improve the organizational structure of the PDMA, proposing a new structure including key functionalities, which had previously been missing (please see Figure 3). In addition, the plan recommended purchase of necessary equipment, which would make the PDMA more efficient and responsive. The recommendations of the plan were aimed at making the PDMA more effective in responding to disasters in a comprehensive manner, with effective collaboration between relevant line departments at the subnational level. The GoB approved and notified the plan, creating the institutional basis for its implementation. 77. As with the DRM Plan, the implementation of the Capacity Enhancement Plan was not an objective of the BDMP. However, the PDMA has made efforts to implement the recommendations of the plan specifically those pertaining to organizational restructuring through the hiring of additional staff (albeit on contract-basis). 78. A key element of the plan was the establishment of R3Ts within the PDMA. R3Ts were trained to provide timely, efficient, effective response to disasters; save lives; and minimize further injuries to people and damage to property. Since its constitution last year, R3Ts have responded rapidly to several disasters in the province, including

search and rescue activities after derailment of Jaffar Express passenger trains near Aab-e-Gum in Kacchi district

assisting in shifting the population and clearing the waterway near Quetta city to avoid flooding the city

response to earth breach due to torrential rain near Quetta city search and recovery of dead bodies of drowning victims at Kochlak Dam, Saryab Well, Yaro

Mandon Dam, and Pishin Dam rescue of 54 persons and recovery of five bodies with the assistance of Levies and Frontier Corps

(FC) personnel during flash flooding at Zard Aloo picnic point in Harnai district provision of medical assistance and support to victims at terror-attack sites as first responders response to automobile accidents near Quetta

79. Since project closure, the PDMA has made a number of efforts to implement the Capacity Enhancement Plan using its own resources. The PDMA has approximately 90 staff, including support staff, of which 51 are regularized civil servants; this number does not include the contract staff hired as part of the R3Ts. The PDMA has recently approached the relevant line departments (Planning and Development, Finance and Chief Secretary) to regularize 34 rescue workers and 12 key experts according to the capacity enhancement plan. It is also planning to regularize an additional seven R3T members. This staffing is part of the roll-out of the emergency (Rescue 1122) services that PDMA is developing for Quetta city.

19

80. Discussions with PDMA staff reveal that, even though the additional positions proposed through the capacity enhancement plan have not been regularized, as an interim solution, existing staff have been given additional charge and are undertaking the functions highlighted in the capacity enhancement plan. The PDMA has also hired contract staff where possible to augment the functions of existing staff as required11.

11 The DG PDMA has the discretion to hire staff below Grade 16 for up to a year on a contract basis.

20

Figure 3: Organizational Structure of the Provincial Disaster Management Authority (PDMA) Proposed through the Balochistan Disaster Management Project

PDMC

PDMA BOARD

DIRECTOR GENERAL

INTERNAL AUDIT

DIRECTOR RELIEF & RESPONSE

DIRECTOR RESCUE

1122

DIRECTOR ADMIN &

PROCUREMENT

Dy. Dir. DRM

Dy. Dir. Relief

Dy. Dir. Response

Rescue 1122 Operations

Emergency Response

PEOC Coordinator

DIRECTOR PLANNING &

COORDINATION

Dy. Dir. Planning &

Coordination

DRM Office

CBDRM Officer

G&CP Office

r

AD P&C

M&E Officer

AD Rev. Mob

AD Training

DMIS Officer

System Admin

Hardware Support

Software Support

Dy. Dir. Admin & Proc.

Dy. Dir. Finance

AD Admin

Proc. Officer

IT Officer

Accounts Officer

Fin. Mgt. & Bgt.

AD Relief Warehouse Officer

21

Preparedness 81. Fifth, BDMP supported the completion of the hazard and risk assessment for the city of Quetta. This assessment, which brought together a number of institutions, including the PDMA, NDMA, Space and Upper Atmosphere Research Commission (SUPARCO), Geological Survey of Pakistan (GSP), Pakistan Meteorological Department (PMD), Survey of Pakistan, and the University of Balochistan, had two main objectives: develop a multi-exposure map for the city of Quetta and formulate a methodology for replicating such studies in other parts of the province and country. Using the risk data collected through this assessment and available geo-referenced data, the PDMA was able to create a DMIS, which collates all disaster-related data and information, and make it readily available in one place. 82. The DMIS is fully operational with baseline data as well as data on hazard and risk exposure entered for the city of Quetta. However, The PDMA has not, to date, replicated the methodology for the hazard and risk assessments in other parts of the province. 83. Since project closure in June 2016, PDMA staff has made efforts to sustain the achievements of the BDMP. The plans developed through the project are in various stages of implementation. As mentioned above, the capacity enhancement and CBDRM plans are in advanced stages of implementation. Although the Provincial Disaster Management Commission has not formally approved the DRM plan, implementation has been initiated with the establishment of Rescue 1122 services in Quetta city (elaborated below). The SOPs developed for the PDMA for disaster response have been prepared and tested in one formal drill and simulation exercise. These procedures have been used daily to respond to security incidents and other disasters. While no clear mechanism has been established to update the DMIS regularly with data on hazard and risk exposure, the DMIS of PDMA Balochistan has been linked to the national platform for hazard and risk assessments, ensuring continued data collection and collation, though this data will not be comprehensive.

Assessment by Third Party Validation Firm

84. As mentioned above, BDMP hired a third-party validation firm to review progress on objectives and to assess implementation modalities. The review found that implementation of the project had improved DRM service delivery, which was restricted to rescue and relief before the project. The project employed innovative information and communications technology–based tools for the first time in a DRM context in Pakistan, including the establishment of mobile units, use of unmanned aerial vehicles, and establishment of the DMIS. The third-party validation report made a number of key recommendations pertaining to the government’s ownership of the PDMA and its role in promoting the DRM agenda in Balochistan. There is widespread recognition of the contributions that the BDMP has made toward building the PDMA’s capacity. The onus now is on the government to ensure that the necessary steps are taken to sustain this capacity. The report also recommended that the PDMA take steps to ensure that human resource capacity built through the project is retained. The report also underscored the need for drills and simulation exercises with line departments to update SOPs regularly. These SOPs have been finalized, notified and adopted by GoB and PDMA Balochistan. If the PDMA leverages resources for its continued development, whether through the government’s annual development program or through development partners, it should be ensured that the best practices introduced through BDMP are adopted for better planning.

3.3 Efficiency

22

(net present value/economic rate of return, cost effectiveness, e.g., unit rate norms, least cost, and comparisons; and financial rate of return) 85. It is difficult to assess the overall efficiency rating of the project. At the time of preparation, the practice for projects being prepared under OP 8.0 was that they did not require a full economic analysis and consequently, no economic analysis was conducted for the project at the time of appraisal. Conducting a formal cost benefit analysis is also difficult because of the nature of project activities, emphasizing capacity building and institutional reform, and the complexity associated with quantifying expected benefits. 86. In essence, the project was designed to strengthen and operationalize the PDMA to enable it to fulfill its mandate. The project has achieved this objective through the interventions highlighted above. The project has introduced a number of innovations, which have been applied in other DRM operations in the country. These have helped improve the capacity to manage not just natural, but also man-made disasters, thereby benefiting the population of Balochistan (13.16 million12) directly and indirectly.

87. Budget execution13: It can be argued that BDMP has improved BDMP capacity for budget execution: PDMA expenditures went from US$ 2.6 million in 2011-12 (before the project was initiated) to US$ 20.1 million in 2014-15 (after project implementation). While part of this increase in budget execution can be explained by PDMA’s response to the flash floods that occurred during this period, approximately 60 percent of this increase is permanent, being utilized for emergency response, establishment of the Rescue 1122 service and strengthening emergency communications systems. 88. Leveraging resources: The project has enabled the PDMA to leverage resources from the government and other development agencies. As part of the rollout of the DRM plan developed through the BDMP, the project has leveraged approximately US$5.8 million from the GoB to implement Rescue 1122 services (described below). Unlike other similar projects, these funds are from the recurrent budget, as well as the development budget. The PDMA has also been able to leverage in-kind support from U.N. agencies, particularly for stockpiling of relief equipment across the province, which has improved preparedness and response capacity. Mobilizing these resources would not have been possible for the PDMA without implementation of the BDMP. 89. Perceptions: Qualitative discussions with staff of the PDMA in 2014 and 2016 were undertaken to assess the evolution of the authority during and after implementation of the BDMP. Key findings of the survey are summarized in Table 3. Regarding the role of the BDMP in improving capacity of the PDMA, staff unanimously felt that the project has had a strong effect, although the need for expanding project activities at the district level continues. This perception has not changed since 2014. Table 3: Perceptions of Provincial Disaster Management Authority (PDMA) Staff Regarding the Balochistan Disaster Management Project (BDMP)

2014 2016

12 Based on 2011 estimates, extrapolated from the 1998 census. 13 Source: Balochistan Public Sector Development Programme (PSDP). These figures include the expenditures incurred during the PDMA response to the 2013 earthquake in Awaran.

23

Weaknesses of the PDMA

Insufficiency or non-availability of staff

Increased DRM capacity at district level, particularly in areas of community-based DRM and improved communications through strengthening the provincial emergency operations center

Contribution of the PDMA to the population

Distribution of Watan14 cards and provision of shelter to affected populations in aftermath of disasters

Rollout of CBDRM plan, stockpiling, improvement of logistics and communications, particularly through mobile operational support facilities

New skills acquired by staff through BDMP

Filing, warehouse management, public dealing, quick response during emergencies

Management alert systems, risk exposure mapping and hazard profiling, logistics support, provision of relief and project management

90. The project utilized 95 percent of its allocated resources (US$4.77 million out of US$5 million) by the time of closure. The project completed all planned activities and, as elaborated below, has resulted in a number of unintended consequences, which have strengthened the PDMA to prepare for and respond to disasters. Moreover, by connecting with citizens through the CBDRM initiative, the project has helped create a strong positive image of the PDMA as a robust institution of disaster management. 91. Fiduciary arrangements were streamlined after an initial delay of nine months. These delays were caused by weak capacity within the PDMA and GoB and lack of familiarity with World Bank procedures. The necessary capacity was built within the PDMA and key procurements were identified and prioritized on a fast-track basis. No other major fiduciary issues were identified during the lifetime of the project.

92. Project management costs are estimated to be US$ 500,000, approximately 10 percent of total disbursements. However, since project management capacity was embedded within the line functions of the PDMA, this cost can also indirectly be attributed to strengthening of the PDMA, which was the overall objective of the project.

3.4 Justification of Overall Outcome Rating (combining relevance, achievement of PDOs, and efficiency) Based on the assessment below, the overall outcome rating of the project is estimated to be satisfactory (Please see Table 3 below). Table 3: Justification of Overall Rating

Rating Relevance High Objective High

14 These are issued by the Benazir Income Support Programme (BISP) enabling beneficiaries to access financial resources.

24

Design High Implementation High Efficacy High PDO 1 (response mechanism) High PDO 2 (awareness and hazard and risk assessment)

High

PDO 3 (CBDRM) High Efficiency Total Disbursed US$ 4.77 million (95%) Final Rating Satisfactory

93. Relevance of objectives, design and implementation: The objectives, design and implementation of the project were relevant to the context at the provincial and national level. The project reached its desired objective and improved capacity of the PDMA to prepare for and respond to natural and man-made disasters through improved capacity for preparedness and response and institutionally strengthening the DRM agenda in Balochistan. The project has emphasized ex-ante DRM measures such as data collection and collation for preparedness. Moreover, the project is the first step in supporting the Government of Pakistan and GoB in implementing NDM Act 2010. The implementation process has increased awareness of the DRM agenda within the Government of Pakistan as well as the World Bank and lessons learned from BDMP have been applied to other operations in Punjab and Sindh.

94. Efficacy: The project achieved all its targets vis-à-vis the Results Framework, surpassing those related to the CBDRM plan. Table 4 shows achievements against the PDO indicators.

Table 4: Summary of PDO Achievement PDO Area/Result

Indicator Achievement Rating

Project Development Objective Indicators Improved disaster response mechanism at PDMA Balochistan

SOPs for disaster response prepared and operationalized

SOPs for emergency response prepared and tested during the first simulation drill, further refined based on results of drill, and approved by PDMA management

S

Number of drills and simulation exercises completed to assess operational coordination, preparedness, and emergency protocols

2 S

Increased awareness of the hazard and risk environment in Quetta

DMIS database established and populated by data collected during hazard and risk assessment

Data generated from hazard and risk assessment of Quetta collated in newly established DMIS; geo-referenced data of various sectors collated in DMIS to serve as baseline

S

Enhanced capacity at PDMA Balochistan for implementing

Number of master trainers trained

20 S

Number of training sessions for responders in selected union councils in Quetta

5 S

25

CBDRM initiatives

Instruction materials on CBDRM available for dissemination to communities

20,000 HS

95. Efficiency: Although it is difficult to ascertain efficiency of the project or conduct a formal cost benefit analysis, the project has improved the robustness of the PDMA and enhanced its capacity for budget execution and leveraging resources from GoB and other development partners. Implemented as a pilot, the project was catalytic in generating a number of unintended consequences outlined in the section below. 96. There are several examples that demonstrate PDMA’s efficiency in responding to disasters and these are elaborated in detail in Section 3.5. Capacity building initiatives have enhanced awareness and understanding of DRM amongst PDMA staff. This has improved their ability to plan for and respond to disasters and enhanced the maturity of the institution as a whole. Equipment purchased through the project has helped in responding to a number of disasters: hydraulic cutters used to rescue citizens from a train crash, UAVs were used to rescue victims during recent security incidents, mobile operational support units have been used to improve coordination not just in Balochistan but also in KP and other parts of the country, boats were used to rescue citizens from flash floods and tents were used for shelter during earthquakes. The PEOC and communications strategy developed by the project has been used to improve communications and awareness amongst citizens regarding the threat of disasters or response in the aftermath. The simulation drills conducted through the project have improved the coordination between relevant line departments.

3.5 Overarching Themes, Other Outcomes and Impacts (if any, where not previously covered or to amplify discussion above) 97. Conflict Sensitivity and State-Citizen Trust: The project has contributed towards enhancing state-citizen trust, which is the higher-level objective of the MDTF in six ways. First, anecdotal evidence shows that BDMP has been successful in enhancing state-citizen trust (higher-level objective of the MDTF and Strategic Objective 1 of the PCNA) by building the capacity of the PDMA to respond to crises through the CBDRM plan, which enables the PDMA to engage directly with communities at the grassroots level. Second, the project supported the purchase of two mobile operational support facilities, mobile office units that not only enhance state visibility, but also ensure immediate connectivity in the event that communications links are not functioning. This has enhanced the PDMA’s coordination function. The facilities are set up in 10-wheel trucks that augment PDMA Balochistan’s existing fleet of 20 trucks and have been used for relief provision in six incidents already in Balochistan, as well as in other parts of the country, including KP and other parts of the country. These trucks are fitted with trackers that ensure that the contents are not stolen. Third, the project has established and trained R3Ts for rapid deployment to respond to various disasters and scenarios, capability previously absent in the province. These teams have been subsumed within the Rescue 1122 service developed for Quetta. Fourth, based on lessons learned from the Philippines and the response to cyclones, the project introduced use of unmanned aerial vehicles for the first time in Pakistan that will enhance the PDMA’s role in surveillance, damage assessment, and rescue and have been used already in the aftermath of recent security incidents to locate victims. Fifth, the project led to the solarization and greening of the PDMA to enable 24-hour connectivity, which has improved coordination and ensures seamless service delivery in the event of a disaster. Sixth, the project developed a mobile application for android phones to enable information dissemination on preparedness and appropriate response mechanisms. The application is aimed to ensure timely communications with affected populations.

26

98. Knowledge and outreach: BDMP developed a comprehensive behavioral change communications strategy to promote awareness and change perceptions about disaster mitigation and response. As part of this strategy, the project also filmed a documentary on the role of the PDMA that was broadcast on local television. Instruction materials distributed through the CBDRM program were developed through this strategy. 99. Institutional Strengthening: BDMP support has been instrumental in enabling PDMA to leverage support from other sources. In 2013-14, the PDMA approached the WFP for warehousing facilities, which are used to stock relief goods. The WFP also provided PDMA with 12 Flospans (pre-fabricated frameless structures used for stockpiling of relief supplies and equipment), each with a capacity to store between 300 and 500 ton of relief goods. Through BDMP and WFP support, the PDMA has been able to establish six relief hubs in six districts across the province (Loralai, Sibbi, Dera Murad Jamali, Khuzdar, Lasbella and Gwadar). The equipment stored at these hubs and in these buildings has been identified as critical in the logistics plan developed by the BDMP. These facilities contribute directly to preparedness and rapid response. 100. As part of the implementation of the DRM plan developed through the BDMP, the PDMA has launched the Rescue 1122 service in Quetta, which is expected to subsume the R3Ts established and trained through the project. The government had released US$1.9 million for the implementation of Phase 1 of this operation and has committed to release another US$3.9 million for the launch of Phase 2, which envisages development of four satellite offices in the city of Quetta. It is expected that the data center and the PEOC established under the BDMP will become the headquarters of this service. Discussions with PDMA staff reveal that the development of the Rescue 1122 service would not have been possible without the capacity developed through the BDMP.

101. Gender and Inclusion aspects: PDMA Balochistan constituted a gender and child cell in May 2012 with the support of the U.N. Entity for Gender Equality and the Empowerment of Women. The role of this cell is to incorporate gender concerns in disaster risk reduction efforts in the province. PDMA Balochistan also formed a management committee for the gender and child cell consisting of a chairperson and two members. The U.N. Entity for Gender Equality and the Empowerment of Women organized a 1-day gender sensitivity training for 12 staff members at COMSATS for PDMA staff. Although this training was not directly implemented through the BDMP, the capacity developed through the project created the platform for such training to be delivered and used to improve service delivery. 102. Within the World Bank, the project has been helpful in promoting cross-global practice (GP) collaboration between education and DRM, which helps capture synergies. The MDTF portfolio in Balochistan consists of three main projects: the GSP, the PGEBP, and the BDMP. PGEBP has been involved in the construction of schools in 12 districts of Pakistan that have become ‘launch pads’ for the implementation of the CBDRM plan. This collaboration provided a substantial multiplier to BDMP in terms of outreach.

103. Comparing PDMA Balochistan to other PDMAs in Pakistan: Amongst the provincial disaster management authorities, which were notified by the provincial governments mandated under the NDM Act 2010, PDMA Balochistan is the foremost authority benefiting from the long-term and holistic engagement aimed at capacity building and institutional strengthening. A long-term systematic support received through BDMP has assisted PDMA Balochistan to emerge as one of the progressive authorities among peers. Under BDMP support, Balochistan was the first province to update provincial DRM Plan, initially developed in 2008 with the support of UNDP.

27

Punjab and Sindh are set to update plans under Bank’s latest engagements with the respective authorities. The continuity of support under BDMP ensured Balochistan to lead in development of Logistics Plan and CBDRM Plan, and in conducting hazard and risk assessment of Quetta. Hazard and risk assessments of Lahore and Karachi were conducted with the support of Bank through technical assistances. Punjab is the only other province to develop a DMIS, a web-based dashboard developed with the support of WFP, operationalized during emergencies to compile damages & loss, and response data. BDMP supported PDMA Balochistan in desegregating PDMA functions and acquiring necessary manpower and capacity to deal with disasters, however other provincial disaster management authorities still face understaffing.

104. Comparing BDMP to other projects within the MDTF portfolio, it is clear that BDMP has performed very well, having utilized 95 percent of its resources and with a satisfactory rating for performance. Of the 11 projects in the overall portfolio, seven have closed and only two have been given a satisfactory rating (excluding BDMP). Total disbursements of BDMP stand at 95 percent; only two other projects (KP and FATA rural roads projects) exceed this number.

4. Assessment of Risk to Development Outcome 105. The overall rating of risk to development outcome is estimated to be high. 106. Although efforts are being made to sustain the achievements of the BDMP, the success of these efforts depends upon the continued commitment of the GoB to the DRM agenda, which remains uncertain. 107. Implementation of the capacity enhancement plan: Although the PDMA is making every effort to ensure that the recommendations of the capacity enhancement plan are implemented in full, particularly those pertaining to the regularization of key staff, it is uncertain whether the government will create and/or regularize the proposed positions. The PDMA has made its recommendations, but the decision to revise provincial civil service structures is complex and requires high-level political commitment, which may not be forthcoming. To mitigate this risk, the PDMA is hiring staff on contract basis to ensure continuity of its operations. Also, the project facilitated purchase of equipment as detailed in the capacity enhancement plan, but no mechanism has been developed for its maintenance. 108. Maintaining the DMIS: Although the project has established an effective DMIS with a comprehensive baseline information 15 entered, data used to update this system are collected through regular hazard and risk assessments, the first of which was conducted through the project. The PDMA has not institutionalized a system for regular hazard and risk assessments that can feed into the DMIS. There is also no system to track population changes and increases and decreases in public infrastructure, which are essential for improved planning. The project conducted a hazard and risk assessment only for Quetta district but there is a demonstrated need to conduct similar assessments for other disaster-prone areas of Balochistan to ensure comprehensive coverage of the province. 109. As a way to mitigate this risk, the DMIS that the BDMP created has been linked to the national risk assessment and sharing platform, which collates risk- and exposure-related data from various agencies that can be updated based on information provided through dedicated units within

15 This baseline data has been collected through the hazard and risk assessment conducted through BDMP.

28

the NDMA. The World Bank task team has also supported the creation of a national working group on risk assessments, of which Balochistan PDMA is a part. 110. Government Ownership and Commitment: The institutional arrangements of the PDMA lent it substantial credibility and ensured ownership of BDMP activities, but there is a concern that it is mainly driven by individuals such as the current home minister. When these individuals change, it is possible that the ownership of the PDMA and more broadly the DRM agenda, within the overall GoB may also change. GoB’s ambivalence towards DRM was already evident when a change in government leadership led to changing priorities: away from building capacity of the PDMA and DDMAs to resilience. The current PDMA management has made extensive efforts to sustain the momentum created through the project and expand activities initiated through the BDMP, but these efforts have not yet come to fruition, which may influence the future effectiveness of the PDMA as a whole. 111. Nevertheless, in assessing BDMP’s contribution towards sustainability, it is important to recognize that scope and objectives of the project. The project was prepared for US$ 5 million with a very defined set of objectives, contributing towards building capacity of the PDMA to leverage resources from other sources, including the Government as well as development partners. BDMP aimed to and has built the capacity of the PDMA Balochistan to prepare for and respond to disasters in the province. Adding further depth to these achievements is the prerogative of the Government and there is evidence that the Government of Balochistan is being supported by the World Bank in mainstreaming the DRM agenda across relevant sectors. 112. The reliance on external consultants was a necessity at the start of BDMP, to: i) inject capacity particularly for specific skills sets not present within the PDMA Balochistan at the time of project initiation; and, ii) ensure capacity transfer. The implementation of BDMP ensured that the PDMA was able to fill in vacant positions in a timely manner and this process was augmented by the Capacity Enhancement Plan developed by BDMP. Any consultants being hired by the project are slowly being phased out but contractual staff will continue to be hired on a need basis. 113. Within the World Bank, the DRM agenda has been brought to the fore. The current CPS calls for operational DRM and early warning systems in at least two provinces, the milestones of which are multi hazard risk assessments and preparation of provincial DRM plans. Including DRM within the CPS in this manner reflects the CMUs commitment to scale-up the ‘BDMP experience’ in other provinces where these activities have not been implemented at all. 114. The project was implemented on a pilot basis for two main reasons. First, this was the first time the World Bank was engaging with the PDMA Balochistan, which was a nascent organization with limited capacity for resource absorption and budget execution. A larger project would not have been successful, neither in terms of results achieved, nor in terms of disbursements. Second, the approach to initiate a pilot was in line with the MDTF mantra of ‘think big-start small’.

115. One possible way to ensure continuity of activities initiated through the BDMP and to expand project activities at the district level is to integrate this component into the on-going International Development Association (IDA)-funded Integrated Water Resource Management Project being implemented in the Nari and Poralai river basins of Balochistan, with a financing envelope of US$200 million. A similar approach has been developed in Sindh through the Sindh Resilience Project, although it is too early to assess its effectiveness, which will require further dialogue with the client and technical assessment.

29

5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues)

5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Satisfactory 116. The task team designed a strategically relevant project which was fully in line with the World Bank CPS 2010-14 and PCNA recommendations16, particularly those related to enhancing state visibility and state capacity for service delivery (Strategic Objectives 1 and 3). The project was the first dedicated disaster risk management project of the Bank in Pakistan aimed at strengthening institutional capacity for preventive disaster risk mitigation and preparation. Piloting this approach in Balochistan was the rational choice because Balochistan is disaster prone and has a large population that is exposed to natural and man-made disasters. Designed as a pilot, the BDMP experience has increased overall knowledge in the sector that has been applied to other projects prepared and being implemented in Punjab and Sindh. 117. The project design was technically sound and was based on international and national lessons learned in the field of disaster risk management, and the results framework captured a coherent results chain, ensuring realistic progress toward the PDOs. Implementation arrangements were designed to ensure sustainable capacity transfer, particularly for line functions and government ownership was ensured from the start. The emphasis on CBDRM facilitated citizen engagement. Monitoring and evaluation arrangements were set up not only to track performance of the project itself, but also to set up a system for the GoB to improve preparedness. A thorough risk assessment completed at the outset ensured smooth implementation and management of bottlenecks as they emerged. 118. A key design innovation that the BDMP introduced in Pakistan was the CERC, a zero-sum component that could be used to channel resources on an emergency basis as required (e.g., in a disaster). 119. The design of the BDMP was fully in line with the recommendations of the World Development Report 2011, with three key elements: emphasizing flexibility and inclusiveness; ensuring a strong link between the need for intra-agency reform, better service delivery, and development outcomes; and addressing key institutional challenges. The project was designed to cover multiple areas, including community development, inclusion of women and children, and enhancing transparency. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Satisfactory

16 Although the PCNA was conducted for KP and FATA, it was unanimously agreed that the PCNA recommendations were relevant also for Balochistan. The project was designed in the context of the PCNA but was also fully aligned with the Balochistan Development Needs Assessment (BDNA), which was published approximately six months after the project was declared effective.

30

120. The World Bank team was proactive throughout project implementation in monitoring implementation and achievement of results, ensuring progress towards PDOs. In the context of Balochistan, where the security situation has remained uncertain, task team provided supervision with support from a third-party validation firm. At the time of project initiation, the team visited Quetta frequently for supervision, although given adverse security conditions during the latter part of project implementation, the team was unable to visit Quetta, so reverse missions were organized wherein PDMA staff were invited to Islamabad for supervision. All project-related documentation was effectively maintained, enabling the team to provide adequate, timely follow-up. Day-to-day support was also provided as required. Given the nature of the project, no major safeguard problems were triggered. Challenges related to fiduciary management emerged during implementation, but these were resolved through training of PDMA staff, day-to-day supervision by the task team and timely support. A dedicated accounts officer was assigned to the PDMA to ensure separation of key functions, enhance transparency and resolve problems as they emerged. The MTR was conducted as scheduled and no major problems were identified. The project was extended for a year beyond its original closing date, but this was in line with the overall MDTF extensions and ensured effective contract management. 121. Although the CERC was not used during implementation, the project created awareness of this instrument in Pakistan, and it was included in new projects, particularly those being implemented in areas prone to natural or man-made disasters. 122. The restrictions placed on the task team to visit Quetta limited its ability to undertake meaningful dialogue with the Client on the issue of sustainability, transition, and a possible next phase. Even though reverse missions were organized, they were project specific and did not allow for in-depth discussion of strategic challenges. Preparations for MDTF Round II were initiated after the MTR in 2014 and expansion and extension of BDMP was highlighted as a priority of the government at the time, but with a change in government, priorities shifted toward water management, and it was decided that the BDMP would not be extended. A more-meaningful and -sustained interaction with the client could have helped address the question of transition and sustainability after it became clear that the BDMP would not be extended beyond June 2016. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 123. The overall rating for Bank performance is Satisfactory based on rating of satisfactory both for quality at entry and quality of supervision.

5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 124. The GoB was strongly committed to the project during the preparation and implementation phase, particularly through the establishment of the necessary legal and institutional framework, for the PDMA to perform its role effectively. After 2005 and the response to the earthquake, there was strong recognition of a need to operationalize disaster risk management and response capacity within the provinces. In Balochistan, the government’s commitment to DRM was evident from the fact that the Balochistan PDMA had been notified, albeit with a skeletal structure in place (three professional staff) and no defined rules of business. As mentioned above, the PDMA was closely affiliated with the Home Ministry, which strengthened its position as an autonomous body and

31

created an enabling environment for strong collaboration with the World Bank to ensure an effective project. 125. During the course of implementation, with the change in government, development priorities of the province shifted toward water management and resilience, away from capacity building of PDMA/DDMAs. This was in line with the World Bank approach being developed in Punjab and Sindh, but with the reduction in the scale of the MDTF and the government’s decision not to take any further IDA loans, the provincial government, through the Planning and Development Department (P&DD), decided to limit its engagement with the World Bank to the governance and policy reform program. 126. Despite changing priorities, the proposed BDMP expansion through MDTF Round II, estimated at US$10 million, could have been approved if coordination between the PDMA and P&DD had been effective. According to the P&DD, any expansion of the BDMP would have had to be channeled through the Governance and Policy Reform Program that the P&DD is implementing through the Implementation Support Unit, which was set up through the GSP funded in MDTF Round I. The GSP had tested this implementation modality for small investments in selected line departments, and there was concern that the Implementation Support Unit would not be able to manage a large amount of funding using this modality. In addition, involving the P&DD directly for implementation in the manner proposed would not only undermine the institutional mandate of the PDMA, but would also create a layer of bureaucracy, inefficiencies, and increased transactions costs. Such arrangements would also take the P&DD away from its core function of planning and coordination. As a result, it was difficult for the PDMA to have a meaningful dialogue with the World Bank on BDMP expansion. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 127. The implementing agency, PDMA Balochistan, remained committed throughout the life of the project. At initiation stage, the Director General PDMA at the time took his own initiative to mobilise resources from the MDTF, which entailed extensive liaison with the Planning Commission. The PDMA did not have familiarity with these procedures and received limited guidance from P&DD. Once the project was under preparation, there was a recognition that PDMA staff was not familiar with World Bank guidelines. To address this issue, the PDMA teams proactively liaised with the task team on a daily basis to ensure speedy resolution of any issues. During implementation, PDMA worked closely with the Third Party Validation firm based in Peshawar as well as other technical resources provided through the project, to ensure capacity transfer and to identify implementation bottlenecks in a timely manner. When the task team was unable to travel to Quetta due to security limitations, PDMA staff readily undertook ‘reverse missions’ and travelled to Islamabad, which ensured adequate supervision of project activities. Even after closure of the project, the PDMA has continued its efforts to sustain the momentum gained through the BDMP to ensure sustainability of results. As shown in Table 2 in Section 3.3, the PDMA has evolved into a mature institution, as reflected in the attitude, perceptions, and understanding of the staff toward the DRM. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory

32

128. The overall rating for Borrower performance is Moderately Satisfactory based on rating of moderately satisfactory for government performance and satisfactory for implementing agency performance.

6. Lessons Learned (both project-specific and of wide general application) 129. The following key lessons have been identified. Design and Implementation 130. Sequencing of activities is critical for the design and implementation of projects that are aimed at capacity building and institutional strengthening, especially in a low-capacity environment. The BDMP had a simple design with concrete objectives, structured in a manner to allow the project to build on successes gradually and as capacity was built. Capacity building initiatives helped the PDMA to implement the plans that were developed through the project. This process relied heavily on monitoring and data collection related to hazard and risk exposure. 131. Mechanisms to ensure sustainability should be built into project design. These could be in the form of a transition plan to ensure continued commitment of the government and the World Bank. In the case of BDMP, although the PDMA is making every effort to build on the achievements of the project, it remains uncertain whether GoB will remain sufficiently committed to sustain these activities. Sustainability can also be improved by fostering linkages and coordination between the PDMA and other line departments working on resilience such as irrigation (water resource management), agriculture (food security), civil works (infrastructure) etc.

Lessons for Bank Operations 132. Timing of engagement is critical. The achievements of BDMP are heavily dependent upon the fact that the project became effective immediately after the notification of the PDMA Balochistan. The project was able to benefit from the interest and momentum established at the national and provincial levels to support disaster management authorities at all levels and the marginal returns of engaging with a nascent organization such as the PDMA were high. A small engagement like the BDMP can have significant impact, if the World Bank stands ready to engage when an opportunity arises. 133. The level of effort required to prepare projects is independent of the size of the project and nature of projects (e.g. lending vs. grant). This can influence commitment of the World Bank as well as the Government towards larger projects, which offer scale and tangible benefits. Going forward, with the shift towards agile banking, guidelines should be developed to streamline policies and procedures for smaller projects and a conscious effort should be made to implement these.

134. When implementing projects that seek to build disaster management capacity, the most important initiative is a strong capacity enhancement plan, which envisages technological innovation, human resource development and provision of goods.

135. The implementation of BDMP reaffirms the need for beneficiary consultation and feedback.

33

Lessons for World Bank Operations in Fragile, Conflict and Violence Affected Environments 136. Flexibility is critical in addressing the dynamic and rapidly evolving context of FCV environments and within project design, this can be ensured through the inclusion of the CERC. Although the BDMP did not use the CERC during its implementation, this component would have benefited projects such as the FATA Rural Livelihoods and Community Infrastructure Project, FATA Urban Centres Project, and KP Southern Area Development Project, which had to be restructured to address the evolving needs of temporarily displaced persons resulting from military operations initiated in 2014. Such restructuring was cumbersome and time consuming to process and affected the implementation trajectory of the projects. 137. The CERC should remain a zero-sum component with no resource allocation from the outset to avoid committing resources from the outset, as such allocation would be contrary to the aim of the CERC i.e. enhancing flexibility. 138. Project supervision in FCV environments is complex and resource intensive. Arrangements such as ‘reverse missions’ and recruitment of firms for third party validation/monitoring are useful ways to identify and address issues related to implementation bottlenecks and results in situations where the task team may not be able to visit project sites and can be used to augment the capacity of the task team. However, to ensure the effectiveness of such arrangements, it is important to design contracts for such agents/firms very carefully and these firms should be utilized on a day-to-day basis by the task team. In addition, these efforts remain project-specific and proactive effort is required to ensure strategic dialogue on issues that may not be directly related to the project but have a bearing on the sector as a whole. Also, supervision requires significant staff time as well as supervision funds to cover increased security costs.

139. Results frameworks in the FCV context should be realistic and simple, based on a thorough assessment of prevailing implementation arrangements and risks. Establishment of delivery mechanisms in FCV environments should be considered a result in itself and can be measured using a mix of process and output-based indicators.

Mainstreaming the DRM Agenda 140. One way to leverage commitment and foster relevance of the DRM agenda is to mainstream it across other sectors. Although projects such as the BDMP are a necessary step toward building provincial capacity in DRM, there is need to leverage partnerships across other line departments within the government and GPs within the World Bank to enhance outreach and ensure effective planning for disaster management. Embedding governance and capacity building interventions in larger projects, as is the case in Sindh and Punjab can help garner support for DRM and ensure effectiveness. 141. Including a CERC, also in non-DRM specific projects, can enable fast and flexible support following a disaster and is another way to mainstream DRM.

7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing agencies 142. The findings and lessons learned presented in this ICR are based on discussions with the Client at a workshop held on November 25, 2016. Also, please refer to Annex 4 for a summary of the Client’s ICR.

34

143. The implementing agency raised two major challenges: continuation of BDMP and expansion of project activities at the district level, as discussed during the MDTF Round II Consultation Workshop held in Quetta in October 2014, and development of sectoral DRM plans. The activities are essential in developing sustainable capacity for disaster preparedness and response in Balochistan.

35

Annex 1. Project Costs and Financing

(a) Project Cost by Component (in USD Million equivalent)

Components Appraisal Estimate

(USD millions)

Actual/Latest Estimate (USD

millions)

Percentage of Appraisal

Component 1: Institutional Strengthening of PDMA Balochistan

2.5 50

Component 2: Hazard and Risk Assessment

1.0 20

Component 3: Development and Piloting of CBDRM Program

1.5 30

Component 4: Contingent Emergency Response Component

0.00 0

(b) Financing

Source of Funds Type of

Cofinancing

Appraisal Estimate

(USD millions)

Actual/Latest Estimate

(USD millions)

Percentage of Appraisal

Trust Funds 0.00 0.00 MDTF for Crisi Affected Areas of NWFP/FATA/Balochistan

5.00 0.00 .00

36

Annex 2. Outputs by Component17

Result Evaluation Criteria/Indicator Original Target Achievement Project Development Objective Indicators

Improved disaster response mechanism at PDMA Balochistan

SOPs for disaster response prepared and operationalized

Final SOPs approved and implemented

SOPs for emergency response prepared and tested during the first simulation drill, further refined based on results of drill, and approved by PDMA management


2 2

Optional Comment: The SOPs for emergency response were prepared and tested during the first simulation drill, further refined based on the results of the drill, formally notified and institutionalized and adopted by the GoB and PDMA Balochistan. These SOPs were prepared based on international best practices and lessons learned and also contextualized to ensure relevance in Balochistan. They have been formally notified by the Government of Balochistan and adopted by the PDMA. The project conducted two extensive drills, based on lessons learned and international best practice, aimed at testing the coordination and response capacity of the PDMA Balochistan. Various scenarios were constructed including for floods and disasters and the PDMA led the exercise to coordinate response across relevant line departments. Such drills were useful also because they enabled the PDMA to test its communications equipment and its ability to mobilise relief equipment across the province. Given that the project is not being expanded, it is unclear whether these drills will be carried out on a regular basis. This could also affect the revision of SOPs, which in turn, is based on the drills.

Increased awareness of the hazard and risk environment in Quetta

DMIS database established and populated by data collected during hazard and risk assessment

Collection in DMIS of data generated through hazard and risk assessment of Quetta completed

Data generated from hazard and risk assessment of Quetta collated in newly established DMIS; geo-

17 The last ISR in the archived in the system does not reflect the updated achievements of the project. This Results Framework has been updated based on discussions with the PDMA staff and based on reports submitted by the Third Party Validation (TPV) firm.

37

referenced data of various sectors collated in DMIS to serve as baseline

Optional Comment: This data is already being used by the PDMA to better prepare for cyclical flash floods in Quetta. To ensure sustainability, the DMIS has been linked with organizations at national level including SUPARCO and Geological Survey of Pakistan to ensure data sharing to the extent possible.

Enhanced capacity at PDMA Balochistan for implementing CBDRM initiatives

Number of master trainers trained 15 20

Number of training sessions for responders in selected union councils in Quetta

4 5


10,000 20,000

Optional Comment: The project surpassed its targets for this indicator, training 20 master trainers against a target of 15, organizing 5 training sessions against a target of 4 and distributing 20,000 instruction materials against a target of 10,000. The outreach for the last evaluation criterion was high because these materials were distributed through the schools supported by the MDTF supported Promoting Girls Education in Balochistan Project (PGEBP) across 12 districts. The curriculum for trainings conducted through the CBDRM initiative was developed based on national and international best practices. Technical specialists were hired to deliver the trainings to staff of GoB, local NGOs and staff of public schools. The CBDRM material was prepared as part of the overall behavioural change communications strategy developed by the project. This material has been vital in enhancing community knowledge regarding preparedness and, after the October 2015 earthquake, have been adopted as the national standard by the National Disaster Management Authority (NDMA).

Intermediate Results

Component 1 – Institutional Strengthening of PDMA Balochistan

Enhanced capacity at PDMA Balochistan to collect, manage, and communicate information

DMIS database established DMIS database set up DMIS established and data generated from hazard and risk assessment of Quetta collated in DMIS; geo-referenced data of various sectors collated in DMIS to serve as baseline

Enhanced communications systems in place at the Provincial Emergency Operations Centre

Emergency communications equipment operationalized and integrated into response regime

Improved communications systems in place, including short messaging service alert system, call management system, fleet tracking system, communication SOPs, directory listings.

38

The communications equipment purchased through the project was integrated within the response framework of the PDMA, emphasizing coordination across line departments and PDMA teams across the province.

Development and effective implementation of disaster response procedures at Balochistan Disaster Management Authority

Balochistan DRM plan 2008 reviewed and updated

Draft strategic updates agreed on after discussion with stakeholders

Balochistan DRM plan 2014-19 prepared for presentation and approval at next Provincial Disaster Management Commission, which Chief Minister chairs

PDMA capacity assessment undertaken and enhancement plan prepared

Capacity enhancement plan discussed with stakeholders and finalized

Capacity assessment completed and capacity enhancement plan implemented through procurement of various equipment as indicated in plan

SOPs for PDMA on disaster response prepared Proposed systems tested and plans revised


2 2

Number of exposure visits by PDMA Balochistan staff undertaken and knowledge products produced

3 4 (two in Pakistan and two in Thailand)

Component 2: Hazard and Risk Assessment

Increased technical capacity at PDMA to manage implementation and dissemination of hazard and risk assessments

Scoping exercise completed and methodology for hazard and risk assessment finalized

Methodology for hazard and risk assessment finalized

Methodology finalized; hazard and risk assessment for Quetta completed.

Data collected and collated in DMIS repository Collection in DMIS of data generated through hazard and risk assessment of Quetta completed

Data generated from hazard and risk assessment of Quetta collated in newly established DMIS; geo-referenced data of various sectors

39

collated in DMIS to serve as baseline

Information collected during hazard and risk assessment on sources of risk available to all departments and agencies responsible for development planning in provincial capital

Results from hazard and risk assessment published and disseminated to departments and agencies responsible for development planning

Hazard and risk assessment of Quetta completed, baseline data collated in DMIS will be updated through national risk assessment sharing platform

Component 3: Development and Piloting of CBDRM Programme

Increased capacity of PDMA Balochistan to engage with local communities for improved preparedness

Provincial CBDRM program developed Provincial CBDRM program agreed upon with stakeholders and finalized

Provincial CBDRM plan developed and available at PDMA, along with training modules

Number of training sessions conducted by master trainers in selected union councils in Quetta

5 5

Logistics plan developed for emergency stockpiles in selected districts


Logistics plan developed; checklist for emergency stockpiles part of plan

Number of pilot stockpiles established in Quetta based on logistics plan

2 2


10,000 20,000

Component 4: Contingent Emergency Response Component

NA NA NA NA

40

Annex 3. Grant Preparation and Implementation Support/Supervision Processes

(a) Task Team members

Names Title Unit Responsibility/Spe

cialty Lending/Grant Preparation Haris Khan Task Team Leader, DRM Specialist SASDU

Raja Rehan Arshad Lead Operations Officer SASDU

Shahnaz Arshad Senior Urban Specialist SASDU

Suhaib Rasheed DRM Consultant SASDU

Shiraz Ali Shah DRM Consultant SASDU

Ahsan Tehsin DRM Consultant SASDU

Zahir Ali DRM Consultant SASDU

Rehan Hyder Senior Procurement Specialist SARPS

Javaid Afzal Senior Environment Specialist SARDI

Samina M. Islam Social Development Consultant SASDS

Waseem Kazmi Financial Management Specialist SARFM

Anwar A. Bhatti Financial Analyst SACPK

Chau-Ching Shen Senior Finance Officer CTRFC

Shabir Ahmad Program Assistant SASDO

Shahnaz Meraj Program Assistant SASDO

Sameena Dost Senior Counsel LEGES

Jack Campbell Peer Reviewer GFDRR

Armando Guzman Peer Reviewer LCSUW

Supervision/ICR

Haris Khan Task Team Leader, Sr. DRM Specialist

GSURR

Marc S. Forni GSURR Ahsan Tehsin DRM Specialist GSURR Zohaib Jamal Khan Durrani DRM Consultant GSURR Maha Ahmed Rural Development Specialist GFA

(b) Staff Time and Cost

Stage of Project Cycle Staff Time and Cost (Bank Budget Only)

No. of staff weeks USD Thousands (including travel and consultant costs)

Lending

Total: 0.00 Supervision/ICR

Total: 0.00

41

Annex 5. Summary of Grantee's ICR and/or Comments on Draft ICR

1. Project Context, Development Objectives, and Design The Balochistan Disaster Management Project is a project under the Multi Donor Trust Fund amounting to US$5.0 million and aimed at the institutional strengthening of the Provincial Disaster Management Authority (PDMA) Balochistan.

2. Key Factors Affecting Implementation and Outcomes

During the course of implementation of the project, the main factors that affected the project:

1. frequent transfer of PDMA management 2. natural disasters for which the project team was mobilized 3. law and order situation of the province 4. number of activities new to the province or country or have limited information, therefore,

difficulty in getting the product, information, or required service

Factor that might affect outcomes:

1. Lack of sustainability because of lack of human resources 2. recurring cost of services 3. maintenance and usability of equipment

3. Assessment of Outcomes (Annex 2. Outputs According to Component)

1.1 Component 1: Institutional Strengthening of Provincial Disaster Management Authority (PDMA) Balochistan

Intermediate Result Indicator

Output Indicator Year 3 Target Status

Enhanced capacity at PDMA Balochistan to collect, manage, and communicate information

Enhanced communications systems in place at PEOC

Emergency communications

equipment operationalized and integrated

within response regime

Well-equipped PEOC has been established and operationalized and is supported by data center

Development and effective implementation of disaster response procedures at PDMA

SOPs for PDMA on Disaster response prepared

Proposed systems tested and plans

revised

SOPs have been tested through drills, but one more drill to be conducted before monsoon season

Number of drills or simulation exercises completed to assess

2 One drill completed, one

42

operational coordination, preparedness, and emergency protocols

drill to be conducted before monsoon season

Number of exposure visits by PDMA Balochistan staff undertaken and knowledge products produced

4

2 national exposure visits planned for April 1 to 7 and April 12 to 18, 2015. PDMA staff and officers from line departments

1.2 Component 2: Hazard and Risk Assessment



Increased technical capacity at PDMA to manage implementation and dissemination of hazard and risk assessments

Scoping exercise completed and methodology for hazard and risk assessment finalized

Methodology for hazard and risk assessment finalized

Assessment completed

Data collected and collated in DMIS repository

Collection in DMIS of data generated through hazard and risk assessment of Quetta completed

Data generated during assessment collated in DMIS

Information collected during the hazard and risk assessment on sources of risk available to all departments and agencies responsible for development planning in the provincial capital

Results from hazard and risk assessment published and disseminated to departments and agencies responsible for development planning

Provincial workshop will be held and report disseminated among stakeholders as assessment is completed

1.3 Component 3: Development and Piloting of CBDRM Program



Increased capacity of PDMA Balochistan to engage with local communities for greater preparedness

Provincial CBDRM program developed

Provincial CBDRM program agreed upon with stakeholders and finalized

CBDRM program developed after consultation with all stakeholders

Number of training sessions conducted by

5 5

43

master trainers in selected union councils in Quetta Logistics plan developed for emergency stockpiles in selected districts


Logistics plan developed, including checklist for emergency stockpiles

Number of pilot stockpiles established in Quetta based on logistics plan

2 2


10,000 PDMA have printed 20,000 IEC material disseminated in community, 3,000 awareness material distributed during community training, overall target of 10,000 Nos of dissemination met

4. Assessment of Risk to Development Outcome

During negotiation stages of the project, an operational risk assessment was completed. At the completion stage of the project, the risk assessment is as follows.

1. DRM Risks: It was determined that unclear roles and responsibilities are a major risk in DRM, but clear roles and responsibilities are mentioned in the provincial DRM Plan (emergency response SOPs) and have been tested in drills and simulations.

2. Capacity: Project was focused on increasing the capacity of PDMA Balochistan; training sessions (national & international) on different areas were conducted; information technology infrastructure was developed and equipment was procured.

3. Governance: Qualified technical staff were hired for the project. 4. Design: The hazard and risk assessment of Quetta City was completed and; earthquake

contingency planning of Quetta city is in process on the basis of the study. 5. Social and Environmental: There was o risk involved throughout the project phase. 6. Implementation and sustainability: Implementation of project was achieved satisfactorily

because qualified staff were hired, but sustainability of outcomes is important.

5. Assessment of Bank Performance

Bank performance was satisfactory; the Bank DRM project team helped in successful achievement of PDOs.

6. Lessons Learned

1. Work plan must be more realistic while working with consultancy firms. 2. PMU should be separate from implementation agency to avoid delays and undue influence. 3. Design of project should ensure that outcomes are sustainable after completion of projects.

44

Annex 1. Economic and Financial Analysis

US$5.0 million has been invested in disaster management in Baluchistan, investment was in capacity building and development of documents related to DRM. Capacity of PDMA has been improved; systems have been designed and implemented.

Media awareness campaigns were conducted, communities were trained in how to prepare for, respond to, and mitigate disasters, which will save lives and limit economic losses.

A hazard and risk assessment of Quetta City was completed that provides recommendations to save lives from earthquake, flood, and drought. Implementation of the recommendations will boost the economic and financial sectors of the city.

Based on improvement and strengthening of PDMA due to the project, a positive image has been portrayed at the provincial and national levels.

Annex 2. Stakeholder Workshop Report and Results

Five behavioral change communication and advocacy strategy sessions were conducted in Quetta. Some 125 individuals from different districts and sectors (media, international NGOs (INGOs), NGOs, political figurehead) participated in sessions.

Two provincial and two national consultative workshops on DRM and CBDRM were held, with participants from different departments, INGO, NGOs, community-based organizations, and technical agencies attending the workshops.

Two hazard and risk assessment sessions on Quetta City were conducted in Islamabad, with experts from different technical agencies attending and providing input into the assessment process.

Five trainings on CBDRM conducted by 20 master trainers were held in different union councils, with approximately 125 community members trained.

Three consultative sessions were conducted in which the Home Minister, Chief Secretary Balochistan, Home Secretary, Additional Chief Secretary, Chairman Chief Minister Inspection Team, Senior Member, Board of Revenue, and other secretaries were briefed on project achievements.

45

Annex 8. List of Supporting Documents

1. Aide Memoires 2013-162. Implementation Status Reports 2013- 63. Annual Report—Balochistan Disaster Management Project, PDMA Balochistan, 2012-134. Balochistan Consultations Report—World Bank Group Country Partnership Strategy FY2015-

19, 20135. Country Partnership Strategy for Pakistan FY2010-13—World Bank, July 30, 20106. Country Partnership Strategy for Pakistan FY2015-19—World Bank, April 4, 20147. Country Partnership Strategy Progress Report for Pakistan FY2010-14—World Bank,

November 16, 20118. Disaster Risk Management—Pakistan Strategy Note FY2012-16, World Bank, 20139. Disaster Risk Management Plan 2014-19—PDMA Balochistan, 201410. Emergency Project Paper—Balochistan Disaster Management Project, June 201211. Interim Report—Balochistan Disaster Management Project, PDMA Balochistan, 201412. Organizational Capacity Assessment & Enhancement Plan, PDMA Balochistan, January 201413. Progress Review Note—BDMP Progress Review, 05/2015, 08/2015, 11/201514. Report on Perception Survey—PDMA Balochistan, 201415. Third-Party Monitoring and Validation Final Report—Balochistan Disaster Management

Project, PDMA Balochistan, 201516. Balochistan Disaster Management Project17. An Approach towards Conflict Reduction and Stabilization through Disaster Risk Reduction18. BDMP Training Completion Reports19. Balochistan Public Sector Development Programme

The World Bankdocuments.worldbank.org/curated/en/416971488305832727/pdf/ICR00003580... · the world...

Documents

Transcript of The World Bankdocuments.worldbank.org/curated/en/416971488305832727/pdf/ICR00003580... · the world...