Evaluation of the Integrated Safeguards System Draft Inception Report – Annexes

April 2019

Evaluation Task Manager: Monica Lomena-Gelis: m.lomena-gelis@afdb.org

Annex 1 - AfDB Integrated (Environmental and Social) Safeguards System – ISS ............................... 1

Annex 2 - ISS in the context of the Bank’s Project Cycle ..................................................... 10

Annex 3. Reconstructed Theory of change of the Integrated Safeguards System ............... 13

Annex 4. Desk validation tool used to assess the projects in relation to ESS requirements. 15

Annex 5. OS2 assessment proposal (compensation/resettlement of affected population) ... 19

Annex 6. Financial intermediaries operations assessment proposal .................................... 33

Annex 7. – Evaluation matrix ............................................................................................... 56

Annex 1 - AfDB Integrated (Environmental and Social) Safeguards System – ISS

Development of the ISS The African Development Bank (the Bank) acknowledges its Integrated Safeguards System as “a cornerstone of its strategy to promote growth that is socially inclusive and environmentally sustainable, and a powerful tool for identifying risks, reducing development costs and improving project sustainability.”1 The system was approved in December 2013 and became effective July 1, 2014 following an extensive review of the existing environmental and social safeguards policies and processes. The review was initiated in light of Board agreement that the safeguards required updating to meet Bank requirements for the management of environmental and social risk in the current Bank portfolio, to support mainstreaming of environmental and social sustainability for Bank financed projects, and to bring them more into line with other development banks. In particular, the Bank was concerned that the Safeguards should be better aligned with the Bank’s new policy priorities, including the High 5’s, and to support the implementation and key operational priorities of the Bank’s Ten-Year Strategy to promote inclusive green growth (2013 – 22) (see Figure 1).

1 Preamble to AfDB’s Integrated Safeguards System, Policy Statement and Operational Safeguards, Safeguards and Sustainability Series, Volume 1, Issue 1 (Dec.2013)

Figure 1. AfDB’s 2013-22 Ten Year Strategy – Key Operational Priorities and relevance of the ISS

Source: Adapted from AfDB Sustainability Series, Volume 1, Issue 1

The safeguards review was conducted using a highly consultative process, in particular through five regional workshops in Nairobi, Lusaka, Libreville, Abuja, and Rabat with a range of stakeholders including governments, private sector and civil society. The main issues addressed were the need for:

increased alignment with the Bank’s new policies and strategies2

adoption of current international good practice, lessons learned and emerging issues, including climate change

adaptability/versatility of policies and practices to evolving lending products and financing modalities3

greater harmonization with other multi-lateral financing institutions4

improved tailoring of approaches to different clients with varying capacity to implement, and

improved internal processes and resource allocation.

The review process highlighted the following specific issues in the application of the safeguards prior to 2013:

high transaction costs associated with the time required to consult diverse and at times contradictory policy documents, with conflicting priorities

uneven coverage of issues resulting in important issues receiving insufficient attention, including social issues

insufficient coverage of emerging issues, such as climate change

uneven coverage of projects owing to resource constraints and lack of focus on certain types of lending

2 The High 5s, and the ten year strategy to promote inclusive green growth in particular 3 Including greater focus on program based lending (PBO) and financing of financial intermediaries (FIs) 4 There is a general move within the world of international project finance to increase harmonization of standards where practical, to simplify co-financing arrangements and to decrease the burden of multiple requirements on Borrowers

difficulties monitoring borrower compliance with requirements

the approval of some investment proposals without sufficient environmental and social due diligence

lack of E&S risk management in the private sector portfolio, with policy based or program lending, and FIs

the absence of sufficient technical information on many projects, to categorize projects effectively according to environmental and social risk

significant shortfalls in categorization of projects within SAP5 (up to 50% prior to 2014) resulting in ongoing compliance failures throughout the project cycle

significant shortfalls in the number of staff available to manage safeguards in conjunction with an expanding portfolio

associated failure to effectively manage E&S risks across the portfolio with the potential for increased numbers of projects to be referred to the Independent Review Mechanism (IRM)

lack of organized archiving system for projects documents especially environmental and social safeguards documents

a range of additional weaknesses and inconsistencies in application of the ESAP including:

o lack of clarity around roles and responsibilities, and steps required o information management for project monitoring or knowledge generation

lack of Borrower capacity to implement the safeguards on projects, requiring more support from Bank staff.

In response to the review, and issues identified, the existing safeguard policies were consolidated into a single Integrated Safeguards System for implementation across the Bank’s entire portfolio, to simplify and standardize the process. Some of the changes included in the ISS were:

improved policy coherence, including a revised and systematic environmental and social assessment process to help ensure strong and efficient management of project risks and impacts

inclusion of emerging issues, such as cultural heritage, climate change, labour, biodiversity and eco-systems services

inclusion of some important social issues including community impact analysis

clear categorization process for projects

more systematic and meaningful consultation, including specific process requirements, as well as a requirement that consultation is to be free, prior and informed (FPIC) and achieve broad community support for projects

introduction of the Integrated Safeguards Tracking System (ISTS) to facilitate verification of project compliance

greater harmonization with Bank’s strategies, national systems and other MDBs

greater clarity on scope, requirements and responsibilities application of the safeguards to all Bank operations including program-based lending

providing budget support, lending for regional or sector investment programs managed by

the client or borrower, and private sector lending including through financial intermediaries and corporate loans

clear policy on how projects trigger specific policy requirements

mandatory Strategic Environmental and Social Assessment (SESA) for upstream operations such as budget support and client management investment programs

use of Environmental and Social Management Frameworks (ESMFs) for regional and sector wide programs

5 SAP is the Bank’s corporate business management software / database system

use of Environmental and Social Management Systems (ESMS) by Financial Intermediaries (FIs) to systematically manage E&S issues, appropriate to the nature and scale of operations

more flexibility in application to accommodate differing requirements of public and private sector projects, especially timelines

increased focus on strengthening country systems

clearly established requirements for resettlement including the contents of a Resettlement Action Plan (RAP), and

clear definition of vulnerable groups, including requirements for assessment and protection

establishment of a central archiving system for all project documents During the develpment of the ISS, and in a paper outlining the strategic choices made by the Bank in the ISS design6, several issues were highlighted as outstanding challenges that needed to be addressed. The following are a sub-set that, based on document reviews and preliminary interviews during the inception phase, may continue to be relevant today:

Ensuring effective implementation of ESMF’s by Borrowers for regional and sector wide programs

Ensuring effective implementation of ESMS’s by FIs

Improving the timely compliance of all Bank operations with the categorisation requirement, and ensuring 100% compliance

Ensuring adequate levels of supervision of ESMP implementation

Providing additional resources and technical capacity where needed to support effective ISS implementation

6 Integrated Safeguards System Working Progress: Strategic Choices made in the design of the ISS (AfDB, March 2012)

Evolution of the Bank’s organizational structure to deliver the Environmental and Social Safeguards

Figure 2. Evolution of the organizational structure dealing with environmental (and social) mainstreaming / safeguards.

Source: Update from previous IDEV (OPEV) evaluation about environmental mainstreaming in the transport sector (2013)

1996-2002

VP-Operations

Environment & Sustainable

Development Unit (OESU)

2002-June 2006

Policy , Planning & Research Complex

Sustainable Development &

Poverty Reduction Unit (PSDU)

Since June 2006

Regional Operations Complex

Sustainable Development

Division

(ORPC.3)

June 2008 to 2017

Presidency, Chief Operating Officer

Compliance & Safeguards Division

(ORQR.3)

Gender & Social Development

Monitoring Division (ORQR.4)

Sector/ Infrastructure Operations Complex

Natural Resources & Environmental Management

Division (OSAN.4)

Environment andClimate Change

Division (ONEC.3)

Climate Change Unit (CCCC)

operational since 2011

(non-affiliated )

Gender, Climate, & Sustainable Development

Unit (OSUS) (unit dissolved in 2010 )

2017 to present

Senior VP Office

Safeguards & Compliance

Department (SNSC)

VP Power, Energy, Climate and Green

Growh

Climate and Green Growth (PECG2)

VP Agriculture, Human and Social

Development

Gender and women empowerment

(AHGC1)

Civil Society and Community Engagement

(AHGC2)

The State of Environmental and Social Safeguards Instruments in the AfDB The new 2013 Integrated Safeguards System (ISS) comprises an Integrated Safeguards Policy Statement, five Operational Safeguards, and two practical guidance notes. This system builds on, consolidates and integrates the individual safeguard policies, procedures and cross cutting policies that were developed in an ad hoc fashion starting in 1999 (see Figure 2). The ISS supersedes the provisions in previous policies on environmental and social safeguards and compliance7.

Figure 3. Evolution of the Integrated Safeguards System

Source: Adapted from 2017 AfDB Safeguards & Compliance Unit presentation

Goal and Objectives of the ISS The goal of the ISS is to promote the sustainability of project outcomes by protecting the environment and people from the potentially adverse impacts of AfDB-financed projects, thereby supporting long term and sustainable development, or green growth, in Africa. Specifically, the safeguards objectives are to:

Avoid adverse impacts of projects on the environment and affected people, while maximising potential development benefits to the extent possible;

Minimise, mitigate, and/ or compensate for adverse impacts on the environment and affected people when avoidance is not possible; and

Help borrowers/clients to strengthen their safeguards systems and develop the capacity to manage environmental and social risks.

Policy Statement The Policy statement defines the objectives of the ISS, and the general commitments of the Bank to the management of environmental and social risks on projects and programs. It also sets out the basic roles and responsibilities of both the Bank and the Borrower. The commitments include:

7 AfDB Safeguards and Sustainability Series, Volume 1, Issue 1 (Dec. 2013) pg. 5

Systematic assessment of impacts and risks

Application of the ISS to the entire portfolio

Proportionality and adaptive management

Transparency, good governance and inclusivity

Protecting the most vulnerable

Promoting gender equality and poverty reduction

Harmonisation and facilitation of donor co-ordination

Compliance monitoring and supervision of safeguards, and strengthening country systems

Country level grievance and redress mechanism

The Bank’s Independent Review Mechanism (IRM)

The Bank’s approach to safeguards compliance and the management of E&S risk

Operational Safeguards The 5 Operational Safeguards (OS) set out the operational requirements for the Borrower.

They are designed to support the integration of the assessment of environmental and social

impacts into operational processes and promote sustainability and long-term development, by

preventing projects from adversely affecting the environment and local communities and

ensuring that risks are properly minimized, mitigated and/or compensated (see Table 2).

The specific objectives of the Operational Safeguards are to:

Mainstream environmental, climate change and social considerations into Country

Strategy Papers (CSPs) and Regional Integration Strategy Papers (RISPs)

Identify and assess the environmental and social impacts and risks – including those

related to gender, climate change and vulnerability – of bank lending and grant-

financed operations in their areas of influence

Avoid or, if avoidance is not possible, minimize, mitigate and compensate for adverse

impacts on the environment and on affected communities

Provide for stakeholders’ participation during the consultation process so that affected

communities and stakeholders have timely access to information in suitable forms

about Bank operations, and are consulted meaningfully about issues that may affect

them

Ensure the effective management of environmental and social risks in projects during

and after implementation, and

Contribute to strengthening regional member country (RMC) systems for

environmental and social risk management by assessing and building their capacity to

meet AfDB requirements set out in the ISS.

Table 1. The five Operational Safeguards and their applicability

Operational Safeguards Applicability and Objectives

OS1: Environmental and

Social Assessment

This overarching Operational Safeguard applies to all projects. It governs the process of determining a project’s Environmental and Social category and associated assessment requirements, scope of application including on a project’s vulnerability to climate change and associated resilience, assessment of vulnerable groups, public consultation and disclosure, grievance and redress mechanisms, and determines the applicability of the Bank’s other Operational Safeguards.

OS2: Involuntary

Resettlement, Land Acquisition, Population Displacement and Compensation

Applies to any Bank-financed operation that causes the involuntary resettlement of people. It is intended to avoid resettlement where feasible or minimize its impact when necessary; and ensure that when people must be displaced, they are treated fairly, equitably and in a socially and culturally sensitive manner. It covers both physical and economic displacement, compensation at full replacement cost, and the requirement that host communities share in development opportunities.

OS3: Biodiversity and

Ecosystem Services

Applies to Bank operations that: (i) are located in any type of “habitat;” (ii) implicate ecosystems which potentially affected stakeholders depend

upon for survival, sustenance or primary income (ecosystems services);

(iii) extract renewable resources as a primary purpose; or (iv) involve the use and commercialization of an indigenous knowledge

system.

This OS outlines the requirements for borrowers to: (i) identify and implement opportunities to conserve and sustainably

use biodiversity and natural habitats; and (ii) observe, implement and respond to requirements for the

conservation and sustainable management of priority ecosystems, including prevention of introduction of invasive alien species.

OS4: Pollution Prevention

and Control, Greenhouse Gases, Hazardous Materials and Resource Efficiency

Intended to foster high-quality environmental performance, and efficient and sustainable use of natural recourses over the life of a project. Includes a specific section on dealing with waste & hazardous materials, and GHGs. Specific requirements and thresholds are set out in the World Bank Environmental Health and Safety Guidelines (EHS Guidelines) and other applicable international standards.

OS5: Labour Conditions,

Health and Safety Designed to help protect workers’ conditions, rights and protection from abuse or exploitation, building on ILO standards, and the UNICEF Convention on the Rights of the Child

Under OS1, projects are categorized according to the potential for the project to have adverse

environmental and/or social impacts on the environment or society (see Table 3). Category 1

projects are those with the greatest likelihood of adverse impact, and are therefore the greatest

focus for resources going forward. Category 2 projects are projects likely to have detrimental

environmental and social impacts and less adverse than those of Category 1. Category 3

projects are projects with no environmental and social impacts. Category 4 applies to Financial

Intermediaries only.

Table 2. Project Categorization under OS 1 Category Description

1 Bank operations likely to cause significant environmental and social impacts, including any

project requiring a Full Resettlement Action Plan (RAP).

2 Bank operations where likely environmental and social impacts are few in number, site-

specific, largely reversible and readily minimized by applying appropriate management and mitigation measures.

3 Bank operations with negligible adverse environmental and social impacts. Following

categorization, no further action is required, though assessments may be beneficial in managing potentially unintended impacts on affected communities.

4 Bank operations involving lending to an FI that lends to or invests in sub-projects that may

produce adverse environmental or social impacts. FIs are required to develop and implement an Environmental and Social Management System (ESMS).

4 FI-A A FI with a portfolio that is deemed high-risk and may include sub-projects that can be

categorized as Category 1.

4 FI-B A FI with a portfolio that is deemed medium-risk and may include sub-projects that can be

categorized as Category 2.

4 FI-C A FI with a portfolio that is deemed low-risk and includes sub-projects that can be categorized

as Category 3 only.

Practical and Technical Guidance In addition to the OS, the Bank developed practical and technical guidance materials to replace older guidance, and to provide support to Bank staff and Borrowers on the implementation of processes and procedures associated with the ISS:

the Environmental and Social Assessment Procedures (ESAP)8, and

the Integrated Environmental and Social Impact Assessment (IESIA) guidance notes (see Figure 2).

The ESAP replaces the 2001 version, and describes the steps required to implement the ISS at every stage of the project cycle, for both public and private sector lending. The IESIA Guidance Notes provide technical guidance for the Bank and its borrowers on specific methodological approaches or standards and management measures relevant to meeting the requirements of the ISS. It includes guidance on aspects of the project cycle where compliance has been weak, general guidance on ESIA, topic-specific guidance relevant to specific themes and requirements, and sector-specific guidance (brief key sheets).

Preliminary information about the ES risk profile of the Bank’s portfolio

IDEV has not found any data about the Environmental and Social risk profile of the AfDB’s ongoing

portfolio. Recent data from the Compliance Division of the Bank (SNSC) estimate that the ongoing

portfolio at the Bank is mostly composed of category 2 projects (42%), followed by category 1 projects

(30%). Category 3 projects represent 28% of the portfolio with data about the environmental

categorization in Bank’s internal database (SAP)9. It is likely that part of the projects without an ES

categorization in the internal database SAP are category 4 projects.

When using the information about the ES categorization reported by SNSC (previously ORQR) in its

Annual Reports, the profile changes a bit, with category 4 projects becoming a higher percentage of the

approvals in recent years (see Figure below). These figures are only estimates. As the data collection

for this exercise demonstrated, a significant percentage of category 4 projects which underwent the ES

categorization projects and were even approved, were finally not signed or cancelled.

Figure 4. Evolution of the ES category of the portfolio approved by year

Source: self-elaboration on the basis of SNSC data10.

8 This ESAP should not be confused with the other commonly used acronym for Environmental and Social Action Plan 9 These figures were estimated for the SNSC retreat in 2017. The analysis could not use a total of 339 for which the environmental categorization was not included in SAP. It is likely that lots of them are category 4 projects, since the analysis only reported 1 ongoing category 4 projects out of the 116 analyzed. 10 Data reconstructed from the ES categorization conducted by ORQR/SNSC and reported in their Annual Reports (2015 and 2017 does not cover the whole year according to those reports).

23%

20%

18%

31%

31%

32%

32%

27%

28%

34%

23%

16%

17%

14%

15%

22%

12%

5%

2010

2015

2016

2017

cat 1 cat 2 cat 3 cat 4 missing

Annex 2 - ISS in the context of the Bank’s Project Cycle The Bank´s project cycle comprises key stages before Board approval (project identification, preparation and appraisal) and during implementation/completion. The project cycle fits into a country programming one. Across this cycle, different ES outputs are developed by the borrower and validated by the Bank, depending on the type and E&S risk of the intervention. The operationalization of the ISS at the project level entails various due diligence, review and supervision activities that the Bank undertakes to ensure that the borrower comply with the ISS requirements. (See figure below).

Figure 5. ISS across the Project Cycle

Source: 2017 AfDB Safeguards & Compliance Unit presentation

Management of Environmental and Social Risks at Project Identification For public sector projects, the borrower is first expected to provide baseline data and internal screening/scoping documents in accordance with the national system of environmental and social screening during the project identification phase. On the basis of these data, sector departments conduct initial environmental, social and climate change scoping to determine the appropriate Environmental and Social category. Sector Departments submit a Request for Categorization Memorandum to the Bank’s Safeguards and Compliance team (Safeguard’s team). The Safeguards team reviews the categorization and suggests revisions as necessary. If approved, the Safeguards team provides a Validation of Categorization Memorandum. For private sector projects, the Bank will normally enter later in the process, once project identification has been completed.

Management of Environmental and Social Risks at Project Preparation For public sector projects, subsequent to the validation of the proposed categorization, the

Bank notifies the borrower of the project categorization and specifies the various studies

required under Bank policy. The borrower prepares the Terms of Reference for the required

studies, for which the Bank may provide technical assistance funds. It is expected that affected

communities, vulnerable groups and other local stakeholders are consulted on the Terms of

Reference. The Terms of Reference are reviewed by the Sector Team before being reviewed

by the Safeguards team. The Terms of Reference are revised as required and relevant

information is included in the Project Concept Note (PCN). The Safeguards team conducts a

compliance check of the PCN and Terms of Reference as part of the Readiness Review.

For private sector projects, an initial assessment of environmental and social issues is

conducted by the Safeguards team during the Project Concept Review stage. This initial

review is intended to identify potential issues that require examination through more detailed

studies and identify a preliminary Environmental and Social categorization for the project.

Issues identified during this initial review help inform the Terms of Reference for more in-depth

Environmental and Social impact assessments, as required.

Management of Environmental and Social Risks at Project Appraisal At the Project Appraisal Stage, all public sector projects identified as category 1 or 2 must

prepare an Environmental and Social Impact Assessment (ESIA) and Environmental and

Social Management Plan (ESMP), as applicable. During appraisal, a site visit should be

conducted to ensure compliance with national legislation and determine the adequacy of the

assessment, revising the operational safeguards applicable (see below) and the project

categorization as appropriate.

The borrower finalizes the required studies based on comments received, at which point the

Safeguards team reviews the reports for compliance and provides feedback as necessary.

Once cleared by the Safeguards team, the studies are posted publicly before being cleared.

In providing clearance of these assessments, the Safeguards team specifies conditions for

the loan agreement. The information from the studies is then incorporated into the Project

Appraisal Report and Results-Based Logical Framework. The Safeguards team provides a

final clearance through the Readiness Review of the Appraisal Report.

Private sector projects similarly undergo in-depth Environmental and Social Impact

Assessment during the appraisal phase led by the Safeguards team. The Safeguards team is

then responsible for conducting a due diligence review of the studies, verifying: (i) compliance

with Bank policy and national legislation; (ii) that public disclosure requirements have been

met; and (iii) that sufficient financial provisions have been made in the PAR for implementation

of the ESMP. If these requirements are met, the Safeguards team issues an Environmental

and Social Compliance Note, which is sent to the Country Team for review alongside the

Project Appraisal Report.

Management of Environmental and Social Risks during Supervision and at

Completion Both public and private sector projects require close supervision during implementation from

Board approval through completion. According to IESIA Guidelines (Vol.1, p.54), supervision

of ES issues by environmental and social specialists should be done regularly especially for

high-risk Category 1 projects.

Supervision is focused on ensuring effective implementation of the measures set out in the

ESMP. However, to be effective, it may also include ensuring that any specific loan conditions

are fulfilled prior to commencing disbursement, and also, for example, ensuring that issues

from the loan conditions and the ESMP are reflected in the Invitation to Tender for any

contractors, who are usually in charge of the implementation of the ESMP and control of

potential environmental and social impacts.

For Public Sector projects, the Implementation Progress and Results Report (IPRR) is

designed to capture results through indicators tracking the provisions of the Bank’s

Environmental and Social Safeguards requirements. The types of indicators most commonly

used are focused not on the quality of implementation, but on the timeliness of implementation

of the environmental and social management plan (ESMP) and/or the Resettlement Action

Plan (RAP). A specific annex in the ESAP 2015 details the ES information that shall be

incorporated into Section C.1 of the IPRs (compliance with covenants), see table below.

Table 3. ESS information to be included in the IPR.

13

Annex 3. Reconstructed Theory of change of the Integrated Safeguards System

14

15

Annex 4. Desk validation tool used to assess the projects in relation to

ESS requirements. INTEGRATED SAFEGUARDS SYSTEM DESK-BASED EVALUATION AND SCORING TOOL The environmental and social Desk-Based Evaluation and Scoring Exercise (ISS desk validation) will be based on a document review of the sampled Public Sector and Private Sector Operations, including both qualitative and quantitative analyses for specific project characteristics.

Project approved before ISS entry into force (before June 2014), more flexibility on the scoring is applied.

Project approved after ISS entry into force (after July 2014) The objectives of the Exercise are:

1. Determine the extent of compliance with the Bank’s Safeguards requirements; 2. Identify the types of quality challenges during identification, monitoring and mitigation of

environmental and social impacts of Bank projects; 3. Identify challenges to the operationalization of Safeguards requirements at Entry and during

Supervision; 4. Examine trends in environmental-social categorisation over time; and 5. Examine differences between existing ratings for environmental and social dimensions in

supervision and completion reports and the external experts’ assessment.

Bank’s safeguards-related documents/sections of documents reviewed (Please leave only the documents effectively used in the desk validation including their date; delete those that do not apply).

Documents considered in the current Desk-Based Evaluation of ISS

Public Sector Private Sector

Request for categorization and/ or memo of Validation of Categorization

ESIA/ESMP

RAP (Resettlement Action Plan), if applicable

ESCON Memo (Env-Social compliance note, if available)

Readiness Review – PAR – sections about safeguards

Project Appraisal Report – sections about safeguards

General supervision reports (BTOR, aide memoire/IPR)11

Safeguards supervision reports, if any

Project Completion Report, PCR

Request for categorization and/ or memo of Validation of Categorization

ESIA or ESMS for category 4 (env-social management system)

ESCON Memo (Env-Social compliance note, if available)

Project Appraisal Report– sections about safeguards

General supervision reports (BTOR, aide memoire/PSR)1

Safeguards supervision reports, if any

Project Completion Report, XSR

11 The Lattanzio’s sector specialists in charge of the desk validation of the project will flag to the Lattanzio’s safeguards specialists any general supervision report where safeguards issues are mentioned.

16

QUALITY AT ENTRY

Judgement Criteria

Highly Satisfactory Satisfactory Unsatisfactory Highly Unsatisfactory

All elements of the Bank QA processes are integrating social and environmental issues with full compliance with ISS, at Quality at Entry

Most elements of the Bank’s QA processes are integrating social and environmental issues with full compliance with ISS, at Quality at Entry

At least one of the Bank QA processes is not integrating social and environmental issues with full compliance with ISS, at Quality at Entry

Several elements of the Bank’s QA processes are not integrating social and environmental issues with full compliance with ISS, at Quality at Entry

EVALUATION QUESTIONS (use Bank documents only) INDICATORS EXPERT COMMENT

Operational Safeguard 1: Environmental and Social Assessment

1.1 Has the Project’s E & S category been correctly assigned?

Was an ESIA required? (if NO go to Question 1.2) If YES,

Was the Scope appropriate?

Was the Direct Area of Influence appropriate?

Were all potential significant impacts identified?

Were all significant impacts to be mitigated included in ESMP?

1.2 Was an SESA required? (if NO go to Question 1.3) If YES,

Was the Scope appropriate?

Were all potential significant impacts identified?

Were all significant impacts to be mitigated included in the ESMP?

1.3 Was the project screened and categorised following the Bank’s Climate Screening Manual12? (if NO go to Question

1.5) If YES,

1.4 Was an Adaptation Review and Evaluation Procedures (AREP) included in parallel to the ESMP?

1.5 Is the Bank Summary of the ESMP adequate?

Was the full ESMP validation finished before approval or some additional studies were included as conditions in the loan

agreement or any other arrangement?

Is the Summary uniquely referenced?

Is the Summary finalised and signed off?

Does the Summary describe the impacts to be mitigated?

1.6 Is there a fully documented stakeholder and community consultation?

Were community impacts identified and addressed appropriately?

Were vulnerable groups identified and addressed appropriately?

1.7 Has a grievance mechanism been described?

12 The Climate Safeguards System, which is part of the ISS, has been piloted since 2014, and includes climate risk

screening and adaptation review for project categorization, so that potential measures may be identified to protect the investment against climate risks and to promote opportunities for adaptation. Information about the application of the CSS is stored in a different Bank online database from that of the other ISS applications, the Climate change tracking system.

17

Operational Safeguard 2: Involuntary Resettlement

2.1 Was a Resettlement Action Plan needed? (if NO go to Question 3) If YES,

Is the Bank Summary of the RAP adequate?

Is the Summary uniquely referenced?

Is the Summary finalised and signed off?

Does the Summary describe the impacts to be mitigated?

2.2 Has compensation at full replacement cost been applied?

2.3 Is there a likely prospect that the outcome will:

Improve standard of living?

Improve income-earning capacity?

Improve overall means of livelihood?

Address gender adequately?

Address age adequately?

Not disenfranchise Project-Affected People?

Operational Safeguard 3: Biodiversity and Ecosystem Services

3.1 Has the ESIA/ESMP identified any specific impacts on biodiversity and ecosystem services and propose adequate mitigation measures?

3.2 Has the sustainable use of natural resources been taken into account by the Project?

Operational Safeguard 4: Pollution Prevention and Control, Greenhouse Gases, Hazardous Materials and Resource Efficiency

4.1 Does the ESIA/SMP validated by the Bank include a full inventory of impacts arising from pollution, waste or hazardous materials?

4.2 Is there vulnerability analysis and monitoring of GHGs?

4.3 Is there an appropriate monitoring and compensatory framework?

Operational Safeguard 5: Labour Conditions, Health and Safety

5.1 Does the ESIA/SMP validated by the Bank include adequate information about protection for workers’ conditions, rights and protection from abuse and exploitation?

5.2 Does there appear to be adequate health and safety practices?

5.3 Does there appear to be adequate measures taken for avoiding child or forced labour?

DISCLOSURE

18

Were the requirements for disclosure of the ESIA and ESMP summaries on the Bank’s website respected according to the safeguards category of the project?13

QUALITY OF SUPERVISION Judgement Criteria

Highly Satisfactory Satisfactory Unsatisfactory Highly Unsatisfactory

All elements of Bank QA processes are integrating social and environmental issues with full compliance with ISS, at Quality of Supervision

Most elements of the Bank’s QA processes are integrating social and environmental issues with full compliance with ISS, at Quality of Supervision

At least one of the Bank QA processes is not integrating social and environmental issues with full compliance with ISS, at Quality of Supervision

Several elements of the Bank’s QA processes are not integrating social and environmental issues with full compliance with ISS, at Quality of Supervision

EVALUATION QUESTIONS (use Bank documents only) INDICATORS EXPERT COMMENT

Operational Safeguard 1: Environmental and Social Assessment

1.1 Is progress on the implementation of the ESMP satisfactory?

1.2 Is progress on the outcomes of stakeholder and community consultation satisfactory?

1.3 Are community impacts identified and addressed, being followed up satisfactorily?

1.4 Are actions concerning vulnerable groups being followed up satisfactorily?

Operational Safeguard 2: Involuntary Resettlement

2.1 Is progress on implementation of the RAP satisfactory?

2.2 Is progress on the compensation at full replacement cost satisfactory?

2.3 Is improvement of the standard of living satisfactory?

2.4 Is the improvement of income-earning capacity satisfactory?

2.5 Is improvement of the means of livelihood satisfactory?

2.6 Is improvement of the standard of living satisfactory?

2.7 Improve overall means of livelihood?

2.8 Does gender continue to be addressed adequately?

2.9 Does age continue to be addressed adequately?

Operational Safeguard 3: Biodiversity and Ecosystem Services

3.1 Is the project mitigating adequately its impacts on biodiversity?

3.2 Does the sustainable use of natural resources continue to be promoted by the Project?

Operational Safeguard 4: Pollution Prevention and Control, Greenhouse Gases, Hazardous Materials and Resource Efficiency

4.1 Is the inventory of impacts arising from pollution, waste or hazardous materials being maintained?

4.2 Is there ongoing monitoring of GHGs?

4.3 Is there ongoing monitoring of the compensatory framework?

Operational Safeguard 5:

13 For Category 1 projects, these shall be disclosed for 120 days for public sector projects and at least for 60 days for

private sector operations. All category 2 operations shall be disclosed for 30 days before Board deliberations (page 3 of ESAP, 2015).

19

Labour Conditions, Health and Safety

5.1 Is there ongoing adequate protection for workers’ conditions, rights and protection from abuse and exploitation?

5.2 Are there ongoing adequate health and safety practices?

5.3 Are there ongoing adequate measures taken for avoiding child or forced labour?

Annex 5. OS2 assessment proposal (compensation/resettlement of

affected population) By mid-January 2019, IDEV managed to identify a sample of 41 projects which were approved since ISS

became effective (July 2014) and which triggered OS2. Among those, 19 were included in the review of

the first stage of the evaluation14.

The assessment of OS2 compliance in the framework of this evaluation will involve a three-stage

evaluation process:

1) Rapid assessment of the projects which have triggered OS2 which have been approved by the Board after July 2014 to the end of 2017.

2) Deep Dives on selected themes 3) Interviews with key stakeholders

1. Rapid Assessment A rapid assessment will be carried out on the universe of Projects having been approved between July 2014 and the end of 2017. It was agreed not to include projects approved in 2018 to maximize the chances to get projects with a significant advance of the resettlement/compensation of the population. The ideal set of documentation reviewed at this stage is summarized in the table below. Table 4 Typology of Documents to be collected as part of this assessment

Bank Generated Borrower Generated Others (for instance, external party report, other donor,

national authorities…)

Pre-Bank’s approval

- Project Appraisal Report (PAR)

- RAP Summary

- Environmental and Social Impact Assessment (including ESMP)

- Resettlement Action Plan (RAP)

- External Due Diligence reports

Implementation - Implementation Progress Report (IPR)15

- Aide Memoire (AM) - Back to Office Reports (BTOR)

- Advance reports (quarterly/annual) of the implementation of the RAP

- Lenders Technical Advisors reports

- Lenders E&S Monitoring Consultants reports

14 There are 18 review fiches since one project entailed two loans but the same RAP and ESIA (Mauritania and Senegal Rosso multinational bridge). 15 Or Project Supervision Reports (PSR) for private sector projects.

20

Completion - Environmental and Social Completion Report (ESCR)

- Project Completion Report (PCR)

- RAP Completion Report - Lenders Technical Advisors reports

- Lenders E&S Monitoring Consultants reports

The list of projects and the state of documentation found is available in a separate Excel.

This rapid assessment tool has been largely crafted on the basis of the AfDB ISS Guidance material (2015)

and will assess:

1) Project Preparation and Appraisal a. Assessment of Borrower’s RAP b. Assessment of Bank summaries (RAP Summary and PAR sections about the RAP) c. Assessment of Loan Agreements

2) Project Implementation a. Assessment of IPR, as well as Aide Memoires/BTOR (or PSR for private sector projects)

when available 3) Project Completion

a. Assessment of ESCR and PCR (or XSR for private sector projects), The Rapid Assessment Tool template is included at the end of this Annex.

2. Deep Dives

For a subset of key thematic issues, detailed Deep Dives will be carried out. The selection of deep dive key

themes was carried out during the inception phase of this assignment and drawn from a combination of

(i) identification of key issues based on IRM documentation, (ii) key issues identified in the AfDB

Involuntary Resettlement Policy: Review of Implementation (2015), (iii) the results of this assignment’s

Rapid Assessment. The final selection of Deep Dive key themes was consulted with the Evaluation

Reference Group Members.

Theme #1: Stakeholder Engagement

OS 1 and 2 and the Volume 2: guidance on safeguards issues (2015) provide detailed accounts of the

requirements for, and associated guidance related to, participation, consultation and Broad Community

Support (BCS) as well as the management of grievances in the context of Bank financed operations

involving involuntary resettlement in particular, and also in the context of Bank-financed projects more

generally. RAPs should describe how the participation requirements of OS 2 have been fulfilled, and

specifically how the requirement for BCS has been accomplished in the project context.

IRM reports have highlighted this theme as a key area related to complaints received. Comparators also

report this theme as a recurrent source of complaints. Therefore, IDEV ISS Evaluation included this issue

in Question 4.4 of the Evaluation Matrix. Consistent with OS 2, the Stakeholder Engagement theme will

primarily focus on stakeholder engagement with Project Affected Persons (PAPs) rather than broader CSO

consultations aspects which is a broader and higher level point than OS 2 that should be done at the

Project level (as part of OS 1).

21

Early findings from BDEV’s OS 2 Rapid Assessment show significant discrepancies between Bank policy and guidance and Project documentation on public participation, including demonstrating broad community support, consultation and participation during implementation and specific approaches taken for consultation with vulnerable groups. The objective of this deep dive will be focused on assessing to what extent the project design documents reported on adequate stakeholder engagement activities performed before project approval and what is described to ensure stakeholder engagement throughout project implementation. Therefore, the Deep Dive will explore how this requirement is met by a subset of projects, highlighting ‘good practices’, what are comparator organizations policies and practices with regards to consultation (e.g. Free, Prior and Informed Consent – FPIC). This will inform recommendations on how the AfDB could reinforce this area. Theme #2: Vulnerable Groups OS 2 states that member countries and other borrowers/clients are responsible for protecting the physical, social and economic integrity of vulnerable groups. There is a specific section on “vulnerable groups’ identification and inclusion in development’ in Volume 2: guidance on safeguards issues (2015). The Resettlement Policy Review of Implementation (2015) noted that “all PAPs are generally treated equally, without special attention to vulnerable groups”. The preliminary findings of BDEV’s OS 2 Rapid Assessment has identified that resettlement planning and implementation provide very limited attention to vulnerable groups, despite the policy and guidelines of the AfDB. Vulnerable groups are not systematically identified in baseline assessments, are not provided with specific consultation opportunities and seldom provided with specific compensation or resettlement assistance. This Deep Dive will explore how this requirement is met, what are comparator organizations policies and practices with regards to vulnerable groups. This will inform evaluation recommendations on this area. Theme #3: Supervising OS 2 implementation & implementation challenges

The effective implementation of OS2-related requirements present significant challenges on the ground, which may hinder the realization of all the mitigation measures included in Resettlement Action Plans.

BDEV’s rapid assessment has identified significant deficiencies in the reporting of supervisions exerted by the Bank regarding OS 2. RAP implementation advance is very cursory in IPRs and BTOR/Aide Memoires, for projects triggering OS 2. Despite being included systematically in Loan Agreements of projects triggering OS 2, very few borrower RAP implementation reports could be assessed16.

The deep dive will explore in more detail implementation supervision processes for OS 2, including the participation of E&S officers in supervision missions and, to the extent possible, the skills set of the officer supporting Task Managers. The deep dive will also try to capture any available data about RAP implementation challenges, although preliminary findings suggest that this information is very few and spares in Borrower and Bank supervision reports. Finally, it will also describe how comparator organizations are supervising resettlement issues (including the World Bank’s newly introduced Tracking Social Performance system). This will inform recommendations on how the AfDB could reinforce this area.

16 Despite having started data collection in December 2018, the BDEV received only four (4) individual Borrower’s

RAP progress reports out of a total of 36 projects. All Bank staff involved in those interventions (E&S officers and

Task/Portfolio Managers) has been contacted. The lack of an integrated archiving system hampered data collection.

22

3. Interviews Interviews with Bank’s Safeguards Specialists will be held to inform the Rapid Assessment and the Deep Dives (Project Task Managers and Borrowers for the deep dives, E&S department staff, staff from the Bank’s Independent Review Mechanism, comparator institutions. Interview protocols will be developed later. [Note: the protocols will be developed after desk review of documents to develop initial desk based rapid assessment and deep dives] A focus group with task managers whose projects triggered OS2 and social safeguards will be handled to discuss the preliminary findings from the desk review of the 36 projects.

23

Rapid Assessment Tool (template) for projects which triggered OS2

IDEV’s OS2 compliance Rapid Assessment Tool for the ISS Evaluation (2019)

This document presents a rapid assessment tool to assess the quality and comprehensiveness of the Bank’s and Borrower’s documentation related to OS2 at

key stages of the Project Cycle against the policy and ISS guidance materials.

Project Stage Documents solicited by IDEV for the Rapid Assessments

Documents received and used by IDEV for the Rapid Assessment

(specify document title and date) Project Preparation & Appraisal Borrower’s RAP

Bank’s RAP summary

Bank’s PAR sections covering OS2

Loan agreement related to OS2

Project Implementation Bank’s IPRs or PSRs/BTOR/aide memoires

Project Completion Bank’s completion reports (ESCR end PCR/XSR) covering OS2

24

1.Project Preparation & Appraisal

Assessment of the quality of the Borrower’s RAP: The borrower is responsible for preparing Resettlement Actions Plan, when applicable, during Project Preparation/Appraisal

Dimension /

linkage to Evaluation

Matrix (EM)

Criteria17 Outcome [ Yes,

Partially or No18 ]

Rationale 19

1. Project Description

Adequate general description of the project and its area of influence, including a map

2. Potential Impacts

Adequate description of the project components or activities that would give rise to involuntary resettlement, the zone of impact of such activities, and the alternatives considered to avoid or minimize involuntary resettlement20.

3. Organizational / Institutional

arrangement to ensure RAP

implementation

Adequate description of the institutional arrangements to ensure adequate and timely implementation of the RAP (responsibility of the executing agency, third party stakeholder or, interagency coordination if necessary).

Adequate description of the RAP implementation process, including a detailed timetable21 and an estimation of costs and overall budget22.

17 The criteria are largely based on ISS Guidance Materials, 2015 18 The assessor will use professional opinion to assign each criterion as having been met (Yes), partially met (Partially) or Not Met (No). This will be justified in the rationale column. The assessment outcome is defined as follows – Yes: requirement has been largely met, no gaping holes are identified; Partially: requirement has been addressed but has one or two gaping holes; No: requirement has not been met, having 3 or more gaping holes. [OPTION: In some instances, these criteria can be applied differently and will be justified in the rationale column]. 19 Specify RAP page where this is addressed and add any other justification for the assessment, including examples/narrative when possible which could be used in the evaluation reports (examples of good or not-so-good practices). 20 The “avoid or minimize” process should be clearly and comprehensively demonstrated in the RAP document. 21 The RAP should provide an implementation schedule covering all resettlement activities, from preparation through implementation, including target dates for achievement of expected benefits to displaced people and host communities, and target dates for terminating the various forms of assistance. 22 The RAP should provide detailed tables indicating the breakdown of cost estimates for all resettlement activities, including allowances for inflation and other contingencies, a timetable for expenditures, sources of funds, and arrangements for the timely flow of funds.

25

(to inform EQ

4.2 of the evaluation

matrix)

Adequate the monitoring and evaluation system for the RAP. This should comprise monitoring23 during implementation as well as ex-post evaluation24.

Adequate description of the capacity of the executing agency to implement the RAP (capacity of staff and necessary financial resource), including capacity building measures (with an associated timetable and budget) when needed

4. Community participation and

meaningful consultation

[related to EQ4.4 of the EM]

Adequate written proof of meaningful consultation25

Adequate description of how community participation and consultation will happen during implementation.

Adequate information about the specific approach taken to consultation with vulnerable groups26

5. Project-level grievance requirements - [Related to EQ4.5 of the EM]

Adequate detailed description of the grievance redress mechanism that has been developed

23 The RAP should include arrangements for monitoring of resettlement activities by the implementing agency, supplemented by independent third party monitors as appropriate. The section should lay out the timing of monitoring activities (including quarterly reviews, in-depth reviews of mid-term progress etc.) and should be consistent with overall project scheduling. This section should also describe how sufficient monitoring information will be obtained, what the performance monitoring indicators will be, and how the monitoring outcomes will be recorded and documented. 24 The RAP should explain that an independent ex-post evaluation – in the form of a RAP completion audit or RAP implementation completion report – will be carried out by the borrower or client and the Bank - and include relevant information in the Project Completion Report (PCR). It should be clarified that this PCR will be followed by the Bank’s own PCR, and a commitment should be made in the FRAP that, if either of these assessments reveals that any key objectives of the FRAP were not achieved, follow-up measures shall be developed to remedy the situation. 25 An explicitly written statement should be provided in the RAP that embodies the agreements reached from the negotiations with affected people (including displaced persons, host communities, government representatives, NGOs and other stakeholders), as a guarantee that consultations were conducted in a meaningful way and that BCS was obtained for the involuntary resettlement process being carried out. In addition, meeting minutes, focus group minutes, interview minutes and attendance records should be kept and appended to the RAP. A summary of what was discussed in each consultation, and how the outcomes of the consultations were fed into the resettlement planning process. 26 Vulnerable status can be determined by identifying a group’s likelihood of facing harder conditions as a result of the project, owing to such factors as gender, economic status, ethnicity, religion, cultural behaviour, sexual orientation, language or health condition. Depending on the specific context of the project, vulnerable groups may thus include female-headed households, those below the poverty line; the landless, some categories of children (orphans, homeless), marginalised social groups and indigenous peoples, those without legal title to assets; ethnic, religious and linguistic minorities; and those who are physically handicapped. Guidance on the identification of vulnerable groups and vulnerability assessments is found in the Integrated Environmental and Social Impact Assessment (IESIA) Guidance Notes (ISS Policy Statement and OS, page 26. AfDB, 2013). The RAP should spell out the specific approach taken to consultation with vulnerable groups, including Indigenous Peoples and those affected by gender vulnerabilities.

26

6. Integration with host communities

Adequate written proof that a meaningful host community capacity assessment has been carried out, and that potentially negative impacts on host communities have been addressed by the project27, including the development of conflict resolution mechanisms to resolve integration issues, if deemed necessary.

7. Baseline data about the PAPs

Adequate level of detail of the socioeconomic baseline conditions in existence prior to the implementation of the involuntary resettlement process28.

8. Legal framework

applicable to the RAP

Adequate description of the applicable legal and administrative procedures related to resettlement in the domestic context29

Adequate description of the international standards to which the resettlement operation is seeking to adhere, how these align with / differ from domestic legislative requirements, and how the gaps between the two are to be addressed in the project context.

9. Description of eligibility criteria

of PAPs

Adequate detailed eligibility criteria used in the project context; namely eligibility criteria that define all affected people for compensation and other resettlement assistance, including vulnerable groups.

10. Valuation of, and

RAP should describe the proposed types and levels of compensation under local laws and the supplementary measures required to achieve full replacement cost for lost assets30.

27 The RAP should also provide details of the measures being taken to augment public services (e.g. education, water, heath and production) in host communities, in order to make them comparable to the services provided to those subject to displacement. 28 A baseline survey should include a population census that covers 100% of those facing displacement impacts (including seasonal resource users that may not be present in

the project area of influence during the time of the survey), and that gathers information on: a) existing public infrastructure and services;

b) production and land tenure systems (including common property and non-title based land ownership or other local allocation systems); c) household organization, d) livelihoods

(including natural assets upon which the affected people may depend for a portion of their livelihoods); e) standards of living; and

––gender and age disaggregated information pertaining to the economic, social and cultural conditions of the affected population, as well as a asset inventory that covers 100%

of those facing displacement impacts, and that seeks to assess the magnitude of the expected losses, both in terms of total and partial losses to individual as well as group

assets. The baseline survey should identify and provide detailed information on disadvantaged or vulnerable individual and groups, for whom special provisions may need to be

made.

29 This should include a description of the remedies available to displaced persons in the judicial process, the normal time frame for such procedures, and available alternative dispute resolution mechanisms that may be relevant to the project. 30 The RAP should detail the methodology to be used in valuing losses to determine their full replacement cost. It is crucial that a certified professional valuator is detailed local market evaluation study that is written up and justified appropriately in the RAP. The RAP should describe in detail the packages of compensation and other resettlement measures which will be provided to ensure that standards of living, income-earning capacity, production levels and overall means of livelihood for all categories of eligible group

27

compensation for losses

11. Management of influx

The RAP should detail any measures being implemented to prevent influx of ineligible persons at the selected sites, together with the legal arrangements being put in place for regularizing tenure and transferring titles to the displaced persons.

12. Env. management of the resettlement site

Adequate assessment of the environmental impacts of the proposed resettlement site and measures to mitigate and manage these impacts.

Assessment of the quality of the Bank’s summary of the RAP

The Bank prepares RAP summaries of the borrower’s RAP documentation and provides a synopsis of the RAP in the Project Appraisal Report. The RAP summaries are disclosed before presentation of the Projects to the Board. Assessment of the Bank’s RAP summaries

The RAP summary should include key aspects to confirm the compliance of the Borrower’s RAP with the requirements of OS2. While the contents of the ESIA summary is included in the ESAP, 2015 (annexe 8), the ESAP does not provide specific guidance on the RAP summaries, only the FRAP/ARAP outlines are provided (Annex 13: Format of Report: Full Resettlement Action Plan /Abbreviated Resettlement Action Plan). IDEV will confirm with SNSC if there is any other internal guidance for ESS specialists about the minimum content and the types of analyses to be included in the summaries.

Dimension Criteria Outcome [Yes, Partially, No]

Rationale

1. Project Description Same as (borrower )RAP assessment (section 2.1)

2. Potential Impacts Same as RAP assessment (section 2.1)

3. Organizational / Institutional arrangement to ensure RAP implementation

Same as RAP assessment (section 2.1)

4. Community participation and meaningful consultation Same as RAP assessment (section 2.1)

are improved beyond pre-project levels. The RAP should summarize the compensation packages developed, in the form of an entitlements matrix that identifies: a) all categories of affected people; b) all types of loss associated with each category; and c) all options for the types of compensation and resettlement assistance to which each category is entitled.

28

5. Project-level grievance requirements Same as RAP assessment (section 2.1)

6. Integration with host communities Same as RAP assessment (section 2.1)

7. Baseline data about the PAPs Same as RAP assessment (section 2.1)

8. Legal framework applicable to the RAP Same as RAP assessment (section 2.1)

9. Description of eligibility criteria of PAPs Same as RAP assessment (section 2.1)

10. Valuation of, and compensation for, losses Same as RAP assessment (section 2.1)

11. Management of influx Same as RAP assessment (section 2.1)

12. Environmental management of the resettlement site Same as RAP assessment (section 2.1)

13. Compliance with the disclosure requirements deadlines according to the ES category31

The RAP summaries should be disclosed through the Bank’s website and field offices32

Assessment of Bank’s PAR sections related to OS2

According to the ESAP, 2015, section 4.6 of the PAR, covering environmental and social impacts should cover the resettlement aspects. The ESAP, 2015, provides guidance on inclusion RAP related information which informs the assessment table below (see ESAP, 2015, annex 23 Environmental and Social Content in Project Appraisal Report). It is also expected that the PAR includes components related to environmental and social aspects in the Project logframe and costing (ESAP, 2015, Step 4.6a).

Criteria Outcome [Yes/Partially/No]

Rationale

Does the PAR mention that a FRAP / ARAP has been prepared / reviewed / disclosed33?

Is the completion date of the RAP indicated in the PAR?

Is the review date of the RAP indicated in the PAR?

Is the disclosure date of the RAP indicated in the PAR?

Are the key resettlement issues that need to be addressed described in the PAR?

Are the main measures proposed to address these resettlement issues described in the PAR?

31 Information to add includes PAR date, date of the RAP summary (if specified) and date of website publication. 32 At least 120 days before presentation to the Board for Category 1 projects (for public sector projects) and 30 days for Category 2 projects (unspecififed public/private). [Note: guidance unclear on public and private sector differentiation] see ESAP 2015, step 4.6a. This assessment is based on the website disclosure only – the field office disclosure could not be assessed. 33 The PAR should describe the disclosure done by the client on the RAP documentation. [NOTE: the guidance is slightly ambiguous if it refers to Borrower’s RAP or the Bank’s summaries. For this assignment, this section is understood to mean the disclosure of the Borrower’s documentation by the Borrower]

29

Are resettlement aspects included in the log frame?

Are resettlement costs included in the Project costing?

Assessment of the inclusion of RAP measures in the Bank’s loan agreement

Loan agreements should include conditions and covenants to cover any outstanding environmental and social concerns that need to be satisfied prior to or after project approval34.

Type Measure35 Applied [Yes/No]

Condition precedent Detailed resettlement action plan

Detailed compensation plan

Acquisition of land right

Submission of a works and compensation schedule and of satisfactory evidence that all Project affected persons in respect of [civil works in a given lot] have been compensated36

Submission of satisfactory evidence indicating that the resources allocated for the compensation and/or resettlement of PAPs have been deposited in a dedicated account in a bank37

Other Conditions Acquisition of Land completed

Evidence of compensation

Proof of environmental acceptability

Undertakings/Commitments Implementation of RAP

Compensation

Periodic Report (ESMP/ESIA/RAP)

Completion Report (ESMP/ESIA/RAP)

34 It should be noted that according to ESAP (Step 5.1), environmental and social loan conditions and covenants should be kept to a minimum. The majority of environmental and social issues should have been resolved prior to this Project Cycle phase. 35 The list of measures will be adjusted to reflect the universe of measures related to OS2 used in loan agreements. 36 Standard RAP clauses prepared by the legal dept. (send to IDEV by SNSC) 37 Standard RAP clauses prepared by the legal dept. (send to IDEV by SNSC)

30

2. Project Implementation

Assessment of the Quality of information to justify the ES rating in the IPR or other Bank’ s supervision report.

As part of the Implementation Progress and Results Report, the Bank should report on environmental and social safeguards in section C.1 on legal covenants. Under this section, a compliance rating should be included and an assessment of the ESMP/RAP implementation should be provided. [Note: the guidance includes only ESMP but it is assumed here that this also includes the RAP when applicable].

Criteria Outcome [Yes/Partially/No]

Rationale

Does the IPR provide a rating on compliance with environmental and social safeguards?38

Is the ES rating in the IPR properly justified?

Does the IPR provide information about the level of compliance of the RAP-related loan conditions?

Is the rating supported by an assessment of implementation of the RAP39

Does the IPR reported the RAP implementation as a source of delay of the overall project implementation? Explain the reasons in the comments column

Does the IPR provide any information about the results of the RAP in terms of livelihood restoration or enhancement?

38 The IPR template includes a row for E&S safeguards overall – it does not specifically address OS2/resettlement issues. 39 The guidance form the ESAP, 2015 annex 23 suggests an assessment should be detailed on E&S issues in section C.1b to describe implementation progress of the ESMP/RAP and outstanding issues. The rating options include the following: (4) Highly Satisfactory: All safeguards measures – as specified in the ESMP – are expected to be met at the time of reporting ; (3) Satisfactory: At least 75% of safeguards measures – as specified in the ESMP – are expected to be met at the time of reporting. Minor delays in compliance (usually 6 – 12 months) are being experienced for conditions not yet met. Actions to address the issues related to unmet conditions are under implementation ; (2) Unsatisfactory: Between 50% and 70% of safeguards measures – as specified in the ESMP – are expected to be met at the same time of reporting. Substantial delays in compliance (usually 6 – 12 months) are being experienced for conditions not yet met. Corrective actions have to be implemented and closely monitored and (1) Highly Unsatisfactory: Less than 50% of safeguard measures – as specified in the ESMP – are expected to be met at the time of reporting. Major delays in compliance (over 12 months) are being experienced for conditions not yet met. Immediate management attention is required and sanctions are envisaged. The IPR template suggests a full report on compliance with covenants should be included in Annex 3 of the IPR.

31

Assessment of the Quality of information in the Aide Memoire/BTOR

Criteria Outcome [Yes/Partially/No]

Rationale

Do the Aide Memoire/BTOR provide an adequate assessment of the RAP implementation?

Is the RAP information included in the aide-memoire and BTOR aligned with the one included in IPRs?

3. Project Completion

Upon completion of a Project, the Bank safeguards specialist should prepare an Environmental and Social Completion Report (ESCR) (ESAP, 2015, Step 7.1). The Project Completion Report (PCR) includes a summary of the ESCR. Private sector projects usually develop an Expanded Supervision Report (XSR) as a completion report40. Assessment of the ESCR and PCR

The format for this content is provided in the ESAP, 2015 (annex 25: environmental and social content in Project Completion Report) and forms the basis for the assessment table below.

Criteria Outcome [Yes/Partially/No] Rationale

Adequacy of description of the RAP41

Adequacy of the assessment of RAP implementation42

Adequacy of the evaluation of results of RAP implementation43

Adequacy of description of other impacts not foreseen in the RAP44

40 As this evaluation purposefully samples projects approved after July 2014 (when ISS became effective), not many completed projects are expected. 41 This section shall outline a brief overview of the Resettlement Action Plan. Particular reference shall be given to mitigation measures that were proposed in order to ensure that the operation is in line with the Bank’s safeguard policies. 42 This section shall provide a thorough analysis of the environmental and social performance rating of the project. The assessment shall be based on the RAP prepared at the project appraisal stage. The assessment shall include: the number of mitigation measures implemented, the extent of implementation, the timeliness and the extent of compliance with the Bank’s E&S safeguard policies and procedures. 43 This section of the report shall evaluate the results of the RAP implementation by clearly indicating the performance of all parties involved (Borrower, Bank, Contractors, Consultants, etc.) in implementing the RAP, whilst giving reasons for successful implementation or failure thereof. 44 Impacts not foreseen at the time of compiling the RAP shall be outlined in this section. Information shall also be provided on the nature and scale of these impacts, as well as how they were mitigated or minimized.

32

Adequacy of description of lessons learned45

45 Lessons drawn from managing the environmental and social issues of this project shall be described in this section.

33

Annex 6. Financial intermediaries operations assessment proposal

The portfolio of the Bank of category 4 projects

As detailed in Annex 1, category 4 are projects involving investment of Bank Group’s funds through Financial Intermediaries (FI) in subprojects that may result in adverse Environmental and Social (ES) impacts. FIs include banks, insurance, reinsurance, and leasing companies, microfinance providers, private equity funds and investment funds that use the Bank’s funds to lend or provide equity finance to their clients. FIs also include private or public sector companies that receive corporate loans or loans for investment plans. FIs operations have many sub-projects for which the resources provided by the AfDB may be used for. In these situations, the FI has the full responsibility to ensure compliance of the subprojects with AfDB’s ISS. For that, the individual subproject need to be screened and appropriate due diligence, monitoring and overall ES management performed in relation to the potential ES risk associated to the subprojects. The Bank’s FI portfolio is channeled through three main types of instruments: lines of credit, equity (funds) and guarantees. The FI portfolio of the Bank Group has been significant for the past decade. As presented in the figure below, lines of credit (LOC) represent an important part of the FI operations of the Bank, ranging from a maximum of 93% in 2011 to 29% in 2018. From 2013, the Bank started increasing the use of guarantee and this resulted in the decrease of LOC and equity funds. The figure below presents the composition of the FI portfolio, which was reconstructed according to the approvals by the Board of Directors. This includes investments which may have been cancelled later for different reasons. Figure 6.Amount allocated to Financial Intermediary operations, in UA million46 (2008-2018).

Source: IDEV’s calculations based on data from Board’s approvals

46 Using the list of Board approvals in the intranet, the evaluation team reconstructed the list of Financial Intermediary operations filtering by type of instrument (line of credit, equity and guaranty), assuming they are the majority of category 4 projects according to ISS. Corporate loans, which sometimes are classified as 4, were not included in this calculation.

298

557

314

698

402 394 412622

1159

585

291

141

142

190

51

134 157 133

73

154

70

76

25

12

743

108

777

239

333

626

0

200

400

600

800

1000

1200

1400

1600

1800

2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018

Line of Credit Equity Guarantee

34

Lines of Credit (LOC) at the AfDB

LOC are long-term loans, either in local or foreign currency, provided by an International Financial Institution

(IFI) to a Financial Intermediary (FI) for on-lending to their customers, also referred to as sub-borrowers.

They aim to improve access to finance for the private sector and to foster the development of financial

markets by enhancing the financial and technical capacity of FIs. LOC is one of the instruments used to

finance private sector47. LOCs provide liquidity and contribute to institutional development and capacity

building. The long-term tenure of the LOCs allow the borrowing institutions to extend the maturity of their

lending operations. LOCs also allowed the Bank to indirectly and efficiently reach a larger number of

enterprises than would otherwise be possible, particularly the SMEs.

Different channels are used to disburse the LOC: commercial banks, microfinance institutions, Development Financial Institutions - DFIs (national and regional development banks), and SMEs. According to IDEV’s calculation based board approvals data, over the period 2011- June 2014, the Bank has approved 28 LOC for UA 1,582 Million. The amount approved almost remains the same (UA 1,487 Million) between July 2014 and October 2018 for 24 operations. Multinational LOCs received the highest amount followed by West Africa, Southern Africa and East Africa. According to a recent IDEV evaluation synthesis48 of LOC, there are several reasons for this geographical distribution, including the strategic choices of AfDB, the high risks for some countries, the size of markets, the quality of counterparts, the competitiveness of Bank pricing, etc.

Equity

Equity is an instrument used by the Bank to promote both private and financial sector development through

cooperation with DFIs and other private institutions.49 Equity is also used by the Bank as an instrument to

promote efficient use of resources and play a catalytic role in attracting investors and lenders to financially

viable projects. The Bank uses this status to hold equity in any public or private enterprises without any

managerial function. By doing so, the Bank can bring in other investors with capital, managerial skills and

technical know-how contributing to the development of capital market in the member countries. The Bank

can also choose to invest through quasi-equity instruments. Equity is usually extended in the Bank's lending

currencies. The final figures of the ongoing equity portfolio will be confirmed for the evaluation report.

Guarantees

Guarantee is an instrument used by the Bank to facilitate the access of the enterprise to local or foreign

currency funding on attractive terms and maturities not otherwise easily available (AfDB, 2003). For the

Bank, guarantees carry risks similar to those for the loans to enterprises and therefore, the projects for

which the guarantees are issued by the Bank, are appraised, processed and supervised in the same way

as projects loans. The Bank’s guarantees are classified into two categories: Partial Credit Guarantees

(PCGs) and Partial Risk Guarantees (PRGs). The PCGs cover a portion of scheduled repayments of private

sector loans or bonds against the risk of default. It can be used to support mobilization of private funds for

project finance, financial intermediation and policy-based finance. PRGs cover private lenders against the

risk of a government, or a government-owned agency, failing to perform its obligations vis-à-vis a private

sector project. According to IDEV’s calculation based on Board approvals, from July 2014 until October

2018, the Bank has approved 07 partial risk guarantee operations for UA 672 Million. Only one of the five

operations was approved for Cote d’Ivoire, the rest being multinational operations.

47 2005 AfDB Evaluation Guidelines for Private Sector’s Lines of Credit Operations. 48 2018 IDEV LOC Synthesis Evaluation. Available at http://idev.afdb.org/en/document/do-lines-credit-attain-their-development-objectives-evaluation-synthesis-2010-2017 49 2003 AfDB. Bank Group Financial sector policy

35

Bank’s Environmental and Social Safeguards (ESS) procedures to manage ES risks

related to category 4 projects As explained in Annex 2, the Bank´s project cycle comprises key stages before Board approval (project identification, preparation and appraisal) and during implementation/completion. The operationalization of the ESS (Environmental and Social Safeguards) at the project level entails various due diligence, review and supervision activities that the Bank undertakes to ensure that the borrower comply with the ISS requirements. Once the project is categorized as ‘category 4” according to the ES risks (see Annex 1), the client is required50 to develop an Environmental and Social Management System (ESMS) that incorporates:

Environmental and Social Policy that states FI commitment to manage ES risks;

Screening, categorization and assessment procedure that categorizes transactions to the Bank’s OS 1 Categories to define the scope of the ES assessment and ensures compliance with the Negative List and Host Country ESHS Laws and Regulations. Financial intermediary subprojects equivalent to Category 1 and Category 2 are subject to the relevant OS requirements, as if they were directly financed Category 1 or Category 2 projects;

Organizational and managerial capacity to implement the ESMS effectively, including dedicated environmental and social staff in the case of a high-risk FI;

Financial resources to achieve required social and environmental performance;

Requirements for monitoring and reporting on subprojects. The FI will disclose a summary of its ESMS to the public and report any major environmental, social or health and safety incident, such as a serious non-compliance, pollution incident, community impact, worker fatality or equivalent to the Bank as soon as is feasible. For Category 1 subprojects, the Bank’s Safeguards & Compliance Division shall review and clear the SESA / ESIA and ESMP (and where applicable the FRAP / ARAP). For Category 2 subprojects, the Safeguards & Compliance Division shall review and clear the ESMP summary (and where applicable the ARAP. In addition, the reporting requirements are based on the FI categorization as below: Category FI-A: The portfolio is deemed high-risk and may include sub-projects that can be categorized as Category 1. The FI should report to the Bank on a regular (quarterly) basis indicating how subprojects have been categorized and providing details of environmental and social assessment agreed with clients for subproject deemed to be equivalent to category 1 or 2. Reporting should also include monitoring of the client’s implementation of ESMP/FRAP. The client will retain qualified and experienced external experts to verify its monitoring information. Category FI-B: The portfolio is deemed medium-risk and may include sub-projects that can be categorized as Category 2. The FI should report to the Bank on an annual on the results of their screening, categorization, application of ESA procedures and ESMP measures agreed with clients. It should also report on monitoring of client’s implementation of agreed mitigation measures. Category FI-C: The portfolio is deemed low-risk and includes sub-projects that can be categorized as Category 3 only. The FI should report to the Bank on an annual basis confirming that its portfolio of subprojects still present minimal risk of environmental and social impact, taking account the possible cumulative effects of its portfolio.

50 According to the ISS, the ESAP and General Guidance on Implementation of OS 1 for Financial Intermediaries

36

Table 5. Bank’s ESS quality assurance requirements across the project cycle for category 4 projects

Cycle stage Bank’s ESS Quality Assurance requirement ES output to be reviewed in this evaluation

Pro

ject

ide

ntifica

tion

, p

rep

ara

tio

n a

nd

app

rais

al (b

efo

re a

pp

rova

l)

Sector Departments conduct screening/scoping to determine the project category. SNSC reviews and validates the category. Sector Departments notify client/borrower of category; client/borrower prepares TOR for ESMS with Sector Departments; Sector Departments integrate TOR of ESMS or existing ESMS into PCN; SNSC engages in compliance check and PCN is subject to Readiness Review (for public FIs). ESAP does not mention what happens with private FIs The borrower develops an ESMS or improves an existing ESMS; Sector Departments conduct due diligence on FI’s ESMS; borrower discloses ESMS or makes available ESMS summary in accordance with in country system; SNSC engages ESMS compliance check; Sector Departments insert synopsis of ESMS/due diligence information document in the PAR and store it in ISTS; PAR is subject to Readiness Review (for public sector FIs); SNSC engages in PAR compliance check. Sector Departments prepare loan conditions and covenants, legal department provides assistance. E&S loan conditions and covenants are integrated in the project loan agreements, which are stored in ISTS.

- Notification of category to client/borrower - ESMS - Disclose or available ESMS -Integration of ESMS summary or due diligence information into PAR -Integration of loan conditions and covenants into Project Loan Agreement/PAR

Pro

ject

imp

lem

en

tatio

n a

nd

su

pe

rvis

ion

51

Sector Departments review Quarterly Implementation Reports done by the borrower and supervise the implementation of the ESMS, with clearance from the SNSC. Whenever non-compliance is observed or unexpected impacts arise, the sector departments request the borrower to review the ESMS. Those changes must be cleared by the Compliance and Safeguards Division. If specific circumstances, SNSC initiates full compliance review.

ES information in PSR52

Pro

ject

co

mp

letio

n Sector Departments integrate ES content into

completion reports which is reviewed by the Compliance and Safeguards Division

XSR with ES content

Self-elaboration for the evaluation on the basis of ESAP 2015, pages 10-11.

51 In addition to periodic project supervision (conducted by sector departments and supported by ES safeguards specialists for category 1, and other projects upon request), SNSC and CRMU conducts additional compliance reviews. 52 The environmental procedures of the Bank for private sector operations (2000) stablish that the Bank’s supervision reports should include a section about the respect of the ES clauses included in the investment contracts. The ESAP 2015 (applicable to private and public sector operations) is less explicit for the private sector. It does not provide a template for the ES section in the PSR.

37

Management of ES risks of category 4 projects by other MDBs and available evidence

from evaluations A recent IDEV evaluation synthesis of LOC reported that the quality and consistency of enforcement of the environmental and social standards remains a problem53. Inadequate monitoring and reporting on the effective implementation of ES standards constitute an issue for LOC. Also, LOC are more challenging than direct investments because the Bank have to rely on the FIs to mitigate ES risks at the sub-project level. The same remark was made by the South Africa CSPE54 which concluded that the lack of ES details at the sub-project level is acute for LOCs. The evaluation synthesis also noted that FIs are not inclined to internalize ES as they do not consider this to be part of their business. The capacity of FIs to understand and implement ES analysis in their operations depend on many factors, including the general institutional setting (e.g. existence and enforcement of environmental law) and previous experience with other MDBs requiring compliance with specific environmental and social safeguards. The Bank cannot protect itself against the reputational risk of financing projects in non-eligible sectors55. In

this respect, both the initial ES risk assessment of the potential clients of FIs and continuous monitoring

become critical. ES risk classifications at project appraisal can be done on a list of projects selected by the

FIs or on the relevant portfolio. In the former scenario, FIs might be tempted to swap high-risk projects for

low-risk projects, in order to comply with the Bank’s ES policy. IDEV’s evaluation synthesis recommends

the Bank to focus on facilitating a cultural change within clients of FIs to increase their level of understanding

and management of ES risk, rather than focusing on the compliance with ES standards in sub-project lists.

The current evaluation will confirm or nuance, if necessary, this conclusion. Additionally, the Bank needs

to secure support from FIs’ higher management level and provide better guidance and training to FI staff.

This should also ensure that the quality of FIs’ ES reporting, which most evaluations deem largely

unsatisfactory, improves. The following section summarizes the AfDB’s experience found so far.

A preliminary summary of the experience of other MDBs is presented at the end of this Annex. This will be

confirmed and further developed, as necessary, through the current assignment.

AfDB experience in ES capacity building Like many other DFIs, AfDB supports FIs under the condition that ES issues are well managed. In 2011, a

research conducted by the AfDB showed that only 38% of a sample of 13 FI clients had an ESMS in place56.

As the main explanatory factors for this situation, the report highlighted low FI internal capacity to handle

ES issues and lack of willingness to invest in ES management. Although many FIs expressed their interest

in ES capacity building, most of them were still concentrated in their financial objectives, according to this

study.

The AfDB funded a series of training sessions on ES management in the African financial sector in 2012

with the aim to support FI in building ES sustainability into their operations. Ten training workshops were

53 2018 IDEV LOC Synthesis Evaluation (draft version) 54 2017 AfDB South Africa CSPE 55 According to the 2013 AfDB ISS policy statement, the non-eligible list includes the following items: alcoholic beverages, tobacco, radioactive materials, platinum, pearls, precious stones, gold and related products, nuclear reactors and related products, weapons, ammunition and other goods used for military and/or paramilitary purposes, luxury consumer goods, and goods harmful to the environment. 56 AfDB: E&S Management in the African Financial Sector: Training and consultancy on environmental and

social management in financial intermediaries and microfinance institutions in Africa. An AfDB-FAPA project Project

Completion Report

38

organized across Africa bringing over 160 participants from 101 FIs (MFIs, SME, Commercial banks,

Central banks, National/Regional Development banks and leasing companies).

The results of the trainings show that approximatively 40% of the participant-FIs have an ESMS which was

implemented. However, most of the FIs admitted to not fully applying the ESMS, but rather, certain aspects

of it such as: the exclusion list, Corporate Social Responsibility (CSR) programs, use of screening tools.

The study also noted that they are some exceptions in countries like Nigeria, South Africa and Rwanda

where the governments or central banks are playing a key role in promoting ES management. The

workshops also came out with two mains drivers for ESMS implementation: the FI’s investors and the

legal environment.

This study grouped FI participants into three categories: re-active (those who see ES issues as something

imposed by regulators or investors), active (those who know that ES issues are inevitable and are taking

necessary actions to include it into their operations), and pro-active (those who see ES issues as part of

the core FI business strategy, including sustainability in the governance of their organization). Another

finding was related to the fact that 80% of the active group has formulated E&S values in

mission / vision, but only 25% has an ESMS in place with their main challenge being lack

of knowledge. Also 70% of the re-active participants had a CSR program and their main

challenge was the lack of strategic alignment. For the pro-active group, the main

challenge remained to monitor and report on its own carbon footprint.

Finally, the study and training programme identified a small group of FIs (two of them are

in the list of category 4 projects considered for this evaluation: Wema Bank and LAPO

Microfinance Bank Ltd, both in Nigeria) to participate in a follow up in-house coaching

session, based on the implementation/adaptation of the FI’s ESMS. While most of the FIs

selected reacted positively, the final report of this experience reported that maintaining

the coaching trajectory was quite difficult and this was due to the fact that most of these

FIs are dealing with market pressure, competition and regulatory scrutiny.

FI operations under evaluation To build the data base of FI operations to be considered in the ISS Evaluation, the evaluation team used

the information in the Bank’s intranet about the Board’s approval (see the first section of this Annex). The

cut-off date used was July 2014, date of entry into force of the ISS. All projects approved from July 2014 to

end of 2018 were included in this list. The second filter used was about the financing instrument of those

projects (lines of credit, equity and guarantees). The evaluation team verified in the Project Appraisal

Reports that the selected projects were effectively categorized as 4, according to the Integrated Safeguards

System. This yielded 91 category 4 projects approved during that period.

During the documentation collection phase, 33 of these 91 projects were removed based on the information

from the Task Managers/Investment Officers because the loan agreement has not been signed as February

2019 or the operation was cancelled. The final list includes 58 category 4 projects, with a total of UA

2,743 Million investment, constitute the universe, which will be evaluated. The Figure below presents

the distribution of these 58 projects per instrument. Lines of credit represent an important share of category

4 projects in the Bank accounting (54% of the total investment during the period), followed by loans (25%).

39

Figure 7. Amount and number of investment per instrument during the evaluation period

Source: IDEV’s Calculation based on board approvals

As reported in the Figure below, around 50% of the category 4 operations (both in terms of number and

amount allocated) are multinational. Specific information about multinational FI operations will be included

later. The Southern Africa Region concentrates the second largest amount invested in category 4 projects

(UA 570 Million for 5 projects), followed by the Eastern Region (UA 381 Million for 8 projects) and the

Western Region (UA 318 Million for 11 projects). The amount of investments as well as the number of

investments in the two remaining Regions (Northern and Central Regions) are significantly lower, with only

one project in Central Africa Region for UA 10 Million and three projects in Northern region for UA 60 Million.

Apart from the Multinational projects, the analysis of the regional distribution of these investments show

that 7 out of the 11 West Africa Region projects were exclusively from Nigeria (for a total of 59% of the

amount of category 4 operations in West Africa). Also 4 out of the 8 Eastern Africa projects were exclusively

from Kenya (72% of the amount allocated to the Region was devoted). This justifies the choice of Nigeria

and Kenya as cases studies for this evaluation, including field visits.

Figure 8. Geographical distribution of category 4 projects approved between July 2014 and December 2018 and ongoing as February 2019.

0

10

20

30

0

500

1000

1500

2000

Lines of credit EquityInvestments

Guarantees Loans (includingcorporate)

Amount and number of investment per instrument

Amount (UA Million) Number of operations

1

8

35

11

30

0

5

10

15

20

25

30

0

200

400

600

800

1000

1200

1400

1600

Central Eastern Northern Southern Western Multinational

Amount allocated (UA Million) Number of projects

40

Source: Calculation from Board approval data

The figure to the right presents the description of

the category 4 operations per sector, where

financial sector projects have the largest amount,

with a high concentration of multinational

investments. The two biggest investments in

financial sector are: Development Bank of

Namibia-UA 262 Million (Namibia) and Banco de

Poupança e Credito-UA 231 Million (Angola), all

being lines of credit. Agricultural sector receives

the second largest investment, thanks to two

major multinational investments (ETC group

limited-UA 144 Million and Export Trading Group-

UA 71 Million). The remaining 10% are distributed

between the other sectors: Transport (4%), Multi-

sector (3%), Industry-Mining (1%), Social (1%),

Power (1%) and Communication (1%).

Figure 9. Amount allocated per sector for category 4 projects

Source: Calculation from Board approval data

The ES sub-categorization in the PARs of the 58 category 4 projects is not uniform. The majority (36) of

PARs included the Bank’s ISS sub-classification (4A, 4B and 4C); 15 PARs only included “4” as the

category; six PARs used a 4-1, 4-2, 4-3 classification; and one project was identified as FI-B. The evaluation

will further explore if this has any implications in terms of the implementation and monitoring of

environmental and social impacts of those interventions.

Methodology to assess the level of ISS compliance from the emerging practice of

the Financial intermediaries lending (category 4)

Desk review and validation of projects to assess their compliance with ISS The first stage of the evaluation, already completed for the Evaluation of quality Assurance across the project cycle, assessed the compliance of a sample of projects with the Bank’s environmental and social safeguards across the project cycle. The sample was composed of 89 projects approved between 2011 and 2017 which were categorized as 1, 2 or 4. The desk review of the category 4 projects in this sample could not be undertaken due to problems to access key ES documentation. Only eight category 4 projects initially sampled could be included in the review conducted in stage 1 of the evaluation.

The current assignment will conduct an analysis of category 4 projects approved after ISS became effective (July 2014). A specific desk validation tool is proposed to assess the quality of the ES information in Bank and FI’s documents through a desk review. The Rapid Assessment Tool (RAT) with a rating system has been developed to assess the following aspects about compliance with the ISS and ESAP and Bank’s legal requirements in investment agreements:

Finance78%

Agriculture 12%

Multi-Sector3%

Transport 4%

41

1. Quality of FI’s Environmental and Social Management System (ESMS): the assessment will specifically look at the quality of the information provided in the ESMS in relation to: 1.1. E&S Policies and management commitment 1.2. Screening, categorization and assessment procedure in the ESMS 1.3. Organizational capacity and staffing of the FI to implement the ESMS 1.4. Resources 1.5. Expertise and competence of the safeguard staff 1.6. Monitoring and reporting systems

2. Actual ESMS Implementation Performance: The assessment summarizes the individual indicators below: 2.1. Categorization of subprojects 2.2. Clearance by AfDB of category 1 and 2 subprojects (to be confirmed by SNSC) 2.3. Disclosure of ESMS and in cat 1 subprojects ESIA, ESMP and RAP before approval 2.4. Negative List 2.5. Host-country E&S laws 2.6. Verification of subprojects’ compliance with AfDB Safeguards Requirements (cat 1 and 2), (to be confirmed by SNSC) 2.7. Annual Environmental Performance Reports (AEPRs)

3. Safeguard work quality at entry/appraisal: 3.1. Adequacy of the categorization of the project (Annex 1 of ESAP, 2015) 3.2. AfDB Due Diligence of the ESMS and the FI’s organizational capacity (ISS Annex 1 & 19) 3.3. AfDB support to FIs to set up appropriate ESMS and build E&S management capacity 3.4. Ensuring disclosure of ESMS 3.5. Translation of Safeguard requirements to legal documents (financing agreements)

4. Safeguard work quality at supervision: 4.1. Quality of ES information in AfDB’s supervision reports and review of FI’s reports, actions

to enhance reporting 4.2. Supervision of ESMS implementation 4.3. Clearance/Verification of high risk subprojects compliance with Safeguards 4.4. Capacity building to FIs during supervision e.g. by missions and visiting subprojects 4.5. Quality of ES content in the XSR (if operation achieved maturity)

The quality of Bank’s ES due diligence will be assessed not only before Board approval, but also during implementation and completion (interviews with comparators should also be used as a benchmark here)57. The assessments will also evaluate, whether cofinancing MDB’s have improved the FI’s ES performance. Best practices and Lessons Learned, when adequate, will also be identified and duly reported in the evaluation reports.

Interviews and focus group discussions with key AfDB’s staff Key stakeholders will be interviewed during the mission to AfDB headquarters in Abidjan, including Board Executive Directors, senior management, sector directors, ESS specialists of SNSC, and actors interviewed for IDEV’s recent evaluation synthesis on lines of credit. The stakeholder interviews will be carried out and recorded using a semi-structured approach, with interviews guided by a framework of questions aligned to the evaluation matrix, and based on the potential insights or expertise of the individual stakeholder or group.

Field Missions The desk review of Bank and FI’s ES documents will be complemented by field missions to two regional member countries with substantive FI portfolio. Nigeria and Kenya are candidates; this will be confirmed

57 For instance, Stage 1 found that ESS experts of the Bank did not participate in any supervision mission of the 8 category 4 projects, however, completion reports had richer ES information.

42

with management. The field mission will include visits to the FI’s headquarters and to some sub-borrowers premises. The template used for the field missions to evaluate FI’s ESMS and its performance, and AfDB’s work quality has the same structure as the Rapid Assessment Tool presented above, but a detailed analysis of performance under each sub-indicator is given in addition to the summary assessments. In addition, separate ES assessment templates will be developed to assess the performance of FIs and its subprojects. With reservations for traveling, 4-6 FIs will be visited. The field visits will start at FI’s premises with 1-2 hours introduction of the evaluation objectives, presentation of the ESMS (responsible persons, organization, and subprojects’ appraisal & monitoring procedures) and key E&S monitoring documentation, and will continue with site visits to subprojects. The walk-through site visits at the subproject companies will be accompanied with the FI staff and usually take one to three hours and focus on compliance with ISS requirements, monitoring and permit documentation, mitigation and control of pollution, Health & Safety etc. After the visits to the subprojects, a wrap-up discussion will be held at FIs’ premises on key findings and conclusions. The field visits are conducted and recorded using the templates prepared separately for the FI and its subprojects.

43

Deep-dive Safeguards Assessment tool for FI operations (for country visits) Project Name

XXX

Country

XXX

Project ID

XXX

ISS Category in PAR

FI-X

Borrower (FI) Loan amount Type of FI Board approval date

Type of instrument

Line of credit/corporate

loan/guarantee/others

FI received a previous

loan for an MDB/AfDB?

From whom?

When?

Loan Agreement date

FI received TA/capacity building

from another MDB?

Participated in the AfDB-

FAPA project in 2015?

From other? When?

Table of Content

1. ...................... SAFEGUARD PERFORMANCE AND AfDB’s SAFEGUARD WORK QUALITY 43

1. ............................................................. DETAILED SAFEGUARD PERFORMANCE ANALYSIS 45

Rating system to evaluate Safeguard Performance –FI projects 45

1. SAFEGUARD PERFORMANCE AND AfDB’s SAFEGUARD WORK QUALITY

SAFEGUARD PERFORMANCE

Rating

Safeguard performance and rating. Project approval date and categorization. AfDB safeguard requirements.

Summary assessment and rating of ESMS and its implementation. Portfolio projects and their categorization and

compliance. Assessment of FI’s corporate responsiveness and sustainability agenda.

Rationale and background

Approval date, project objectives, type and volume of AfDB financing, executing agency, disbursement, financing

status (active, closed).

AfDB’s SAFEGUARD WORK QUALITY AT PROJECT SCREENING, PREPARATION & APPRAISAL

Rating

Rationale and rating of AfDB’s work quality at appraisal. Categorization of the project, AfDB Due Diligence of FI

risks and their mitigation process and E&S management capacity (ESAP,2015, Annex 19, Review of ESMS

Document for FI), disclosure of the ESMS. AfDB support to FIs to set up appropriate ESMS and build E&S

management capacity. Translation/integration of Safeguard requirements to legal documents (Project Loan

Agreements)

AfDB’s SAFEGUARD WORK QUALITY AT SUPERVISION AND COMPLETION

Rating

Rationale and rating of AfDB’s work quality at supervision. Quality of monitoring report (e.g. ES information in

PSRs) reviews and AfDB actions to enhance reporting if needed. Supervision of ESMS implementation.

Clearance/Verification of high risk subprojects compliance with Safeguards. FI capacity building e.g. by visiting

subprojects. Quality of ES content in the XSR.

44

ENVIRONMENTAL AND SOCIAL LESSONS

Headline to effectively summarize the lesson, max 2 lines

WHAT AFDB EXPECTED AT APPROVAL – AfDB expected that... (include project name, country or region, sector,

contextual information and clear expectation)

WHAT ACTUALLY HAPPENED AND WHY – (relate back to expectation)

LESSONS FOR FUTURE OPERATIONS – (supported by the previous section, written generically without referring to the

specific project, use words like :”should”, one or more lessons)

Documents available for this assessment:

Project Stage Documents solicited by IDEV for the Rapid

Assessments

Documents received and used by

IDEV for the Rapid Assessment

(specify document title and date)

Project

Preparation &

Appraisal

Project Appraisal Report (PAR)

ADOA

Loan agreement

Environmental and Social Management

System (ESMS)

Project

Implementation

Bank’s ASR/PSRs/BTOR

Borrowers’ ES performance progress reports

Project

Completion

Bank’s completion reports (ESCR end

PCR/XSR)

AfDB’s SAFEGUARD WORK QUALITY Rating

1. Safeguard Work Quality at Appraisal

1.1. Categorization of the project (Annex 1 of ESAP, 2015) 1.2. AfDB Due Diligence of the ESMS and the FI’s organizational capacity (ISS Annex 1 and 19) 1.3. AfDB support to FIs to set up appropriate ESMS and build E&S management capacity 1.4. Ensuring disclosure of ESMS 1.5. Translation of Safeguard requirements to legal documents (financing agreement)

2. Safeguard Work Quality at Supervision and Completion

2.1. Quality of AfDB’s ES supervision reports and review of FI’s reports, actions to enhance reporting 2.2. Supervision of ESMS implementation 58 2.3. Verification of high risk subprojects compliance with Safeguards 2.4. Capacity building to FIs during supervision e.g. by missions and visiting subprojects 2.5. Quality of ES content in the XSR59

58 The ESAP only includes a template for ES content in public sector supervision reports (IPR: Implementation

Progress and Results Report), Annex 23, but not for private sector operations. 59 The ESAP only includes a template for ES content in public sector supervision reports (PCR: Project Completion

45

1. DETAILED SAFEGUARD PERFORMANCE ANALYSIS

Rating system to evaluate Safeguard Performance –Category 4 (FI) projects The Safeguard assessment based primarily on AfDB’s specified standards in effect at approval, mainly the ISS and ESAP and the specific

conditions set in the loan agreements of each intervention. The assessment should consider the adequacy of the FI’s ESMS and its

implementation. The environmental and social performance of subprojects / fund investee companies should also be considered. The

evaluation needs to clearly define (i) compliance benchmarks (derived from statutory and regulatory requirements and AfDB standards in

investment agreements); (ii) non-mandatory performance indicators subject to defined targets and standards; and (iii) other positive and

negative changes that can be attributed to the project. The rating scale and definitions are presented below:

• Highly Satisfactory (HS): The ESMS has been developed and implemented in a timely and satisfactory manner and in full compliance with

AfDB requirements; institutional capacity is strong and there is sufficient funding to ensure the environmental and social sustainability of the

operation. The FI has provided transparent and timely reports, verifying that the project has consistently met AfDB’s at approval requirements,

and as applicable, sub-projects have been appropriately appraised and supervised and their adverse environmental and social impacts have

been mitigated, and the safeguard performance is deemed acceptable in view of the IFI’s current requirements.

• Satisfactory (S): The ESMS has been developed and implemented in a timely and satisfactory manner and in material compliance with AfDB

requirements; institutional capacity and funding are deemed sufficient to ensure the environmental and social sustainability of the operation. If

required, the E&S performance of sub-projects are in material compliance with the AfDB’s requirements

• Unsatisfactory (U): The ESMS has been developed and implemented with major delays or in an unsatisfactory manner; institutional capacity

and funding are deemed insufficient to ensure the environmental and social sustainability of the operation.

• Highly Unsatisfactory (HU): The ESMS has not been developed and implemented; institutional capacity and funding are not available to

ensure the environmental and social sustainability of the operation. The FI is not in material compliance with AfDB’s at-approval requirements

and mitigation prospects are uncertain or unlikely, or sub-projects’ non-compliance resulted in substantial and permanent environmental

damage.

• No opinion possible (NOP): Where, after best efforts, the relevant information to establish material compliance (or lack thereof) cannot be obtained, a rating of “no opinion possible” may be assigned. This rating should be used as a last resort, after reasonable effort has been made to obtain the necessary information. A sponsor's failure to report on material negative Safeguard events, or repeated refusal to cooperate on reporting on these issues, should result in a less than satisfactory or unsatisfactory rating. • Not applicable (NA): If the project was classified as category FI-C (no adverse impact) and that categorization has remained valid over the life of the project so far (and is likely to remain so going forward), then the correct rating is not applicable. If, despite its category FI-C categorization, the project has in fact had actual or potential environmental and social impacts, then it should be rated accordingly. • Non Evaluable (NE): If the project has not yet reached required operational maturity for the evaluation, the rating is Non Evaluable; for example, the first supervision reports have not yet been submitted and project performance cannot be yet evaluated. AfDB’s Safeguard Work Quality at appraisal and supervision

Highly Satisfactory Best practice example; AfDB kept itself fully informed and used information to proactively improve performance

Satisfactory: Met materially AfDB requirements in the ISS and ESAP, AfDB kept itself sufficiently informed

Unsatisfactory: Material shortfall in at least one important area

Highly Unsatisfactory: Material shortfalls in several areas or mistake/negligence in at least one important area Sources: Staff Guidance on Project Completion Reporting and Rating (2012), ECG Big Book on Evaluation Good Practice Standards (2012)

Report), Annex 25, but not for private sector operations (XSR: Expanded Supervision Report). 59 The ESAP only includes a template for ES content

46

Summary of Safeguard performance Rating

1. ESMS: Safeguard policies and procedures, resources and capacity

1.1. E&S Policies and management commitment

1.2. Screening, categorization and assessment procedure in the ESMS 1.3. Organizational capacity and staffing of the FI to implement the ESMS 1.4. Resources 1.5. Expertise and competence of the safeguard staff 1.6. Monitoring and reporting systems

2. Actual ESMS Implementation and Sub-projects’ Performance

1.1. Categorization of subprojects

2.3. Disclosure of ESMS and in cat 1 subprojects ESIA, ESMP and RAP before approval

2.4. Compliance with the Negative List 2.5. Compliance with Host-country E&S laws 2.6. Subprojects’ compliance with AfDB Safeguards/ISS Requirements (cat 1 and 2) 2.7. Annual Environmental Performance Reports (AEPRs)

3. Assessment of FI’s corporate responsiveness and sustainability agenda60

Safeguard Performance

1. ESMS: Safeguard policies and procedures, resources and capacity

AfDB requirement:

The Environmental Management System (ESMS) is assessed and rated based on the compliance of FI’s

ESMS according to AfDB’s requirements.

AfDB Integrated Safeguards System (2013)

Each Category 4 financial intermediary is required to:

• Have adequate corporate environmental and social governance policies, apply the Bank’s OSs to its

Category 1- and Category 2-equivalent subprojects, and comply with local environmental and social

requirements (Host Country ESHS Laws and Regulations);

• Develop and maintain an ESMS in line with the Bank’s OSs that is appropriate for the scale and nature

of its operations— recognizing that the operations of financial intermediaries vary considerably and in

some cases may pose minimal environmental and social risk (particularly those of reinsurance

companies, which may only need to develop a corporate environmental and social policy);

• Demonstrate that it has the management commitment, organizational capacity, resources and

expertise to implement its ESMS for its subprojects; and

• Develop and disclose a summary of the ESMS to the public on its website and make use of the

Bank’s Negative List (as defined in the ISS), which includes goods that are harmful to the environment,

when soliciting a loan or a grant and before the loan can be approved. The Bank carries out due diligence

on the ESMS and the financial intermediary’s organizational capacity.

AfDB Environmental and Social Assessment Procedures (2015)

Annex 6 — Description of instruments: Environmental and Social Assessment (ESA) studies: An ESMS

should integrate environmental and social impact and risk management into the FI’s business processes

so that the FI can manage potential environmental and social impacts of subprojects by ensuring the

conduct of environmental and social due diligence prior to financing subprojects and adequate monitoring

during the term of the loan agreement. The format for an ESMS report is provided in Annex 12.

60 See rating system Reactive (R ), Active (A) and Proactive (P) in the Section 3

47

Information on how to prepare an ESMS is provided in the Guidance Note on ESMS.

Step 3.2b: For Category 4 projects, the client / borrower shall develop the TORs for an Environmental

and Social Management System (ESMS) with Sector Departments and/ or consultants if it does not

already have an ESMS (see Annex 12) that is in compliance with the Bank’s OSs and appropriate for the

scale and nature of its operations. This must be developed prior to Board approval. The existing ESMS

should indicate the scope of the due diligence and relevant in country system that is required or the work

needed to develop a new one. Step 4.1b: The borrower is notified of the project’s category. The FI shall

either develop an ESMS (if it does not already have one) or improve its existing ESMS, and ensure that

this ESMS is in full compliance with the Bank’s ISS, as well as being appropriate for the scale and nature

of the FI’s operations. The FI must also be able to demonstrate that it has the organizational capacity,

resources and expertise to implement its ESMS for its sub-projects.

Loan Agreement (XXX)

The safeguard requirements in the Loan Agreement

Rating at evaluation: XXX

Project Performance :

Brief assessment on compliance and rating.

Summary assessment on performance in meeting AfDB requirements for ESMS.

1.1. E&S Policies and management commitment

AfDB requirement:

Volume 1: General Guidance on Implementation of OS 1: ESMSs for Financial Intermediaries

The FI’s policy should state its commitment to managing environmental and social risks to which it might

be exposed to as a result of transactions and clients in its portfolio and to respecting local laws and

regulations.

AfDB Environmental and Social Assessment Procedures (2015) Annex 12

This section of the ESMS shall describe the FI’s environmental and social policies that apply broadly

across its organization, as well as its management’s commitment. The FI’s E&S policies sha ll ensure

that measures and actions to assess identified impacts and risks as well as proposed mitigation and

enhancement measures as appropriate are in place. Further to that they shall favor the avoidance and

prevention of impacts over minimization, mitigation, or compensation, wherever technically and

financially feasible. The FI shall be able to demonstrate its commitment to sound environmental and

social management through its previous or ongoing operations. Have the FI’s climate change policies

being outlined in the ESMS?

Rating at evaluation : XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

1.2. Screening, categorization and assessment procedure

AfDB requirement:

AfDB Environmental and Social Assessment Procedures (2015) Annex 12

Screening: The FI shall determine E&S risk level of the operation. The criteria used to arrive at this

judgment shall be outlined in this section. Examples of criteria that are commonly used include, but are

not limited to the type and scale of the subprojects and their location.

Categorization: Operational Safeguard 1 – Environmental and social assessment” gives guidance for

project categorization (categories 1, 2 and 3).

Environmental and Social Assessment (ESA): The FI shall conduct an integrated ESA of the potential

risks and impacts of the operation, including labor, health, and safety. The Assessment process shall be

based on current information, including an accurate description of the undertaking, and the appropriate

social and environmental baseline data. Applicable social and environmental national and international

48

laws and regulations of the jurisdictions in which the activities operate will also be taken into account.

Implementation: In this section of the ESMS the FI shall identify specific mitigation measures and actions

necessary for the operation to comply with applicable national laws, regulations and the Bank’s safeguard

policies. To this end the FI shall prepare an action plan that is technically feasible and cost effective in

order to implement the mitigation and enhancement measures identified. These measures and actions

will reflect the outcomes of consultation on social and environmental risks and adverse impacts. The

action plan shall include: (i) corrective actions to be undertaken; (ii) prioritize these actions; (iii) address

the time-line for their implementation; (iv) be disclosed to the affected communities; and (v) describe the

schedule and mechanism for external reporting on the client’s implementation of the Action Plan

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

1.3. Organizational capacity and staffing

AfDB requirement:

AfDB Environmental and Social Assessment Procedures (2015) Annex 12

Organizational capacity: The FI shall outline the organizational structure that defines roles,

responsibilities, accountability and authority to implement the management program, including the action

plan. Specific personnel including management representative(s), with clear lines of responsibility and

authority shall be included in this section. Key social and environmental responsibilities shall be stated

as well as methods of communication to the relevant personnel and to the rest of the organization.

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

1.4. Disclosure of ESMS

AfDB requirement:

ISS (2013, page 25)

Each FI is required to develop and disclose a summary of the ESMS to the public on its website.

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

1.5. Resources

AfDB requirement:

AfDB Environmental and Social Assessment Procedures (2015) Annex 12

Resources: This part of the ESMS shall state the financial resources that the FI’s management has

provided on an ongoing basis to achieve effective and continuous social and environmental performance

of the FI’s operations. (For example, the FI shall indicate a percentage of its annual expenditure reserve

to address E&S issues).

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

1.6. Expertise and competence of the Safeguard staff

AfDB requirement:

AfDB Environmental and Social Assessment Procedures (2015) Annex 12

The FI shall present human resources assigned by the FI for environmental and social management

purposes. The number of E&S staff in the FI, their skills and knowledge as E&S officers, the analysis of

their specific job functions shall be included in this section. In addition, the training that they have received

in the past as well as training needs (if any with regard to the expected impacts of the proposed operation)

49

shall be provided.

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

1.7. Monitoring and reporting systems

AfDB requirement:

AfDB Environmental and Social Assessment Procedures (2015) Annex 12

Monitoring and supervision: The FI shall outline the procedures to monitor and measure the effectiveness

of the management program in this section of the ESMS. For sub-projects with significant impacts that

are diverse, irreversible, or unprecedented, the client will retain qualified and experienced external

experts to verify its monitoring information. The extent of monitoring should be commensurate with the

project’s risks and impacts and with the project’s compliance requirements. Monitoring shall be adjusted

according to performance experience and feedback. The FI shall document monitoring results, and

identify and reflect the necessary corrective and preventive actions in the amended management

program. The FI shall implement these corrective and preventive actions, and follow up on these actions

to ensure their effectiveness

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

2. Actual ESMS Implementation and Sub-projects’ Performance

This section shall provide a thorough analysis of the environmental and social performance of the

project/operation during implementation. The assessment will be based on the requirements of the ESMS

at appraisal stage, the findings from annual reports and regular Bank’s supervision mission reports.

Implementation of the ESMS is assessed and rated based on the FI’s performance in screening,

categorization, clearance and disclosure, monitoring and reporting of subprojects, and meeting other

AfDB’s requirements (e.g. Negative List, Host Country E&S Laws and Operational Safeguards). The FI

should clearly report on Safeguard risks of its portfolio and compliance with AfDB requirements for the

subprojects that have been financed with AfDB funds. The FI shall document monitoring results, and

identify and reflect the necessary corrective and preventive actions, implement these corrective and

preventive actions, and follow up on these actions to ensure their effectiveness.

Rating at evaluation: XXX

Project Performance

Brief summary assessment on performance in meeting AfDB requirements and rating.

2.1. Categorization of subprojects

AfDB requirement:

AfDB Integrated Safeguards System (2013):

Financial intermediary subprojects equivalent to Category 1 and Category 2 are subject to the relevant

OS requirements, as if they were directly financed Category 1 or Category 2 projects. Each Category 4

financial intermediary is required to apply the Bank’s OSs to its Category 1- and Category 2-equivalent

subprojects. “Operational Safeguard 1 – Environmental and social assessment” gives guidance for

project categorization as below:

Category 1: Bank operations likely to cause significant environmental and social impacts

Category 2: Bank operations likely to cause less adverse environmental and social impacts than Cat1

Category 3: Bank operations with negligible adverse environmental and social risks

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

50

Assessment on correctness of project categorization. Summaries on compliance and key findings on

reports as presented in Table 1.

Table 1. Subprojects, annual reports and categorization

OS Category triggered

Subproject Report OS2

IR

OS3

Biod.

OS4

Poll.

OS5

L,HS

2.2. Clearance by AfDB of category 1 and 2 projects before sub-project approval

AfDB requirement:

AfDB Integrated Safeguards System (2013)

Financial intermediary subprojects equivalent to Category 1 and Category 2 are subject to the relevant

OS requirements, as if they were directly financed Category 1 or Category 2 projects. 4.5a, For Category

1 projects, the Safeguards & Compliance Division shall review and clear the SESA / ESIA and ESMP

(and where applicable the FRAP / ARAP). For Category 2 projects, the Safeguards & Compliance

Division shall review and clear the ESMP summary (and where applicable the ARAP).

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

2.3. Disclosure ESMS and in cat 1 projects, ESIA, ESMP and RP 60/120 days before approval

AfDB requirement:

AfDB Integrated Safeguards System (2013)

Each Category 4 financial intermediary is required to develop and disclose a summary of the ESMS to

the public on its website. For Category 4 projects involving financial intermediaries or corporate loans,

the financial intermediary ensures that the subprojects that require ESIAs undergo the same information

disclosure process as other private sector Category 1 projects funded by the Bank. Financial

intermediaries are required to notify the responsible sector department in the Bank if a subproject is

deemed to be Category 1, and this information is passed on to the relevant Compliance and Safeguards

function of the Bank.

For Category 1 public sector projects, final and cleared versions of key environmental and social

assessment documents (SESA/ESIA, ESMP and Resettlement Action Plan summaries) are disclosed at

least 120 days before Board consideration, and final versions of Category 1 private sector projects are

disclosed at least 60 days before Board consideration.

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

2.4. Compliance with the Negative List

AfDB requirement:

AfDB Integrated Safeguards System (2013)

The Bank defines goods harmful to the physical and social environment, and excludes them—in addition

to the items explicitly mentioned in the Negative List—from its operations for both the public and private

sectors.

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

51

2.5. Compliance with Host-country E&S laws

AfDB requirement:

AfDB Integrated Safeguards System (2013)

Each Category 4 financial intermediary is required to comply with local environmental and social

requirements.

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

2.6. Subprojects’ compliance with AfDB Safeguards OS 2-5/ISS Requirements (cat 1 and 2)

AfDB requirement:

AfDB Integrated Safeguards System (2013)

Financial intermediary subprojects equivalent to Category 1 and Category 2 are subject to the relevant

OS requirements, as if they were directly financed Category 1 or Category 2 projects. In addition for a

category 1 project, if an OS is triggered, the requirements of this specific OS should be met by

the project. For Category 4 projects involving financial intermediaries or corporate loans, the financial

intermediary ensures that the subprojects that require ESIAs undergo the same information disclosure

process as other private sector Category 1 projects funded by the Bank. Financial intermediaries are

required to notify the responsible sector department in the Bank if a subproject is deemed to be Category

1, and this information is passed on to the relevant Compliance and Safeguards function of the Bank.

AfDB Environmental and Social Assessment Procedures, ESAP, 2015: page 25)

Step 6.1: The borrower is responsible for the implementation of the ESMP / ESMS and shall

diligently monitor it by: i) ensuring that the indicators identified in the project implementation

documents are respected; ii) ensuring compliance with the Bank’s ISS; iii) ensuring adherence to the

environmental and social covenants of the financing agreement. The borrower shall then report to the

Sector Departments / Private Sector Department on the implementation of the ESMP / ESMS, as part of

the Quarterly Implementation Reports that it submits to the Bank. These reports shall clearly identify the

results achieved in implementing the ESMP and key management and monitoring tasks.

Step 6.2: The Sector Departments / Private Sector Department shall review the Quarterly

Implementation Reports and engage in supervision missions, through which they shall assess and report

on ESMP / ESMS implementation. Whenever non-compliance is observed or unexpected impacts arise,

the Sector Departments shall request the borrower to review the ESMP / ESMS in collaboration with

relevant stakeholders, as appropriate. Changes to the ESMP / ESMS must be cleared by the Sector

Departments / Private Sector Department and the Safeguards and Compliance Division before being

implemented. Responsibility: Sector Departments / Private Sector Department and the Safeguards and

Compliance Division (for supervising and verifying compliance) and the borrower (for reviewing and

amending ESMP / ESMS where necessary). Output: Environmental and social information in

Implementation Progress and Results Report (IPRR)

See reporting requirements in the Section 2.6

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

2.7. Annual Environmental Performance Reports (AEPRs)

AfDB requirement:

Volume 1: General Guidance on Implementation of OS 1: ESMSs for Financial Intermediaries

The requirements for Environmental and Social Monitoring and Reporting by FI are presented below:

The FI should report any major environmental, social or health and safety incident, such as a serious

non-compliance, pollution incident, community impact, worker fatality or equivalent to the Bank as soon

as is feasible.

52

FI-A, High Risk

The FI should report to the Bank on a regular (quarterly) basis indicating how subprojects have been

categorized and providing details of environmental and social assessment agreed with clients for

subproject deemed to be equivalent to category 1 or 2. Reporting should also include monitoring of the

client’s implementation of ESMP/FRAP.

FI-B, Medium Risk

The FI should report to the Bank on an annual basis on the results of their screening, categorization,

application of ESA procedures and ESMP measures agreed with clients. It should also report on

monitoring of client’s implementation of agreed mitigation measures.

FI-C, Low risk

The FI should report to the Bank on an annual basis confirming that its portfolio of subprojects still present

minimal risk of environmental and social impact, taking account the possible cumulative effects of its

portfolio.

Rating at evaluation: XXX

Project Performance

Brief assessment on performance in meeting AfDB requirements and rating.

3. Assessment of FI’s corporate responsiveness and sustainability agenda

E&S Management in the African Financial Sector. Training and consultancy on environmental

and social management in financial intermediaries and microfinance institutions in Africa. An

AfDB-FAPA project (2015)

The report above gives guidance to assess of FI’s corporate responsiveness and sustainability agenda

as below:

Re-active (R) are those that see E&S or Sustainability as something that imposed on them by regulators

or investors or consumer lobbies. Their response is defensive aimed at reputation control without

substantially addressing the issues. It may involve various forms of philanthropy or community support.

In some cases this response involves an element of ‘green washing’. Re-active firms are concerned with

generating outputs, with no voluntary consideration for outcomes and impact.

Active (A) are those that have accepted the inevitability of E&S and are taking active steps to include it

into their operation, mostly out of reputation and risk management concerns toward internal and external

stakeholders (including Regulator compliance). Typically, they consider sustainability, CSR or E&S as a

stand-alone initiative, which are regarded as ‘add on’ to their business. Active firms have opened their

eyes to creating positive outcomes.

Pro-active (P) are those who see shared value as core business strategy, enabling them to create new

products and develop new markets that cater for new societal and environmental needs. They don’t just

avoid doing harm; they actively use their products and services to do good. Sustainability principles are

included in the governance (‘inclusive governance’) and culture of the organization. Pro-active

organizations are concerned with creating positive output, outcomes and impact; even though they

realize that positive impacts can be realized in partnership with other stakeholders.

Rating at evaluation: XXX

Project Performance

Brief assessment on FI’s corporate responsiveness and sustainability agenda.

53

Practices of comparators to manage ES risks of category 4 projects Institution Framework to address ES risks of FIs and recent evaluations ES categorization of FIs

International Finance Corporation (IFC), World Bank Group61

A key aspect of IFC’s approach to the E&S risk management in the FI sector is to ensure that the FIs providing project or long-term corporate finance are identified, and make sure that the E&S risks from such activities are assessed and appropriately mitigated. IFC’s FI clients are involved in various financial activities including, but not limited to, corporate finance; project finance; lending to micro, small, and medium enterprises; trade finance; and housing finance, among others, each with different E&S risk. The process is organized in two steps:

- IFC reviews the business activities of prospective FI clients to identify areas where the FI could be exposed to potential E&S risk as a result of the borrowers/investees in its portfolio and/or prospective business.

- The FI is then required to undertake lending/investment-level actions commensurate with the level of ES risk related to the FI activities supported by IFC, ranging from a simple review against IFC’s Exclusion List to application of the Performance Standards.

IFC requires all FI clients to develop and operate an Environmental and Social Management System (ESMS) commensurate with the level of ES risk in their portfolio and prospective

business activities. The ESMS should incorporate relevant principles of IFC’s Performance Standard 1 on Assessment and Management of Environmental and Social Risks and Impacts. The scope and complexity of the ESMS will depend on the E&S risk of the FI’s lending/investment activities. The Independent Evaluation Group (IEG) of the World Bank conducted an evaluation62 in 2008 to evaluate the support of IFC to MSMEs through FIs (from 1994 to 2006). IEG visited 20 MSME-FIs and 65 subprojects that were selected by IEG from a list (prepared by the MSME-FIs) with the highest potential EHS risks. The evaluation found that in the frontier countries (Low-income and/or high-risk countries), 83% of subprojects had satisfactory Environmental, Health and Safety (EHS) performance, while 50% of the Micro, Small and Medium Enterprise-FIs (MSME-FI) themselves did. The evaluation explained this by the fact that many MSME-FIs did not follow regular guidelines such as: implementation of good environmental management system, training of staff, proper screen of subprojects before providing loan and regular submission of EHS reports to IFC.

Category FI-1: when an FI’s existing or proposed

portfolio includes, or is expected to include, substantial financial exposure to business activities with potential significant adverse environmental or social risks or impacts that are diverse, irreversible, or unprecedented. Category FI-2: when an FI’s existing or proposed

portfolio comprises, or is expected to comprise, business activities that have potential limited adverse environmental or social risks or impacts that are few in number, generally site-specific, largely reversible, and readily addressed through mitigation measures; or includes a very limited number of business activities with potential significant adverse environmental or social risks or impacts that are diverse, irreversible, or unprecedented. Category FI-3: when an FI’s existing or proposed

portfolio includes financial exposure to business activities that predominantly have minimal or no adverse environmental or social impacts.

European Investment Bank (EIB)63

The nature of ES due diligence varies depending on the type64 of FI. However, all projects financed through financial intermediaries are covenanted to comply with appropriate environmental and social legislation; within the EU, EU legislation, outside the EU, national legislation, with reference where appropriate to alignment with EU legislation and EIB

61 https://www.ifc.org/wps/wcm/connect/38d1a68049ddf966af3cbfda80c2ddf3/IN+on+FIs_Revised+April+11+2017.pdf?MOD=AJPERES 62 https://openknowledge.worldbank.org/bitstream/handle/10986/6485/448160PUB0Box327413B01official0use0only1.pdf?sequence=1&isAllowed=y 63 2013. European Investment Bank, Environmental and Social Handbook. 64 Mid-Cap Loan, Global loans and Funds

54

environmental and social standards. On the request of operations directorate, project directorate may carry out an environmental and social review of a particular global loans operation, including an assessment of the environmental social risk management capacity of the intermediary. It may also carry out an environmental and social review of a particular sub-project when requested by operations directorate. Specific monitoring requirements must be designed to address the E&S impacts identified during the due diligence including any emerging impacts identified during the course of EIB-support to the operation. Where climate issues are identified as relevant and included in the global loans documentation, reporting on climate elements may be required. The financial intermediary shall undertake to promote compliance of the sub-projects with relevant national and EU law. A recent EIB evaluation of FIs operations (from 2010 to 2015) reported that although EIB ES safeguards are always taken into consideration, they are not always followed satisfactorily65. The quality of ES sheets was found to be low and the evaluation could not find a trace of dialogue between the EIB and the FI with an aim of improving them. The evaluation gave an example of a technical assistance which included ES training but this was not considered by the FI, as they considered others areas to be a priority. Several FIs pointed out that they selected projects with low E&S risk to present to the EIB and systematically excluded those with high E&S risks, thereby circumventing the more demanding ES requirements, such as an independent EIB assessment of environmental impact and risks. This behavior could theoretically lead to the exclusion of potentially higher impact – but more “complicated” – projects.

IDB Invest, or Inter-

American Investment Corporation (IIC), a member of the Inter-American Development Bank (IDB) Group66

IDB Invest supports financial intermediaries such as a bank, investment fund or another specialized institution, such as a factoring company, a leasing company, or microfinance institution. IDB Invest delegates to the FI the primary responsibility for assuring the requirements of this policy are met by the individual sub-projects. IDB Invest retains contractual rights to verify that the FI effectively implements an ESMS that achieves such compliance and reports thereon to IDB Invest. The ES Specialist shall identify what actions the client would need to undertake to ensure adequately managing ES aspects of sub-loans, including gaps in the clients ES policies and procedures, ES skills and competencies, and training of the FI’s staff working on ES issues. IDB Invest’s ES Specialist will ensure that these elements are addressed in the client’s ESMS. IDB uses a screening and monitoring tool (Mr Blue) to screen FI operations up front and to monitor FIs’ safeguards performance. IDB conducted an evaluation in 2016 focusing on the ES policy framework as it applies to FIs operations approved from 2005 to 201467. According to the evaluation, the weakness of information reported by FIs constitutes a serious issue for IDB’s supervision of FIs’ ES performance. The reporting format for FIs often lacks basic information

FI-1 operations are those where the potential risk

is high: the FI’s current or future portfolio financed as part of IDB Invest investment has the potential to include Category A sub-projects FI-2 operations are those where the risk potential

is considered medium: the FI’s current or future portfolio is limited to Category B or C sub-projects. FI-3 operations are those where the risk potential

is considered low: the FI’s current or future portfolio is dominated by Category C and lower risk Category B sub-projects.

65 http://www.eib.org/attachments/ev/ev_intermediated_lending_if_acp_en.pdf 66 https://publications.iadb.org/handle/11319/7541 67 2016 IADB Evaluation of Financial Intermediaries

55

on the sub-loans. The FIs rarely presented a breakdown of their own risk classification. The IDB evaluation found evidence of cases where the FI’s risk classification and mitigation mechanism clearly indicated that the FI’s ESMS was not able to respond independently and appropriately to IDB’s Safeguard Policy. However, the evaluation showed that more than half of FIs applies their ESMS to all loans. It also found that commercial banks are more advanced in the ESMS procedures and application than national development banks.

Asian Development Bank (ADB)68

ADB does not have direct oversight of or strong leverage in relation to subprojects, because subprojects are often unknown when a FI is appraised and funds are dispersed widely to many subprojects. Financial intermediation financing can entail several layers that complicate social and environmental risk management and its supervision. For projects involving investment of ADB funds to or through FI, ADB conducts safeguard due diligence to assess the potential ES impacts and risks associated with the FI's existing and likely future portfolio, and its commitment and capacity in ES management. All FIs must ensure that their investments are in compliance with applicable national laws and regulations and will apply the prohibited investment activities list to subprojects financed by ADB. If the FI’s intervention have minimal or no adverse ES risks, the FI project is classified as category C. All other FIs will be required to have in place or establish an appropriate ESMS commensurate with the nature and risks of the FI's likely future portfolio to be maintained as part of the FI's overall management system. ADB will assess the adequacy of the FI’s capacity to manage ES impacts and risks. Where there are gaps in the FI’s capacity, the ADB and FI will establish a time-bound plan to address identified gaps. ADB conducted a safeguard operational review of 40 FI projects from 2010 to 2012 and found that the requirement for the description of the FI reporting to ADB was not rigorously followed in the ESMS69. ESMS documentation for 10 projects (out of the 18 that required ESMS) provided details of their reporting activities, both from the sub-borrower to the FI and from the FI to ADB. In seven other projects, the ESMS described reporting from the borrower client to the FI but there was no adequate description of arrangements for reporting to ADB. One ESMS document did not mention monitoring and reporting responsibilities. According to this evaluation, monitoring and reporting requirements by the sub-borrower to the FI, the contractor to the sub-borrower, and the FI to ADB should be described in greater detail in the ESMS documentation, especially if no ESMS exists at the time of project preparation.

Projects are classified as category FI if they involve a credit line through a financial intermediary or an equity investment in a financial intermediary. The financial intermediary must apply an environmental management system, unless all subprojects will result in insignificant impacts.

68 2009 Asian Development Bank, Safeguards Policy Statement https://www.adb.org/sites/default/files/institutional-document/32056/safeguard-policy-statement-june2009.pdf 69 2014 Asian Development Bank, Safeguards Operational Review. ADB processes, portfolio, country systems and financial intermediaries. Corporate evaluation study. https://www.adb.org/sites/default/files/evaluation-document/89401/files/ces-safeguards.pdf

56

Annex 7. – Evaluation matrix Criterion and

overarching question Specific Evaluation Questions (EQ) Indicators/Judgement criteria Proposed Methodology

I. Relevance

1. Is the Bank’s ISS aligned with comparator organizations and identified best practices and covering all relevant safeguards emerging safeguards issues?

1.1 To what extent is the ISS aligned with other MDBs70 and best practice standards (from a design and content perspective)? 1.2. How effectively does the ISS address emerging safeguards issues (workers and community health and safety, biodiversity

offsets, influx71, gender based violence, climate

change mitigation and adaptation?

Level of alignment and coverage on issues in relation to the ability of co-financing and to include the concept of not only “do not harm”, but also “do good”72 and the concept of “associated facilities”73

Analysis of the type of safeguards approach: principles-oriented vs procedural74

Level of depth with which ESS issues are treated and associated guidance75

Document Review of ISS and Safeguards documents of other MDBs Comparators tables like the GEF review of comparative standards approach76 and benchmarking indicators template Interviews with Bank staff and with ES staff of other MDBs and experts

70 Ideally, for the purposes of this evaluation, comparator organizations will include organizations with similar geographical coverage and types of funding operation, or that have a direct relationship with the AfDB, including: (i) World Bank (new ESF) and the IFC; (iii) Inter-American Development Bank and IDB Invest; (iv) Development Bank of Southern Africa; (v) European Bank for Reconstruction and Development (EBRD); (vI) European Investment Bank (EIB). 71 Managing the risks of adverse impacts on communities from temporary project induced labor influx (World Bank, 2017) and Handbook for addressing project-induced migration (IFC). 72 Moving beyond environmental impact mitigation, ie. ‘do no harm’, to a more encompassing approach to achieve sustainability, ie. ‘do good’. Although Environmental and Social Safeguards are usually conceived as “do no harm” policies; with a strong legalistic and rights connotation (a rules-based administrative system), focused on preventing negative environmental and social project outcomes and mitigate when negative outcomes are unavoidable, the new ISS includes the idea of “do good”. The evaluation will assess to what extent this concept is aligned with other MDB’s Safeguards system and how well it is codified to ensure its implementation. 73 The Bank’s Policy statement and operational safeguards (2015) states that the area of influence encompasses the “associated facilities dependent on the project that are not funded by the project and that would not have been implemented if the project did not exist”. 74 The evaluation where the AfDB’s ISS is located in a continuum from principles-oriented approach (focus on the results of implementation and solutions-oriented) and up-front preparation requirements or procedural approach (process-focus, tick-the-box attitude and enforcement approach). 75 The way other organizations articulate Human Rights and the gender agenda with the social safeguards requirements will be analyzed, in the framework of the AfDB’s Gender Marker System piloted for all public operations since 2018. Are environmental and social aspects effectively on an equal footing? 76 Focus on whether the standards are materially consistent (and appropriate to the African context), rather than consistent to the letter (2017 Review of GEF Policy on Agency Minimum Standards on Environmental and Social Safeguards).

57

Criterion and overarching question

Specific Evaluation Questions (EQ) Indicators/Judgement criteria Proposed Methodology

I. Relevance

(continuation) 2. Is the Bank´s ISS effectively supporting the Bank´s strategic priorities and relevant to the current portfolio and the institutional changes?

2.1. To what extent is the Bank’s ISS relevant to the Bank’s strategic goals in relation to inclusive and green growth and implementation of the High 5s and the new changes from the DBDM?

2.2. To what extent is the Bank’s ISS relevant to the types of lending operations that the Bank currently has in place, including project finance, policy-based lending operations (PBOs) and Financial intermediaries (FI)?

Level of alignment of ISS outcome indicators with the goals of the Bank´s Ten-Year Strategy, the High 5s and ongoing DBDM reforms Level of alignment with comparator MDBs in relation to the inclusion, depth of treatment, associated guidance materials and resources to address ES risks of PBOs and FI

Document review Interview with Bank staff, especially senior management. Interview with ES staff of other MDBs and experts, interviews with operational staff managing the “new” lending operations

58

Criterion and overarch. Question

Specific Evaluation Questions (EQ) Indicators/Judgement criteria Proposed Methodology

II. Efficiency

3. What are the Bank´s internal factors that facilitate or constrain implementation of the ISS as designed in relation to resources (human and financial), capacity (skills), systems, process and tools, and institutional incentives?

3.1. How adequate has been the allocation of resources and staffing (and their skills) for environmental and social safeguards work in comparison to comparators77?

3.2. How adequate are the available

guidelines, instruments and incentives to reflect the requirements of the ISS?

3.3. To what extent is there sufficient training

and technical support to ensure efficient ISS implementation?

3.4 Are appropriate systems and tools

available to ensure the quality control of ES outputs, to conduct periodic quality review?

3.5 Is there a common understanding of the objectives, principles, and requirements of the safeguards by all Bank staff? To what extent do they perceive the process to be clear, relevant, credible and useful?

3.6 To what extent the Bank has developed

systems to disseminate lessons learned emerging from the application of the ISS?

AfDB ratio of safeguards staff to projects vs. comparators, # of lead specialists Existence of subject matter specialists for key complex safeguards issues. AfDB budget for safeguards implementation vs. comparators Ratio of ESS specialists staff (environment, social) vs. comparators Ratios of staff time spent on project preparation vs. supervision Quantity of training and assistance about ESS provided to Bank´s operation staff Quality of training received by ESS specialists since ISS approval Evidence of clearly defined process and accountabilities / incentives’ system78

Extent to which stakeholders consider ISS processes as clear, relevant, credible and useful79. Types of systems and tools in place to collect and disseminate lessons learned.

Interviews with comparator organizations Interviews with Bank staff Use of benchmarking indicators Document review, specifically OS Guidance materials and IESIA HR data about staff and training Stakeholder survey Analysis of the compliance review audits conducted by the Bank´s safeguards department, other general audits and IRM spotchecks Analysis of the knowledge management system in place to ensure identification and dissemination of key lessons learned in the sector and safeguards department, including the level of functionality of ITST.

77 The evaluation will compare the organizational structure of the AfDB Safeguards Division, and the level of seniority of its senior management in comparison to other organizations, and come up with conclusions about its consequences to enforce the ISS. It will also cover how the staff is organized to deliver efficiently and how the work of different departments and complexes of the Bank is articulated to ensure ISS compliance. 78 From the continuum of non-attentive to safeguards/reluctant (safeguards considered as a transaction cost or burden), through acceptance but isolated in a small team to a shared agenda or a clear value proposition. 79 The survey and interviews will ask task Managers about how ISS services could improve and what is the main value added from the ISS currently.

59

Criterion and overarching Question

Specific Evaluation Questions (EQ) Indicators/Judgement criteria Proposed Methodology

III. Emerging evidence on Effectiveness

4. To what extent are

the Bank´s quality assurance processes contributing to ensure compliance with ISS objectives (mitigation of environmental and social risks of Bank-funded projects)?

4.1. How adequate is the Bank’s categorization process in managing risk? 4.2. What is the overall quality of the ES outputs validated by the Bank prior to Board approval80, including strategic ES assessments and ES Management Systems of FI? 4.3. What are the types of ES loan conditions and covenants included in the loan agreements and how are they enforced? 4.4. To what extent is the Bank in compliance with the ISS disclosure and meaningful consultation requirements before project approval and during implementation, especially in relation to resettlement and compensation?

Number and proportion of mis-categorized projects, projects approved with ISS-related waivers, inconsistent category, projects re-categorized later Presence of systemic omissions with regard to identification of ES risks when compared with best practice Overall quality of the Bank´s validated ESA studies in relation to experts’ opinion Quality of SESA for PBOs and sector-wide programmes and ESMS of FIs Qualitative analysis of the ES loan conditions and follow up in supervision reports Number of projects where meaningful consultation has been achieved, including specific actions to reach the most vulnerable. Assessment of timeliness of the review, approval and disclosure of

Desk review of a sample of projects using a desk validation tool based on the Operational Safeguards (OS), 2015 In-depth application of the dimensions about OS2 of the desk validation tool , including elements of the methodology used for the Bank’s 2015 Review of implementation of the Involuntary resettlement policy Country visits and elaboration of case studies Interviews with Bank´s staff and a sub-sample of borrowers

80 The ESAP 2015 recognizes different types of safeguards tools at appraisal depending on the characteristics and ES risks of the project: ESIA and RAP, ESMP (actionable document with the key ES mitigation measures), Strategic ESA (SESA) and ESMF, ESMS for category 4 projects. The Bank’s experience validating SESA at sector or country level will also be analyzed.

60

III. Emerging evidence on Effectiveness (continuation)

4. To what extent are the Bank´s quality assurance processes contributing to ensure compliance with ISS objectives (mitigation of environmental and social risks of Bank-funded projects)?

4.5. How are project-level grievances requirements been met in a sample of projects, especially those involving resettlement and compensation of persons affected by projects (PAP)? 4.6. To what extent is the Bank in compliance with the ISS OS2 requirements (involuntary resettlement, land acquisition, population displacement and compensation)?81

4.7. To what extent the Bank’s supervision/completion documentation is able to capture the level of implementation of ES mitigation measures agreed at approval? 82

ESA studies according to ISS requirements Projects with a project-level grievance mechanism working, including informal arrangements In-depth analysis of a sample of projects which triggered OS2, including review of borrowers‘ documents Proportion of project supervisions/ completion where a safeguards specialist was associated83

Proportion of project supervision reports / aide memoires which raised key ESS issues84 Analysis of the candor of ES ratings in the IPR/PSR and completion reports Analysis of value added of special supervision for environmentally or socially sensitive projects?

81 Some emerging safeguards issues raised by other MDBs are economic displacement and the effects on host communities. 82 The evaluation will consider the concept of “adaptive management” when applicable. It will consider the ES documents modification during implementation and will report on the proportion of the reviewed projects whose ES documentation at appraisal was updated or modified during implementation. To the extent possible, it will also try to explore how clear the roles and responsibilities between the Bank and the borrower are in relation to the implementation of ESS. 83 If enough information is available, the evaluation will explore the concept of “third party monitoring” used for supervising the implementation of the environmental and social mitigation measures. For instance, the first phase of the evaluation identified various ES implementation arrangements mentioned in the PARs: inclusion of a specialist in the project implementation unit, an external company or consultant, an agreement signed with the National Environmental Management Agency to ensure oversight of the implementation of the ESMP, a lender technical advisor for private sector operations. 84 information about ES covenants, information about ESMP implementation (individual mitigation measures

61

Criterion and overarching

Question

Specific Evaluation Questions (EQ) Indicators/Judgement criteria Proposed Methodology

III.Emerging evidence on Effectiveness (continuation)

5. What is the level of ISS compliance from the emerging practice of the Financial intermediaries lending (category 4)?

5.1. How effectively is ISS being implemented for FIs (category 4 according to ISS) before Board approval and during implementation and completion? 5.2. Does the Bank have the specific expertise to provide the FI-specific support to ensure ISS compliance of FI operations?

Proportion of FI projects where existing ESMS and procedures for screening of sub-projects were reviewed prior to lending and adequately summarized in Bank´s documents (analysis of comments of the Bank’s ESS specialist, comments from other donors, if any) Number of FI projects where timely and quality reports are provided to the Bank (information about the ES risk of sub-borrowers’ portfolios). Number of FI projects that are ISS compliant Level of expertise about FIs

operations by safeguards staff

Level of support provided to FIs to set up appropriate ESMS (comparison with other organizations)

Document review Client interviews Bank staff interviews

Desk review of a sample of projects (ESA documentation at appraisal and during supervision) Visit to a sample of FI, and clients with portfolio with potential high/medium impact

62

Criterion and overarching

Question

Specific Evaluation Questions (EQ) Indicators/Judgement criteria Proposed Methodology

III.Emerging evidence on Effectiveness (continuation)

6. What is the level of advance in relation to the ISS commitments to mainstream climate

change?

6.1. How well are the climate change

requirements included in the ISS integrated in the

Environmental and Social Assessment

Procedures (ESAP) ?

6.2. What has been the level of implementation of

the tools to operationalize the ISS commitment to

mainstreaming climate change?

Analysis of the level of integration

of climate change issues in ESAP

processes and the Bank’s

Operations Manual.

To what extent CC has been

mainstreamed in the

ESIA/SESA/ESMPs of projects?85

Assessment of a sample of climate

screening reports of 181 projects

stored in the intranet:

https://climatess.afdb.org

Assessment of a sample of the

Adaptation Review Reports

(AREP) of 31 projects stored in the

intranet: https://climatess.afdb.org

Assessment of the implementation

of the GHG accounting tool

Document review Bank staff interviews

85 If possible, analysis of a sample of ToR of the ESIA/SESA/ESMP of the borrower and the reports themselves to see how well they cover climate change issues and PAR and Technical Annexes (For instance, specific cc chapter in the PAR Technical Annexes of public projects, not linked to the ESS annex).