Evaluation of the Bank’s Country Strategy and Program 2004...

An IDEV Country Strategy Evaluation

South Africa: Evaluation of the Bank’s Country Strategy and Program 2004–2015 Sum

mary Report – Redacted version

Independent Development Evaluation

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

South Africa: Evaluation of the Bank’s Country

Strategy and Program 2004–2015Summary Report

Redacted version

January 2017

IDEV conducts different types of evaluations to achieve its

strategic objectives

Thematic Evaluations Project Cluster Evaluations

Regional Integration Stra

tegy

Evaluations

Project Perfo

rmance Evaluations

(Public Secto

r)Impact Evaluations

Project Performance Evaluations

(Private Sector)

Coun

try S

trate

gy E

valu

atio

n

Evaluation Syntheses

Corporate Evaluations

Sect

or E

valu

atio

ns

Coun

try

Stra

tegy

Eva

luat

ions





An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

South Africa: Evaluation of the Bank’s Country

Strategy and Program 2004–2015Summary Report

Redacted version

January 2017

ACKNOWLEDGEMENTS

Task manager Penelope Jackson, Principal Evaluation Officer

Team member Akua Arthur-Kissi, Evaluation Officer

Consultants Ecorys Group; Andrew Danino, Team Leader

Internal peer reviewer Girma Kumbi, Principal Evaluation Officer

External peer reviewer Prof. James McDavid, Evaluation Reviewer

Internal Bank reference group Southern Africa Resource Center (SARC); Energy, Environment and Climate Change Department (ONEC); Private Sector Department (OPSD); and Financial Sector Development Department (OFSD)

Knowledge management officer Jayne Musumba, Principal Knowledge Management OfficerNajade Lindsay, Junior Consultant, Communications and Knowledge Management

Other assistance / contributions provided by Wolassa Kumo, Country Economist, SARC, Boubacar Ly, Evaluation Consultant

Special thanks to All Bank staff, and country team members in particular Kennedy Mbekeani, SARC, Lilian Macharia, OFSD, Ernest Tettey, Farai Kanonda, ONEC, Peter Sturmheit, SARC, Jonathan Banda, OPSD/Malawi Field Office, for their contribution and good cooperation during the evaluation phases. Staff of development partners, representatives and external stakeholders, clients, particularly the National Treasury officials and partners and private sector clients for their time for interviews, to arrange site visits, and to comment on evaluation findings.

Division manager Rafika Amira

Evaluator-General Rakesh Nangia

© 2017 African Development Bank Group All rights reserved – January 2017

South Africa: Evaluation of the Bank’s Country Strategy and Program 2004–2015 – Summary Report – Redacted version An IDEV Country Strategy Evaluation, January 2017

DisclaimerUnless expressly stated otherwise, the findings, interpretations and conclusions expressed in this publication are those of the various authors of the publication and are not necessarily those of the Management of the African Development Bank (the “Bank”) and the African Development Fund (the “Fund”), Boards of Directors, Boards of Governors or the countries they represent.

Use of this publication is at the reader’s sole risk. The content of this publication is provided without warranty of any kind, either express or implied, including without limitation warranties of merchantability, fitness for a particular purpose, and non-infringement of third-party rights. The Bank specifically does not make any warranties or representations as to the accuracy, completeness, reliability or current validity of any information contained in the publication. Under no circumstances including, but not limited to, negligence, shall the Bank be liable for any loss, damage, liability or expense incurred or suffered which is claimed to result directly or indirectly from use of this publication or reliance on its content.

This publication may contain advice, opinions, and statements of various information and content providers. The Bank does not represent or endorse the accuracy, completeness, reliability or current validity of any advice, opinion, statement or other information provided by any information or content provider or other person or entity. Reliance upon any such opinion, advice, statement, or other information shall also be at the reader’s own risk.

About the AfDBThe overarching objective of the African Development Bank Group is to spur sustainable economic development and social progress in its regional member countries (RMCs), thus contributing to poverty reduction. The Bank Group achieves this objective by mobilizing and allocating resources for investment in RMCs and providing policy advice and technical assistance to support development efforts.

About Independent Development Evaluation (IDEV)The mission of Independent Development Evaluation at the AfDB is to enhance the development effectiveness of the institution in its regional member countries through independent and instrumental evaluations and partnerships for sharing knowledge.

Independent Development Evaluation (IDEV)African Development Bank GroupAvenue Joseph Anoma, 01 BP 1387, Abidjan 01, Côte d’Ivoire Phone: +225 20 26 20 41 Fax: +225 20 21 31 00 E-mail: [email protected] idev.afdb.org

Layout & production: Visual Identity – www.visualidentity.co.uk Original language: English – Translation: AfDB Language Services Department

mailto:idevhelpdesk%40afdb.org%20?subject=

http://idev.afdb.org/

http://www.visualidentity.co.uk

Acknowledgements iAbbreviations and Acronyms vExecutive Summary 1Management Response 9

Introduction 23

The South African context 25

Is the Bank managing itself to maximize performance in South Africa? 29Country Strategy overview and design 29Portfolio focus 30Portfolio coherence 32Innovation 32Knowledge and policy advice 33Partnerships 34Leverage 36Efficiency 36Managing for results 39

To what extent has the Bank’s engagement in South Africa achieved expected results? 43Relevance 43Effectiveness 44Sustainability 50Crosscutting issues 53

Overview of conclusions and recommendations 57

Annexes 61South Africa Portfolio 62Evaluation Questions 65Methodology 65Country Context: Economic and Development Indicators 75TimeEfficiency 84

Contents

Contents

List of figuresFigure 1 Ratings based on PRAs only (% of projects by number) 2Figure 2 Overall ratings against the four standard criteria 3Figure 3 Portfolio concentration in 2004–2015 approvals 30Figure 4 Evolution of ambitions for knowledge and analytical work in South Africa CSPEs 33Figure5 EfficiencyratingsfromPRAs 36Figure 6 PRA ratings for the two relevance sub-criteria 44Figure7 PRAratingsforthetwoeffectivenesssub-criteria 45Figure8 Overviewofdeliveryagainstinputs,outputs,andoutcomesatcountrylevel 46Figure 9 PRA ratings for the two groups of sustainability criteria 51Figure 10 Results framework integration of crosscutting issues in South Africa versus other country portfolios 53

List of tablesTable1 Fastestandslowestcasesintheportfolio 38Table 2 Selected deliverables against output targets for energy projects assessed 46Table 3 Overview of cross cutting issue consideration in strategy and project appraisal 52

List of boxesBox 1 Coverage of Evaluation Questions 23Box2 LeveragingfinanceintheSouthAfricaportfolio 35Box3 Factorsaffectingtimeefficiency 38Box4 Client:alessoninappraisingtoensurethatriskstoeffectivenessareconsidered 47Box 5 Implicit versus explicit objectives and good banking versus good development

bankingwithfinancialintermediation 48Box6 Factorsaffectingoutputandoutcomedelivery 49Box 7 Unintended project impacts: the Medupi Power Plant 50Box8 Conclusionsaboutstrengthsandweaknesses 57

vAbbreviations and Acronyms

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Abbreviations and Acronyms

AFD Agence française de développement

AfDB African Development Bank

AWF African Water Facility

BA Board Approval

BBBEE Broad-Based Black Economic Empowerment

BTOR Back-to-Office Report

CEDR Comprehensive Evaluation of Development Results

CODE Committee on Operations and Development Effectiveness

CPIP Country Portfolio Improvement Plan

CPPR Country Portfolio Performance Review

CSPE Country Strategy and Program Evaluation

CSP Country Strategy Paper; Concentrated Solar Power

CTF Clean Technology Fund

DBSA Development Bank of Southern Africa

DFI Development Financing Institution

DO Development Objective/Outcomes

DoE Department of Energy, South Africa

DoT Department of Transport, South Africa

DP Development Partner

DPE Department of Public Enterprise

DTI Department of Trade and Industry, South Africa

E&S Environment & Social

EIB European Investment Bank

ESIA Environmental and Social Impact Assessment

ESW Economic and Sector Work

FAPA Fund for African Private Sector Assistance

FGD Flue Gas Desulphurization

GDP Gross Domestic Product

GHG Green House Gases

GoSA Government of South Africa

GTLP Global Trade Liquidity Program

GW Gigawatt

IDC Industrial Development Corporation, South Africa

IDEV Independent Development Evaluation

IFC International Finance Corporation

IP Implementation Progress

IPP Independent Power Producer

IPPPP Independent Power Producers Procurement Program

vi South Africa: Evaluation of the Bank’s Country Strategy and Program 2004–2015 – Summary Report – Redacted version

IPR Implementation Progress Report

IRP Integrated Resource Plan

JICA Japan International Cooperation Agency

KfW German Development Bank (Kreditanstalt für Wiederaufbau)

LA Loan Agreement

LoC Line of Credit

MIC Middle Income Country

MTR Mid-Term Review

MTSF Medium-Term Strategic Framework

MW Megawatt

NDP National Development Plan

NEPAD New Partnership for Africa’s Development

NERSA National Energy Regulator of South Africa

NGO Non-Governmental Organization

OFSD Financial Sector Development Department (AfDB)

O&M Operations and Maintenance

ONEC Energy, Environment and Climate Change Department (AfDB)

OPSD Private Sector Department (AfDB)

OSBP One-Stop Border Post

PAR Project Appraisal Report

PCR Project Completion Report

PRA Project Results Assessment

PSD Private Sector Development

PSR Project Status Report

PV Photovoltaic

QaE Quality-at-Entry

RE Renewable Energy

REIPP Renewable Energy Independent Power Producer

REIPPPP Renewable Energy Independent Power Producer Procurement Program

RISP Regional Integration Strategy Paper

RMC Regional Member Country

SA South Africa

SADC Southern African Development Community

SANRAL South African National Roads Agency

SAP Systems, Applications & Products – in data processing

SARC Southern Africa Resource Center (AfDB)

SME Small and Medium Enterprise

SOE State-Owned Enterprise

TA Technical Assistance

TF Trust Fund

TFR Transnet Freight Rail

UA Unit of Account

USD US Dollar

1Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Executive Summary

Introduction

This report summarizes the findings of the inde-pendent evaluation of the African Development Bank’s (AfDB or Bank) Country Strategy and Program in South Africa for the period 2004 to 2015. The evaluation is intended to inform the next country strategy paper (CSP) due in 2018; and to contribute to accountability and learning. To provide solid evidence for its findings, the evaluation included thorough project assessments as well as review of the broader portfolio of strategies and of non-lending activities. It answers questions about how the Bank is managing itself, what it has achieved, and the lessons learned based on what has helped or hindered performance. Finally, it proposes recommendations designed to help the Bank improve its future performance in South Africa.

Overview of key findings

The evaluation found a Bank that is learning to adapt to the developed, competitive South African market. It highlighted the fact that while the scope and scale of Bank operations cannot make a significant national impact, they are nevertheless important for the Bank. The findings showed that the Bank was moving in a relevant direction given its objectives, but that gaps exist in key areas and that ambitions were ill matched to available resources, instruments and demand. The evaluation raised concerns about efficiency (timeli-ness and processes) and effectiveness (in relation to Bank targets). The findings and conclusions indicate that current policies and practices are inadequate if the Bank wants to remain relevant and grow its portfolio in South Africa. A more tailored approach, appropriate resourcing of the team and a broader menu of innova-tive funding facilities, are required.

The evaluation themes provide insights for future business development in South Africa, and

possibly in other MICs. These include better identi-fying the Bank’s comparative advantage and compet-itiveness, the suitability of its financing instruments and activities. The experience in South Africa around energy, leveraging finance, syndication and lines of credit could also serve operations elsewhere.

How the Bank has managed its operations in South Africa

The evaluation found that the Bank set ambitious, relevant objectives. However, it has not been able to deliver on all of them. Objectives were aligned overall to the Bank’s strategic priorities and those of the Government of South Africa (GoSA). However, they did not integrate the important issue of ensuring inclusive growth in a highly unequal society, which is a priority that both the Bank and South Africa have made explicit. The Bank was unable to deliver a port-folio that transformed all its strategic priorities into operations and activities, although the volume of its portfolio grew overall. The Bank did not effectively judge its comparative advantage in the South African context, especially regarding the competitiveness of its private sector finance and the government’s willingness to provide sovereign guarantees. The Bank’s expansion was blocked as a result, particu-larly in water and transport, as were its capacity and resources to be more than a financier in areas such as knowledge work and capacity development. The port-folio in South Africa focused on infrastructure (mainly energy) and finance by default as much as by design (see Portfolio Focus), which affected the Bank’s abil-ity to produce results across all of the areas that its strategies identified as priorities (see Effectiveness).

The Bank’s approach to partnership has been selective. The focus has been to build a close rela-tionship with the National Treasury (NT), to develop business opportunities rather than to target policy

2 South Africa: Evaluation of the Bank’s Country Strategy and Program 2004–2015 – Summary Report – Redacted version

changes or to build closer alliances with other devel-opment partners (DPs) at the strategic level. Relations with the private sector and other DPs have not been pursued with equal energy. This reflected the limited human resources as well a deliberate, strategic deci-sion about where the Bank was likely to generate busi-ness (see Partnerships).

The Bank has sought to innovate in its approach and to leverage additional funding. While the posi-tive examples of leverage did not yet amount to a systematic, fully successful approach they did provide a rich source of learning, highlighted the potential added value that the Bank and its staff can have in this area, as well as management’s interest. Clients acknowledged and appreciated the Bank’s willing-ness to be the first at the table and to attempt to lever-age other finance. With regards innovation, there is scope for the Bank to do more within its operations (see Innovation and Leverage).

The Bank has not realized its aspiration to be more than a financier in South Africa, nor has it added

value by generating and sharing knowledge or supporting capacity development. Stakehold-ers see the Bank as a financier, not as a knowledge producer or supporter of their capacity. Significant strategic ambitions regarding capacity development and knowledge generation and sharing were not resourced, and the Bank did not identify a compar-ative advantage and the relevant tools or expertise to help it perform in these areas. (See Knowledge and Policy Advice and Effectiveness).

With regards monitoring and managing for results, the picture in South Africa resembled the picture elsewhere. With a few exceptions in infrastructure operations, the focus of monitoring remained imple-mentation and financial data rather than progress towards development results. The advances of the last five years in approaches to monitoring public sector projects have not been mirrored in private sector projects. For example, public sector log frame indicators were more focused and required clearer monitoring data from clients than in the private sector. The approach to understanding and reporting the

Figure 1: Ratings based on PRAs only (% of projects by number)

% of projects achieving satisfactory or higher

% of projects achieving moderately satisfactory or higher

Relevance

EffectivenessSustainability

100

50

0

Efficiency

Source: AfDB, SAP.

3Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

results of lines of credit (LoC) in particular was found wanting (see Managing for Results).

Assessing Bank achievements and performance

The evaluation addressed four standard areas that were rated on the basis of evidence and assess-ments at project, sector, and country levels. (See Annex B, six-point rating scale). Figure 1 summarizes the ratings for projects comprising 76% of the port-folio. These ratings were a crucial building block for the overall rating (Figure 2). Both project and overall ratings indicate that efficiency and effectiveness were the two main areas of weakness

The evaluation found the program, its strategies and the individual projects, to be broadly relevant and well aligned with the priorities of the Bank and of the Government of South Africa (GoSA). However, this high degree of relevance in what the Bank was seeking to achieve was not always reflected in the design of the pipeline of operations or individual projects in terms of their appropriateness, quality, and appraisal. Consequently, they were rated moderately satisfactory (see Relevance).

The evaluation of efficiency examined cost effi-ciency, profitability and timeliness. The picture was mixed. The evaluation found some positive stories in individual projects but there was insufficient informa-tion to assess the cost effectiveness of the program

as a whole. The story was more negative regarding timeliness: Major delays occurred at various stages of the project process from appraisal through implemen-tation. The factors affecting timeliness were recurrent: long, bureaucratic processes and under-resourcing of task managers and of relevant support department staff (e.g. legal, disbursement, etc.). While there was considerable variation across the portfolio, delay was a perennial issue across the portfolio (timeliness has been poor for small grant approvals (Middle Income Country Trust Fund [MIC TF] and for major project execution). Furthermore, clients and stakeholders, and even Bank staff complained that Bank processes were too slow. The Bank is not perceived to be the nimble partner that South African institutions want, which contributed to an overall efficiency rating of moderately unsatisfactory (see Efficiency).

Effectiveness was considered at output and outcome levels, by the results achieved in the port-folio and in other non-lending activities. The levels of achievement varied by sector. In infrastructure, output delivery was generally solid albeit often late. In the energy sector, for example, project delays, among other things, thwarted the expected increase in power generation to prevent load shedding, which was not averted as planned but projects are on track to deliver the expected level of power at a later date. The picture was more mixed in the finance sector – with considerable variation in project outcomes. Some less well performing projects had inappropriate indi-cators while others missed the strategy’s outcome focus of developing SMEs. With regards knowledge work and capacity development, high ambitions were not tethered to a solid analysis of the Bank’s compar-ative advantage, to the existence of appropriate tools or to the necessary expertise or resources. Overall effectiveness was rated moderately unsatisfactory (see Effectiveness).

Sustainability was considered from a financial, technical, environmental, and social perspec-tive. It was found to be less of a concern than in some other country programs, although data was limited. Monitoring reports often lacked the informa-tion needed to assess the likelihood of sustainability,

Figure 2: Overall ratings against the four standard criteria

Relevance: moderately satisfactory

Effectiveness: Moderately unsatisfactory

Efficiency: Moderately unsatisfactory

Sustainability: moderately satisfactory


but the lack of environmental and social detail at the sub-project level is acute for LoCs. Neverthe-less, based on available information, environmental and social sustainability were judged to be at least moderately satisfactory in most cases. In terms of ongoing profitability, the nature of the finance sector projects was such that the Bank itself took on little risk. However, some loan portfolios included non-per-forming sub-project loans, although their size was limited in most cases. With regards to infrastruc-ture projects, high technical quality bodes well for sustainability although there was some variation on the environmental side. The environmental impacts of the Medupi Power Station, for example, were especially challenging. The extent to which the CSPs examined sustainability improved somewhat over time. Sustainability was rated as moderately satis-factory (see Sustainability).

Mainstreaming crosscutting issues remained challenging. The Bank’s work in South Africa did not explicitly integrate gender or inclusive growth issues

– yet shared growth is a stated priority for both the Bank and GoSA. By contrast, the strategy and port-folio improved over time in terms of considering envi-ronmental and green growth issues and compared favorably with other countries. Regional integration was central to the most recent CSPs but was only partially evident in the design of program operations and activities (see Crosscutting issues).

Broader themes based on experience to inform a future approach

Finding the right niche in the South African context. Two interlinked themes emerged with respect to strat-egy and to finding the right niche for the Bank in the competitive South African context.

❙ First, there was a mismatch between the Bank’s ambition and the resources that it allocated, with respect to its capacity and expertise for developing new business and for achieving objectives such as knowledge generation, capacity development, and gender mainstreaming. The Bank seeks to be

more than a financier yet it has not organized itself to make this a reality.

❙ Second, strategic objectives and the planned port-folio were based on an inadequate understanding of where the Bank could add value, including with respect to competitive pricing in local currency in a context where there are few sovereign guarantees and dynamic local bond markets. The Bank has thus fallen into a niche dominated by lending (i) to a small number of SOEs, and (ii) in foreign currency for on lending outside of South Africa. Both areas are valid for continued investment. However, there is only limited room to further expand the Bank’s portfolio if it maintains the same approach.

Striking the balance between good banking and good development banking. The Bank was successful in deploying funds to third countries with-out taking on risk, through its lines of credit. However, that was rarely an explicit objective of the regional LoCs. In reality, these operations were used to build up an initially slim portfolio and to make inroads into the South African market. The increase in profitable, low-risk lending was generally successful (over the period, the finance sector accounted for over 800 million UA). However, it is much harder to demon-strate success from a development banking perspec-tive of ensuring that funds contributed to inclusive and green growth by creating jobs, etc. Linked to this was the difference between what the Bank argued that it wanted to achieve with its LoCs during appraisal, and what the loan agreements stated. This made it diffi-cult to monitor against original objectives especially where borrowers were unaware of the Bank’s explicit objectives. Furthermore, there was no link with CSP objectives of job creation and small and medium enterprise (SME) support in South Africa in the LoCs that are used only outside the country. Intermediaries were interested in working together to improve results reporting, but the Bank has not yet taken advantage of this mutually beneficial opportunity (see Efficiency and Effectiveness).

Organizing resources and processes effectively and efficiently. The Bank’s decentralization in the

5Executive Summary

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

South Africa context has been insufficient: It leaves gaps in capacity, in available expertise, in due dili-gence and business development. Headquarters did not act upon previous requests to enhance capac-ity of the Southern Africa Resource Center (SARC). Looking forward, the implementation of the Bank’s new business development model can benefit South Africa program especially if it addresses key bottlenecks in capacity such as business develop-ment, disbursement, legal, operations assistance, and (if these remain central to the next strategy) the resources to work on knowledge and capacity (see Knowledge and policy advice). Timeliness, which was identified as one of the weakest areas portfolio-wide, has affected the delivery of results and the Bank’s reputation with clients. Such reputational damage is serious in the South African context given that the Bank is a small player looking to gain ground in a highly competitive market.

Recommendations to improve Bank performance

The evaluation proposes the following recommen-dations on the basis of its findings and conclusions. Management should consider implementing these recommendations within the next CSP cycle.

1. Initiate steps to ensure that the next CSP is based on a more detailed analysis of the (i) Bank’s comparative advantage and constraints in South Africa; (ii) the resources required to deliver in selected areas of focus. In particular the review could cover:

❙ Key areas of new business development and of repeat business, based on a thorough analysis of where and when Bank funding instruments will be competitive. This means resisting pressure to add sectors that may interest the Bank but for which it has no comparative advantage and it means thinking innovatively about competitive funding instruments.

❙ Whether or not the Bank has a comparative advantage in (i) knowledge generation and shar-ing and; (ii) capacity development, and if it does, in what specific areas.

❙ Consider a more systematic appreciation of client capacity and options so that the Bank can help develop capacity in the areas that are key to operations delivery.

2. Address the mismatch between the implicit and explicit objectives of LoCs and between appraisals and loan agreements in private sector operations by adopting a more coher-ent, realistic approach which is also aligned with the objectives of the next CSP. This will require frank discussions with the Board and senior management on the banking/risk-sharing role of LoCs versus their develop-ment potential, and designing operations that strike the right balance. All objectives should be stated in the appraisal to allow for a fuller understanding and to acknowledge success with respect to objectives such as risk sharing. Closing the appraisal and loan agreement gap may require more upfront work, but it should reduce the lag that frequently occurs between approval and signature.

3. Build on the South African experience to inform revisions to the Bank’s approach to private sector operation design and moni-toring, to allow for better understanding of results. Current project status reports and back to office reports (BTORs) lack information about progress towards development results, which also stems from the initial design and appraisal procedures. The approach for LoCs in particular needs to be revisited to ensure that the neces-sary reporting data from clients is known upfront and that measurable indicators are used. In South Africa, the Bank has an opportunity to collaborate with existing clients to develop a more robust approach that would benefit both


the intermediary and the Bank by enhancing the ability to understand, track, and report results. A new approach can also draw from experience in the Bank’s public sector supervision revisions and from the International Finance Corporation (IFC) and other partners.

4. Use the South Africa experience to support the Bank in identifying new flexible funding mechanisms better suited for MICs. Differ-ent types of risk participation may be one avenue to explore, more targeted LoCs may be another. This could be done jointly with a range of departments and be central to the business development strategy.

5. Ensure a complete decentralization process and business process streamlining. The expe-rience in South Africa is symptomatic of a broader issue that Bank management is now seeking to address through a new business model. SARC would benefit from in-house disbursement and legal skills and from expertise linked to a given strategic focus (e.g. more business development

officers on the group for private sector opera-tions), especially if it wants to continue to pursue the knowledge agenda. In terms of delegating approval authority, the current situation has not allowed the Bank to be the nimble partner it wants to be. The experience of the MIC TF makes it clear that delegating this and other small grant facilities would be an obvious, helpful step.

6. Share the lessons from South Africa’s experi-ence in the energy sector. South Africa’s expe-rience in investing and diversifying its energy sector as well as the Bank’s engagement are rich with lessons relevant to other RMCs. Given its continental mandate and its prioritization of energy, the Bank is well placed to facilitate a learning exchange that can cover (i) differ-ent types of energy generation; (ii) public and private (independent power producer) roles; (iii) tariff setting and the role of a strong regulator; (iv) contract and project management for new building and refurbishment, and (v) community relations and management in major infrastruc-ture development.

9Management Response

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Management Response

Management welcomes the Independent Development Evaluation’s (IDEV) report on the Bank’s development assistance to South Africa during the period 2004–2015, covering three Country Strategy Papers (CSPs): 2003–2005 extended to 2007; 2008–2012, and 2013–2017. The Evaluation provides a timely assessment of the CSPs’ efficiency, relevance, effectiveness and sustainability in the key sectors of energy, transport, water, mining, finance, and regional integration. It has also assessed the performance of CSPs in light of cross-cut-ting themes, including: capacity development, inequality, gender, and environment. Overall, Management agrees with the key findings and recommendations of the evaluation, particularly the need to ensure a complete decentralization process and business process streamlining to make the Bank a more nimble partner, as well as initiating steps to ensure thorough analysis of the Bank’s comparative advantage and the competitiveness of its funding instruments in South Africa. Some of the other issues raised by the evalua-tion will be addressed as the Bank is implementing the new Development and Business Delivery Model. The evaluation provides useful lessons and opportunity for the Bank to improve its operations in South Africa. The recommendations of the IDEV report will inform the Bank’s next CSP 2018–2022 for South Africa.

Relevance

Management agrees with IDEV that all three CSPs were fully aligned with the Bank’s priorities and South Africa’s development strategies, and that the Bank operations undertaken in this period were relevant to the country’s development goals and objectives. Management also agrees that although the design of individual projects are of good quality standard overall, there is a need to further strengthen project design with respect to client capacity, targeting, technical design, procurement, contracting, and log frames, indicators and targets. The next CSP (2018–2022) will preserve the alignment with the country’s National Development Plan, the Industrial Policy Action Plan and other relevant sector development priorities.

Effectiveness

Management is encouraged by IDEV’s findings that output delivery in infrastructure projects is broadly satisfactory and that the Sere Wind Farm energy project was rated highly satisfactory as energy gener-ation figures surpassed those planned at project appraisal. Management is pleased by the fact that,

together with other development partners, the Bank assisted the government to build Sere Wind Farm, the country’s first large-scale renewable energy project with an average yearly energy generation capacity of about 298,000 MWh contributing to reducing green-house gas emissions by nearly 6 million tons over the course of its expected 20-year operating life.

Management is also encouraged by IDEV’s findings that in spite of delays in the implementation of ongo-ing non-renewable energy projects, some projects that have been completed have exceeded targets significantly contributing to household access to reliable and affordable electricity supply in the coun-try. Management also notes IDEV’s observation that a number of challenges have affected implemen-tation of the ongoing energy projects, but they are broadly on track to deliver the planned output in the medium term. The Eskom Medupi project as well as the renewable energy project were constrained by several challenges, including design gaps, client or contractor capacity limitations, and start-up and implementation delays. Management appreciates IDEV’s attention to the major implementation difficul-ties encountered at the start of these projects in the evaluation of their performances.


The Bank, through its Middle Income Countries’ Technical Assistance Fund, will conti-nue to support client capacity building. Moreover, as agreed in the portfolio improvement plan, to improve project quality at entry, the Bank will ensure that compre-hensive feasibility studies have been conducted to inform project preparation, that project designs are not overloaded and midterm reviews are systemat-ically conducted. The Bank will continue to provide short term training to public and private sector offi-cials and staff of project executing entities to ensure clear understanding of Bank audit, disbursement and procurement rules and procedures.

Management agrees with IDEV’s finding that the Bank’s interventions in transport sector which resulted in a successful acquisition of more and efficient locomotives has contributed to increased volume of commodity rail freight transported. Although IDEV rated overall effecti-veness moder-ately unsatisfactory, evidence in this and other sectors point to better results. For instance, total rail freight transported increased by 7.7 percent between 2014 and 2015 reaching an all-time high of 226.6 tonnes. Bank investment accounted for about 4 percent in average annual freight rail capital expenditure by Transnet. More resources have been provided to the transport sector in the current year to augment results achieved so far.

Management also acknowledges IDEV’s findings that the non-sovereign corporate loan to Eskom of USD 500 million provided in 2008 by the private sector window of the Bank has yielded positive results as electricity generation output exceeded targets.

Management equally concurs with the evaluation finding that the Bank’s private sector operations in the finance sector have been largely success-ful in enabling the South African financial institu-tions to grow their portfolio locally and to increase their engagement in other RMCs especially the low income countries. The Bank may consider to capi-talise on the successes achieved in the energy and finance sectors and consolidate these by pursuing its interventions in these areas in the future CSP.

Management also notes the evaluation’s concern about the discrepancy between the good develop-mental banking objectives, such as job creation and environmental and social responsibility, included in the PARs approved by the Board and the loan agree-ments of the financial intermediation entities, which rarely include these objectives.

The Bank views support to South African financial intermediaries to invest across the region as an essen-tial and integral component of the Bank’s regional integration strategies. In this regard, Management is pleased to acknowledge that the South African finan-cial sector is one of the key drivers of regional financial integration in the SADC, as well as in Eastern Africa. Management acknowledges that Bank processes have been evolving and improving over the 10 year review period. Thus, it is noted that while some of the earlier approved Lines of Credit (LOCs) could not meet all the current requirements, at present, all new LOCs Agreements require reporting on selected develop-ment indicators that balance the ability of the finan-cial institution to realistically monitor and access the required information and reduce transaction costs in terms of time and resource cost burden on the clients. Management concurs that there is a need to review and differentiate the current approach to monitoring LOCs from that of project financing to ensure that it is realistic and in line with market realities and prac-tices. Potential alternatives to addressing the existing monitoring challenges – including through monitoring some of the development impacts and outcomes by implementing Bank funded case studies, expanded supervision reports and project completion reports – are already on going. This would enable the Bank to more reliably measure the impact and outcomes of its private sector support without unduly encumbering the implementation process on the part of the client. The Bank’s implementation of environmental safe-guards has improved over time which explains the lack of reporting in earlier LOCs on this front. However, all LOCs Agreements now systematically require regular reporting on environmental and social issues.

Going forward, the Bank will strive to consolidate interventions in the current High 5 priority areas to


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

assist the government’s efforts to improve the lives of the majority of the South African population.

Management acknowledges IDEV’s evaluation find-ings that the Bank’s planned know-ledge and analyt-ical works have only been partly delivered and that the Bank has limited room to target policy change in the South African context partly due to the developed capacity of institutions in the country. Management also agrees with the IDEV finding that partnership and leverage have been strong with central government as this is the key entry point to develop business for the Bank. Bank efforts to leverage finance have also been partly successful with positive results across the portfolio, indicating the Bank is adding value and learning from experience.

Management concurs with IDEV that the Bank’s intervention in South Africa is not significant enough for any meaningful analysis of its contribution to national impact indicators. The effectiveness eval-uation, therefore, only concerned itself with outputs and outcomes but not impacts of these operations. Management is cognizant that, where possible, the evaluation assessed local unintended impacts of specific large projects.

Management also notes that the rating provided by the evaluation report is at an aggregate level, but that the analysis did not make available a review at each individual project level. This would have increased the transparency of the evaluation and allowed manage-ment to better understand the findings.

Efficiency

Management agrees with the evaluation findings that the efficiency of implementing Bank interventions during the evaluation period were constrained by time instead of cost. Management concurs with the eval-uation finding that, in spite of decentralisation, there has been limited improvements in time efficiency with the result that the client still perceives the Bank as bureaucratic and slow. Efficiency constraints spanned across all sectors of Bank interventions in the country.

Management appreciates evaluation’s findings that lack of full implementation of decentralisa-tion contributed to slow improvements in time effi-ciency and, in some cases, promoted inefficiencies. Management concurs that in spite of decentralisation, the Regional Resource Centre continuously relies on the Headquarters on key decision making processes including any legal or disbursement related matters and for a wide range of approval authorities, while for some approvals, decentralization added rather than removed a layer. Management agrees with the evaluation finding that the decentralisation process served the Bank and South Africa well but it was not completed. Capabilities remained heavily concentrated at headquarters, delegation of author-ity needed to be strengthened, and Bank processes needed to be simplified. However, an enhancement will be made to the process with the rollout of the new Development and Business Delivery Model.

Management concurs with IDEV’s finding that procurement is the one area where the Bank has decentralized significantly, and placed a regional coordinator and a procurement officer on the ground and that in the last five years, delays relating to procurement in the South Africa portfolio are gener-ally not on the side of the Bank, as witnessed by a client and a development partner who reported a quick turnaround (compared to other partners) for the processing of no objections.

Management agrees with the finding of the evalua-tion regarding factors beyond Bank processes that reduce timeliness of project implementation such as: (1) Project start-up delays including clients meeting the Bank’s pre-commitment conditions and disburse-ment conditions; (2) Project approval process layers and the bureaucratic hurdles therein; (3) Procure-ment and or contracting issues relating to contract and contractor management particularly with pack-aging contracts into project components in complex infrastructure operations; (4) Poor Bank supervision and monitoring resulting in limited lines of commu-nication between the Bank and the client providing needed information and responses to act quickly and adequately on issues; and (5) Bank and client


capacity (at various stages) due to inadequate time and resources from the Bank to address client needs in administering loans.

Sustainability

Management agrees with the evaluation findings that the Bank operations undertaken during the three CSP periods in the country were technically and financially sustainable. An overwhelming majority of projects received a rating of moderately satisfactory or higher for overall sustainability, an overall performance for South Africa which is typically above that of a broad range of countries previously evaluated by IDEV.

Management concurs with IDEV that all three energy sector projects are technically very sound primarily because of Eskom’s particular attention for the higher quality of projects it implements. The lifespan of the Medupi project is over 50 years while that of the Sere Wind Farm project is over 20 years which demon-strates well the sustainability of Bank financed oper-ations in the country.

Management agrees with the IDEV observation that for the private sector projects, profitability or business success is the main measure of sustainability and for the lines of credit, the intermediaries take on most of the risk. In this regard, Management concurs with the findings of the evaluation that all finance projects have achieved moderately satisfactory ratings while the overwhelming majority have achieved a satisfac-tory rating on sustainability.

Management agrees with the IDEV findings that the energy sector projects have performed well in terms of environmental and social sustainability measures despite challenges arising from delays in funding Flue Gas Desulphurisation (FGD) for the Eskom Medupi project which may undermine air quality at least in the first five years of operation of the project while water stress may grow as a proposed operation remains unfunded. Management also concurs with the evalu-ation findings that the renewable energy project has performed well in terms of social and environmental

sustainability and that it complies with the relatively solid national environmental legislation, is implement-ing an environmental authorization and environmental management plan, and has a third party environmen-tal control officer acting for the Department of Envi-ronment who conducts regular audits.

Given the importance of the Bank’s energy portfolio in South Africa and the sensitivities surrounding the use of coal for electricity generation in the country, management wishes to highlight that the loan agree-ment for Eskom’s Medupi project calls for the client to submit to the Bank the strategic endorsement and timing on FGD retrofit and the water permit from the Department of Water Affairs and for Mokolo Dam Phase 1 and the integrated water permit for Croco-dile water transfer Scheme. Both of these conditions have been fulfilled to the letter. IDEV has overlooked this in the evaluation. Additionally, the water compo-nent is being developed to ensure that the needed water is available as and when the Units come on line. The Government has shown its commitment to the water transfer scheme by implementing the first phase, albeit without formally confirming the financ-ing plan. The timing of the implementation of the FGD is still in line with the Eskom commitment at the time of appraisal, six years after the commission of each unit. Eskom is continuously raising funds to meet its overall funding requirements, including for the FGD. Whilst no specific FGD funding has been identified, funds are fungible and as long as Eskom is able to raise funds on the market, which is the case right now, it will be able to fund the FDG program in line with its commitment to lenders.

Management concurs with the findings of the evalu-ation that the Bank has succeeded in supporting the development of environmental and social standards for the Lines of Credit projects. The Bank supported two commercial banks to apply the Equator Princi-ples, which recognised that they did so in response to International Financing Institutions’ suggestion to do so. Management also acknowledges the IDEV findings that information to determine the quality of environ-mental and social standards at the final sub-pro-ject level for the projects which were funded by the


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

intermediaries that received Lines of Credit from the Bank were inadequate. This is due to the fact that the information available on lines of credit is at the intermediary level rather than at the final sub-project level in most cases. In the future, the Bank will ensure that intermediaries will provide information on envi-ronmental and social standards at sub-project level during supervision and monitoring.

Cross Cutting Issues

Gender and Environment

Management agrees with the evaluation’s findings that the current CSP for South Africa has adequately dealt with environmental and green growth issues and that energy and renewable energy play an important role in the Bank’s strategy and portfolio. Management agrees with the IDEV findings that the credit provided by the Bank to support a client to improve its envi-ronmental monitoring and systems and efforts has targeted sub-projects focusing on clean energy and has therefore significantly contributed to improved environmental and social outcomes. Manage-ment notes IDEV’s findings that gender should have been covered more comprehensively in opera-tions, although it does feature sufficiently in the list of economic and sector works that have been deliv-ered. Inclusive growth is one of the objectives of the Bank’s Ten Year Strategy and Bank’s interventions in the country were aimed at ensuring inclusive growth.

In addition to strong gender mainstreaming measures in all recent Bank funded projects in the country, new and on-going activities are addressing the inclusion of gender using different approaches. These include, gender-focused operations where interventions in the SME sector to empower women entrepreneurs are directly funded, technical support for local women entrepreneurs, and support to specialised activities aimed at creating employment for youth and women.

Going forward, in line with the Bank’s new Gender Strategy 2014/18: Investing in Gender Equality

for Africa’s Transformation and the Government’s National Policy Framework for Women’s Empower-ment and Gender Equality, the design and implemen-tation of operations will pay more attention to gender equality and women empowerment through support to entrepreneurial activities and job creation. When-ever there is an opportunity, gender focused opera-tions will also be undertaken. Furthermore, the Bank acknowledges and works within the Broad-based Black Economic Empowerment (BBBEE) framework. The BBBEE addresses government development objectives to promote economic growth and trans-formation by enabling meaningful participation of black and other previously disadvantaged people in the economy to bring about equitable income distri-bution and create equal opportunities. Furthermore, the Bank has contributed to the government’s objec-tive to achieve inclusive growth through infrastruc-ture development.

Capacity Development

Management acknowledges IDEV’s findings that capacity development was a major theme of the last two CSPs, although several investment projects were not planned to include a capacity building component and some of the planned capacity development activ-ities in some major projects have not achieved results. Also Management notes that not as many small grants were used in this area as anticipated in the strategy, partly because of the limited number of small grant instruments available and the lack of appetite to use what is available because of the nature of the process associated with applying for those grants.

Going forward, in the remaining period of the current CSP and in the next CSP 2018–2022, stronger emphasis will be placed on inclusion of capacity development in major operations where capacity needs would have been identified at appraisal. As the Bank implements the new Development and Business Delivery Model, Bank processes will be streamlined and will enable efficient use of grant resources to provide capacity building support to stakeholders in need.


Regional Integration: Management agrees with IDEV’s findings that the Bank support to develop a national policy on One Stop Border Post has not only delivered the planned output but is also a rele-vant intervention to harness regional integration. The policy is currently awaiting approval by the Cabinet and Parliament.

Management notes the evaluation’s conclusions that the Regional Lines of Credit are involved in invest-ments in every part of the continent, rather than focusing on integrating a region. It is the view of Management that the regional Lines of Credit were essential to leverage South Africa’s institutional capacity to harness regional integration across the SADC region, as well as the continent. This is in line with the Bank’s Regional Integration Strategy and Policy (2014–2023).

Management also notes that Bank interventions in the regional infrastructure, in particular the energy sector have achieved results since 2015 and will meet targets in the medium term while interventions in freight rail have contri-buted to improved freight tonnages transported by Transnet. Going forward, the Bank will continue to support infrastructure projects in line with two of the High 5 priorities, namely, Inte-grating Africa, and Lighting up and Powering Africa.

Managing Bank’s Assistance

Managing for Results

Management concurs with evaluation findings that the managing-for-results approach was not adequately developed in the first two CSPs, and only the current CSP 2013–2017 adopted an appropriate results-based framework, monitoring and evaluation systems, and the use of lessons to guide strategy formulation and project design and that efforts to enable the moni-toring of development results have also improved over the period. Management agrees with the IDEV findings that in line with changes in the rest of the Bank, the improvements in monitoring public sector

operations have not occurred in the private sector. Monitoring results from lines of credit in the South Africa portfolio needs to be strengthened. Manage-ment concurs with the evaluation findings that under the current CSP, the Bank and other development partners are increasingly conducting joint monitoring of projects and that the findings are increasingly being used to inform the government and to provide tech-nical support to the client. Management also agrees with the evaluation findings that under the current CSP, where projects are funded by both private and public windows, as in the case of Medupi, the public and private sector teams are coordinating their site super-vision and monitoring programs. In addition, currently, in agreement with the government, all supervision programs for public sector projects are combined and conducted within an agreed two weeks period in each quarter of the year.

Innovation, Leverage and Partnerships

Management concurs with IDEV’s findings that inno-vation in terms of technologies, methods of funding and leverage, and ways of targeting specific groups at strategy and individual project levels has increased and included some interesting interventions such as (i) the Bank’s involvement in the renewable energy independent private power generation program (REIPPP), (ii) support to new technology (concen-trated solar power) albeit severe delays because of technology and procurement reasons; (iii) syndication to leverage more funds from the private sector; (iv) targeting disadvantaged groups and affordable hous-ing with small loans (Commercial Bank B sub-debt), and (v) combining local and foreign currency loans.

Management appreciates the evaluation’s findings that the Bank has succeeded in leveraging finance both through syndication, i.e. B-loan structures and by being the first development finance institution to be on the table to provide funding to both public and private operations thereby enhancing investor confi-dence, and that it has received praises from clients in this respect. Management agrees with the eval-uation findings that the Bank succeeded in building strong partnership with the core of the government, in


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

particular, the National Treasury, thanks to the open-ing of the country office and the Regional Resource Centre since 2009. Management also acknowledges that strategic and high level efforts were made to establish stronger partnerships with the private sector and other development partners, and that the Bank actively partners with development partners on the energy sector projects and public finance manage-ment group although the role of donor coordination groups remains limited as government prefers to manage relations bilaterally.

Portfolio focus and coherence

Management acknowledges the evaluation findings that the Bank’s current portfolio in South Africa is more focused and concentrated in two sectors – infra-structure (energy and transport) and finance – due to the limited areas where the Bank has a comparative advantage in the South African context and internal capacity constraints arising from limited resources to support enhanced engagement in all key sectors. Management concurs with the IDEV findings that there are gaps between the sectors identified in the strategies and the portfolio primarily because of the government policy that prohibits the national govern-ment from borrowing directly from the Bank. However, the Government encourages State Owned Enter-prises to borrow directly from the Bank. Other reasons include the lack of a Bank lending policy to municipal-ities and provincial governments. Management notes

the evaluation findings that on non-lending activi-ties, although the latest two strategies highlighted the importance of knowledge generation and shar-ing, the strategies did not make an in-depth assess-ment regarding whether the Bank had the expertise, resources, capacity or instruments to add value in this area. In addition, where strategy documents acknowl-edged that resources would be required, these were eventually not made available.

Management wishes to reiterate that South Africa has a well-developed institutional capacity for knowl-edge generation. Its public research institutions such as the Human Science Research Council and Council for Scientific and Industrial Research are amongst the best in the continent, producing cutting-edge public research in areas that are crucial to development. In addition, most of South Africa’s higher learning insti-tutions are also amongst the best in Africa and have well established capacity in development research.

Management would in developing the next CSP survey using some of the South Africa think tanks to carry out specific ESWs that can help guide its dialogue with the Government and the private sector.

Going forward, the Bank’s next CSP for South Africa covering the period 2018–2022 will benefit from the evaluation findings. Key lessons and recommenda-tions will be fully integrated into the CSP. Specific actions and lessons taken by SARC are flagged in the following Management Action Record.


Summary of Management Action

Recommendation Management Response

The Bank’s next CSP 2018–2022 for South Africa will be drafted in 2017. This CSP will benefit from IDEV’s analysis and recommendations. Key lessons will be fully integrated into the final draft of the CSP. Specific actions and lessons taken by the Southern Africa Resource Centre (SARC) are highlighted in the following Management Action Record.

RECOMMENDATION 1: Focus on selected areas where the Bank has a comparative advantage.

Comment – Initiate steps to ensure that the next CSP is based on a more detailed analysis of the Bank’s comparative advantage and constraints in South Africa and of the resources required to deliver in selected areas of focus, and on providing them. In particular the review could cover:❙ Key areas of new business

development and of repeat business, based on a thorough analysis of where and when Bank funding instruments will be competitive. This also means resisting pressure to add sectors that may be of interest to the Bank but for which it has no comparative advantage in South Africa.

❙ Assessing carefully if the Bank has a comparative advantage in knowledge generation and sharing and, if it does, in what areas. (For example, the regional perspective or more specifically linked to Bank-funded operations). Similarly, whether or not the Bank has the tools to support capacity development.

❙ Consider a more systematic consideration of client capacity is needed and options for the Bank to help develop capacity in areas that are key to operations delivery should be central to the operation appraisal.

Agreed. The next CSP for 2018–2022 will be informed by more detailed analysis of Bank’s comparative advantage and constraints in South Africa and will fully respond to IDEV’s recommendation.

Further Actions:

More thorough studies will be conducted leading up to the next CSP 2018–2022 to identify the Bank’s comparative advantage in sectors of intervention including in the areas of knowledge generation and gaps in client capacity, as well as key development challenges facing South Africa. The studies will be completed in the first half of 2017 in time to inform the presentation to be given to OpsCom and CODE members on the combined CSP completion report and pillars for the new CSP.

The selectivity criteria that will inform the CSP strategic choices will include:

South Africa’s most pressing development challenges;

Alignment with South Africa’s National Development Plan and sector development strategies;

Alignment with the Bank’s corporate strategic framework and priorities, notably the Bank’s Ten Year Strategy and the High 5s;

Outcome of stakeholder consultations;

Sectoral focus of development partner interventions;

Building on the Bank’s past achievements, comparative advantage and aid effectiveness;

Lessons learned from previous engagement; and

Outcome of feasibility studies.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

RECOMMENDATION 2: Address the mismatch in objectives of Lines of Credit.

Comment – Address the mismatch between the implicit and explicit objectives of lines of credit and between appraisals and loan agreements in private sector operations by adopting a more coherent, realistic approach to addressing them as is done for public sector operations. Future private sector operations in South Africa (and, by extension, elsewhere) should close the gap between the objectives and indicators in the PAR and requirements in the loan agreements. It could also consider realistic and appropriate choice of indicators to match the operation type. This will require frank discussions with the Board and senior management on the banking/risk-sharing role of LoCs versus their development potential and designing operations that strike the right balance. All objectives should be stated in the appraisal to allow for a fuller understanding and to give the credit to success with respect to objectives such as risk sharing. Closing the appraisal and loan agreement gap may require more upfront work but it should reduce the lag that often occurs between approval and signature.

Agreed. Going forward the Bank will develop new guidelines for processing of lines of credit. The CSP 2018–2022 for South Africa will take into account the updated guidelines, to the extent that they would be made available by the time of the CSP preparation.

Further Actions:

The indicators of Lines of Credit interventions will be reviewed and aligned with operational and market realities. This would include: (i) adopting a differentiated approach of assessing progress towards attainment of objectives of the LOCs from its expected development outcomes and impacts; (ii) differentiating indicators for LOCs (intermediated projects) from those of (direct) Project finance; and, (iii) Reducing the number and type of indicators to those that are core to the objectives of the LOC and that can be realistically monitored by beneficiary institutions. The Bank will also seek to reduce the time from approval to signature in the areas that are within its control and efforts to identify and prioritize those processes that can be finalized before Board are on-going. However, it should also be noted that some processes that take time and cause delays are required only after Board approval e.g. guarantee ratification by parliament or non-objection letters amongst others which are done based on the Board approval. It will also look into the issue of risk sharing to channel funds to priority sectors in higher risk countries. These measures would bridge the gap between appraisal information and LOC Agreement contents. It would also minimize subsequent monitoring and reporting challenges affecting LOCs as highlighted elsewhere. Among the considerations is shifting the modality of collecting data on development outcomes and impacts towards monitoring during Bank supervision missions and the preparation of case studies and Bank commissioned studies. This effort is part of the work conducted by the Bank to comprehensively monitor and report on operations’ progress, which involves providing staff with the right capacity and tools to this end. Regarding the risk sharing objectives, Bank operations are guided by the Bank’s risk capital utilisation framework which sets limits on the types and quantum of risk the Bank can undertake. The leveraging of Financial Institutions in lower risk countries to channel funds to priority sectors in higher risk countries is a risk diversification/mitigation strategy of the Bank rather than a specific objective of each respective LOC.


RECOMMENDATION 3: Use South African experience to inform revisions to the Bank’s approach to private sector operations, design and monitoring.

Comment – Build on the South African experience to inform revisions to the Bank’s approach to private sector operations, design and monitoring. Current project status reports and BTORs lack information about progress towards development results, which also stems from the initial design and appraisal procedures. The approach for lines of credit in particular needs to be revisited to ensure that the necessary reporting data from clients is known upfront and that measurable indicators are used. In the South African context, the Bank has an opportunity to collaborate with existing clients to develop a more robust approach that would benefit both the intermediary and the Bank by enhancing the ability to understand, track, and report results. A new approach can also draw from experience from the Bank’s public sector supervision revisions and from the IFC and other partners.

Agreed. The Bank will make the required revisions to the guidelines and strategies that guide the Bank’s private sector operations based on experiences from South Africa. Some of the earlier approved Lines of Credit (LOCs) would not meet all the current requirements. Currently, all new LOC Agreements require reporting on selected development indicators that balance the ability of the financial institution to realistically monitor and access the required information and reduce cumbersomeness in terms of time and resource cost burden on the clients.

Further Actions:

The Bank Group’s Results Measurement Framework for 2016–2025, to be approved in early 2017, is introducing initiatives to improve the quality-at-entry and implementation of non-sovereign operations. It will clarify the logic of intervention of private sector operations. A logical framework will be rolled out to better capture the results of private sector projects. It will be streamlined to be more intuitively used by project teams, looking at a select set of outputs and outcomes. This framework will be used as a monitoring table throughout the life of the project. Private sector project preparation will also be informed with ex-ante data. Project teams will keep the indicators used in the project’s Additionality and Development Outcome Assessment (ADOA) report as part of their own set of indicators to track project progress, and project teams will provide information on the extent to which the ADOA assessment informed the design of the project. This will incentivise teams to use and benefit from the ADOA work. Specifically on LOCs, the Bank will review and differentiate the current approach to monitoring operations using this financial instruments from that of project financing to ensure that it is realistic and is in line with market realities and practices. Potential alternatives to addressing the existing monitoring challenges include the monitoring of some of the development impacts and outcomes through Bank funded case studies, expanded supervision reports and project completion reports. This would enable the Bank to more reliably measure the impact and outcomes of its support without unduly encumbering the implementation process on the part of the client. Going forward, during the Due Diligence Mission, the Project Appraisal Team will clarify this information to the Recipient. Subsequent to the first disbursement, and signing of the project transfer form (PTF), a kick-off meeting between the Bank and the LOC Recipient should take place. During the kick-off meeting, the ADOA template and reporting templates will be shared with the client. There shall also be meeting correspondence between the parties, acknowledging the items that were discussed (including reporting requirements). Follow-on disbursements will not be made unless the recipient is in compliance [SARC, 2017]. Furthermore, once the reporting template is agreed upon, the recipient of the LOC will need to receive continuous capacity building on reporting. This will be done during the supervision mission. In addition, the LOC agreement will include a flexibility whereby the template could be changed to report on emerging issues by the task manager [SARC, 2017].


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

RECOMMENDATION 4: Identify new flexible funding mechanisms better suited for MICs.

Comment – Use the South Africa experience to support the Bank in identifying new flexible funding mechanisms better suited for MICs – allowing the Bank more inroads to develop business in South Africa. Different types of risk participation may be one avenue to explore, and it could be done jointly with a range of departments and be central to the business development strategy.

Partially Agreed. The Bank has employed innovative funding mechanisms in South Africa including syndication, co-financing, Clean Technology Fund, Partial Credit Guarantees (PCGs), hedging support options, and private equity participation and is currently exploring ways of applying PCG to either: (i) a bond issuance and/or (ii) foreign exchange swaps. In addition, the Bank is also in discussions with USAID/Power Africa about the possibility of applying concurrent PCGs to enhance commercial bank liquidity. The Bank is effectively marketing these instruments and FTRY has made presentations to stakeholders in South Africa in several occasions.

Further Actions:

Going forward, the Bank will strengthen the use of existing instruments and will explore the use of Partial Risk Guarantees. Partial Risk Guarantees cover private lenders against the risk of the government, or a government owned agency, failing to perform its obligations vis-à-vis a private project. Unlike the ADF guarantees that benefit from an extensive leverage, the ADB guarantees provide a very small leverage which does not incentivize MICs to access them. Currently, the Bank is undertaking the task to revise the guarantee policy to enhance the Bank Group’s operational policy framework on guarantees. This work will take into account the needs to improve the leverage of ADB guarantees.Nonetheless, it should be noted that the success of new flexible funding mechanisms will depend on the underlying macroeconomic fundamentals in the MICs in question. Moreover, South Africa has demonstrated limited appetite for PBOs. In line with its policy of gradual fiscal consolidation, the government discourages borrowing by public institutions which are dependent on National Treasury for their budgets. Instead, it encourages State Owned Enterprises to borrow against their balance sheet or with sovereign guarantees from central government to fund key public infrastructure projects.SARC has been seeking guidance concerning possibilities of lending to sub national governments in South Africa. This could widen the horizon of financial products offered in the country. Municipal financing options should therefore be explored further. The existing Bank guidelines already allow to lend to sub national governments.


RECOMMENDATION 5: Complete the decentralization process.

Comment – Ensure a complete decentralization process and business process streamlining. The experience in South Africa is symptomatic of a broader issue in the Bank that management is now seeking to address through a new business model. SARC would benefit from in-house disbursement and legal skills and expertise linked to a given strategic focus (e.g. for private sector operations, more business development officers on the ground). If the Bank wants to continue to pursue the knowledge agenda, it will need expertise aligned with specific areas. In terms of delegation of approval authority, the current situation has not allowed the Bank to be the nimble partner it wants to be. The specific experience of the MIC trust funds (MIC TF) makes it clear that delegating this and other small grant facilities would be an obvious and helpful step.

Agreed. In the midterm review of the decentralization process, the Bank acknowledged that decentralization has served the organization and its Regional Member Countries well but that it has to be enhanced in several areas, including portfolio management and delegation of authority. The Bank notes that its capabilities are heavily concentrated at its headquarters, diminishing the ability to make development impact, because it has left regions and countries without the full resources or accountability required to develop business and to optimally manage the existing portfolio. The Bank also notes that many of its processes are overly complex and take much longer than those of its peers, which impacts on the time it takes to process projects and respond to requests from its clients.

Further Actions:

Going forward the Bank will implement further decentralization measures as it rolls out the new Development and Business delivery Model. The following decentralisation actions will be implemented to help improve efficiency of Bank processes and make the Bank a more nimble development partner that it wants to be: The Regional Hub will promote deeper policy and sector dialogue with stakeholders,

develop new business opportunities, improve donor coordination, enhance the efficiency and effectiveness of operations, and increase impact.

More public and private sector staff with the requisite expertise will be deployed to the Regional Hub to drive the business operations and the increase in number of sector managers will effectively drive the operational delivery and speed of execution of the Bank projects.

Legal and disbursement experts with the requisite skills will be deployed to the Regional Hub.

Project development process will be streamlined and combined with enhanced delegated authority to the Regional Hub.

The implementation of the new Development and Business Delivery Model will complete the decentralization process and streamlining of business processes.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

RECOMMENDATION 6: Share the lessons from South Africa’s experience in the energy sector.

Comment – South Africa’s experience in investing and diversifying its energy sector and the Bank’s engagement is rich with lessons relevant to other RMCs. Given its continental mandate and its prioritization of energy, the Bank is well placed to facilitate a learning exchange that can cover (i) different types of energy generation, (ii) public and private (independent power producer) roles; (iii) tariff setting and the role of a strong regulator; (iv) contract and project management for both new building and refurbishment, and (v) community relations and management in major infrastructure development.

Agreed. South Africa is the largest producer of electricity in the African continent and its development story in the energy sector needs to be shared across the continent. In particular, South Africa’s Renewable Energy Independent Power Producers Procurement Program (REIPPPP) has established the most successful PPP model in the energy sector that could be possibly replicated in the rest of the continent. The key success factors include good governance, robust legal, regulatory and policy framework, as well as very strong and transparent procurement process.

Further Actions:

South Africa’s successes in the energy sector will be shared with other RMCs as follows: Lessons learned from South Africa in Project completion reports are already being

shared through a number of ongoing projects in the region, and lessons from South Africa are used to inform the design of new projects in the region. This will continue going forward.

Lessons learned in the areas of the themes of the 12 flagship programs under the Bank’s New Deal for Energy in Africa such as types of energy generation, roles of public and private independent power producers, and tariff setting and the role of strong regulator will be shared through these flagships programs in the form of case studies, presentations and any other appropriate modes of communication.

Organize exchange visits of project management teams from across Africa to Eskom, South Africa’s Department of Energy, the Renewable Energy Independent Power Producers Procurement Program Unit, the Energy Regulator and other energy sector institutions to share experiences in energy sector policy, generation modalities, tariff policy formulation, contract and project management, regulatory framework and community relations and management.

There are several good practices in different countries in the area of renewable energy. The Bank has to play a key role in inventorying, studying and sharing them with the RMCs.

23Introduction

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Introduction

This report provides a summary of the findings of the first independent evaluation of the AfDB’s Country Strategy and Program (CSP) in South Africa, which began in 2015 and concluded in 2016. It is based on the intermediary working papers produced for the evaluation, including individual PRAs, portfolio assess-ment, and a technical report. This summary report presents the key themes emerging from the evalua-tion questions and findings for Board consideration. It is timely for several reasons: (i) it provides independ-ent evidence of achievements and challenges to date in advance of the development of the next CSP; (ii) the South African experience provides lessons for future engagement with other advanced MICs; (iii) it provides information relevant to the Bank’s implementation of its new business model, and (iv) the evidence was used as a building block for the Bank’s Comprehen-sive Evaluation of Development Results (CEDR).

Evaluation purpose and scope

The evaluation serves to enhance accountability and learning. More specifically, it provided evidence about the results of the Bank’s engagement in South Africa and how it has managed its engagement. It also identified the internal and external factors that affect performance as well as lessons and recommenda-tions to inform the design and implementation of future strategies and operations.

The evaluation period (2004/15) covered three CSPs, the establishment of the SARC, and an operations portfolio of over UA 3 billion 1 (see Annex A, Portfolio) that is one of the Bank’s largest and atypical. The eval-uation addressed two groups of questions: How does the Bank manage itself so as to maximize its perfor-mance? What has the Bank achieved? Box 1 summa-rizes the topics covered by the evaluation questions (see Annex B, Evaluation Questions).

Methodology

The evaluation was theory-based and multi-layered. It used mixed data collection methods. For individ-ual projects and at country level, it used a theory of change (ToC) (see Annex C, Methodology). To answer its questions, the evaluation used building blocks of evidence based on (i) in-depth PRAs of a sample of projects that covered 76% 2 of the portfolio (by volume), each of which was based on a robust theo-ry-based methodology, a standardized rating scale and multiple lines of evidence that were validated through further consultations and, (ii) a review of the portfolio as a whole and of the strategic aspects cutting across it. The evaluation combined several data collection techniques: interviews, focus groups, site visits, document review, and additional consul-tations at inception and validation stages. Over 117 persons, most of whom were in South Africa, were

Managing for performance

❙ Design

❙ Partnerships and leverage

❙ Knowledge and policy advice

❙ Managing for results

❙ Efficiency

Achievement of Results

❙ Relevance

❙ Effectiveness

❙ Sustainability

❙ Cross cutting issues

Overarching

❙ Added Value

❙ Lessons

Box 1: Coverage of Evaluation Questions


consulted. The team also used data from the CEDR to allow comparison with other countries.

Performance was summarized against four main eval-uation criteria: i) relevance; ii) effectiveness; iii) effi-ciency, and iv) sustainability. Each criterion received an overall rating on a six-point scale ranging from highly satisfactory to highly unsatisfactory (see Annex C, Table C2) on the basis of multiple sources of infor-mation, including in-depth PRAs (each of which was also rated), evidence from across the portfolio (using qualitative and quantitative data), strategies, and activities. Other evaluation questions were not explic-itly rated but the findings are clearly stated.

Limitations

The evaluation design took the limitations into consid-eration. Where there was not enough information available to make a robust assessment, this is stated unambiguously. Key limitations included:

❙ Poor documentation and document storage, especially on monitoring and supervision. The situation was acute for the earlier part of the eval-uation period where lack of documentation was compounded by a lack of key informants with insti-tutional memory.

❙ The AfDB is a relatively small player in the South African context and portfolio relatively young. Its contribution to development results at the national level was necessarily limited. In understanding effectiveness, the evaluation therefore focused on intermediate outcomes rather than on final development outcomes in part because the port-folio was not sufficiently mature, with few fully completed operations.

❙ A major part of the South Africa portfolio (LoCs) is on-lent outside South Africa to countries across the continent but site visits could not be conducted outside South Africa.

25The South African context

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

The South African context

This section provides a brief overview of the over-all context in South Africa and of aspects relating to sectors most relevant to the Bank. For further socio-economic data, see Annex D.

Economic and policy context

South Africa is an upper MIC, and the third largest economy in Africa with a GDP of USD 312.8 billion. It has relatively good levels of infrastructure despite challenges in some areas, especially in energy supply and regional disparity. The private sector is well developed when it comes to large businesses by contrast to SMEs. In the country’s well-devel-oped financial sector, four large banks lead the field. The exposure of South African banks to the conti-nent has expanded rapidly in recent years. There are 130 state-owned enterprises (SOEs) 3 that play central roles in all sectors. For most of these, the Government’s Department of Public Enterprises is the shareholder.

Growth exceeded 4% before the 2008–2009 global crisis but became negative and has not recovered since. In 2014, GDP growth was below 2%, and 1.3% 4 in 2015. The fiscal deficit has been stable at around 4% of GDP. The debt-to-GDP ratio has increased slightly, excluding contingent liabilities. The National Treasury does not want to see any further rise.5 Fitch downgraded the sovereign rating to BBB in 2015, reflecting concerns about economic performance and outlook.

The Government of South Africa’s (GoSA) does not borrow directly from international finance institu-tions (IFIs) or from bilateral donors. Lending goes to SOEs, local IFIs and the private sector. External development assistance plays a marginal role in the GoSA budget, accounting for well under 1% of GNI.6

While considered an upper MIC, South Africa showed worsening levels of inequality by some measures over the evaluation period.7 It remains one of the most unequal countries on earth. Notwithstanding the GoSA launch of initiatives to tackle inequal-ity, poverty, and unemployment 8 and a reduction in the proportion of people living below the poverty line, unemployment levels have stagnated. Youth, women, black people, and specific regions of the country have been hit the hardest.

During the evaluation period, the focus and direc-tion of GoSA development strategies were somewhat consistent, in particular the focus has been on job creation and reducing inequality through economic growth coupled with targeted programs, including strong emphasis on support to previously disadvan-taged groups through a program known as Broad Based Black Economic Empowerment (BBBEE).

The 2006 Accelerated and Shared Growth Initiative did not meet its key goals of halving unemployment by 2014 and of raising economic growth rates. The Medium-Term Strategic Framework (MTSF) 2009–2014 placed emphasis on (i) rural development, (ii) the green agenda, and (iii) a government-driven devel-opment process and the establishment of a National Planning Commission that prepared the National Development Plan 2030.The 2011 New Growth Path was built around five “jobs drivers” that are expected to generate jobs in the formal and informal sectors. The 2012 National Development Plan (NDP) 2030 identi-fies two major goals: the elimination of income poverty and reduction in inequality. It also includes targets for universal access to energy and clean water and sees South Africa playing a lead in continental integration.

BBBEE policies, particularly enterprise ownership and procurement, cut across all government practices. Having made exceptions to allow the AfDB, the WB, the


EIB, and others to apply their own procurement rules, GoSA recently made it clear that no further exceptions will be made,9 despite the fact that some development partners will not able to finance under GoSA procure-ment rules (AfDB will be able to continue to work under South African procurement rules under its new policy). This gives an indication of the strength of ownership of the country’s own development agenda.

A snapshot of development challenges in key sectors

Energy: South Africa developed as a regional power-house thanks to cheap coal. However, demand outstripped supply and the reserve margin fell from 27.1% in 1999 to 5.6% in 2007. The load shedding that resulted 10 peaked in 2015 with frequent black out hours of between 2 to 4 hours per day. South Afri-ca’s electricity shortages cost close to one percentage point in annual economic growth and are considered the “biggest challenge” to the economy.11 A capacity expansion program led by ESKOM (the responsible SOE) was initiated in 2007. In 2011, the government adopted an Integrated Resource Plan to double over-all capacity and reduce reliance on coal (from 81% to 46%), with a target of 21% renewables by 2030. ESKOM is developing four power plants of which two are new coal-fired.12 Meanwhile, the market for renewable energy generation is opening up with the Independent Power Producers Procurement Program (IPPPP) established in 2011. To date, ninety-two IPPPP projects are set to generate 6326 megawatts.

Transport: South Africa is trade and transport-in-tensive. It relies on the imports and exports flowing through its eight major (capacity-constrained) ports, and its transport network is an important part of some major corridors that are unlocking the broader region.13 The roads in the economic hubs are of good quality relative to the poor quality roads that comprise around 20% of the paved secondary road network and 50% of gravel roads.14 Freight transport relies heavily (88% of all tonnage) on roads 15 on which overloaded heavy vehicles cause about 60% of all damage.16 South Africa’s rail network is dominated by

freight, but includes suburban and intercity passenger rail services. In 2005, Transnet embarked on a stra-tegic turnaround program and the bulk of funding is being invested in freight transport.

Water: Growing demands on South Africa’s finite water resources create competition among agricul-tural, industrial, power generation, mining, commer-cial, and domestic needs.17 Some major projects are planned to harness greater supply to meet growing demand. Since 2001, the government has expanded water services to 96% of the population, extending free water to poor families. The challenges now – as acknowledged in a 2003 strategic framework for small-scale and major water infrastructure plus a 2005 national sanitation strategy – are to reach the remaining minority and to manage maintenance, to upgrade and upscale, but municipalities have limited capacity and expertise.

Mining: Mining contributed 8% of GDP in 2015 and employed 462,000 people as at end 2015. From 2001–2008, however, the South African mining industry shrank by 1% per year and has since come under further pressure because of declining global commodity prices.18 Labor relations and high costs associated with deep mining and infrastructure bottlenecks are identified as contributing to stagna-tion.19 The 2004 Broad Based Socio-Economic Char-ter for the South African Mining Industry was updated in 2010, to facilitate fuller participation in line with BBEE policies. In 2007, the Cabinet approved a Beneficiation Strategy. Another bill that has been in the pipeline for several years to transfer mining and prospecting rights and to advance beneficiation has created uncertainty about the policy and regulatory environment of the mining industry.

Finance: The financial sector is large and the most sophisticated in Africa. The four big South African banks hold over 80% of the market.20 The Johannes-burg Stock Exchange is in the global top 20.21 Total financial sector assets of about 298% of GDP exceed those of most other emerging markets.22 Commercial banks with assets of 112% of GDP make up the single largest segment of the financial system. The retail

27The South African context

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

segment for the commercial banks accounts for a penetration rate of 75% compared to a sub-Saharan Africa rate of 25%. Nearly 95% of banking assets are domestic although exposure to other African coun-tries is expanding. The local capital markets include an active market in government securities and bonds; and debt instruments issued by parastatals, corpora-tions, and commercial banks, among others. Insur-ance companies are also major players in the South African financial sector. Yet, access to finance remains limited for certain segments of society. According to a May 2013 World Bank report, 12 million adults in South Africa had no bank accounts and millions more had only limited access to formal financial services. Lending to SMEs remains low.

Regional integration: Many sectors are implicitly linked to regional integration priorities: South African financial institutions lend across the continent; energy security is linked to that of neighboring countries; transport infrastructure is linked to regional trade, and some major water development initiatives also have regional dimensions. Infrastructure and systems at most border posts require investment to accommo-date the growth of freight and passenger movement. The GoSA has expressed its commitment to regional integration and to working with regional economic communities such as the Southern African Develop-ment Community (SADC), but it has also tightened up on border controls in recent years, notably in relation to economic migration.

29Is the Bank managing itself to maximize performance in South Africa?

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Is the Bank managing itself to maximize performance in South Africa?

The section examines how well the Bank is manag-ing itself to achieve results. The evaluation questions (Annex B) are based on the theory underlying expecta-tions of results (Annex C). Findings are based on trian-gulation of evidence collected at strategic and portfolio levels and in-depth assessments of 13 projects.

Country Strategy overview and design

Finding: Bank strategies were consistent and somewhat focused during the period and their quality has improved. However, shortcomings were identified.

The evaluation period covered three strategies (the current CSP covers up to 2017) that are generally consistent, with some specific changes in strate-gic thinking 23 in line with broader changes in Bank CSPs. The first CSP was based on a divide between public and private sector operations; the second CSP was built around private sector competitiveness, regional integration, and knowledge management and capacity development. The third CSP was built on two pillars (infrastructure, and support for regional integration) plus crosscutting issues (capacity, knowl-edge, gender and governance), which are aligned with the 2011–2015 regional integration strategy papers (RISPs) and with the infrastructure and capacity build-ing pillars. The silo approach to public and private oper-ations evident in the first CSP disappeared in the more recent strategies, although private equity received only cursory mention. Despite structural changes to the strategies, private sector development, infrastruc-ture, and regional integration have remained core to the aspirations throughout the period. The emphasis on knowledge work and capacity development grew as

of 2008. Some strategic changes involved relabeling or re-categorizing priorities rather than major shifts.24

The priorities are broadly aligned with Bank and government priorities, with few exceptions. The priorities identified in the strategies and reflected in the planned pipeline of projects are broadly aligned with the Bank’s strategic priorities that were agreed at corporate level when each strategy was developed, and with those of the GoSA 25 but sought to focus on fewer priorities and sectors in line with Bank guidance. However, all strategies during the period focused more on setting out the general country context and less on the government’s strategy or on the Bank’s program.

Given the context of inequality, the government’s emphasis on BBBEE and the Bank’s emphasis on inclusive growth and, the explicit attempt to address inequality or to ensure inclusive growth is conspicuous by its absence. This was noted in the quality-at-entry (QaE) assessment of the most recent CSP; it was evident from a project review, and was highlighted by the government in recent consul-tations. Similarly, the CSP and the portfolio of projects are at the early stages of integrating gender quality into analysis and planning. This shortcoming was acknowledged by Bank staff.

The quality of the latest CSP was judged mixed overall but satisfactory, and in line with ratings from an IDEV evaluation of QaE of CSPs. In consultations, government stakeholders expressed their apprecia-tion of the Bank’s increased efforts to involve them in consultation about its strategic priorities, includ-ing the recent CSP mid-term review. National Treas-ury (NT) officials also confirmed that the broad pillars of the existing strategy remained relevant and that


past changes (shifting knowledge and capacity from a pillar to a crosscutting issue) were also appropri-ate given the lack of progress (see Country Strategy overview and design). The weaknesses that were noted included the lack of analysis of the sustaina-bility of past interventions, and lack of analytical work to back the strategy, inclusive growth, governance, gender, risk assessment, consultation, and local part-ner dialogue.26

Portfolio focus

Finding: The Bank’s current portfolio in South Africa is more focused than envisaged in the CSPs, concentrated in two sectors: infrastructure (energy) and finance. This is practical but also reveals the mismatch between Bank ambitions and achievements. It reflects the limited areas where the Bank has demonstrated a comparative advantage in the South African context and its internal capacity constraints.

At UA 3.721 billion,27 the Bank’s portfolio in South Africa was the second largest country portfo-lio over the period. A single operation (Medupi)

accounted for UA 1.3 billion, and a further UA 614.8 million goes to the same client through the private sector window.28 Figure 3 illustrates that by volume, the 26 operations in the portfolio focused on two types of projects: infrastructure (mostly energy, public and private) and finance (private). In terms of public or private operations, the volume is roughly equally split, with 53% going to the private sector and 47% going to the public sector (of which 84% relates to Medupi). (Figure 3 does not include the additional UA 30.4 million that was on invested by private equity funds supported by the Bank in a broad range of sectors 29 and classed as multinational operations.)

Gaps existed between the Bank’s strategic ambi-tions defined in its CSPs and the portfolio it has been able to develop. Since 2008, the portfolio has prioritized the Bank’s contribution through knowledge work and capacity development along with some work in higher education and training. In practice, the portfolio is dominated by power generation (mainly with the monopoly power-generating company) and finance (mainly LoCs to three commercial and three development finance institutions). There were two projects in the transport sector (with the same client) and one in the mining sector in the period.30 The Bank

Figure 3: Portfolio concentration in 2004–2015 approvals

Power

Finance

Other Majority in “other” category goes to other infrastructure – notably one major transport (rail) project, the rest comprises investments in mining, water, multi-sector and communications.

Sector comprised of 10 lines of credit to seven financial institutions (Including three commercial Banks and four DFIs).

Sector includes three public & three private projects.

0 500,000

489,387

842,545

2,318,410

1,000,000 1,500,000 2,000,000 2,500,000

Source: Desk review and SAP portfolio data.

Clustering of portfolio by sector in UA 1000s (netloans)


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

was not able to secure projects in several areas. In i) water, where major investments through SOEs were not successful partly because Bank support would not be competitive for the SOE without a sovereign guarantee (though small grants have been made through AWF); (ii) in air travel, where plans to support the South African Airways were shelved; (iii) in trans-port other than rail freight; (iv) in higher education and training;31 (v) in knowledge and analytical work where the deliveries have not been at the scale envisaged, and (vi) in capacity development, which was a high priority in the 2008 strategy, was retained as a cross-cutting theme in the 2013 strategy but does not figure prominently in the portfolio (accounting for 0.1% over the period) or the projects.32

The private equity portfolio was the exception to this pattern of concentration of sectors in which Bank-supported funds were investing. Roughly one fifth of the UA 493 million of private equity funds being on invested in South Africa, of which around UA 30.4 million came from the Bank, went to telecoms, agri-business and to services.33 However, none of these were strategic priorities in the CSPs.

The difference between the strategy and the portfolio indicated a disjuncture between strate-gic ambition and a full analysis of feasibility and of comparative advantage. Despite the fact that desk reviews emphasized the fact that efforts were made to identify the Bank’s comparative advantage in the most recent strategy, the portfolio developed unevenly because of an insufficient understanding of the opportunities in the South African context and because of the tools and resources available to the Bank. In particular:

❙ The GoSA does not borrow from the Bank. To date, the Bank has not lent to local government enti-ties. The government is highly selective in issuing sovereign guarantees for SOEs 34 without which the Bank’s private sector lending in local currency is not considered competitively priced for the South African market by 65% of the clients who were interviewed (despite the generous tenor). In practice, the Bank was able to lend to two SOEs: i)

ESKOM (energy) combining loans with and with-out sovereign guarantees and in multiple curren-cies, and ii) Transnet (rail), which had a strong balance sheet so needed no sovereign guarantee and benefitted from the long tenor, diversification of its debt and potential syndication. The Bank has been most competitive in its private sector pricing in providing foreign currency loans, and has thus developed a series of LoCs to South African intermediaries for on-lending outside of South Africa.

❙ The program has suffered from a lack of capac-ity and focus on business development. A busi-ness development officer for the finance sector was sent to SARC only in 2012; it was not until October 2015 that a business development officer for private sector investment operations was sent (both cover the entire southern region). Before their arrival, some prospective clients were unclear even about their entry point for discussing Bank funding.

❙ The small grant tools have been problematic. The MIC TF is the principal small grant facility availa-ble in South Africa. The others, such as the FAPA35 grant, are available for the private sector. The MIC TF scheme has proven slow and cumbersome even though the amount of funds is small (see Effi-ciency). Some planned studies and grants were not realized at least in part because it was burden-some to get the funding into place.

❙ The emphasis on knowledge and capacity in the CSPs since 2008 failed to consider whether or not the Bank had the expertise, resources, capacity, or instruments to add value in this area. Where the strategy documents acknowledged that resources would be required, they were not made available (see Knowledge and policy advice). For capac-ity development, the GoSA confirmed that it was appropriate to make this a crosscutting issue rather than a pillar in the 2013 strategy given that (i) the Bank had made limited inroads and (ii) that such work would need to be very well targeted. Across the portfolio, one small project focused on capacity building (in statistics) 36 and two were


planned to include capacity building compo-nents with financial intermediaries (though these components were not delivered).37

Portfolio coherence

Finding: The coherence of the portfolio as a whole was strengthened by its focus on two main sectors but weakened by the Bank’s tendency to work in silos. There was evidence that the team is now seeking to address this legacy.

Coherence between public and private sector thinking has improved in the CSPs, however, despite some advances in the last years, the silos between public and private sector opera-tions have persisted:

❙ Public and private teams still work separately although there have been recent efforts to work together for investments in energy because the Bank has public and private loans with the same client.38

❙ Public and private clients raised the issue of rela-tionship management, including internal commu-nication.39 This refers to the role of different teams interacting with the same client, including public versus private; origination versus portfolio moni-toring, and ad hoc interactions with others includ-ing senior management. In addition, the frequent changes in task managers in some cases created a lack of consistency in Bank requirements over time.

❙ Overarching portfolio management and monitor-ing in South Africa was focused on public sector operations – not covering private sector projects, lines of credit or equity. Independently of this, a single private sector portfolio officer working on investment projects in the region and the officers covering the finance sector are based in Nigeria, Malawi, and Zambia.

Proactive attempts to find synergies between other components of the portfolio, notably the infrastruc-ture and finance sides, and the national and regional

have not been found. The portfolio implicitly covered support for regional integration with continent-wide financing and infrastructure that has regional impli-cations. However, with the exception of the One Stop Border Posts study, the projects’ overall focus and objectives did not explicitly target these dimensions.

Innovation

Finding: There was evidence of some innovation in strategies and individual projects. However, innovation in the Bank’s portfolio and approach is not yet systematic. Some opportunities to innovate to enhance development results have been missed.

Innovative aspects were found in 11 of the 26 oper-ations in the portfolio: in technologies, methods of funding and leverage, and ways of targeting specific groups.40 Particularly interesting, including for Bank operations in other countries are (i) the Bank’s involve-ment in the Renewable Energy Independent Power Producer Program (REIPP) program for private renew-able energy generation, albeit a small investment 41 (ii) support to new technology (concentrated solar power) albeit severely delayed because of technology and procurement reasons; (iii) attempts at syndica-tion to leverage more funds from the private sector, although the first of these did not perform as expected and the second (provision for a B-loan of USD 950 million in addition to Eskom corporate loan) has yet to be finalized; (iv) targeting disadvantaged groups and affordable housing with small loans (Commercial Bank B sub-debt), and (v) combining local and foreign currency loans.

The overall approach to LoCs was strategic but few operations were found to be internally inno-vative. The Bank found a niche by providing foreign currency for LoC to the South African financial insti-tutions (FI) which is on-lent outside of South Africa,42 allowing it to deploy foreign exchange to companies in low income countries (LIC) through intermediar-ies and increasing Bank finance to high risk environ-ments without affecting its risk profile. However, the


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

opportunity to be innovative in these operations was not fully exploited. Most LoCs in the operations did not find ways to focus on target groups such as SMEs or niche sectors (see Effectiveness) in practice, nor have the operations added value to the financial interme-diary by supporting better measurement of develop-ment results or capacity. In 7 of 9 cases, funds were used for major investments and not for targeting small business. In some cases, sub-loans went to projects that the Bank did or could have supported directly.43

Knowledge and policy advice

Finding: The Bank’s ambitions for contributing knowledge work in South Africa have grown since 2008. But in practice, less knowledge work was delivered than was envisaged and its impact was unclear. Despite some positive examples, the Bank did not generally live up to its aspirations to become a major knowledge generator or broker. Its ambitions were not matched with sufficient resources or by a background analysis to confirm that the Bank has a comparative advantage in this area in the South African context.

On paper, knowledge and analytical work was a feature of the Bank’s strategic approach in South Africa throughout the period, though more

prominently from 2008. Figure 4 shows that knowl-edge and capacity constitute a pillar of the 2008 CSP and that the need for resources to deliver and linkages with the CSP objectives was acknowledged. In 2013, the importance of knowledge sharing (in addition to generation) was also emphasized, including between South Africa and other countries in the region. The latter two strategies were also more specific about planned deliverables than the first.

Delivery of free-standing analytical work did not meet the ambitions set out in the strategies. Nine internally generated sector focus knowledge products were concluded over the period, almost all since 2009. The majority of these were economic papers led by the Bank’s country economist for South Africa.44 Despite the acknowledgement in the 2008 CSP that the Bank would require human and financial resources to successfully position itself as a knowledge Bank in South Africa, only one person worked part time on economic papers and funds for other studies that have been drawn mainly from the MIC TF, which was both slow and cumbersome.45

Review of the portfolio showed that the Bank had not integrated knowledge and analytical work into its operations. Of 26 projects, seven included knowl-edge aspects and all of these were small projects; no knowledge work was integrated into any of the

Figure 4: Evolution of ambitions for knowledge and analytical work in South Africa CSPs

Only high level mention of importance of knowledge and analytical work.

Knowledge and capacity become a pillar.

Specific studies are identified.

Need for resources, including MIC grants noted.

Knowledge becomes a crosscutting issue. Specific studies identified. Sharing knowledge between South Africa and other countries also noted.

2003–2007 2008–2012 2013–2017

Source: Desk review and SAP portfolio data.


major investment projects. Two of the six projects that included knowledge aspects used the MIC small grant fund to commission knowledge-related products that were intended to influence policy.46 Others were attempted or are still in process.47 The other projects that included some aspect of knowledge work – knowledge generation or simple sharing – related to the African Water Facility (AWF) and to the statistical capacity project but were not big-ticket operations.

Stakeholders view the Bank as purely a financier. For knowledge work in areas where Bank investments were focused, clients, staff, and partners consistently responded in interviews that the Bank was seen as a financier and not as a knowledge broker or producer.48

If the Bank wants to add value in knowledge in the South African context, it will need to clearly focus on: (i) where it is directing its investments, and (ii) the regional dimension and the sharing of knowledge between South Africa and the region, for example through SADC. There was internal acknowledgement that even with a tighter focus, delivery and impact would continue to be undermined unless there was sufficient resourcing and a more strategic approach.

Much of the knowledge work produced was rele-vant for the priorities of the Bank and of South Africa. However, its impact was unclear. There was little evidence of any proactive Bank effort to dissem-inate beyond initial emails to key clients and website postings, with the exception of African Economic Outlook. The attempt to establish a website to show-case SARC knowledge work and activities never went live. Plans to have country/region specific sites from SARC were not put into practice because Bank head-quarters preferred to keep dissemination centralized. Yet the World Bank produced hundreds of knowledge products in the same period that were all accessi-ble through the South Africa part of their website targeting local readers. The two studies funded by the MIC Fund have had only limited impact to date. The MIC grant to Broadband Infraco (BBI) was used to develop a strategy for an SOE that has since been merged with another, making it irrelevant. The One Stop Border Post study that is highly relevant to Bank priorities was completed and has been with GoSA

for over a year, but it was unclear whether or not it would have an impact on the government approach. However, some stakeholders remarked that SARC has hosted a number of useful workshops and semi-nars, which reflects well on nascent attempts to aid knowledge sharing.49

There was a mismatch between the ambitions for knowledge and analytical work, including the link to Bank operations and government policy in the strategy, and Bank achievements in practice, as the SARC team acknowledged. The ambitions laid out in the 2008 and 2013 strategies were based on three assumptions: (i) human resources would be made available to produce in-house knowledge products (whereas in fact only one staff member is available and the majority of that person’s time is taken up with mandatory corporate knowledge products); (ii) financial resources would be accessible to commis-sion studies (whereas the MIC TF has been slow and cumbersome, lowering the interest of both staff and GoSA), and (iii) the Bank correctly identified knowl-edge needs that it was best placed to fill (in fact such analysis was not conducted).

Partnerships

Finding: The picture was mixed at the strategic or country level. The Bank focused on the govern-ment as opposed to developing other relation-ships and achieved positive partnerships in some projects. However, the lack of engagement in other projects was noteworthy.

The Bank successfully developed a relationship with the National Treasury (NT), the heart of the government. The relationship has improved over time, moving from little interaction during the first part of the evaluation period when the Bank was not present on the ground, to frequent contact today.50 Working relationships were built from 2009 onwards and a more strategic relationship developed once SARC was created. Relationships with line ministries were less developed, and generally functioned via the NT for coordination and business development.51


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Seeing itself as protecting a privileged position of trust with the GoSA, the Bank has been attentive to align-ing itself with GoSA priorities rather than lobbying for specific policy changes, given the country context.54

Relationships with other potential partners – private sector and other international partners

– were less well developed. During SARC’s first years, strategic and high-level efforts were made to establish stronger partnerships with the private sector and other DPs. In addition, private sector origination teams based at headquarters led some of the Bank’s first inroads into the South African market, but clients reported a drop-off in high-level strategic engage-ment in recent years. For other DPs, GoSA preferred to manage bilateral relationships and there are few donor coordination groups. The Bank was engaged

in one group on public financial management. With the exception of consultation for the CSP and the mid-term review where DPs were consulted bilater-ally, engagement was limited and related to specific projects. DPs expressed interest in more proactive engagement from the Bank.55

At project level, there were some partnerships and coordination in the energy sector. While slow to form their partnership, the AfDB and World Bank have worked closely in the last two years in moni-toring the biggest project in the portfolio – Medupi.56 Each organization has benefitted from the different areas of expertise: the World Bank added a techni-cal (boiler) expert to its team in recent years and the AFDB added value on the financial and procurement sides. After this experience of coordination, staff

The Bank has sought to leverage private sector funds using its B-loan facility in two cases. The first was with Transnet, where results were limited not because the commercial banks expressed no interest but because Transnet found cheap finance elsewhere.52 The second, with ESKOM, is ongoing 53 but on the back of the Bank’s USD 375 million loan, ESKOM has the interest of 10 commercial banks that were attracted by the ability to piggy-back on the AfDB’s preferred creditor status to provide an additional USD 950 million. It was too early to say whether this attempt would be entirely successful.

Discussion with clients on both the finance and infrastructure sides of the portfolio indicated an appreciation of a Bank ready to be the first to fund and by doing so, inspiring confidence in others. Comments on AfDB’s early involvement were heard from the FIs, the Development Bank of Southern Africa (DBSA) and the South Africa Industrial Development Corporation with which AfDB has been engaged for more than a decade. In the case of the Commercial Bank A, the NT has encouraged DP support in recent years, but it was only after the AfDB LoC that the World Bank began appraising its own support. ESKOM and Transnet both noted AfDB involvement early on in their experiences with IFIs. This not only inspired the confidence of other investors and helped them to diversify their balance sheets, but also provided the experience of working with an IFI, including more stringent requirements. As a result, it was a learning experience for them.

In the 9 of 11 cases that were examined in depth, the Bank’s LoCs were used mainly for on-lending outside of South Africa. The Bank saw this as a channel to leverage South African investment into other RMCs by providing good pricing and tenure. In the case of large commercial banks, the funds were too small to have a causal effect; in the case of the IDC and the DBSA, Bank finance may have helped the growth of their international investment arms. However, it should be noted that these two institutions have little experience outside South Africa and the portfolios of projects supported included notable non-performing loans (see Effectiveness).

The AWF small grant projects in South Africa are driven by the concept of demonstrating successful small-scale projects and encouraging GoSA to scale-up where the projects are successful. However, it was not yet possible to determine whether or not this worked.

The Bank also brought in Clean Technology Fund resources for the two public sector renewable energy projects to complement ADB funding.

Box 2: Leveraging finance in the South Africa portfolio

Source: Interviews with AfDB staff, clients and document review.


from both organizations as well as the client and the government all expressed regret that such project coordination had not been achieved earlier. For two other energy projects, the AfDB aligned with World Bank procurement procedures. AFD and the Climate Technology Fund (via AfDB) were also engaged 57 but this coordination contributed to delays. In the South African context, future alignment is likely to be with GoSA procedures rather than those of another DP. The AfDB’s new procurement policy positions enables it to do this whereas some other DPs may find their hands tied in major investment projects in future.58

In contrast, in the finance sector, there was little evidence of coordination or even any engagement with other DPs, even when they lent to the same clients. Both the IFC and the European Investment Bank (EIB) expressed interest in greater engagement from AfDB teams to share and coordinate information.59 The Bank was supporting the same intermediaries in many cases but information was not shared. Better engage-ment with another DP working with the Commercial Bank A could have highlighted its ongoing capacity support, which AfDB missed at appraisal.60 The excep-tions were the two fast track operations with commercial banks after the financial crisis led by IFC, but coordina-tion between AfDB and IFC took place at headquarters rather than at the country level; IFC took a leadership role in appraisal and monitoring and AfDB aligned.

Leverage

Finding: Examples of explicit and implicit attempts to leverage finance existed in the port-folio. The AfDB was acknowledged as being the first at the table in several cases, which has strengthened investor confidence.

Explicit attempts to leverage via syndication included one partially successful experience and a new attempt that was in process. In other cases, the Bank supported financial leveraging from others simply by being the first IFI at the table. While this did not confirm a systematic approach to leveraging, it did show attempts and the potential for future leveraging (Box 2 summarizes the main cases).

Efficiency

Finding: Efficiency received a rating of Moderately Unsatisfactory overall because of time rather than cost inefficiency. The evidence did not indicate an improvement in time efficiency over time despite decentralization. Stakeholders still perceived the Bank to be bureaucratic and slow.

For efficiency overall, two thirds of all projects reviewed achieved a rating of moderately satisfactory

Figure 5: Efficiency ratings from PRAs

Timeliness

HU U MU MS S HS

Cost effectiveness/ Bank profitability

0% 20% 40% 60% 80% 100%

Note: HU: Highly unsatisfactory; U: Unsatisfactory; MU: Moderately unsatisfactory; MS: Moderately satisfactory; S: Satisfactory; HS: Highly satisfactory.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

or higher whereas for time efficiency, less than half the projects received a rating of at least moderately satisfactory (see Figure 5).

Overall 62% of South African projects achieved a rating of moderately satisfactory or higher compared to 66% for projects reviewed across 14 countries.61 The overall figures do not reflect the details, however:

❙ The range in project ratings was large, from unsat-isfactory to satisfactory. Eight of 13 projects received a rating of moderately satisfactory or higher of which only three were satisfactory and none were highly satisfactory. There was no discernable difference by sector or by public/private operations. With only 13 data points, it was not possible to conclude greater efficiency for one-sub-group over another.

❙ For cost efficiency, ratings ranged between moderately unsatisfactory and satisfactory, with 8 of 13 projects achieving a satisfactory rating or higher.62 Medupi was the only case that received a rating of moderately unsatisfactory rating; while it saw massive cost escalations it was still expected to be a cost effective solution (i.e. economically viable but at a reduced economic internal rate of return compared to alternatives).63 Both of the other infrastructure projects were satisfactory on cost effectiveness. Seven of the nine finance projects were satisfactory or highly satisfactory for cost efficiency.

❙ Time efficiency ratings spanned the scale. Less than half of the 13 projects were rated moderately satisfactory or higher, only two were rated satis-factory or highly satisfactory. Given that 7 of the 13 projects were rated between highly unsatisfac-tory and moderately unsatisfactory, time efficiency was the most negatively rated aspect across the projects. Two of the three energy sector projects (one public, one private) received poor ratings for timeliness, with delays of up to five years so far. The Sere Wind Project began operations in 2015 and was rated moderately satisfactory because the Bank chose to decouple it from the severely

delayed Upington solar project. The majority of the finance projects received mid-range ratings. Delays occurred at every step of the process from approval to disbursement to implementation.

❙ There was no pattern suggesting that recently approved projects achieved greater cost or time efficiency than projects approved at the beginning of the evaluation period.

Other data regarding efficiency support the low ratings on timeliness noted in the in-depth project reviews, including the data available on small projects across the portfolio and client feedback. Stakeholders and clients in particular, as well as GoSA and AfDB staff, raised the issue of timeliness in relation to Bank processes. Disbursement delays were also raised as an issue for the overall SARC portfolio in the MTR of decentralization. Specific experiences in South Africa included the following:

❙ MIC grants were relatively small yet could take as long to approve as large investment projects. This significantly reduced their utility in the South Afri-can context (see Annex E, Table E2 showing time elapsed for portfolio operations).

❙ The sole attempt to make use of the FAPA grant on the private sector side was abandoned. The client confirmed in an interview that they found the paral-lel process to be too bureaucratic and cumbersome given the minor additional support on offer.

❙ Timeliness indicators varied across the portfolio (Annex E, Timeliness). Table 1 sets out the short-est and longest cases for key steps in the process and shows that there is no clear pattern for public versus private or for smaller versus larger projects.

❙ A comparison of the two solar power projects in the portfolio was instructive. The private sector project was approved in June 2014 and disbursed in March 2015 while the public sector project was approved in May 2011 but had yet to disburse at the time of writing (due to issues around procure-ment and the choice of technology).


❙ The Bank had repeat lending to four institu-tions in its private sector portfolio and to two in the public sector. The clients lamented the fact that the Bank repeated the process used for initial lending for repeat lending. Private sector clients also found that the process required for changes to an agreed loan, which often required returning to the Board, were unnecessary and

bureaucratic compared to the processes of other IFIs with which they work.64

Stakeholders reported that they expected decen-tralization to make for a nimble Bank, but that this has not happened. For example, for any legal or disbursement-related issues and for a wide range of approval authorities, SARC reverted to headquarters.

Table 1: Fastest and slowest cases in the portfolio

Source: Analysis of project milestone data constructed from SAP data and project document review.

* Some projects are yet to disburse.

Stage Projects

From project concept note to Board approval Fastest: Xina Solar Power (1.3 months) and Commercial Bank A Global Trade Liquidity Program (1.8 months) an IFC led transaction.Slowest: Sere Wind Farm and CSP Upington (initially appraised as a single renewables project) at 22 months. DBSA V (16.6 months).

From Board approval to loan agreement (legal signature)

Fastest: Medupi power plant (0.5 months) and IDC LOC 1 (0.7 months).Slowest: Kalagadi Manganese (31.6 months) and One Stop Border Post study (23.2 months).

From loan agreement to first disbursement Fastest: Xina Solar Power (0.9 months) and Integrated water harvesting (1.2 months) AWF.Slowest: Sere Wind Farm (21 months) and DBSA IV (13.7 months)*.

The analysis of the 13 projects included in the in-depth assessments supplemented by a desk review of 6 other projects, shows that some factors are more common than others in explaining poor time efficiency. The factors cited most frequently in desk reviews, interviews, focus groups, and workshop discussions included the following:

❙ Project start-up delays including clients meeting the Bank’s pre-commitment and disbursement conditions: e.g. Eskom operations, Sere Wind, CSP Upington, IDC, and Transnet II. These stemmed from weaknesses in pre-approval assessments.

❙ Project approval process layers and their bureaucratic hurdles: disbursement, procurement, and legal requirements affected energy project delays and the efficiency of most grant operations.

❙ Procurement: some projects have been delayed by procurement concerns (such as CSP Upington).

❙ Contracting issues relating to contract and contractor management particularly with packaging contracts into project components in complex infrastructure operations: e.g. Medupi large number of contract packages contributed to difficulty in managing them and delays.

❙ Poor Bank supervision and monitoring referred to the limited lines of communication between the Bank and the client that provided needed information and responses so as to act quickly and adequately on issues.

❙ Bank and partner/client capacity concerned inadequate time and resources from the Bank to address client needs in administering loans. Weak client capacity to meet project requirements was also a factor: e.g. Commercial Bank A and DBSA, needed support to meet conditions to process loan and grant approvals on time.

Box 3: Factors affecting time efficiency

Source: 13 PRAs, additional interviews, project milestone data (portfolio) analysis, and document review.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

For some approvals, decentralization added rather than removed a layer.65 Some task managers were SARC-based while others remained at headquarters through most of the period. Headquarters did not act on 2013 requests to restructure and bolster the SARC team,66 nor did it fill all key positions that had been made vacant. Procurement was the one area where the Bank decentralized further and put a regional coor-dinator and a procurement officer on the ground. In the last five years, delays relating to procurement in the South Africa portfolio were generally not on the side of the Bank. Indeed one client and a DP reported a quick turnaround (compared to other partners) for non-ob-jections.67 Beyond Bank processes, other internal and external factors were identified as reducing the timeli-ness of project implementation (see Box 3).

There was not enough information to demonstrate whether the Bank managed itself cost-efficiently. No stakeholder or other evidence suggested wastefulness or over-resourcing in the South African context. Indeed, issues of under-resourcing in key areas were raised. However, some staff raised the issue of wasteful use of task managers’ time who spent a large proportion of their time on administrative tasks that might be done more cheaply and quickly by dedicated operations assistants, there were no such assistants in SARC.68 Issues relating to continued high mission costs have already been iden-tified in the mid-term review of decentralization.69

Managing for results

Finding: Some aspects of a system managed for results appeared in the Bank’s activities in South Africa at country, portfolio, and project levels. However, the focus tended to be on disburse-ment and implementation rather than on devel-opment results. In line with changes in the rest of the Bank, the improvements in monitoring public sector operations have not occurred in the private sector. The evaluation highlighted a particular concern for monitoring results for LoCs in the South Africa portfolio.

Efforts to gather and learn from experience at country level were more prevalent in recent years than during the first half of the evaluation period. For example, CSP mid-term reviews and country port-folio improvement plans were done only in the second half of the period and showed an effort to integrate lessons. Efforts to enable the monitoring of devel-opment results also improved over the period: While the first CSP did not include a log frame, the second one did, albeit with poorly quantified or measurea-ble indicators and without baselines. The current CSP includes a box of lessons from the Country portfolio performance plan (CPIP), although a recent evalua-tion of CSPs 70 assessed it as insufficient. In addition, a review of those lessons three years on indicated that some had not been applied, such as finding ways to reach South African SMEs 71 and to support PPPs.

The portfolio was evolving in a similar manner. Interviews pointed to an increasing use of Execu-tive Dashboard reports to chase problematic or slow projects. The country program officer and SARC management used the tool proactively and used its indicators in their performance assessments. However, these reports focused on progress in implementation and did not cover progress on development results. Country portfolio performance reports mostly empha-sized public sector operations, which means that they did not cover most of the major projects in the South Africa portfolio well. The 2015 CPIP also examined progress against 2012 recommendations, including ongoing work to improve the quality of supervision from technical, fiduciary, and safeguards standpoints. This monitoring may also have contributed to better disbursement rates, which rose from 43% in 2012 to 75% in 2015, although this also related to a small number of energy projects.

For individual projects, the picture was mixed and the private sector portfolio raised concerns. Across the Bank, changes have been introduced in project level monitoring on the public sector side that have not affected private sector project monitoring: This was evident in the South Africa portfolio.72 There were few implementation performance reports (which also


include monitoring progress towards development outcomes) for South African projects. On the private sector side, short project status reports and BTORs were produced and some were centrally stored by the portfolio management team and not by SARC. Staff reported that previous lesson-sharing workshops involving portfolio officers and the business develop-ment team were helpful to them but that there have been no such workshops in the past two years.73

There is a difference between monitoring frequency and quality. The portfolio achieved the Bank’s regular standard of 1.5–2 supervisions per year. However, the evaluation examined the quality of supervision for 11 private sector projects 74 and found the following:

❙ Supervision quality rated between unsatisfac-tory and moderately satisfactory; but only four projects achieved the moderately satisfac-tory bar. Three of these had commercial banks as clients and two were led by another IFI. For finance projects PSRs and BTORs focused on the financial health of the intermediary and included annual financial data but only minimal coverage of developmental aspects. Any development results information depended on the data provided by the intermediary, varying in quality and detail. In the last two years, financial intermediaries were asked to complete development outcomes templates, however some of these were incom-plete. Comparison with the IFC approach in South Africa indicated that it is possible to improve the quality and consistency of results data collection and intermediaries’ reporting with clarity on what is required upfront.75

❙ The Bank’s portfolio management and the evalu-ation team found some data inconsistencies, for example, regarding LoCs used to refinance exist-ing projects, e.g. Commercial Bank A.76

The supervision and management of the two private sector infrastructure investment projects were also assessed as moderately unsatisfac-tory; although for different reasons compared to

finance projects. In particular, supervision report-ing for Transnet was assessed as poor for its lack of consistent reporting, which made it difficult to prop-erly track implementation and outcomes. Eskom Hold-ings showed similar reporting issues of quality and completeness (PSRs, BTORs). Delays were acknowl-edged in the supervisions but were given inadequate attention, and the limited availability of critical reports counted among the poor monitoring issues noted.

Across the public and private projects reviewed, there was little evidence that monitoring and post-review data were used to improve potential development outcomes. In the case of Commercial Bank A, the fact that another DP had already provided TA was noted but no action was taken to find an alter-native use for the planned TA. For Medupi, AfDB and other DPs are now conducting joint monitoring and the findings are being used to inform the government shareholder and to try and provide technical support to the client, however this was not the case in the early years. For IDC and DBSA, the design of the successor lines did not make use of the lessons raised during the implementation of previous LOCs, particularly for the capacity to lend successfully outside of South Africa and the difficulties in providing adequate report-ing data. In some cases, monitoring records did not correctly pick up on non-performing sub-loans and the required action.

Another issue about monitoring public and private projects raised in interviews, particu-larly with clients, and also evident from docu-ment review was the need to improve internal coordination. ESKOM has arranged over 14 77 site visits for AfDB in the last 2 years.78 As explained under coherence, public and private sector teams now coordinate their site visits for Medupi (both also visit other sites). As to relationship manage-ment, clients perceived that staff turnover and, on the private sector side, the gap between origination and monitoring teams, affected institutional memory and therefore the quality of monitoring as part of a broader ongoing relationship. Going forward, DFIs expressed interest in working with the Bank to mutu-ally strengthen reporting systems.

43To what extent has the Bank’s engagement in South Africa achieved expected results?

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

To what extent has the Bank’s engagement in South Africa achieved expected results?

Relevance

Finding: Overall, the objectives of the Bank’s strat-egy and portfolio were relevant to the Bank, to the government, to clients, and to beneficiaries. However, in practice, the range of areas identified as being relevant in the design of the project pipe-line was not followed through (e.g. several water projects and capacity development operations). Specific project design weaknesses were identi-fied regarding achieving project objectives. The evaluation assessed the Bank as broadly operat-ing in appropriate areas but with a project design and portfolio that were not always appropriate for delivering on objectives. Overall, the relevance of the strategy and program was rated as moder-ately satisfactory.

Country level strategies were assessed as being relevant to the strategic priorities of the Bank and of GoSA at the time and also at the time of the evalua-tion (see Country Strategy overview and design). In terms of the planned versus the actual portfolio, Bank strategies envisaged a more comprehensive, varied portfolio than the one that it developed, although it engaged in areas that were relevant to overall objec-tives and was well focused (see Portfolio Focus). The challenge for the Bank was to deliver against stra-tegic objectives relating to parts of the portfolio that it was not able to develop. If activities were broadly relevant to stated objectives, the pipeline portfolio was not designed to achieve them, partly because not all projects in the pipeline were relevant to the Bank’s comparative advantage, tools, or resources in South Africa. As also noted, the strategies and the portfo-lio addressed the crosscutting issue of inequality and inclusive growth only implicitly.

All projects were rated satisfactory or higher in terms of the relevance of their objectives to the Bank and to stakeholder priorities. For example, all energy projects assessed responded appropriately to national priorities to increase energy generation and combined renewable and non-renewable sources. Both of these align with the Bank’s energy generation objectives.79 On the finance side, the LoCs beyond South African borders aligned with the GoSA’s interest in expanding South African business across the conti-nent and with the Bank’s interests for financial inclu-sion and private sector development continent-wide, its private sector strategy update of the time, and with the more recent financial sector development strategy. The generally good alignment among the objectives of projects, of the Bank, and of the client, was in line with findings in other countries.80 More broadly across the portfolio, a couple of projects did not align with CSP objectives but were relevant to other government priorities (e.g. mining).

Regarding the adequacy of the design of indi-vidual projects to achieve their stated objectives, most projects (86%) achieved a rating of moder-ately satisfactory or higher. This is above average for all countries assessed by IDEV but below the bar for MICs (see Figure 6). However, only two thirds of the projects were rated satisfactory.

The issues identified in the evaluation of project design included the following:

❙ Overlooking client capacity and other constraints, for example, in relation to IDC capacity to lend successfully outside South Africa or failing to note that loans to small emerging farmers would not be competitive because of other government


support to this group in the Commercial Bank A case. The limited understanding of client capac-ity was also clear in cases that were not included in the in-depth assessment, such as the Kalagadi mining project, which remains severely delayed as a result and threatened with cancellation.

❙ Lack of targeting in some LoCs with intermedi-aries permitted to on-lend to any size of business in a range of sectors anywhere on the continent without a specific focus to ensure objectives (e.g. reaching SMEs and job creation) were met, for example, in the IDC, DBSA, Commercial Bank A LoC and Commercial Bank B LoC (as opposed to the targeting in the Commercial Bank B Sub-Debt and Commercial Bank A LoC).

❙ Technical design, procurement and contract-ing. Procurement has been an issue in all energy projects that were assessed. This was rarely due to the Bank’s procurement systems, but rather because the Bank aligned behind others’ procurement systems without due consideration of the consequences (delays, flawed procure-ment, etc.). In the case of Medupi, the Bank financed the project after ESKOM had conducted

the relevant procurement, which reduced the usual due diligence of the Bank’s processes, although an assessment was conducted to ensure that the procurement was satisfactory at the time of appraisal. The company that won the contract underperformed significantly in deliv-ering key components funded by the Bank, and the process has also been embroiled in inves-tigations by various entities, including the US Securities Exchange Commission and the Bank’s anti-corruption department.81

❙ Problematic log-frames, indicators and targets. Some include vague measures, some try to make a link between small inputs and national or even continent-wide changes (IDC 2 suggested that USD 200 million going to 7 projects would be attributable to changes in continent wide exports, employment and growth figures with specific targets allocated), and another confuses spend-ing with outputs.82 In the case of private sector projects the objectives and targets from the PARs are not necessarily also reflected in the Loan Agreement to which the client adheres, and there-fore do not necessarily even feature as an objec-tive they are working towards.83

Figure 6: PRA ratings for the two relevance sub-criteria

1b. Relevance of project design

HU U MU MS S HS

1a. Relevance of project objectives

0% 20% 40% 60% 80% 100%



An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Figure 7: PRA ratings for the two effectiveness sub-criteria

2b. Achievement of outcomes

HU U MU MS S HS

2a. Acivement of outputs

0% 20% 40% 60% 80% 100%


Effectiveness

Findings: Effectiveness, measured in terms of producing planned outputs and outcomes for both individual projects and across strategic objec-tives was mixed. Each project was rated based on the outputs and outcomes that were explic-itly planned at appraisal versus those that were actually achieved 84 (see Figure 7). Achievements against the overarching objectives set out in the CSPs are assessed, including non-lending activi-ties such as knowledge work. Overall, the Bank’s activities were delivering well against some but not all of its stated goals, with challenges in lend-ing and non-lending activities. Given the signifi-cant variation, the overall rating for effectiveness was moderately unsatisfactory.

For the Bank’s CSP goals in South Africa over-all, a pattern emerged of achievements that were stronger in some areas than in others. Strong outputs and outcomes in energy and rail transport contributed to infrastructure efforts but the picture was very mixed else-where particularly, with respect to domestic access to finance for SMEs, (since most LoCs did not target SMEs and/or were on-lent outside South Africa) and for knowl-edge work (see Figure 8; Annex C for full ToC).

Outputs

Output achievement was mixed across the projects, with few clear patterns. Overall, with 77% of the projects rated moderately satisfactory or higher for output achievement (though only a third reached the satisfactory bar), programs were performing slightly less well than in other countries examined (83%). Given the small number of data points, it was not possible to assess whether there was any signif-icant difference between small and large projects or whether there was a trend of improved output achievement for new projects compared to older ones. In the energy sector, stakeholders observed differ-ences in performance based on the manageability of project size (comparing Medupi and Sere, for exam-ple) but given the range of factors affecting results, it was difficult to generalize from so few cases.85

Output delivery in infrastructure projects was broadly satisfactory despite some serious delays. All four projects assessed achieved moderately satis-factory or higher on output achievement (one was highly satisfactory: Sere Wind Farm’s energy generation figures surpassed those planned at project appraisal). Trans-net’s freight transportation volumes were also higher for commodities thanks to the successful acquisition of more, efficient locomotives. However, two other major


Table 2: Selected deliverables against output targets for energy projects assessed

Source: PRAs. Note: Non-exhaustive list of indicators selected for their importance.

Project/target Achieved or on track to be achieved

Medupi Coal Power Plant ❙ 4764 MW of power to the grid by 2015.

By 2015, only 1 of 6 units was online, delivering 600 MWs. However, the plant is broadly on track to deliver the full amount by new target date of 2020. (Target likely to be met or nearly met, albeit late).

❙ 18000 jobs during construction and 680 permanent positions.

18000 jobs during construction, 495 current employees, but still expecting the requirement of 680 in 2020 with operational capacity. (Target likely to be met, albeit late).

❙ Installation of FGD system to reduce air emissions from point of operation.

FGD not installed in time for current or future operations for some years and not funded at the time of evaluation. (FGD will not be present in time to reduce emissions in the first 5+ years of operation; currently funding for FGD unsecured, so target considered unmet).

Sere Wind Farm ❙ 100 MW to grid and 220 GWh.

100 MW to grid and 298 GWh (target surpassed).

ESKOM corporate loan ❙ Return-to-service plants (Komati, Camden,

Grootvlei): 3600 MW by October 2011.

Completed, 3741 MW (target exceeded).

❙ Pumped storage plant (Ingula): 1332 MW by December 2012.

First unit (MW) now planned for 2016. (Target unmet). Delays experienced, new target date of 2017.

Figure 8: Overview of delivery against inputs, outputs, and outcomes at country level

Investment in infrastructure for energy (including green energy) transport, water and sanitation in SA

Increased energy generation capacity, including renewables

Increased freight (rail) transportation capacity and air capacity and services

Increased water and sanitation capacity for people and industry

Increased financial capacity amongst SA intermediaries to on lend or invest in SMEs in key sectors in SA

Increased financial capacity among SA intermediaries to on lend or invest in SMEs in key sectors in the wider region

One stop border policy and first posts established

Increased awareness and skills, knowledge-sharing, collaboration, gender equity, good governance

Increased household and business access to secure electricity

Increased household and business access to secure water

Greater volumes of freight transported by rail for import/export

Increased access to finance for business, especially SMEs, in SA

Increased access to finance for business, especially SMEs, in wider region

Intermediaries become part of stronger financial system with sound environmental and social footing

Improved border clearance efficiency

Increased knowledge-sharing, greater gender equity, and strengthened frameworks and practices

Investment in infrastructure for transport including rail and air

Investment in major water infrastructure

Lines of credit and equity to SA institutions for the wider region

Lines of credit and equity for use in SA

Policy advice and knowledge work focused on knowledge-sharing, infrastructure and regional integration, gender equity and governance

INPUT OUTPUT OUTCOMES

Note: Green highlights areas of greatest success in delivery; red indicates areas of least success.

Source: Total portfolio analysis wherever possible according to program logic.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

public infrastructure projects were so seriously delayed that they were excluded from the in-depth assessment for want of sufficient progress to examine progress towards results (Kalagadi Mining Project and Concen-trated Solar Power Upington).86 The Medupi Power Plant project was seriously delayed but a new project schedule made it likely to deliver energy generation in the order of that planned at appraisal. Concerns about other planned outputs remained, however (see Table 2).

The picture was different for the finance sector projects. One third did not achieve the

moderately satisfactory bar for output delivery. The evaluation was made strictly against objectives stated in the appraisal not on implicit objectives in project documents or loan agreement objectives as opposed to appraisal document. As noted in 3.9, the results targets in project appraisals approved by the Board and the targets identified in the legal agreements with the clients were often dissonant. In some cases, clients were not made aware of some of the Bank’s operational targets 87 in others they found them onerous and not in line with their business model.

One 2012 LoC was designed with 50% of the UA 58 million to go to emerging farmers. At the time of appraisal, it was known that GoSA had subsidized a loan scheme for this group, which made the funding comparatively uncompetitive. The LoC was nonetheless approved, and three years later the 50% remained undisbursed. (At the time of writing, a waiver was agreed to allow the intermediary to spend this money differently and two potential large companies had been identified). The other 50% was disbursed as refinancing to a major international company that nominally allocated the funding to a wood processing plant with a good development story attached to it that included direct and indirect job creation, other local development such as infrastructure in close collaboration with local government, strong environmental and social standards implemented. However, the factory and the associated benefits were created 5 years before the LoC lending. It was therefore inaccurate to report (as internal Bank monitoring did) that the jobs and other benefits were created with Bank funds.

The LoC was one of few to include a TA component, but it was never used. The appraisal did not pick up that another DP had already provided such support. A Bank supervision report had noted that this tranche was not being used but it was unclear whether or not any action was taken. In interviews, the intermediary’s officials were unaware that Bank support for TA was available.

Neither of the two objectives – supporting the intermediary to develop its capacity to support, or providing finance to emerging farmers (as opposed to its traditional corporate lending base) – were achieved. The project was based on incorrect assumptions and was ultimately ineffective from a development perspective and against the targets approved by the Board. From a straightforward banking perspective, however, the loans made good profit sense for AfDB and the intermediary.

Output area Target Achieved Comments

Proportion of LoC to emerging farmers

+50% 0% This 50% was being reallocated at the time of the visit to large established clients rather than to emerging or disadvantaged farmers. Execution rate: 0%

Number of loans to emerging farmers, commercial farmers, cooperatives

2,460 1 1 commercial refinancing loan to a major corporation. Up to 3 more planned. Execution rate: less than 1%

TA 50% of LB staff trained25% of EF clients trained

0

0

TA not done

Execution rate: 0%

Box 4: A lesson in appraising to ensure that risks to effectiveness are considered


In practice, the Bank gave intermediaries significant freedom to deploy funds in the majority of the LoCs reviewed, which means that specific outputs cited at appraisal – e.g. as sub-loans to specific sectors, a focus on SME lending – were not met. In some, a few large loans went to a range of sectors and may have included major investment projects that the Bank was supporting directly 88 and refinancing loans already on the intermediary’s books. The most extreme case here is described in Box 4. In other cases, the number/size of the sub-loans indicated that there was no follow through on the original intention to support SMEs. For exam-ple, DBSA V funds were used for 6 sub-loans against a target of 41. In other cases, the original targets were already low, indicating that funds were always destined for larger investment projects (e.g. the Commercial Bank B and Commercial Bank A LoCs) in contrast to the larger number of transactions in the IFC-led operations with the same institutions. For example, in the Commerical Bank A Global Trade Liquidity Program, 40–60 transactions were planned and the target was surpassed.

Outcomes

At the level of outcomes, just over half of the projects examined achieved moderately satis-factory or higher ratings. Overall, this figure (54%)

compared poorly with data available for a wider set of countries where close to 75% of the projects met the moderately satisfactory bar.89

Regarding the difference between the infrastruc-ture and finance sides of the portfolio, three of the four infrastructure projects were rated moder-ately satisfactory or higher. The wind power project

– which also exceeded some of its output targets – contributed to the green economy in South Africa. The transport operation responded to the strategic planned investment in infrastructure for the trans-port sector, having as one of its primary objectives to increase freight transportation by shifting from road freight to rail. The project has contributed to private sector development, increasing efficiency and relia-bility of rail freight, and supporting export of minerals.

None of the finance projects achieved ratings higher than moderately satisfactory and 5 of the 9 did not reach that bar. This level of achievement should be put into perspective: the assessment was made against planned objectives in project appraisals, which may differ from loan agreements and other strategic objectives not made explicit in the appraisal targets (Box 5). Project ratings were affected both by under performance and by the choice of indicators at appraisal. In assessing

When assessing the effectiveness of Bank LoCs, the stated objectives for outputs and outcomes was the starting point. Success cannot be measured against objectives that are not explicit.

However, this misses one broadly successful area. The Bank’s private sector team sought to use its engagement with South African financial institutions to build up a portfolio in the country and to increase engagement in other RMCs, especially low-income countries (LIC) without affecting the Bank’s prized risk rating. Judging by these three points, which are critical from a good banking perspective, the portfolio of finance operations has been largely successful. However, the PARs approved by the Board related more specifically to good development banking objectives (job creation, support to SMEs in particular areas of focus, supporting environmental and social responsibility). These kinds of objectives were rarely included in loan agreements and not always monitored.

In effect, there was a mismatch between what the Board approved and what was actually agreed, Moreover, the Board is not always fully informed of the LoC’s implicit objectives of channeling more funds to high-risk countries without taking on that risk, where the operations have been more successful.

Box 5: Implicit versus explicit objectives and good banking versus good development banking with financial intermediation

Source: Interviews and documentation review.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

finance sector outcomes with respect to the logic of the overall country program, the evidence supported the focus in finance operations on contributing to the provision of financial capacity among South African intermediaries. Performance was moderately satis-factory. The evidence showed limited achievement in contributing to increasing access to finance for business, especially to SMEs in the wider region and not at all to South Africa. However, the bulk of the finance operations illustrated support to interme-diaries to achieve stronger financial situations with sound environmental and social footing.

Box 6 summarizes factors identified as affecting effectiveness at output and outcome levels, drawn from in depth PRAs and other evidence.

For the equity portfolio, two broad proxy indicators of effectiveness were noted. First, the funds were relatively young so the internal rates of return were

premature, and the total value paid gave an intermedi-ary indication of fund performance only. On this indica-tor, 8 of 11 funds had values greater than one. Second, the success in targeting SMEs rather than big busi-ness showed that 66% of the investments (by number) were under USD 1 million, indicating a focus on smaller enterprises than those seen in the LoC. 12% were investments of over USD 10 million.90

Factoring in non-lending activities and looking at progress towards high-level objectives

Beyond the effectiveness of the Bank’s portfolio of operations, non-lending activities also contrib-uted to overall effectiveness at country level and indicate limited effectiveness. Underlying reasons included capacity and resources to deliver, prioritiza-tion, and the context. In particular:

Analysis of the 13 projects included in the in-depth assessments supplemented by a desk review of 5 other projects in the portfolio highlighted certain recurrent factors that explained poor performance. The most frequent reasons that also pertained to poor output or outcome delivery included:

1. Delays in project start-up and implementation can affect effectiveness. For example, energy projects to prevent load shedding experienced delays that resulted in the failure to deliver (Sere Wind Program) the technology to resolve the energy problem. This limited the relevance of the program when it was completed.

2. Design floors or gaps. Issues that are overlooked at appraisal can come back to bite when judging effectiveness. For example the Sere Wind operation had to be split into two projects to progress; the Kalagadi project had a funding gap that created implementation delays; in the Commercial Bank A operation, the Bank’s condition did not match the Commercial Bank A’s business style, which led to difficulties in administering the loan. Finally, the Commercial Bank A LoC where the Bank had no influence on sub-project performance as no limits were set on project size, sector type, or focus areas. As a result, the client had too much flexibility in using the funds.

3. Bank staff capacity is linked to the quality of appraisal, monitoring, and institutional arrangements that affected the achievement of the outcome or created unintended negative effects. For example, findings of limited resources and expertise allocated for these activities, and the lack of commitment of relevant experts (legal, disbursement, procurement, sectors) throughout the project cycle.

4. Client and contractor capacity to implement as planned, especially in the energy projects; Eskom Medupi, Sere Wind in particular, and some finance operations (LoCs to IDC, DBSA and Commercial Bank A: failure to attract black farmers).

5. Poorly identified indicators to monitor and measure effectiveness realistically and poorly planned logical frameworks, e.g. finance operations DBSA and IDC operations, with unrealistic targets.

Box 6: Factors affecting output and outcome delivery

Source: PRAs, interviews, document review.


❙ Planned knowledge and analytical work was only partially delivered (indicating limited output achieve-ment for this area). For the work that was delivered, there was scant evidence of outcomes in terms of effects on operations or policies. The Bank has limited room to target policy change in the South African context and little knowledge work explic-itly related to its major investments. As explained in the “Knowledge and Policy Advice” section, the Bank has not been effective in building a reputation as a knowledge generator or broker partly due to the developed capacity of institutions in the country.

❙ Partnership and leverage were strong in some areas and weak in others. There is a strong part-nership with central government, but opportu-nities to build partnerships elsewhere were exploited only on a project-by-project basis, thus enhancing potential effectiveness only in those cases. Bank efforts to leverage finance

have not been fully successful but there were positive examples across the portfolio indicat-ing the Bank was adding value and learning from experience.

❙ Capacity development was an important part of the theory behind the Bank’s strategy, at least since 2008. However, there were close to no deliveries in this area. The Bank and clients agreed that progress was limited (see the “Effec-tiveness” section, on non-lending activities).

Where possible, local unintended impacts were examined. Box 7 summarizes the case of Medupi. Such impacts were most visible for large infrastruc-ture projects, whereas minimal information was avail-able for the LoC’s sub-projects’ unintended impacts. This does not indicate that there were no impacts but rather that they are not known and not adequately addressed by Bank or by intermediary monitoring.

❙ Water stress. ESKOM has opted to use wet FGD technology that increases additional water stress on the area. A second phase Mokolo Crocodile River project is planned but not yet funded.

❙ Heritage sites and grave desecration. Local residents complained that ESKOM disturbed graves whose presence was not picked up by the original ESIA (conducted before AfDB finance), causing serious disquiet in some parts of the local community. The matter has since been investigated (by the AfDB and an independent party on behalf of ESKOM) and work is ongoing to resolve complaints and build a site where community members can pay respects to the dead.

❙ Revised resettlement guidelines for Eskom. ESKOM indicated that AfDB gave constructive detailed input to help Eskom revise its resettlement guidelines, was a direct result of the AfDB investigation into project complaints.

❙ Demobilization. At the peak of construction, 18,000 people were employed on site. Eskom developed a demobilization strategy and a Medupi Leadership Initiative. Community feedback acknowledged these initiatives but indicated they were inadequate for absorbing the demobilized work force. The scale of the challenge was unforeseen which made management and communication about employment inadequate and contributed to a period of unrest in the area.

❙ Urbanization of Lephalale. The influx of people exceeded the municipality’s infrastructure capacity. Piped water now has to be supplemented by trucked-in water for which people stand in long queues (confirmed by local government representative and DEA team leader). The urbanization is irreversible, but the plant no longer provides the same number of jobs.

❙ Economic. The late delivery of the Medupi project has had an unintended negative impact on the economy, and construction costs are likely to rise and be passed on to the private and business consumer once the regulator adjusts tariffs.

Box 7: Unintended project impacts: the Medupi Power Plant

Source: Medupi PRA.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sustainability

Finding: Overall, the picture on sustainability is mixed but information for one half the portfolio is difficult to obtain. From the perspective of the Bank as a lender, financial and technical sustaina-bility is generally at least moderately satisfactory. The portfolio is mixed on environmental and social sustainability measures. CSPs discuss some rele-vant issues but do not consider the sustainability of past interventions. Overall, a rating of moder-ately satisfactory is appropriate.

At project level, 85% of the projects including all infrastructure projects assessed received a rating of moderately satisfactory or higher for overall sustainability (Figure 9). These ratings compared well with those across a broader range of countries (74% achieved MS+).91 However, on the finance side of the portfolio, the rating gave credit for the Bank’s profitability (the risk was passed on to intermediaries making this invariably strong) and for environmental and social concerns at intermediary level (little data was available about individual sub-projects).

On prospects for financial and technical sustaina-bility, some projects exhibited noteworthy features:

❙ All three energy projects examined in depth were with ESKOM, whose attention to quality supported a strong rating on technical soundness, an aspect

that the sustainability assessment considered. For example, Medupi is expected to have a 50-year life span and the Sere Wind Farm to have at least a 20-year lifespan, boding well for sustainability. Cost and time overruns have been problematic, however, and independent experts have accused ESKOM of “gold plating” its technology and equip-ment choices.92

❙ For private sector projects, profitability or busi-ness success was the main measure of sustain-ability and for the LoCs, the intermediaries take on most of the risk. Thus, despite noteworthy non-performing loans in two cases, the rating was affected only when the volume had an impact on the intermediary’s overall balance sheet. Nothing suggested that the intermediary would default as a result since the Bank loan will remain profitable. Therefore, all finance projects reached the moderately satisfactory bar and most of them (7 of 9) surpassed the satisfactory bar.

Regarding environmental and social sustainabil-ity, two issues arising from the projects should be highlighted – variation amongst infrastruc-ture projects and lack of information for finance projects. First, environmental and social sustaina-bility ratings varied substantially in the infrastruc-ture projects. At one extreme, the Sere Wind project performed well: it complied with the relatively solid

Figure 9: PRA ratings for the two groups of sustainability criteria

Financial/business and other aspects of sustainability

HU U MU MS S HS

Environmental and social sustainability

0% 20% 40% 60% 80% 100%



national environmental legislation, was implementing an environmental authorization and environmental management plan, had a third party environmen-tal control officer acting for the DEA who conducted regular audits, and ESKOM environmental and health and safety officers (reporting zero health and safety incidents by the time of the evaluation team visit), and ESKOM was working with the endangered wildlife trust on a bird and bat monitoring program. Medupi also had ESKOM and DEA environmental control officers on site who were active in monitoring and developing the environmental management plan. The AfDB, ESKOM, and DEA have been proactive in environmental management in the past two years.93 However, the project’s environmental sustainability was downgraded overall given its current situation: (i) the Flue Gas Desulphurization (FGD) is not yet funded and air emissions will suffer as a result for the first five or more years of operation, and (ii) water

stress in the area will grow and the proposed solution remained unfunded.

Second, for the LoCs, the available information was at the intermediary level rather than at the final sub-project level in most cases. The Bank succeeded in supporting the development of envi-ronmental and social standards, including having the two commercial banks apply the Equator Princi-ples and acknowledge that IFIs encouraged them to apply the principles.94 The intermediaries’ quality of environmental and social standards was examined but there was insufficient data to provide any assur-ance that each individual sub-project fully adhered to guidelines and averted adverse environmental and social impacts. For DBSA, environmental and monitoring and evaluation capacity were reduced after restructuring, a negative direction that the Bank’s monitoring did not note.

Issue

CSP

1

CSP

2

CSP

3 No. of projects (of 24)*

Comments regarding implementation

Green growth/environment

7 Includes 3 renewable projects. Coverage of environment in some of the lines of credit through pushing stronger E+S systems and standards, including equator principles. (Commercial Bank B (2 operations), Commercial Bank A).

Gender 3 Only in some cases where gender mentioned is reporting information then available particularly credit facility to Commercial Bank B targeting Black SME sub projects. One project (Kalagadi) focused on a BWOE, but no results yet due to delayed start up.

Inclusive growth/inequality

5 Project objectives related to some alignment through emphasis on BBBEE, addressing the deprived population though data not always collected to show progress.

Regional integration

7 Regional LOCs are involved in investments in all corners of the continent, rather than focusing on integrating a region. Support in energy and rail freight are a step towards regional objectives but results not visible yet. Project with a clear focus was the study on one stop border posts which has been completed but not yet had any impact on GoSA.

Capacity development

8 Only two of the major investment projects in the PRA included capacity components and both were not delivered (Commercial Bank A and DBSA IV). Across the portfolio more broadly 9 of 25 projects made some reference at appraisal to a capacity or technical assistance role, most of these were small demonstration projects and grants. The only other large investment project with a capacity development component was Kalagadi mining project – which is too early in implementation to assess results.

Table 3: Overview of cross cutting issue consideration in strategy and project appraisal

Source: CSPs and PARs.

Note: Red = no coverage, yellow = minimal coverage, green = good coverage.

* Analysis was conducted before inclusion of the two 2015 approvals.


An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

At the strategic level, sustainability was not a main feature of the CSPs. The latest CSP does not discuss the sustainability of past interventions. However, it is worth noting that half of the South Africa portfolio is relatively young, and there were few completed projects to analyze when the CSP was drafted. Environmental sustainability was partially covered, as discussed below.

Crosscutting issues

Finding: The strategies, portfolio, and results varied in their coverage of crosscutting issues. For each crosscutting issue examined, coverage in the strategies increased but the majority of projects in the portfolio did not explicitly focus on the issue. There was limited information on implementation and results in these areas.

At the level of country strategies, there is a general improvement over time in the inclusion of all of the cross cutting issues reviewed (see Table 3). However, even the most recent CSPs varied with

respect to the quality of their analysis: from satis-factory for regional integration to moderately unsat-isfactory for gender. The latest CSP acknowledged environmental and green growth issues, which were rated moderately satisfactory and linked in part to the important role of energy and renewable energy in the strategy and portfolio.95

At project level, only a minority of projects included an explicit focus on each crosscutting issue. Gender and inclusive growth were least well covered, although gender does feature in the list of economic and sector work that has been deliv-ered.96 The current team acknowledged the need to mainstream gender better, but is currently examin-ing new projects approved in 2015 and pointed to rare cases such as Kalagadi where one component has a gender dimension. The Bank also published a gender related paper in 2009, though it is not clear what impact it had. Economic and sector work was also planned for green growth related topics but was not delivered. Given that capacity development was a major theme of the last two CSPs, it was tell-ing that most major investment projects were not

Figure 10: Results framework integration of crosscutting issues in South Africa versus other country portfolios

0

10

20

30

40

50

60

70

% of project including outcome indicators on expanding the

economic base across genders

% of projects including outcome indicators on contributing directly

to environmental sustainability

% of projects including outcome indicators on expanding the

economic base across regions

% of projects including outcome indicators on expanding the

economic base across ages, e.g. youth

Note: choice of indicator based on CEDR requirements, % of projects by number.

Source: IDEV data analysis for CEDR.

All 14 countries South Africa


planned to include a capacity component and that the two that were planned did not happen. Not as many small grants were used in this area as antic-ipated in the strategy partly because of the limited number of small grant tools available and the lack of appetite to use what was available because of the well-known cumbersome processes.

South Africa is not the only country where the Bank has struggled to integrate crosscutting issues into project objectives, but it does better than average on environmental sustainability and regional disparity. Only 3 of the 13 projects explic-itly ensured that benefits were considered for both genders, which is below the cross-country average. Feedback from staff and clients confirmed the lack of

progress on gender equality in the South Africa port-folio (see Figure 10).

Information on actual results achieved for cross cutting issues was limited. On the environmen-tal and social side, credit was given where the Bank supported the client to improve its environmental monitoring and systems and where for example clean energy projects were targeted through a credit facil-ity. Of the 13 projects assessed in depth, 10 were rated moderately satisfactory or better on environ-mental and social issues (6 were satisfactory and 4 were moderately satisfactory). However, the failure to consider or deliver capacity development in within projects, boded poorly for sustainability from an insti-tutional perspective.

57Overview of conclusions and recommendations

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Overview of conclusions and recommendations

This section summarizes the main conclusions and proposes a few recommendations to support the Bank in learning from this experience to improve future performance.

Overall conclusions

The evidence gathered by this evaluation pointed to a mixed picture of how the Bank managed its engage-ment and what was achieved. It is not surprising that the Bank is still learning how best to manage its engagement in the atypical South Africa context. The fact that Bank performance varied by area is also important for accountability and learning. Box 8 summarizes strength and weakness identified in the evaluation.

In terms of the four project evaluation questions explicitly rated at country level, relevance and sustainability were rated moderately satisfactory, and efficiency and effectiveness were rated moderately unsatisfactory (Figure 1).

Lessons and themes to inform future engagement

For efficiency and effectiveness, project level factors thwarting performance were identified. In terms of time efficiency the five most frequent factors are illustrated in Box 3. For effectiveness, the findings identified five factors that most frequently influence performance (see Box 6). Certain themes cut across the evaluation findings and are important for looking ahead:

❙ Establishing the Bank’s comparative advan-tage in a competitive context. The Bank has been unable to deliver in all areas planned, partly because it misjudged its own comparative advan-tage or niche in the South African context. This included in sectors where it intended to lend but found that it could not, as well as non-lending work such as knowledge generation and sharing.

❙ Striking the right balance between good bank-ing (profitability, sound risk management) and good development banking (achieving and

Strengths Weaknesses

❙ Relevant strategies and operations focused on two key areas. ❙ Evidence of innovation in some projects, though not all. ❙ Useful experience in leveraging finance. ❙ Patchy progress in managing for results. ❙ Building strong partnerships with some parts of central government. ❙ Relatively strong performance in terms of both financial sustainability

and environmental and social sustainability, with some variation.

❙ Over promising on delivery of operations in all the areas identified as important in CSPs.

❙ Being more than a financier by the knowledge Bank, and source of capacity support that it aspires to be.

❙ Planning for and monitoring development results for lines of credit. ❙ Time efficiency – despite decentralization the Bank

is still not a quick and nimble partner and this is highlighted as a problem for its attractiveness to borrowers in the competitive South African context.

❙ Achieving planned outcomes within individual projects, especially within planned timeframes.

Box 8: Conclusions about strengths and weaknesses

Source: Compiled by evaluation team from evaluation findings.


understanding development results). For the finance side of the portfolio the use of LoCs to on lend outside South Africa was successful for growing the portfolio and reaching LICs without taking on risk. However, this has not helped the Bank achieve objectives related to domestic job creation or access to finance in South Africa, and information on the sub-loans in other countries is insufficient for the Bank to understand the devel-opment results it is contributing to.

❙ Organizing to increase efficiency and effec-tiveness. Despite the existence of SARC in South Africa, the management of the South Africa program was not properly decentralized. There are important capacity gaps and authority has remained mainly centralized. Inefficiencies damage the Bank’s reputation with private and public clients. Finally, the potential for the Bank to help South Africa share experience with others is an important forward-looking theme. The Bank has not yet maximized its position as a continental institution or its potential to share useful experi-ences from South Africa – e.g. in the energy sector and on leveraging finance from the private sector and elsewhere.

Recommendations

The evaluation proposes the following recommenda-tions on the basis of its findings and analysis.

1. Initiate steps to ensure that the next CSP is based on a more detailed analysis of the (i) Bank’s comparative advantage and constraints in South Africa; (ii) the resources required to deliver in selected areas of focus. In particular the review could cover:

❙ Key areas of new business development and of repeat business, based on a thorough analysis of where and when Bank funding instruments will be competitive. This means resisting pressure to add sectors that may interest the Bank but for which it has no comparative advantage and it

means thinking innovatively about competitive funding instruments.

❙ Whether or not the Bank has a comparative advan-tage in (i) knowledge generation and sharing and; (ii) capacity development, and if it does, in what specific areas.

❙ Consider a more systematic appreciation of client capacity and options so that the Bank can help develop capacity in the areas that are key to oper-ations delivery.

2. Address the mismatch between the implicit and explicit objectives of LoCs and between appraisals and loan agreements in private sector operations by adopting a more coher-ent, realistic approach which is also aligned with the objectives of the next CSP. This will require frank discussions with the Board and senior management on the banking/risk-sharing role of LoCs versus their develop-ment potential, and designing operations that strike the right balance. All objectives should be stated in the appraisal to allow for a fuller understanding and to acknowledge success with respect to objectives such as risk sharing. Closing the appraisal and loan agreement gap may require more upfront work, but it should reduce the lag that frequently occurs between approval and signature.

3. Build on the South African experience to inform revisions to the Bank’s approach to private sector operation design and moni-toring, to allow for better understanding of results. Current project status reports and back to office reports (BTORs) lack information about progress towards development results, which also stems from the initial design and appraisal procedures. The approach for LoCs in particular needs to be revisited to ensure that the neces-sary reporting data from clients is known upfront and that measurable indicators are used. In South Africa, the Bank has an opportunity to collaborate with existing clients to develop a

59Overview of conclusions and recommendations

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

more robust approach that would benefit both the intermediary and the Bank by enhancing the ability to understand, track, and report results. A new approach can also draw from experience in the Bank’s public sector supervision revisions and from the International Finance Corporation (IFC) and other partners.

4. Use the South Africa experience to support the Bank in identifying new flexible funding mechanisms better suited for MICs. Different types of risk participation may be one avenue to explore, more innovative and targeted LoCs may be another. This could be done jointly with a range of departments and be central to the busi-ness development strategy.

5. Ensure a complete decentralization process and business process streamlining. The experience in South Africa is symptomatic of a broader issue that Bank management is now seeking to address through a new busi-ness model. SARC would benefit from in-house disbursement and legal skills and from exper-tise linked to a given strategic focus (e.g. more

business development officers on the group for private sector operations), especially if it wants to continue to pursue the knowledge agenda. In terms of delegating approval authority, the current situation has not allowed the Bank to be the nimble partner it wants to be. The expe-rience of the MIC TF makes it clear that delegat-ing this and other small grant facilities would be an obvious, helpful step.

6. Share the lessons from South Africa’s experi-ence in the energy sector. South Africa’s expe-rience in investing and diversifying its energy sector as well as the Bank’s engagement are rich with lessons relevant to other RMCs. Given its continental mandate and its prioritization of energy, the Bank is well placed to facilitate a learning exchange that can cover (i) differ-ent types of energy generation; (ii) public and private (independent power producer) roles; (iii) tariff setting and the role of a strong regulator; (iv) contract and project management for new building and refurbishment, and (v) community relations and management in major infrastruc-ture development.

Annexes


Annex A: South Africa Portfolio

Table A.1 Approved Operations between 2004 and 2015

Divi-sion

Project Name Funding Instrument

Status of Pro-ject

Sector Name

Approval date

Currency Netloan Disb. Ratio (%)

Public Sector Operations

AWTF Integrated Water Harvesting Project

ADB LOAN COMP Water Sup/Sanit

4/14/2009 UAC 299,399.6 100

AWTF Integrated Water Harvesting Project

AWF Grant COMP Water Sup/Sanit

11/10/2011 UAC 80,051.2 100

AWTF Multiple-Use Water S AWF Grant APVD Water Sup/Sanit

7/22/2014 UAC 1,072,686.5 0

AWTF Social Franchising Operations & Maintenance of School Sanita

AWF Grant APVD Water Sup/Sanit

12/12/2014 UAC 950,017.5 0

ONEC2 Eskom Renewable Energy – Sere Wind

ADB LOAN OnGo Power 5/30/2011 UAC 31,884,366.0 20.22

ONEC2 Eskom Renewable Energy – Serewind

CLeanTech Fund

Power 5/30/2011 UAC 35,427,073.4 71.33

ONEC2 Eskom Renewable Energy – Upington CSP

ADB LOAN APVD Power 5/30/2011 UAC 156,398,845.5 0

ONEC2 Eskom Renewable Energy – Upington CSP

CleanTech Fund

APVD Power 5/30/2011 UAC 35,545,192.2 0

ONEC2 Medupi Power Project – Eskom

ADB LOAN OnGo Power 11/25/2009 UAC 740,610,964.2 64.4

ONEC2 Medupi Power Project – Eskom

ADB LOAN OnGo Power 11/25/2009 UAC 620,183,079.5 97

ONRI2 Development of SA Osbp Strategy

ADB-MIC Fund OnGo Multi-Sector

12/22/2011 UAC 178,000.0 100

ESTA Statistical Capacity Building Program Phase II (SCB-II)

ADB-MIC Fund APVD Multi-Sector

7/7/2011 UAC 490,600.0 100

OSHD1 Enterprise Development Pilot Project

ADB-MIC Fund Ongoing Social 4/23/2015 UAC 1,200,000.0 0

Private Sector Operations

OPSM2 Kalagadi Industrial Beneficiation Project

Snr Loan APVD Ind/Mini/Quar

5/18/2011 UAC 105,287,702.8 63

OPSM2 Kalagadi Standby Facility Snr Loan APVD Ind/Mini/Quar

5/18/2011 UAC 38,286,437.4 0

OPSM3 Xina Solar One Project Snr Loan APVD Power 6/23/2014 UAC 51,317,715.0 5

OPSM3 Xina Solar One Project CleanTech Fund

APVD Power 6/23/2014 UAC 29,502,509.5 100

OPSM5 Transnet Ltd Snr Loan Ongoing Transport 6/23/2010 UAC 164,639,383.1 100

63Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Table A.1 Approved Operations between 2004 and 2015 (Continuation)

OPSM3 Transnet Ltd Corporate Loan II

Snr Loan APVD Transport 12/15/2014 UAC 177,304,964.5

OPSM5 Eskom Holdings Limited Snr Loan Ongoing Power 6/28/2007 UAC 355,451,921.6 100

OPSD5 Eskom Holdings – Corporate Loan II

Snr Loan APVD Power 12/15/2015 UAC 262,088,545.5 0

OPSM5 Commercial Bank A of South Africa LoC

LoC Ongoing Finance 9/11/2008 UAC 155,879,122.8 100

OPSM Global Trade Liquidity Program, GTLP – Commercial Bank A Of South Africa Limited (SBSA)


OPSM5 Commercial Bank B LoC Ongoing Finance 9/11/2008 UAC 70,854,146.7 100

OPSM5 Commercial Bank B Subordinated Guarantee (Bond)

LOAN Ongoing Finance 6/28/2007 UAC 61,203,631.0 100

OPSM5 IDC LoC II LoC Ongoing Finance 5/19/2010 UAC 141,708,293.5 100

OPSM5 Non Sovereign Guar.line Of Credit To IDC


OPSM5 Fourth Line Of Credit to DBSA – Development


OPSM5 5th Line Of Credit To DBSA LoC Ongoing Finance 2/3/2011 UAC 212,562,440.2 100

OPSM5 Land and Agricultural Development Bank


OITC3 BBI MIC Grant Request ADB-MIC Fund APVD Commu-nications

2/21/2014 UAC 798,000.0 36.9

OFSD Commercial Bank C Limited-Medium Term Multi-Currency Facility

LoC APVD Finance 12/12/2014 UAC 59,465,528.6 –

TOTAL 3,721,687,168.1

Source: Compiled by evaluation. Exchange rate: 1UA=1.25 EUR; 1 UA= SA Rand 17.45 based on SAP Extraction at August 2015.

Note: Table updated to include ESKOM Corporate Loan II approval in Dec 2015 but not considered in project assessments.


2008

Mul

tinat

iona

l, m

ulti-

sect

or m

id-s

ize c

ompa

nies

31/1

2/20

1431

820

0930

7.90

%24

.15

12.6

615

.85

1.69

15.9

0%27

621

13.1

42.8

15.5

0%3

14.3

2008

Min

imun

25%

equ

ity h

oldi

ngs

in

food

and

agr

ibus

ines

es28

/02/

2015

9520

0815

15.8

0%12

.53

0.2

1.13

1.02

n/a

9012

7.5

50.6

27.2

0%8

6.3

2010

Med

ium

to la

rge

infra

stru

ctur

e pr

ojec

ts31

/12/

2014

254

2009

3011

.80%

25.4

30.

61.

03–2

.00%

247.

98

3135

.514

.30%

311

8

2006

Gene

ral f

und

inve

stin

g ac

ross

all o

f Afri

ca31

/03/

2014

531

100%

1.74

3.30

%48

9.5

1925

.84.

81%

14.

8

2010

Clea

n te

chno

logy

/ene

rgy

in s

outh

ern

Afric

a30

/06/

2014

8820

096.

47.

30%

1.4

0.21

2.23

+ve

60.6

125.

161

100%

125.

1

2010

Cont

rollin

g st

akes

in fo

rest

ry a

nd fo

r-es

try-r

elat

ed c

ompa

nies

31/0

3/20

1416

020

1020

12.5

0%16

.20

n/a

0.88

2.10

%96

.37

13.8

45.8

47.6

0%3

15.3

2008

Micr

o an

d sm

all e

nter

prise

s us

ing

mai

nly

loan

s30

/09/

2014

160

2008

2012

.5%

15.9

53.

23–3

.17

0.81

–15.

70%

103.

528

00.

3720

.519

.80%

640.

32

2010

Gene

ral f

und

inve

stin

g in

larg

e pr

ojec

ts31

/03/

2015

908

2010

303.

30%

15.8

1.54

2.98

1.58

20.0

0%68

6.6

1068

.745

.36.

60%

145

.3

2007

Priva

te c

ompa

nies

in h

ealth

car

e su

pply

chai

n31

/12/

2014

5020

109.

919

.70%

7.86

4.94

1.58

9.30

%38

.49

4.3

3.9

10.1

0%3

1.3

2006

Larg

e sc

ale

infra

stru

ctur

e31

/12/

2014

630

2006

507.

90%

40.5

21.

0410

.70%

415.

510

41.6

180.

243

.40%

360

.1

2008

Pan

Afric

an m

ulti-

sect

or fu

nd31

/03/

2014

188

2009

20.4

10.9

0%19

7.35

–3.8

0.66

–ve

128.

25

25.6

8.7

6.80

%1

8.7

3,44

523

26.

70%

180

312,

633

393

6.7

498.

718

.90%

102

4.9

Start Year

Focus

Data as of

Of fund

$ m

# of loans

Average size $ m

$ mFund

Siz

eCo

mm

itmen

tPa

id In

Retu

rnedAf

DB P

artic

ipat

ion Tota

l Fun

d In

vest

men

ts (r

ealis

ed a

nd u

nrea

lised

) at

cos

t Sou

th A

fric

a

$ m

$ m

$ m

TVPI

IRR

$ m

No

Av $ m

$ m

Year

65Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Annex B: Evaluation Questions

Thematic Question: Is the Bank managing itself in order to maximize its performance in SA?

Design

Question 10: To what extent is the quality of the CSP Satisfactory?Question 11: To what extent has the Bank applied selectivity in designing its country portfolio and focused on areas where it brings added valueQuestion 12: To what extent has the Bank been innovative in adapting its approach to the country’s context and development challenges/needs?Question 13: To what extent are the Bank’s interventions coherent and well-coordinated internally?

Knowledge and policy advice

Question 14: To what extent has the Bank actively engaged in and influenced policy dialogue through relevant advice?Question 15: To what extent has the Bank delivered adequate analytical work in support of its interventions, positioning and advice?

Efficiency

Question 8: To what extent are the Bank’s interventions delivered in an efficient manner?Question 9: To what extent are the Bank’s interventions implemented in a timely manner and in compliance with operational standards?

Partnerships and leverage

Question 16: To what extent are the Bank’s interventions harmonized with those of other DPs avoiding duplication, simplifying procedures etc.)?Question 17: To what extent are the Bank’s interventions and resources bringing in other players and being leveraged for maximizing development effectiveness at country level?

Thematic Question: To What extent has the Bank’s engagement in South Africa achieved results?

Relevance

Evaluation question 1: To what extent are the country strategy and Bank operations aligned with RMC development needs; RMC development strategies and priorities; and the needs of beneficiaries?Evaluation question 2: To what extent are the interventions in the country aligned with the Bank’s strategy and priorities?

Effectiveness

Evaluation question 3: To what extent have the Bank’s interventions achieved their expected resultsEvaluation question 4: To what extent have the Bank’s interventions contributed to the achievement of development objectives and expected development results of the country, including impacts (both intended and unintended)?

Sustainability

Evaluation Question 5: To what extent have achieved benefits continued or will be likely to continue once the Bank’s interventions are completed?

Crosscutting issues

Evaluation Question 6: To what extent are the Bank’s interventions inclusive (i.e., bringing prosperity by expanding the economic base across the barriers of age, gender and geography) in terms of gender equality and regional disparity?

Annex C: Methodology

Evaluation design: a theory-based evaluation focused on learning

The evaluation is theory-based. The ToC (Figure C1) for the country is reconstructed based on a desk review of the past 3 strategies and discussions with Bank staff working on South Africa. The PRAs also employed project level ToC. This approach allowed a focus on assessing actual against planned results while identifying where issues occurred along the results chain as well as the assumptions and risks underlying the theory behind the Bank’s approach.


Assumptions

Resource rich middle income country with high levels of inequality

Growth pole for the region, itself caught in low-growth, middle-income trap

Relatively strong central government. Government not interested in borrowing, but large financing needs amongst SoEs

Strong private sector, dominated by large conglomerates

Strong unions in some sectors.

Capacity constraints in local government and SOEs

Access to finance for government, SOEs and private sector available from both domestic and international sources

Appropriate targeting and level of resources

Right instruments available

Complementarity and coordination with other development partners/sources of finance

Bank (and other partners) provide inputs on time and as planned. This includes Bank’s flexibility in re-aligning to sudden changes of government regulations and policies as well as adaptability in its approach to changing market conditions

Partner meets conditions to allow disbursement on time

Bank inputs are well designed (evidence based/ownership/ design) and sequenced

Mainly internal

Implementation on schedule

Design appropriate and scalable

Sound management in partner organisations

Risks monitored and well managed

Procurement systems efficient and effective

Sufficient capacity available to implement

Training/equipment is effective in enhancing capacity

Appropriate institutional support/capacity/policy space to support use of funds

Knowledge work of sufficient quality, and disseminated

Assumptions

InputContext Output

Investment in infrastructure for energy (including green energy)transport, water and sanitation in SA

Investment in infrastructure for transport including rail and air

Investment in major water infrastructure

Lines of Credit and equity to SA institutions for the wider region

Lines of Credit and equity for use in SA

Policy advice and knowledge work focused on knowledge-sharing, infrastructure and regional integration, gender equity and governance

Increased energy generation capacity, including green capacity

Increased freight (rail) transportation capacity and air capacity and services

Increased water and sanitation capacity for people and industry

Increased financial capacity among SA intermediaries to on lend or invest in SMEs in key sectors in the wider region

Increased financial capacity amongst intermediaries to on lend to or invest in SMEs in key sectors in SA

One stop border policy and first posts established

Increased awareness and skills concerning, knowledge-sharing, collaboration, gender equity and good governance

C.1 Theory of Change

Note: Colour coding introduced only at the end of the evaluation to demonstrate those results chains where the Bank made more (green) or less (orange) progress in achieving results.

67Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Assumptions

Mainly internal Mainly external

Effective service delivery infrastructure and processes

Broader investment in infrastrucutre by government and private sector

Regulatory environment encourages take up of access to finance

Knowledge shared is relevant to needs and there is absorption capacity to use it

Good governance

Workforce has requisite skills to support job creation

SA and regional economies not negatively effected by global economy

No major barriers or stumbling blocks to regional integration, trade, and the free flow of people and capital

Government regulation and policies favorable for successful implementation, and as a result the regulatory environment supports business growth

Private sector development supports inclusive and green growth, not unequal growth or growth that is not environmentally sustainable

Political will for transition to green growth

Political and social stability

Assumptions Assumptions

Intermediate outcome Final outcome Impact

Contribution to improved quality of life for South Africans

Inclusive growth and transition to green growth in SA

Improved regional cooperation and integration

Contribution to increased job creation (reduced unemployment)

Contribution to increased entrepreneurial/ private sector activity in SA

Contribution to private sector development in wider region

Contribution to increased trade within the sub-Saharan region and with the BRIC countries

Increased household and business access to secure electricity

Increased household and business access to secure water

Greater volumes of freight transported by rail for import and export

Increased access to finance for business, especially SMEs, in SA

Increased access to finance for business, especially SMEs, in wider region

Intermediaries become part of stronger financial system with sound environmental and social footing

Improved border clearance efficiency

Increased knowledge-sharing, greater gender equity, and strengthened governance frameworks and practices


A multi-layered approach

The evaluation was based on two main layers: (i) project level assessments (PRAs), and (ii) portfolio and strate-gic level assessments. Thirteen projects identified for PRAs effectively covered 76% of the South Africa portfolio by volume, and barring a small number of small projects such as grants and studies, all projects where it would be reasonable to see results. From a results perspective, these projects therefore covered the full population as opposed to a sample. Box C1 explains how these projects were identified. Box C2 summarizes the criteria used in the PRAs. A six point rating scale was used, the detailed guidance on PRA ratings for both public and private sector projects is available from IDEV.

Projects identified for an in-depth results assessment were i) approved between 2004/15; ii) closed and completed or had high disbursement rates that were examined on a case-by-case basis to see whether it would be reasonable to expect results at the time of the evaluation, and iii) above UA 1 million in value. Studies were excluded from the PRA.

Based on this initial list, the evaluation team discussed with the South Africa team and identified the need for two adjustments. The first was the separation of the two public renewable energy projects – one was fully implemented whereas the other was stuck in procurement stages. Originally approved together, they were separated by the Bank in its own records and IDEV agreed to treat the two separately. (The result was that CSP Upington, a solar power project, was removed from the list). Secondly, one LoC, while not well disbursed, was added to the list since the portion that had been disbursed was well advanced and the other portion was held up. Bank staff and GoSA thought the lessons from the operation were important to include. (The results were the addition of the Commercial Bank A project to the list). The 13 projects for PRA were the following:

Project Name Net loan (UA million)

Eskom Renewable Energy – Sere Wind 67.3.3

Eskom – MEDUPI Power Project 1,360.8

Eskom Holdings Limited 355.5

Transnet Ltd 164.6

Commercial Bank A Of South Africa LoC 155.9

Global Trade Liquidity Program – Commercial Bank A of South Africa Ltd. 70.9

Commercial Bank B LoC 70.9

Commercial Bank B Subordinated Loan 61.2

IDC LOC II 141.7

IDC LOC I 21.3

DBSA 4th LoC 70.9

DBSA 5th LoC 212.6

Land and Agricultural Development Bank of South Africa 60.9

2,814.5

Box C1: Criteria for identifying projects for in-depth PRAs

69Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Data collection and analysis

In addition to project level information, data from the wider portfolio and strategic level were assessed together in order to provide an overall rating for the four evaluation criteria of relevance, effectiveness, effi-ciency and sustainability, a rating at overall level was given on a six-point scale (Table C4, at the end of this annex).

A combination of secondary and primary data collection was employed. Data collection methods included: (i) review of Bank documentation, (ii) review of external documentation; (iii) semi-structured interviews, (iv) focus groups. The evaluation matrix (sets out how each technique was used to answer the evaluation questions.

Key informants

One hundred and seventeen key informants were interviewed (Table C3). Two focus groups were held, one with Bank staff at the beginning of the data collection phase and one with community leaders in energy, one of the Bank’s major investment projects. In both cases, IDEV guidance for interviews and focus group conduct were adhered to.

Public sector Private sector

1. Relevance 1a. Relevance of project objectives 1b. Relevance of project design2. Effectiveness 2a. Achievement of outputs 2b. Achievement of outcomes3. Efficiency 3a. Cost benefit 3b. Cost effectiveness 3c. Timeliness 3d. Implementation progress4. Sustainability 4a.Technical soundness 4b. Financial & economic viability 4c. Institutional sustainability & capacity 4d. Political and governance environment 4e. Ownership and sustainability of partnerships 4f. Environmental and social sustainability 4g. Resilience to exogenous factors & risk management

1. Relevance 1a. Relevance of project objectives 1b. Relevance of project design2. Effectiveness 2a. Achievement of outputs 2b. Achievement of outcomes3. Efficiency 3.a Bank investment profitability 3.b Timeliness 3.c Supervision and administration4. Sustainability 4a. Business success 4b. Environmental and social

Box C2: PRA criteria and sub-criteria for public and private sector projects


Government Institutions

Total SOEs (Clients) Total DPs Total #

Financial Institutions/Private Sector (Clients)

Total AfDB Total

National Treasury 27 TRANSNET, IDC, ESKOM, DBSA

45 AFD, WB, USAID, JICA, KFW

15 Commercial Bank A

8 Executive Director, Advisor

2

Department of Public Enterpris-es, Presidency, National Planning (Evaluation Unit)

SANRAL, NERSA IFC, EIB 5 Commercial Bank B

2 Country Team: OPSD, ONEC, ORPF, OFSD, OSHD, GCRD, OWAS, SEOG, CRMU

22

South African Reserve Bank, Department of Transport, Depart-ment of Trade and Industry

Transnet Freight Engineering Group

Commercial Bank A

4 Group Risk Officer/Former Director OPSM; SARC: Economist, CPO; Ex-Director, OIC

Sub Project: ARM Sub Project: PG Bison Group Ltd, project benefi-ciaries

10

Total 27 45 20 24 24

Grand Total 117

Data collection and validation

The data collection phase took place during July through September-November 2015. At inception, IDEV took part in workshops in SARC as part of the Mid Term Review of the current CSP and used the information that was collected to help inform the design of the evaluation. Two different evaluation team missions followed this.

IDEV organized two emerging findings workshops in South Africa in April 2016 to share an overview of findings with (i) external stakeholders including the government and main clients, and (ii) Bank staff involved in the South Africa Program. These workshops were used to fact check, to validate findings, and to identify any gaps in the review.

The analysis of the data that was collected was underwritten by rigorous triangulation. All findings are based on multiple lines of evidence.

Evaluation governance and documentation

The evaluation benefitted from a Reference Group that provided comments at the inception stage and reviewed the evaluation documentation. The evaluation also benefitted from both an external expert reviewer and an inter-nal peer reviewer who focused on the main background documents and final summary report.

This summary report is built on 18 background documents available from IDEV upon request. They include the following: (i) an inception report; (ii) 13 PRAs; (iii) a country template produced primarily for the CEDR; (iv) two working papers, and a technical report. The interview database is not available to ensure confidentiality for indi-vidual informants. Each PRA was quality assured by an internal panel, using standard guidance.

C.3 Overview of Key Informants

71Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Crite

ria/

sub-

crite

ria

High

ly u

nsat

isfa

ctor

yUn

satis

fact

ory

Mod

erat

ely

unsa

tisfa

ctor

yM

oder

atel

y sa

tisfa

ctor

ySa

tisfa

ctor

yHi

ghly

sat

isfa

ctor

y

Rele

vanc

e –

Over

all r

atin

g ba

sed

on c

ombi

ning

a, b

and

c. (

Eval

uato

r jud

gmen

t not

arit

hmet

ic a

vera

ge).

Exte

nt

to w

hich

ob

ject

ives

of (a

) Ban

k’s

CSPs

and

(b)

inte

rven

tions

ar

e al

igne

d w

ith th

e Ba

nk’s

appl

i-ca

ble

sect

or

stra

tegi

es a

nd

bene

ficia

ry

need

s.

(a) M

ost o

r all o

f the

CSP

s in

the

perio

d ex

hibi

t maj

or

shor

tcom

ings

in th

eir a

lignm

ent

with

: i) t

he B

ank’s

CSP

, ii)

ap-

plica

ble

Bank

sec

tor s

trate

gies

, iii)

the

coun

try’s

deve

lopm

ent

stra

tegi

es, a

nd iv

) the

ben

eficia

ry

or c

lient

nee

ds. O

bjec

tives

of

mos

t Ban

k in

terv

entio

ns h

ave

maj

or s

hortc

omin

gs in

thei

r al

ignm

ent w

ith: i

) the

Ban

k’s

CSP,

ii) ap

plica

ble

Bank

sec

tor

stra

tegi

es, i

ii) th

e co

untry

’s de

velo

pmen

t stra

tegi

es, a

nd

iv) b

enefi

ciary

or c

lient

nee

ds.

(a) H

alf o

r mor

e of

the

CSPs

in th

e pe

riod

exhi

bit

maj

or s

hortc

omin

gs in

thei

r al

ignm

ent w

ith: i

) the

Ban

k’s

CSP,

ii) ap

plica

ble

Bank

sec

tor

stra

tegi

es, i

ii) th

e co

untry

’s de

velo

pmen

t stra

tegi

es, a

nd iv

) th

e be

nefic

iary

or c

lient

nee

ds.

(b) O

bjec

tives

of m

ore

than

ha

lf of

Ban

k in

terv

entio

ns h

ave

maj

or s

hortc

omin

gs in

thei

r al

ignm

ent w

ith: i

) the

Ban

k’s

CSP,

ii) ap

plica

ble

Bank

sec

tor

stra

tegi

es, i

ii) th

e co

untry

’s de

velo

pmen

t stra

tegi

es, a

nd iv

) th

e be

nefic

iary

or c

lient

nee

ds.

(a) H

alf o

r mor

e of

the

CSPs

in

the

perio

d ex

hibi

t sho

rtcom

ings

in

thei

r alig

nmen

t with

: i) t

he

Bank

’s CS

P, ii)

appl

icabl

e Ba

nk s

ecto

r stra

tegi

es, i

ii) th

e co

untry

’s de

velo

pmen

t st

rate

gies

, and

iv) t

he b

enefi

ciary

or

clie

nt n

eeds

. (b)

Obj

ectiv

es

of a

sig

nific

ant n

umbe

r (m

ore

than

25%

) of B

ank

inte

rven

tions

wor

k ha

ve m

inor

or

signi

fican

t sho

rtcom

ings

in th

eir

alig

nmen

t with

: i) t

he B

ank’s

CS

P, ii)

appl

icabl

e Ba

nk s

ecto

r st

rate

gies

, iii)

the

coun

try’s

deve

lopm

ent s

trate

gies

, and

iv)

ben

eficia

ry o

r clie

nt n

eeds

.

(a) H

alf o

r mor

e CS

Ps in

the

perio

d ex

hibi

t no

signi

fican

t sh

ortc

omin

gs in

thei

r alig

nmen

t w

ith: i

) the

Ban

k’s C

SP, i

i) ap

pli-

cabl

e Ba

nk s

ecto

r stra

tegi

es, i

ii) th

e co

untry

’s de

velo

pmen

t stra

t-eg

ies,

and

iv) th

e be

nefic

iary

or

clien

t nee

ds. O

bjec

tives

of m

ore

than

hal

f (50

%) o

f int

erve

ntio

ns

revie

wed

hav

e no

not

able

or

only

min

or s

hortc

omin

gs in

thei

r al

ignm

ent w

ith: i

) the

Ban

k’s

CSP,

ii) ap

plica

ble

Bank

sec

tor

stra

tegi

es, i

ii) th

e co

untry

’s de

velo

pmen

t stra

tegi

es, a

nd

iv) th

e be

nefic

iary

nee

ds.

(a) M

ost (

75%

+) C

SPs

in

the

perio

d ex

hibi

t no

nota

ble

shor

tcom

ings

in th

eir a

lignm

ent

with

: i) t

he B

ank’s

CSP

, ii)

appl

icabl

e Ba

nk s

ecto

r st

rate

gies

, iii)

the

coun

try’s

deve

lopm

ent s

trate

gies

, and

iv)

the

bene

ficia

ry o

r clie

nt n

eeds

. (b

) Obj

ectiv

es o

f mos

t (m

ore

than

75%

) of B

ank

inte

rven

tions

ha

ve n

o no

tabl

e sh

ortc

omin

gs

and

the

rem

aini

ng p

roje

cts

with

min

or s

hortc

omin

gs in

the

alig

nmen

t with

: i) t

he B

ank’s

CS

P, ii)

appl

icabl

e Ba

nk s

ecto

r st

rate

gies

, iii)

the

coun

try’s

deve

lopm

ent s

trate

gies

, and

iv)

the

bene

ficia

ry o

r clie

nt n

eeds

.

(a) A

ll CSP

s in

the

perio

d ex

hibi

t no

not

able

sho

rtcom

ings

in th

eir

alig

nmen

t with

: i) t

he B

ank’s

CS

P, ii)

appl

icabl

e Ba

nk s

ecto

r st

rate

gies

, iii)

the

coun

try’s

deve

lopm

ent s

trate

gies

, and

iv)

the

bene

ficia

ry o

r clie

nt n

eeds

. (b

) Obj

ectiv

es o

f all o

f Ban

k in

terv

entio

ns re

view

ed h

ave

no

nota

ble

shor

tcom

ing

in th

eir

alig

nmen

t with

: i) t

he B

ank’s

CS

P, ii)

appl

icabl

e Ba

nk s

ecto

r st

rate

gies

, iii)

the

coun

try’s

deve

lopm

ent s

trate

gies

, and

iv)

the

bene

ficia

ry o

r clie

nt n

eeds

.

(c) E

xten

t to

whi

ch d

esig

n of

inte

r-ve

ntio

ns is

co

nduc

ive to

th

e ac

hiev

e-m

ent o

f sta

ted

obje

ctive

s.

Desig

n of

mos

t Ban

k in

ter-

vent

ions

is n

ot c

ondu

cive

to

achi

evin

g re

sults

. The

orig

inal

de

sign

of m

ost i

nter

vent

ions

(m

ore

than

75%

) was

eith

er

wea

k or

lost

its

rele

vanc

e du

ring

impl

emen

tatio

n;

maj

or a

djus

tmen

ts to

the

scop

e,

impl

emen

tatio

n ar

rang

emen

ts

or te

chni

cal s

olut

ions

wer

e re

quire

d du

ring

impl

emen

tatio

n,

but t

hese

wer

e do

ne w

ith s

ub-

stan

tial d

elay

s w

hich

neg

ative

ly af

fect

ed th

e ac

hiev

emen

t of t

he

inte

nded

out

com

es a

nd o

utpu

ts.

Desig

n of

mor

e th

an h

alf o

f Ba

nk in

terv

entio

ns re

view

ed

is m

argi

nally

con

duciv

e to

ac

hiev

ing

proj

ects

’ res

ults

. The

or

igin

al d

esig

n of

mor

e th

an h

alf

of p

roje

cts

was

eith

er w

eak

or

lost

its

rele

vanc

e du

ring

impl

e-m

enta

tion;

maj

or a

djus

tmen

ts

to th

e sc

ope,

impl

emen

tatio

n ar

rang

emen

ts o

r tec

hnica

l so

lutio

ns w

ere

requ

ired

durin

g im

plem

enta

tion,

but

thes

e w

ere

done

with

sub

stan

tial d

elay

s w

hich

neg

ative

ly af

fect

ed th

e ac

hiev

emen

t of t

he in

tend

ed

outc

omes

and

out

puts

.

Desig

n of

a s

igni

fican

t num

ber

of p

roje

cts

(mor

e th

an 2

5%) i

s so

mew

hat c

ondu

cive

to a

chie

v-in

g pr

ojec

ts’ r

esul

ts. T

he o

rigin

al

desig

n of

a s

igni

fican

t num

ber

of p

roje

cts

(mor

e th

an 2

5%) w

as

eith

er w

eak

or lo

st it

s re

leva

nce

durin

g im

plem

enta

tion;

m

ajor

adj

ustm

ents

to th

e sc

ope,

im

plem

enta

tion

arra

ngem

ents

or

tech

nica

l sol

utio

ns w

ere

requ

ired

durin

g im

plem

enta

tion,

bu

t the

se w

ere

done

with

sub

-st

antia

l del

ays

whi

ch n

egat

ively

affe

cted

the

achi

evem

ent o

f the

in

tend

ed o

utco

mes

and

out

puts

.

Desig

n of

mor

e th

an h

alf o

f pr

ojec

ts is

larg

ely

cond

ucive

to

ach

ievin

g pr

ojec

ts re

sults

. Th

e ot

hers

wer

e m

oder

atel

y co

nduc

ive to

ach

ievin

g pr

ojec

ts

resu

lts. M

ore

than

hal

f of p

ro-

ject

s ha

ve a

sol

id o

rigin

al d

esig

n an

d re

mai

ned

appr

opria

te

thro

ugho

ut im

plem

enta

tion

and

did

not r

equi

re a

ny o

r req

uire

d m

inor

adj

ustm

ents

to th

e sc

ope,

im

plem

enta

tion

arra

ngem

ents

or

tech

nica

l sol

utio

ns w

ere

requ

ired

to e

nsur

e ou

tput

an

d ou

tcom

e ac

hiev

emen

t.

Desig

n of

mos

t (m

ore

than

75%

) of

pro

ject

s is

fully

con

duciv

e to

ac

hiev

ing

proj

ects

’ res

ults

and

th

e de

sign

of th

e re

mai

ning

25

% is

larg

ely

cond

ucive

to

achi

evem

ent o

f pro

ject

s re

sults

. Th

e m

ajor

ity (m

ore

than

75%

) of

proj

ects

had

a s

olid

orig

inal

de-

sign

and

rem

aine

d ap

prop

riate

th

roug

hout

impl

emen

tatio

n an

d di

d no

t req

uire

any

sig

nific

ant

adju

stm

ents

to th

e sc

ope,

im

plem

enta

tion

arra

ngem

ents

or

tech

nica

l sol

utio

ns to

ens

ure

the

achi

evem

ent o

f the

inte

nded

ou

tcom

es a

nd o

utpu

ts.

Desig

n of

all p

roje

cts

revie

wed

is

fully

con

duciv

e to

ach

ievin

g pr

ojec

ts’ p

lann

ed re

sults

. The

or

igin

al d

esig

n w

as s

olid

and

re-

mai

ned

appr

opria

te th

roug

hout

im

plem

enta

tion;

no

adju

stm

ents

to

the

scop

e, im

plem

enta

tion

arra

ngem

ents

or t

echn

ical

solu

tions

wer

e re

quire

d to

en

sure

the

achi

evem

ent o

f the

in

tend

ed o

utco

mes

and

out

puts

.

Tabl

e C.

4 CS

PE L

evel

Rat

ing

Scal

e


Crite

ria/

sub-

crite

ria

High

ly u

nsat

isfa

ctor

yUn

satis

fact

ory

Mod

erat

ely

unsa

tisfa

ctor

yM

oder

atel

y sa

tisfa

ctor

ySa

tisfa

ctor

yHi

ghly

sat

isfa

ctor

y

Effe

ctive

ness

Ove

rall

ratin

g ba

sed

on c

ombi

ning

a, b

and

c (E

valu

ator

judg

men

t not

arit

hmet

ic a

vera

ge).

(a) E

xten

t to

whi

ch

expe

cted

ou

tput

s ha

ve b

een

achi

eved

, or

are

on

track

to b

e ac

hiev

ed.

Only

a fe

w o

r no

Bank

in

terv

entio

ns re

view

ed h

ave

achi

eved

or a

re li

kely

to

achi

eve

the

inte

nded

out

put

base

d on

the

late

st v

alue

of

the

(PAR

) ind

icat

ors

and

the

anal

ysis

of o

ther

rele

vant

risk

s/fa

ctor

s an

d as

sum

ptio

ns.

A m

inor

ity (l

ess

than

25%

) Ba

nk in

terv

entio

ns re

view

ed

have

ach

ieve

d or

are

like

ly to

ac

hiev

e th

e in

tend

ed o

utpu

t ba

sed

on th

e la

test

val

ue o

f th

e (P

AR) i

ndic

ator

s an

d th

e an

alys

is o

f oth

er re

leva

nt ri

sks/

fact

ors

and

assu

mpt

ions

.

Less

than

hal

f of B

ank

inte

rven

tions

revie

wed

hav

e ac

hiev

ed o

r are

like

ly to

ac

hiev

e th

e in

tend

ed o

utpu

t ba

sed

on th

e la

test

val

ue o

f th

e (P

AR) i

ndic

ator

s an

d th

e an

alys

is o

f oth

er re

leva

nt ri

sks/

fact

ors

and

assu

mpt

ions

.

Half

or m

ore

of B

ank

inte

r-ve

ntio

ns h

ave

achi

eved

or a

re

likel

y to

ach

ieve

inte

nded

out

-pu

ts b

ased

on

the

late

st v

alue

of

the

(PAR

) ind

icat

ors

and

the

anal

ysis

of o

ther

rele

vant

risk

s/fa

ctor

s an

d as

sum

ptio

ns.

Mos

t Ban

k in

terv

entio

ns (m

ore

than

75%

) hav

e ac

hiev

ed

or a

re li

kely

to a

chie

ve th

e in

tend

ed o

utpu

ts b

ased

on

the

late

st v

alue

of t

he (P

AR)

indi

cato

rs a

nd th

e an

alys

is

of o

ther

rele

vant

risk

s/fa

c-to

rs a

nd a

ssum

ptio

ns.

All B

ank

inte

rven

tions

re

view

ed h

ave

achi

eved

or a

re

likel

y to

ach

ieve

the

inte

nded

ou

tput

bas

ed o

n th

e la

test

val

-ue

of t

he (P

AR) i

ndic

ator

s an

d th

e an

alys

is o

f oth

er re

leva

nt

risks

/fact

ors

and

assu

mpt

ions

.

(b) E

xten

t to

whi

ch

expe

cted

in

term

edia

te

outc

omes

ha

ve b

een

achi

eved

, or

are

on

track

to b

e ac

hiev

ed.

Only

a fe

w o

r no

Bank

in

terv

entio

ns re

view

ed h

ave

achi

eved

or a

re li

kely

to

achi

eve

the

inte

nded

out

com

es

base

d on

the

late

st v

alue

of

the

(PAR

) ind

icat

ors

and

the

anal

ysis

of o

ther

rele

vant

risk

s/fa

ctor

s an

d as

sum

ptio

ns.

A m

inor

ity (l

ess

than

25%

) Ba

nk in

terv

entio

ns re

view

ed

have

ach

ieve

d or

are

like

ly to

ac

hiev

e th

e in

tend

ed o

utco

mes

ba

sed

on th

e la

test

val

ue o

f th

e (P

AR) i

ndic

ator

s an

d th

e an

alys

is o

f oth

er re

leva

nt ri

sks/

fact

ors

and

assu

mpt

ions

.

Less

than

hal

f of B

ank

inte

rven

tions

revie

wed

hav

e ac

hiev

ed o

r are

like

ly to

ac

hiev

e th

e in

tend

ed o

utco

mes

ba

sed

on th

e la

test

val

ue o

f th

e (P

AR) i

ndic

ator

s an

d th

e an

alys

is o

f oth

er re

leva

nt ri

sks/

fact

ors

and

assu

mpt

ions

.

Half

or m

ore

of B

ank

inte

r-ve

ntio

ns h

ave

achi

eved

or a

re

likel

y to

ach

ieve

inte

nded

out

-co

mes

bas

ed o

n th

e la

test

val

-ue

of t

he (P

AR) i

ndic

ator

s an

d th

e an

alys

is o

f oth

er re

leva

nt

risks

/fact

ors

and

assu

mpt

ions

.

Mos

t Ban

k in

terv

entio

ns (m

ore

than

75%

) hav

e ac

hiev

ed

or a

re li

kely

to a

chie

ve th

e in

tend

ed o

utco

mes

bas

ed

on th

e la

test

val

ue o

f the

(P

AR) i

ndic

ator

s an

d th

e an

alys

is o

f oth

er re

leva

nt ri

sks/

fact

ors

and

assu

mpt

ions

.

All B

ank

inte

rven

tions

re

view

ed h

ave

achi

eved

or

are

like

ly to

ach

ieve

the

inte

nded

out

com

es b

ased

on

the

late

st v

alue

of t

he

(PAR

) ind

icat

ors

and

the

anal

ysis

of o

ther

rele

vant

risk

s/fa

ctor

s an

d as

sum

ptio

ns.

(c) E

xten

t to

whi

ch B

ank

has

achi

eved

re

sults

ou

tsid

e of

its

lend

ing

portf

olio

.

Non-

lend

ing

aspe

cts

of

the

Bank

’s a

ctivi

ties

have

pr

oduc

ed fe

w g

ood

resu

lts in

th

e fo

llow

ing

area

s: k

now

ledg

e w

ork,

pol

icy

dial

ogue

, bu

ildin

g pa

rtner

ship

s an

d le

vera

ging

add

ition

al fu

nds.

Non-

lend

ing

aspe

cts

of th

e Ba

nk’s

act

ivitie

s ha

ve p

ro-

duce

d so

me

patc

hy (o

r bet

ter)

good

resu

lts in

at l

east

2 o

f th

e fo

llow

ing

area

s: k

now

ledg

e w

ork,

pol

icy

dial

ogue

, bu

ildin

g pa

rtner

ship

s an

d le

vera

ging

add

ition

al fu

nds.

Non-

lend

ing

aspe

cts

of

the

Bank

’s a

ctivi

ties

have

pr

oduc

ed s

ome

good

re

sults

in a

t lea

st 2

of t

he

follo

win

g ar

eas:

kno

wle

dge

wor

k, p

olic

y di

alog

ue,

build

ing

partn

ersh

ips

and

leve

ragi

ng a

dditi

onal

fund

s.

Non-

lend

ing

aspe

cts

of

the

Bank

’s a

ctivi

ties

have

pr

oduc

ed b

road

ly go

od

resu

lts in

at l

east

3 o

f the

fo

llow

ing

area

s: k

now

ledg

e w

ork,

pol

icy

dial

ogue

, bu

ildin

g pa

rtner

ship

s an

d le

vera

ging

add

ition

al fu

nds.

Non-

lend

ing

aspe

cts

of

the

Bank

’s a

ctivi

ties

have

pr

oduc

ed b

road

ly go

od

resu

lts in

at l

east

4 o

f the

fo

llow

ing

area

s: k

now

ledg

e w

ork,

pol

icy

dial

ogue

, bu

ildin

g pa

rtner

ship

s an

d le

vera

ging

add

ition

al fu

nds.

Non-

lend

ing

aspe

cts

of

the

Bank

’s a

ctivi

ties

have

pr

oduc

ed c

onsi

sten

tly

good

resu

lts in

all

of th

e fo

llow

ing

area

s: k

now

ledg

e w

ork,

pol

icy

dial

ogue

, bu

ildin

g pa

rtner

ship

s an

d le

vera

ging

add

ition

al fu

nds.

(Con

tinua

tion)

73Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Crite

ria/

sub-

crite

ria

High

ly u

nsat

isfa

ctor

yUn

satis

fact

ory

Mod

erat

ely

unsa

tisfa

ctor

yM

oder

atel

y sa

tisfa

ctor

ySa

tisfa

ctor

yHi

ghly

sat

isfa

ctor

y

Effic

ienc

y Ov

eral

l rat

ing

base

d on

com

bini

ng a

and

b (E

valu

ator

judg

men

t not

arit

hmet

ic a

vera

ge).

Exte

nt to

w

hich

cos

t ef

ficie

ncy

is

achi

eved

or

on tr

ack

to

be a

chie

ved.

A fe

w o

r no

Bank

revie

wed

ha

ve m

et o

r are

on

track

to,

mee

t or e

xcee

d co

st e

ffect

ive-

ness

or e

quiva

lent

indi

cato

rs.

Stak

ehol

ders

per

ceive

a

broa

dly

cost

effi

cien

t Ban

k.St

akeh

olde

rs p

erce

ive a

co

st in

effic

ient

Ban

k.

Som

e Ba

nk in

terv

entio

ns (l

ess

than

25%

) rev

iew

ed h

ave,

or

are

on

track

to, m

eet o

r ex

ceed

cos

t effe

ctive

ness

or

equ

ivale

nt in

dica

tors

.St

akeh

olde

rs p

erce

ive a

co

st in

effic

ient

Ban

k.

Less

than

Hal

f of B

ank

inte

rven

tions

revie

wed

hav

e,

or a

re o

n tra

ck to

, mee

t or

exce

ed c

ost e

ffect

ivene

ss

or e

quiva

lent

indi

cato

rs.

Stak

ehol

ders

per

ceive

a B

ank

that

is n

ot fu

lly c

ost e

ffect

ive.

Half

or m

ore

Bank

inte

r-ve

ntio

ns re

view

ed h

ave,

or

are

on tr

ack

to, m

eet o

r ex

ceed

cos

t effe

ctive

ness

or

equ

ivale

nt in

dica

tors

.St

akeh

olde

rs p

erce

ive a

br

oadl

y co

st e

ffici

ent B

ank.

Mos

t Ban

k in

terv

entio

ns

revie

wed

(75%

or m

ore)

hav

e,

or a

re o

n tra

ck to

, mee

t or

exce

ed c

ost e

ffect

ivene

ss

or e

quiva

lent

indi

cato

rs.

Stak

ehol

ders

per

ceive

a

cost

effi

cien

t Ban

k.

All B

ank

inte

rven

tions

revie

wed

ha

ve, o

r are

on

track

to, m

eet

or e

xcee

d co

st e

ffect

ivene

ss

or e

quiva

lent

indi

cato

rs.

Stak

ehol

ders

per

ceive

a

high

ly co

st e

ffici

ent B

ank.

Exte

nt to

w

hich

tim

e ef

ficie

ncy

is

achi

eved

or

is o

n tra

ck to

be

ach

ieve

d.

Few

or n

o Ba

nk in

terv

entio

ns

(few

er th

an 1

0%) r

evie

wed

ha

ve m

et o

r are

on

track

to

mee

t im

plem

enta

tion

targ

et d

ates

. All

or a

lmos

t al

l are

beh

ind

sche

dule

.Th

e w

ider

por

tfolio

sho

ws

dela

ys fo

r all

or a

lmos

t the

en

tire

portf

olio

in p

roje

ct p

ro-

cess

ing

and

impl

emen

tatio

n.St

akeh

olde

rs p

erce

ive a

tim

e in

effic

ient

Ban

k.

A m

inor

ity o

f Ban

k in

terv

en-

tions

revie

wed

(few

er th

an

25%

) hav

e m

et o

r are

on

track

to m

eet i

mpl

emen

tatio

n ta

rget

dat

es. T

he m

ajor

ity

are

behi

nd s

ched

ule.

The

wid

er p

ortfo

lio s

how

s de

lays

for t

he m

ajor

ity o

f the

po

rtfol

io in

pro

ject

pro

cess

ing

and

impl

emen

tatio

n.St

akeh

olde

rs p

erce

ive a

tim

e in

effic

ient

Ban

k.

Less

than

hal

f of B

ank

inte

r-ve

ntio

ns re

view

ed h

ave

met

or

are

on

track

to m

eet i

mpl

e-m

enta

tion

targ

et d

ates

. Mor

e th

an h

alf a

re b

ehin

d sc

hedu

le.

The

wid

er p

ortfo

lio s

how

s de

lays

for m

ore

than

hal

f the

po

rtfol

io in

pro

ject

pro

cess

ing

and

impl

emen

tatio

n.St

akeh

olde

rs p

erce

ive a

Ban

k th

at is

not

fully

tim

e-ef

ficie

nt.

Half

or m

ore

of B

ank

inte

rven

-tio

ns re

view

ed h

ave

met

or

are

on tr

ack

to m

eet o

r exc

eed

impl

emen

tatio

n ta

rget

dat

es.

The

wid

er p

ortfo

lio s

how

s de

lays

in fo

r les

s th

an h

alf t

he

portf

olio

in p

roje

ct p

roce

ssin

g an

d im

plem

enta

tion.

Stak

ehol

ders

per

ceive

br

oadl

y tim

e ef

ficie

nt B

ank.

Mos

t Ban

k in

terv

entio

ns (7

5%

or m

ore)

Ban

k in

terv

entio

ns

revie

wed

hav

e m

et o

r are

on

trac

k to

mee

t or e

xcee

d im

plem

enta

tion

targ

et d

ates

The

wid

er p

ortfo

lio s

how

s on

ly a

few

del

ays

in p

roje

ct p

ro-

cess

ing

and

impl

emen

tatio

n.St

akeh

olde

rs p

erce

ive a

tim

e ef

ficie

nt B

ank.

All B

ank

inte

rven

tions

re

view

ed h

ave

met

or a

re

on tr

ack

to m

eet o

r exc

eed

impl

emen

tatio

n ta

rget

dat

esTh

e w

ider

por

tfolio

sho

ws

min

imal

or n

o de

lays

in

pro

ject

pro

cess

ing

and

impl

emen

tatio

n.St

akeh

olde

rs p

erce

ive a

hi

ghly

time

effic

ient

Ban

k.

(Con

tinua

tion)


Crite

ria/

sub-

crite

ria

High

ly u

nsat

isfa

ctor

yUn

satis

fact

ory

Mod

erat

ely

unsa

tisfa

ctor

yM

oder

atel

y sa

tisfa

ctor

ySa

tisfa

ctor

yHi

ghly

sat

isfa

ctor

y

Sust

aina

bilit

y Ov

eral

l rat

ing

base

d on

com

bini

ng a

and

b (E

valu

ator

judg

men

t not

arit

hmet

ic a

vera

ge).

(a) E

xten

t to

whi

ch

tech

nica

l an

d fin

anci

al

sust

aina

bilit

y of

ben

efits

is

like

ly.

It is

hig

hly

likel

y th

at th

e ac

hiev

emen

t of t

he re

sults

of

the

vast

maj

ority

of i

nter

ven-

tions

will

be a

dver

sely

affe

cted

by

fact

ors

rela

ted

to a

ny o

f: th

e te

chni

cal d

esig

n of

the

proj

ect,

the

finan

cial

sus

tain

abilit

y, or

inst

itutio

nal c

apac

ity

It is

like

ly th

at th

e ac

hiev

emen

t of

the

resu

lts o

f mos

t pro

ject

s w

ill be

adv

erse

ly af

fect

ed b

y fa

ctor

s re

late

d to

any

of:

the

tech

nica

l des

ign

of th

e pr

ojec

t, th

e fin

anci

al s

usta

inab

ility,

or in

stitu

tiona

l cap

acity

It is

like

ly th

at th

e ac

hiev

emen

t of

resu

lts o

f hal

f of t

he p

roje

cts

will

be a

dver

sely

affe

cted

by

fact

ors

rela

ted

to a

ny o

f: th

e te

chni

cal d

esig

n of

the

proj

ect,

the

finan

cial

sus

tain

abilit

y, or

inst

itutio

nal c

apac

ity

It is

unl

ikel

y th

at th

e ac

hiev

e-m

ent o

f the

resu

lts o

f mos

t pr

ojec

ts w

ill be

adv

erse

ly af

-fe

cted

by

fact

ors

rela

ted

to a

ny

of: t

he te

chni

cal d

esig

n of

the

proj

ect,

the

finan

cial

sus

tain

a-bi

lity,

or in

stitu

tiona

l cap

acity

.

It is

unl

ikel

y th

at th

e ac

hiev

emen

t of t

he re

sults

of

the

vast

maj

ority

of p

roje

cts

will

be a

dver

sely

affe

cted

by

fact

ors

rela

ted

to a

ny o

f: th

e te

chni

cal d

esig

n of

the

proj

ect,

the

finan

cial

sus

tain

abilit

y, or

inst

itutio

nal c

apac

ity.

It is

hig

hly

unlik

ely

that

the

achi

evem

ent o

f the

resu

lts

of a

ll pr

ojec

ts (7

5% o

r mor

e)

will

be a

dver

sely

affe

cted

by

fact

ors

rela

ted

to a

ny o

f: th

e te

chni

cal d

esig

n of

the

proj

ect,

the

finan

cial

sus

tain

abilit

y, or

inst

itutio

nal c

apac

ity.

(b) E

xten

t to

whi

ch

envir

onm

en-

tal a

nd s

ocia

l su

stai

nabi

lity

achi

evem

ents

ar

e lik

ely.

Mos

t Ban

k in

terv

entio

ns (7

5%

or m

ore)

hav

e no

t put

in p

lace

an

y m

echa

nism

s fo

r eco

nom

ic

and

finan

cial

sus

tain

abilit

y, an

d th

e flo

w o

f env

ironm

enta

l and

so

cial

ben

efits

ass

ocia

ted

with

th

e pr

ojec

t are

not

exp

ecte

d to

con

tinue

afte

r com

plet

ion;

an

d th

e m

ajor

ity p

rodu

ced

nega

tive

unin

tend

ed e

nviro

n-m

enta

l or s

ocia

l im

pact

s.

Half

of B

ank

inte

rven

tions

or

less

hav

e a

few

mec

hani

sms

for e

cono

mic

and

fina

ncia

l su

stai

nabi

lity,

but t

hey

are

not e

xpec

ted

to b

e su

ffici

ent

to e

nsur

e th

e co

ntin

ued

flow

of

env

ironm

enta

l and

soc

ial

bene

fits

asso

ciat

ed w

ith th

e pr

ojec

t afte

r com

plet

ion;

an

d ha

lf or

mor

e pr

oduc

ed

nega

tive

unin

tend

ed e

nviro

n-m

enta

l or s

ocia

l im

pact

s.

Mos

t Ban

k in

terv

entio

ns (7

5%

or m

ore)

hav

e a

few

mec

ha-

nism

s fo

r eco

nom

ic a

nd fi

nan-

cial

sus

tain

abilit

y, bu

t the

y ar

e no

t exp

ecte

d to

be

suffi

cien

t to

ensu

re th

e co

ntin

ued

flow

of

envir

onm

enta

l and

; som

e pr

o-du

ced

nega

tive

unin

tend

ed e

n-vir

onm

enta

l or s

ocia

l im

pact

s.

Mos

t Ban

k in

terv

entio

ns

(75%

or m

ore)

and

mor

e ha

ve s

uffic

ient

mec

hani

sms

for e

cono

mic

and

fina

ncia

l su

stai

nabi

lity

that

are

dee

med

su

ffici

ent t

o en

sure

the

cont

in-

ued

flow

of e

nviro

nmen

tal a

nd

soci

al b

enefi

ts a

ssoc

iate

d w

ith

the

proj

ect a

fter c

ompl

etio

n;

and

prod

uced

few

sig

nific

ant

nega

tive

unin

tend

ed e

nviro

n-m

enta

l or s

ocia

l im

pact

s.

Alm

ost a

ll Ba

nk in

terv

entio

ns

have

in p

lace

suf

ficie

nt

mec

hani

sms

for e

cono

mic

an

d fin

anci

al s

usta

inab

ility

that

are

dee

med

suf

ficie

nt

to e

nsur

e th

e co

ntin

ued

flow

of

env

ironm

enta

l and

soc

ial

bene

fits

asso

ciat

ed w

ith th

e pr

ojec

t afte

r com

plet

ion;

an

d pr

oduc

ed n

o si

gnifi

cant

ne

gativ

e un

inte

nded

env

iron-

men

tal o

r soc

ial i

mpa

cts.

Mos

t pro

ject

s (7

5% o

r m

ore)

hav

e in

pla

ce ro

bust

m

echa

nism

s fo

r eco

nom

ic a

nd

finan

cial

sus

tain

abilit

y th

at a

re

very

like

ly to

ens

ure

the

con-

tinue

d flo

w o

f env

ironm

enta

l an

d so

cial

ben

efits

ass

ocia

ted

with

the

proj

ect a

fter c

ompl

e-tio

n; a

nd p

rodu

ced

no n

otab

le

nega

tive

unin

tend

ed e

nviro

n-m

enta

l or s

ocia

l im

pact

s.

(Con

tinua

tion)

75Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Annex D: Country Context: Economic and Development Indicators

Table D1. South Africa: Selected Macroeconomic Indicators

Indicators Unit 2000 2011 2012 2013 2014 2015(e) 2016(p)

National Accounts

GNI at Current Prices Million US $ 140,976 368,273 403,677 395,817 366,990 … …

GNI per Capita US$ 3,140 7,050 7,640 7,410 6,800 … …

GDP at Current Prices Million US $ 132,964 416,802 397,348 366,060 349,819 308,502 280,622

GDP at 2000 Constant prices

Million US $ 132,964 192,931 197,214 201,577 204,651 207,226 208,761

Real GDP Growth Rate % 4.2 3.2 2.2 2.2 1.5 1.3 0.7

Real per Capita GDP Growth Rate

% 2.6 2.0 1.1 1.1 0.5 0.3 –0.2

Gross Domestic Investment % GDP 15.9 19.1 20.1 20.1 20.4 20.6 21.6

Public Investment % GDP 4.3 6.9 7.1 7.0 7.1 7.2 7.6

Private Investment % GDP 11.6 12.2 13.0 13.1 13.3 13.4 14.0

Gross National Savings % GDP 15.2 16.3 13.6 13.0 13.3 15.4 16.3

Prices and Money

Inflation (CPI) % 7.7 5.0 5.6 5.7 6.1 4.6 6.8

Exchange Rate (Annual Average)

local currency/US $

6.9 7.3 8.2 9.7 10.9 12.8 15.0

Monetary Growth (M2) % 61.9 11.4 4.7 5.7 7.9 7.6 …

Money and Quasi Money as % of GDP

% 81.6 121.5 118.0 115.0 115.6 119.9 …

Government Finance

Total Revenue and Grants % GDP 23.8 27.3 27.3 27.9 27.4 25.0 22.4

Total Expenditure and Net Lending

% GDP 25.8 30.9 31.4 31.7 31.2 28.5 24.8

Overall Deficit (-)/Surplus (+)

% GDP –2.0 –3.6 –4.1 –3.8 –3.6 –3.9 –3.3

External Sector

Exports Volume Growth (Goods)

% 8.9 4.3 0.1 4.6 2.6 3.4 3.0

Imports Volume Growth (Goods)

% 7.7 10.5 6.0 1.8 -0.5 2.2 2.4

Terms of Trade Growth % –0.3 46.6 –8.4 –8.1 –6.7 –6.3 -2.4

Current Account Balance Million US $ –172 –8,996 –19,694 –21,105 –19,041 –13,361 –11,476

Current Account Balance % GDP –0.1 –2.2 –5.0 –5.8 –5.4 –4.3 –4.1

External Reserves months of imports

2.3 4.2 4.3 4.4 4.6 4.6 …


Debt and Financial Flows

Debt Service % exports 61.0 27.3 31.4 37.3 42.3 42.6 41.3

External Debt % GDP 27.8 28.4 35.7 37.3 43.6 42.3 42.3

Net Total Financial Flows Million US $ –494 7,822 5,339 –373 6,005 … …

Net Official Development Assistance

Million US $ 486 1,395 1,066 1,295 1,070 … …

Net Foreign Direct Investment

Million US $ 887 4,243 4,559 8,300 5,712 … …

Real GDP Growth Rate, 2004–2016

0

–1

−2

1

2

3

4

5

6

%

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

Sources: AfDB Statistics Department; IMF: World Economic Outlook, October 2015 and International Financial Statistics, October 2015; AfDB Statistics Department: Development Data Portal Database, March 2016. United Nations: OECD, Reporting System Division.

Note: …: Data Not Available. (e) Estimations (p) Projections.

77Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Inflation (CPI), 2004–2016

0

2

4

6

8

10

12

%

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

Current Account Balance as % of GDP, 2004–2016

2004

2005

2006

2007

2008

2009

2010

2011

2012

2013

2014

2015

2016

–6

–7

–5

–4

–3

–2

–1

0


Table D2. Comparative Socio-Economic Indicators

Year South Africa Africa Developing countries

Developed countries

Basic Indicators

Area ('000 Km²) 2015 1,219 30,067 80,386 53,939

Total Population (millions) 2015 54.5 1,184.5 5,945.0 1,401.5

Urban Population (% of Total) 2015 63.6 39.7 47.0 80.7

Population Density (per Km²) 2015 44.9 40.3 78.5 25.4

GNI per Capita (US $) 2014 6,800 2,045 4,226 38,317

Labor Force Participation* – Total (%) 2015 52.8 66.3 67.7 72.0

Labor Force Participation** – Female (%) 2015 44.9 56.5 53.0 64.5

Gender-Related Development Index Value 2007–2013 0.680 0.801 0.506 0.792

Human Develop. Index (Rank among 188 countries)

2014 116 ... ... ...

Popul. Living Below $ 1.25 a Day (% of Population)

2008–2013 9.4 39.6 17.0 ...

Demographic Indicators

Population Growth Rate – Total (%) 2015 1.0 2.6 1.3 0.6

Population Growth Rate – Urban (%) 2015 1.4 3.6 2.6 0.8

Population < 15 years (%) 2015 29.2 41.0 28.3 17.3

Population ≥ 65 years (%) 2015 5.0 3.5 6.2 16.0

Dependency Ratio (%) 2015 52.1 80.1 54.6 50.5

Sex Ratio (per 100 female) 2015 96.8 100.1 102.8 97.4

Female Population 15–49 years (% of total population)

2015 26.9 24.0 25.8 23.0

Life Expectancy at Birth – Total (years) 2015 57.7 61.2 68.9 79.1

Life Expectancy at Birth – Female (years) 2015 59.5 62.6 70.8 82.1

Crude Birth Rate (per 1,000) 2015 20.4 34.8 21.0 11.6

Crude Death Rate (per 1,000) 2015 12.4 9.3 7.7 8.8

Infant Mortality Rate (per 1,000) 2015 33.6 52.2 35.2 5.8

Child Mortality Rate (per 1,000) 2015 40.5 75.5 47.3 6.8

Total Fertility Rate (per woman) 2015 2.3 4.6 2.6 1.7

Maternal Mortality Rate (per 100,000) 2013 140.0 411.3 230.0 22.0

Women Using Contraception (%) 2014 64.5 35.3 62.1 ...

79Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Education Indicators

Gross Enrolment Ratio (%)

Primary School – Total 2011–2014 100.8 106.4 109.4 101.3

Primary School – Female 2011–2014 98.1 102.6 107.6 101.1

Secondary School – Total 2011–2014 110.8 54.6 69.0 100.2

Secondary School – Female 2011–2014 114.4 51.4 67.7 99.9

Primary School Female Teaching Staff (% of Total)

2012–2014 78.9 45.1 58.1 81.6

Adult literacy Rate – Total (%) 2006–2012 93.7 61.8 80.4 99.2

Adult literacy Rate – Male (%) 2006–2012 95.0 70.7 85.9 99.3

Adult literacy Rate – Female (%) 2006–2012 92.6 53.4 75.2 99.0

Percentage of GDP Spent on Education 2009–2012 6.6 5.3 4.3 5.5

Environmental Indicators

Land Use (Arable Land as % of Total Land Area) 2013 10.3 8.6 11.9 9.4

Agricultural Land (as % of land area) 2013 79.8 43.2 43.4 30.0

Forest (As of Land Area) 2013 7.6 23.3 28.0 34.5

Per Capita CO2 Emissions (metric tons) 2012 9.0 1.1 3.0 11.6

GNI Per Capita US $

2000

2005

2008

2009

2010

2011

2012

2013

2014

Africa South Africa

0

1000

2000

3000

4000

5000

6000

7000

8000

Sources: AfDB Statistics Department Databases; World Bank: World Development Indicators; UNAIDS; UNSD; WHO, UNICEF, UNDP; Country Reports. Last update: november 2015.

Note: n.a.: Not Applicable; …: Data Not Available. * Labor force participation rate, total (% of total population ages 15+) † Labor force participation rate, female (% of female population ages 15+)


Infant Mortality Rate (Per 1000)

0

20

40

60

80

100

2000

2005

2009

2010

2011

2012

2013

2014

2015

Africa South Africa

2.5

Population Growth Rate (%)

0

0.5

1

1.5

2

3

South Africa Africa

2000

2005

2009

2010

2011

2012

2013

2014

2015

Table D3. Progress Toward Achieving The Millennium Development Goals

1990¹ 2000² 2014³

Goal 1: Eradicate extreme poverty and hunger

Employment to population ratio, 15+, total (%) 42.2 40.4 39.2

Malnutrition prevalence, weight for age (% of children under 5) 10.1 8.7 ...

Poverty headcount ratio at $1,25 a day (PPP) (% of population) 21.4 13.7 9.4

Prevalence of undernourishment (% of population) 5.3 5.0 5.0

Goal 2: Achieve universal primary education

Literacy rate, youth female (% of females ages 15–24) 94.3 98.8 99.3

Literacy rate, adult total (% of people ages 15 and above) 82.4 92.9 93.7

Primary completion rate, total (% of relevant age group) 84.8 98.5 ...

Total enrollment, primary (% net) 91.9 89.6 ...

Goal 3: Promote gender equality and empower women

Proportion of seats held by women in national parliaments (%) 30.0 44.5 44.8

Ratio of female to male primary enrollment 96.8 95.1 94.9

Ratio of female to male secondary enrollment 113.0 103.0 106.9

81Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Sources: ADB Statistics Department Databases; World Bank: World Development Indicators; UNAIDS; UNSD; WHO, UNICEF, WRI, UNDP; Country Reports, last update: January, 2017.

Note: n,a,: Not Applicable; …: Data Not Available, * Latest year available in the period 1990–1999; † Latest year available in the period 2000–2009; ‡ Latest year available in the period 2010–2014

Goal 4: Reduce child mortality

Immunization, measles (% of children ages 12–23 months) 75.0 78.0 66.0

Mortality rate, infant (per 1,000 live births) 50.5 40.0 32.8

Mortality rate, under-5 (per 1,000) 71.0 63.4 43.9

Goal 5: Improve maternal health

Births attended by skilled health staff (% of total) 84.4 91.2 ...

Contraceptive prevalence (% of women ages 15–49) 56.9 62.9 64.6

Maternal mortality ratio (modeled estimate, per 100,000 live births) 140.0 160.0 140.0

Goal 6: Combat HIV/AIDS, malaria, and other diseases

Incidence of tuberculosis (per 100,000 people) 509.0 967.0 860.0

Prevalence of HIV, female (% ages 15–24) ... ... 11.9

Prevalence of HIV, male (% ages 15–24) ... ... 5.3

Prevalence of HIV, total (% of population ages 15–49) 13.2 18.9 19.1

Goal 7: Ensure environmental sustainability

CO2 emissions (kg per PPP $ of GDP) 3.0 2.6 2.4

Improved sanitation facilities (% of population with access) 64.3 72.2 74.4

Improved water source (% of population with access) 86.0 93.2 95.1

Goal 8: Develop a global partnership for development

Net total ODA/OA per capita (current US$) 12.2 21.1 24.5

Internet users (per 1000 people) 41.2 100.0 410.0

Mobile cellular subscriptions (per 1000 people) 117.4 912.5 1305.6

Telephone lines (per 1000 people) 124.3 84.9 76.9


Employment to population ratio, 15+, total (%) Primary completion rate, total

Ratio of female to male primary enrollment Mortality rate, infant (per 1000 live births)

0.0 0

150

0 0

20

40

60

50

100

100

150

50

1990 1990

1990 2000 20131990 2000 2012–14

20002000 2013

20.0

40.0

60.0

Progress Toward Achieving The Millennium Development Goals (continuation)

83Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Maternal mortality ratio (modeled estimate, per 100,000 live births)

Incidence of tuberculosis (per 100,000 people)

Improved water source (%) Mobile cellular subscriptions (per 1000 people)

0.0

0 0

500

1000

1500

20

40

60

80

100

0.0

50

100

150

50

100

150

200

1990

1990 19902000 20002012 2012

1990 2000 20132000 2013


Anne

x E:

Tim

e Effi

cien

cy

Tabl

e E1

. Tim

elin

ess

of o

pera

tions

Sector

Transaction

Loan amount (UA millions)

Key

Date

sDu

ratio

n

PEN/PCN

Board Approval

Loan Agreement

1st Disb.

Final Disb.

PEN/PCN to BA (months)

BA to LA (months)

LA to 1st Disb. (months)

Fina

nce

DBSA

LoC

IV71

.115

/3/2

006

21/7

/200

621

/6/2

007

12/8

/08

15/1

2/08

4.2

1113

.7

DBSA

LoC

V21

3.3

15/9

/200

93/

2/20

1117

/10/

2011

28/3

/201

24/

9/20

1316

.68.

45.

3

IDC

LoC

I10

.415

/9/2

003

5/11

/200

426

/11/

2004

11/3

/200

511

/3/2

005

13.7

0.7

3.4

IDC

LoC

II14

2.2

na19

/5/2

010

26/5

/201

128

/9/2

011

23/5

/201

4n/

a12

.24.

1

Com

mer

cial

Ban

k A

LoC

58.3

24/1

0/20

1120

/6/2

012

10/9

/201

24/

4/20

1319

/10/

2015

7.9

2.7

6.8

Com

mer

cial

Ban

k B

– su

bord

inat

ed d

ebt

15

/4/2

007

28/6

/200

7na

6/7/

2007

6/7/

2007

2.4

nana

Com

mer

cial

Ban

k B

LoC

71.1

15/5

/200

811

/9/2

008

27/1

1/20

086/

1/20

0919

/2/2

010

3.9

2.5

1.3

Com

mer

cial

Ban

k A

GTLP

6915

/7/2

009

8/9/

2009

na15

/12/

0915

/12/

091.

8na

na

Com

mer

cial

Ban

k A

LoC

156.

4na

11/9

/200

810

/11/

2008

3/6/

2009

28/1

1/20

11na

26.

7

Aver

age

Fina

nce

7.2

5.6

5.9

85Annexes

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Pow

er/e

nerg

yEs

kom

Hol

ding

s 35

7.9

16/2

/200

728

/6/2

007

10/1

1/20

0830

/12/

2008

NA4.

316

.41.

6

Esko

m M

edup

i (€)

733.

528

/7/2

009

25/1

1/20

0911

/12/

2009

11/8

/201

0NA

3.9

0.5

8

Esko

m M

edup

i (ZA

R)55

1.3

28/7

/200

925

/11/

2009

11/1

2/20

0911

/8/2

010

NA3.

90.

58

Esko

m S

ere

Win

d Fa

rm32

.228

/7/2

009

30/5

/201

125

/9/2

011

27/6

/201

3NA

223.

921

Esko

m S

ere

Win

d Fa

rm C

TF35

.828

/7/2

009

30/5

/201

125

/9/2

011

5/3/

2013

NA22

3.9

17.3

Esko

m U

ping

ton

CSP

157.

528

/7/2

009

30/5

/201

125

/9/2

011

NA

223.

9

Esko

m U

ping

ton

CSP

CTF

35.8

28/7

/200

930

/5/2

011

25/9

/201

1

NA22

3.9

Xina

Sol

ar O

ne45

.615

/5/2

014

23/6

/201

413

/2/2

015

11/3

/201

5NA

1.3

7.7

0.9

Xina

Sol

ar O

ne C

TF27

.915

/5/2

014

23/6

/201

413

/2/2

015

11/3

/201

5NA

1.3

7.7

0.9

Tran

spor

tTr

ansn

et14

0.2

15/1

/201

023

/6/2

010

5/10

/201

127

/3/2

012

NA5.

215

.45.

7

Tran

snet

Loa

n II

179

15/6

/201

415

/12/

2014

10/2

/201

6

NA6

21

Min

ing

Kala

gadi

Man

gane

se10

615

/7/2

010

18/5

/201

17/

1/20

1412

/6/2

014

NA10

.131

.65.

1

Kala

gadi

sta

ndby

faci

lity

38.6

15/7

/201

018

/5/2

011

7/1/

2014

NA

10.1

31.6

Wat

er &

San

Inte

grat

ed W

ater

Har

vest

ing

0.3

14

/4/2

009

27/8

/200

912

/1/2

010

NA

8

IWH

addi

tion

0.1

10

/11/

2011

26/3

/201

22/

5/20

12NA

4.

51.

2

Mul

tiple

/Use

Wat

er S

ervic

es1.

1

22/7

/201

45/

11/2

014

23/9

/201

5NA

3.

5

O&M

Sch

ool S

anita

tion

0.9

12

/12/

2014

NA

Com

ms

Broa

dban

d In

fraco

MIC

Fun

d0.

8

21/2

/201

44/

7/20

1423

/10/

2014

31/1

2/20

05

4.4

8

Mul

ti/se

ctor

One

Stop

Bor

der P

ost P

olic

y0.

2

22/1

2/20

1129

/11/

2013

11/7

/201

431

/3/2

015

23

.2

Stat

Cap

Bui

ld P

rog

Phas

e II

0.5

7/

7/20

1128

/11/

2011

16/4

/201

4

4.

7

Aver

age

Non/

Fina

nce

Loan

s10

.310

.64.

7

Aver

age

Non/

Fina

nce

Gran

ts8

0.2

Tabl

e E1

. Tim

elin

ess

of o

pera

tions

(Con

tinua

tion)


Table E2. Time Efficiency in MIC Grant Administration

Stage Projects

From Board approval to grant signature Fastest: Statistical Capacity SCB II (4.7 months) and Integrated water harvesting, AWF (8 months).Slowest: One Stop Border Post study, MIC (23.2 months).

From grant agreement to first disbursement Fastest: Integrated water harvesting (1.2 months) AWF and Broadband Infraco, MIC (9 months).Slowest: Statistical Capacity Building, MIC (33.7 months) and One Stop Border Post Study, MIC (32 months).

From Board approval to first disbursement Overall ranged from 9 months as fastest processing time (Integrated water harvesting and Broadband Infraco (BBI) to 33.7 months as the slowest. (Statistical Capacity Building Project II).

For grant-funded projects, while one might expect a smaller gap between the different processing stages than in the loan projects, the recorded ranges of 9–32 months elapsed was too long and inefficient given the relative simplicity of the transactions and small amounts involved. Interviews with both task managers, government officials and implementation staff highlighted the following as major reasons for the long delays: ❙ Bank engagement with client: a) inadequate communication lines to provide the necessary information on requirements for fulfillment of grant

conditions, b) Slow response by Bank on clients’ request for clarifications and frequent changes in templates/requirements; c) bureaucratic hurdles in Bank processes, particularly disbursement, procurement, and legal requirements that do not favor country context. Recent example is the Bank requesting a signature at ministerial level for USD 1.2 million to support a tertiary institution.

❙ Clients’ ability to meet conditions because of misinformation or misunderstanding of elements or conditions to be met.

87

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

References

1. Based on net loan figures for all projects approved 2004–14, the portfolio measured UA 3.19 billion.

2. In fact since some of the remaining portfolio is now threatened with cancellation, the PRAs cover an even greater share of the portfolio where we would expect to see results.

3. http://www.gcis.gov.za/content/resourcecentre/contactdirectory/government-structures-and-parastatals

4. http://www.statssa.gov.za/?cat=30

5. At a workshop with AfDB (the mid-term review of the CSP) and with government officials in July 2015, the latter explained the government’s limited appetite for foreign borrowing, citing the current debt-to-income ratio as one reason. This was later confirmed in interviews with the National Treasury. Note that contingent liabilities include the debt of SOEs.

6. According to OECD DAC statistics, the figure is as low as 0.4% although some multilateral contributions are not fully listed. Of this, nearly half went to health and population.

7. The Gini index was 57.8 in 2000 and 65.0 in 2013.

8. GoSA refers to the triple threat of poverty, inequality, and unemployment. Government activity in these areas is evidenced in spending reviews: GOSA spending on welfare as a proportion of GDP is comparable to that of many Northern European countries, according to a cross-country comparison by Bosch et al. 2010. http://sds.ukzn.ac.za/files/WP%2058%20web.pdf

9. Confirmed in interview with National Treasury procurement team, Department of Trade (DTI), AfDB and EIB.

10. Eskom defines load shedding as an intentionally engineered partial electrical power shutdown to prevent a total blackout.

11. President Jacob Zuma, Parliamentary address, 6 August 2015.

12. Two coal-powered plants (Medupi and Kusile), one pumped storage facility (Ingula) and one concentrated solar power plant (Upington).

13. Various corridor initiatives are underway in SADC: those serving developed cargo bases include Trans-Caprivi, Trans-Cunene and Trans-Kalahari stretching inland from Walvis Bay and the Maputo corridor linking Mozambique with South Africa; those running south from Dar es Salaam earmarked as high-growth corridors while the Nacala, Beira, Mtwara and Lobito will become high-growth corridors in the long term in response to the rapid development of mining activities in SADC.

14. National Treasury, 2011 Local Government Budgets and Expenditure Review 2006/07–2012/13.

Endnotes


15. State of Logistics Survey, 2013.

16. CSIR Annual Report, 1997.

17. National Water Resource Strategy, 2013, Department of Water Affairs.

18. http://www.worldbank.org/en/news/press-release/2015/10/20/commodity-prices-expected-to-drop-across-the-board-in-2015-led-by-energy-price-declines

19. Integrated Annual Report 2012, IDC.

20. Standard, First Rand, ABSA (Barclays owned) and Nedbank together account for 83% of South Africa’s commercial banking sector as measured by total assets.

21. JSE eighteenth in the world in 2011 https://www.mfw4a.org/south-africa/financial-sector-profile.html

22. https://www.imf.org/external/pubs/ft/scr/2015/cr1555.pdf

23. CSP 2003/05 was updated allowing extension until 2007; CSP 2008/12; CSP 2013/17.

24. Relabeling point based on document review; comment also made in CSP quality-of-entry.

25. Bank Medium Term Strategy, Private Sector strategy Update, Ten Year Strategy, Financial Sector development Strategy, Energy Policy. For GOSA: National development Plan, ASGISA, the Medium Term Strategic Framework, IRP, National Water Resources Strategy, etc.

26. South Africa documentation for CSP QAE 2014. Aspects mentioned are those for which a rating of unsatisfactory or moderately unsatisfactory was given. Other aspects received either moderately satisfactory or satisfactory.

27. Net loan approvals at Nov 2015 rates. (Revised to include Dec 2015 approval of supplementary loan to Eskom).

28. Figures do not include other December 2015 approval to ESKOM (portfolio covered is up to end 2014).

29. The Bank made the following approvals in 2015: MIC Grant of USD 1.2m (790,000 UA) for Enterprise Development Project and a second private sector corporate loan to ESKOM (USD 268.88 million).

30. The second transport project (Transnet II) was signed in 2016, having been approved in 2014.

31. A small MIC grant with the University of Cape Town has been under negotiation for some time. In April 2016, it was still unsigned.

32. At appraisal, 11 of the 26 projects included minimal or more reference to a capacity development or TA role. See section on cross cutting issues, in part 4.

89

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Endnotes

33. Based on analysis of data provided by Bank PE team on the “underlying investment” locations of the Bank’s full PE portfolio. The Bank has no direct equity in South Africa.

34. Interviews with the national treasury confirm that this approach will continue with recent economic deterioration.

35. FAPA Grant: A multi-donor thematic trust fund that provides grant funding for technical assistance and capacity building to support implementation of the Bank’s Private Sector Development Strategy.

36. This is part of ESTA’s broader program for statistical capacity building seen in most RMCs, not specific to South Africa.

37. DBSA – FAPA grant was planned but dropped by DBSA due to transaction costs, and Land Bank for which part of the loan was for capacity purposes but was not used and later reallocated.

38. 2015–16 loan appraisals for ESKOM show greater effort to coordinate in planning and appraising from the beginning.

39. Engaged government departments (NT, DPME) the two main SOEs the Bank is financing and private sector clients (notably Commercial Bank B, DBSA, Land Bank) raised this issue.

40. Land Bank, Commercial Bank B sub-debt, Commercial Bank A GTLP, Commercial Bank C, Sere wind, CSP Upington, Xina Solar, Transnet 1, O+M School Sanitation

41. The Bank support Xina Solar, which was approved, disbursed and completed relatively quickly. REIPP has been a broadly successful experience for the Government. Those running the program asked the evaluation team if the Bank might be able to help them collate and share lessons from these experiences – both for South Africa’s and other countries. After connecting the relevant persons it is not yet known if the Bank has been able to provide this support, though it aligns well with the sharing knowledge objectives of the current CSP.

42. 7 of the 9 finance projects for which PRAs were conducted focused on on-lending outside of South Africa. The two exceptions were old and relatively small operations (IDC approved in 2004, not fully utilized) and a new line to Land Bank that received a sovereign guarantee (the only LoC in the past decade with this advantage). IFC for example prefers its South Africa LOCs to be on-lent in South Africa, targeting SMEs ETC.

43. Bujagali Dam in Uganda and the New Dawn Satellite Project were supported through Commercial Bank B, and directly by AfDB. This means the same borrower was receiving Bank funds directly and indirectly, with an added intermediary providing different conditions.

44. Two products counted were annual corporate products, i.e. annual flagship reports for each country: CPIA and the African Economic Outlook. The remaining products included economic working papers, policy and economic analysis.


45. Staff and GoSA lamented the slow speed of the MIC process, which is confirmed by milestone dates.

46. These included (i) One Stop Border Posts study intended to help GoSA develop another approach; (ii) a grant to develop a strategy for BBI, a SOE.

47. Including the use of MIC grants to fund a transport strategy and a grant for the University of Cape Town.

48. View almost unanimously expressed in interviews with internal and external stakeholders.

49. It was impossible to obtain a full list of events including workshops on lending products, private sector evaluation, economic context, etc.

50. Both parties confirm the strength of the relationship in interviews, records of meetings, and participant observation.

51. Confirmed in interview with Bank staff and management and the Treasury, and the experience of seeing how relationships work. In some cases, it has taken years to obtain grants, with hold ups on both sides, partly because the grants require Ministerial signature, even where it is not for a ministry, as per Bank current legal requirements: the University of Cape Town is a case in point.

52. ECAs and the China Development Bank: the project involved purchasing Chinese manufactured locomotives.

53. Approved in late 2015, this project was not included in the portfolio under review but it reflected the work the Bank is currently undertaking and the most recent levels of investor interest.

54. Noted in interview with Bank staff/management, confirmed by GoSA NT, that they see AfDB as not pushing their agenda, unlike the bilateral agenda. This was also evident in business development meetings minutes.

55. DPs include EIB, KfW, and IFC, among others.

56. They now conduct joint monitoring missions, including joint feedback sessions and aide memoires for the Medupi Project. (Based on participant observation, interview and document review).

57. Sere wind has been implemented successfully and is providing power to the grid. Concentrated Solar Power (CSP) Upington is still delayed. Technology and procurement issues have led to the serious delays.

58. Confirmed in interviews with NT, AfDB staff and with other DPs (EIB).

59. Interviews with EIB and IFC.

60. AfDB support to the Land Bank included an unused capacity development component. One reason given is that the AFD was already providing such support and the Bank’s addition was not designed to specifically complement existing work as confirmed in interviews with the Land Bank and in monitoring documentation

91

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Endnotes

provided by the Bank portfolio management team. The task manager responsible for originating the loan claimed that he was unable to recall project details and therefore could not be interviewed.

61. IDEV evaluating efficiency across 169 projects in 14 countries; 13 projects were in South Africa.

62. Public and private sector projects are assessed different: the latter focus on profitability and most assessments were positive.

63. Cost escalations are estimated at 42% currently, however the project is still set to be more cost effective than alternatives because (i) coal remains cheap/cheaper than at the time of appraisal; (ii) alternative energy generation to plug gaps including diesel burning remain more expensive; and (iii) cost increases are passed on to the consumer, which has a knock on effect from outcomes for economic growth, as opposed to cost effectiveness in a strict sense.

64. In one case where AfDB and IFC were joint funding, a change was required to the agreement that IFC could agree at management level, while the client (Commercial Bank B) had to wait for AfDB approval (non-objection) at management level and then the Board, slowing the ability to disburse.

65. Interviews with current and former members of SARC management.

66. “Southern Africa Region Staffing Needs”, internal document, October 2012.

67. In relation to CSP and Sere (client: ESKOM, DP: KFW). The procurement team also proactively explained the Bank’s new procurement policy.

68. Raised in interview with staff, management and separately in staff focus group.

69. The Mid-term review of the decentralization road map 2011–2015 found that overall, mission costs had increased (by 10%) rather than decreased up to the end of 2013.

70. IDEV evaluation 2015, “Strategizing for the Africa we want”.

71. Raised again as a challenge by NT in 2016.

72. See IDEV evaluations: 2015 on implementation of Bank commitments raising the levels of progress in improving project supervision in public and private sector operations, previous evaluations – most recently the 2015 evaluation of support to SMEs; and the 2012 evaluation of non-sovereign operations.

73. Raised by two staff members at the 2015 inception workshop, and confirmed with OPSD 4 management.

74. PRAs for private sector projects include a sub-criterion of supervision quality unlike public sector PRAs. This relates to differences in agreed international standards. IDEV is looking to increase the alignment between the two approaches in its 2017 evaluations.

75. Interview with IFC South Africa, and provision of IFC monitoring documents.


76. In the case of Land Bank, AfDB internal reporting claimed 318 jobs created and other outcomes from a factory funded by an AfDB loan. The evaluation team was able to verify a good development story associated with the wood-processing factory, however it was built long before the Land Bank loan supported by the AfDB line of credit. It was in fact to refinance a large corporate that nominally allocated the funds to the factory meeting Land Bank’s priority areas, though the money is fungible. IDEV did not agree that it was appropriate to claim jobs creation and other benefits created years before Bank funding was implicated.

77. Figure considered conservative estimate by ESKOM, it covers various teams visiting different sites.

78. For this reason (and to enable participant observation) IDEV’s two site visits (Sere wind farm, Medupi power station) were coordinated with a scheduled joint monitoring mission, although additional meetings were also held with the client and stakeholders.

79. Medupi predates the 2012 energy policy which emphasizes clean coal but does not rule out coal generation projects.

80. Of 14 countries reviewed for the CEDR (including SA), 98% achieved moderately satisfactory or higher ratings.

81. Despite repeated requests IACD provided no information to IDEV regarding its investigation.

82. For Transnet the main indicator chosen at project design for output was volume of funds spent on buying new trains, which tells us nothing about how many trains were built or value for money, this would normally be considered an activity not an output.

83. Many have not seen or do not recall seeing Bank PARs and log frames; they work on the basis of loan agreements that generally contain less detail and fewer conditions and targets.

84. For ongoing projects effectiveness is assessed based on the current trajectory and likelihoods of achievement against planned outputs and outcomes, see methodology annex.

85. Interviews with Bank staff and other DP staff.

86. As explained in the methodology section, projects were selected for in-depth assessment on the basis of various criteria including a minimal level of disbursement, as it was not appropriate to review progress towards results before inputs were provided. Since then SARC is considering cancelling both projects because of the intractable delays.

87. In several interviews with clients, interviewees claimed not to have seen the log frames included in the PARs and a review of PARs versus loan agreements for the 11 private sector projects highlights the differences.

88. Bujagali Dam and New Dawn Satellite.

93

An ID

EV C

ount

ry S

trat

egy

Eval

uatio

n

Endnotes

89. It is important to note that the lines of credit in the SA portfolio brought outcome scores down, as will be explained. Such lines of credit are not well represented in 12 of the other 13 countries covered by the CEDR analysis used as a comparator.

90. Based on analysis of underlying investment data provided by central equity team.

91. 74% of projects reviewed across 14 countries, covering the same period, achieved MS+ for the composite sustainability rating.

92. An independent consultant engaged by World Bank used this particular term in an interview.

93. The Bank’s investigation is acknowledged for having encouraged closer environmental monitoring by the Bank and ESKOM.

94. AfDB was not the only IFI pushing to apply the Equator Principles during the period, but Commercial Bank A and Commercial Bank B explicitly referred to the Bank’s interest.

95. IDEV 2015 evaluation “Strategizing for the Africa we want”.

96. A 2009 paper on the national gender machinery and gender based violence. There is no evidence to show that this paper was used for policy dialogue or in Bank operations.

Independent Development EvaluationAfrican Development Bank





About this Publication

This evaluation examines the African Development Bank’s engagement and support to South Africa, through its programs and strategies during the period 2004–2015. The assessment sought to provide credible evaluative evidence on the development results of the Bank’s assistance. It also identified factors both internal and external that affect good or poor performance while drawing lessons from the performance to inform future operations in South Africa and potentially in other middle income countries. The Bank’s programs in South Africa over the period covered two main pillars: infrastructure (especially energy) and finance (mostly lines of credit). The operations involved the use of both the public and private sector windows of financing. The Bank’s strategies also included emphasis on non-lending products such as knowledge work.

The assessment draws on different sources of information (such as desk reviews, extensive consultation with key stakeholders, and field visits), and uses both qualitative and quantitative analytical methods. Overall, the assessment revealed, among others, that the Bank is learning to adapt to the developed, competitive South African market. The findings and conclusions indicate that current policies and practices are inadequate if the Bank wants to remain relevant and grow its portfolio in South Africa. Hence a more tailored approach, appropriate resourcing and a broader menu of innovative support, are required going forward.

idev.afdb.org


Layo

ut &

pro

duct

ion:

Vis

ual I

dent

ity –

ww

w.v

isua

liden

tity.

co.u

k

African Development Bank GroupAvenue Joseph Anoma, 01 BP 1387, Abidjan 01, Côte d’IvoirePhone: +225 20 26 20 41E-mail: [email protected]

http://idev.afdb.org/

mailto:idevhelpdesk%40afdb.org%20?subject=

Evaluation of the Bank’s Country Strategy and Program 2004...

Documents

Transcript of Evaluation of the Bank’s Country Strategy and Program 2004...