·==================== =============;:;:;:;:;:;:=:=:::::::::::::::::::;:;:;r~:=~=;:::::::::::=======================================:::::::================;===;==~;====~=~=~=~=~=~:~:~r===~=~=~;:;::~t~:~/~~~~~=~~~~r~~~~r~rr~=~=~= ........ =:=:=:·:::·:=:=:=:::=:=:=:=:·:=:·:·:=:=:·:=:=:=:=:::=::::::::::::::::::~:~:::: ... ::::::::::: ... ::::::::::::::::::::::::::::~:::::::?.:.~:::::~:~:~:::::::::::::::::::::::: ... ::::::=:::===·==:=:=:=:=:=:=:=:=:=:=:====:

International Development Association

Improving Project Implementation

IDAlO Technical Note No. 8 April 1992

1. In response to the request by IDAlO Deputies at their meeting in January 1992, this note describes IDA's recent and ongoing efforts to improve the implementation of its projects .

2. It is the borrower's responsibility to implement projects. IDA's role, as set out in the Articles of Agreement, is to "make arrangements to ensure that the proceeds of any financing are used only for the purposes for which the financing was provided, with due attention to considerations of economy, efficiency and competitive international trade ... " The activities involved in fulfilling this mandate are called supervision.

3. Over the last ten years, IDA has diversified its lending and now supports virtually every sector of economic and social development using a variety of instruments . This diversification has increased the demands on the borrowers' capacity to implement projects. The challenge has been heightened recently by difficult world economic conditions, as well as rapid political change and economic adjustment in many of IDA's

··.·.·.·.·.::::::: ... :;: ... · ·················-·.:::::::

recipient countries. These factors have an important impact on how well projects are implemented. Good supervision cannot overcome them, but it can play a crucial role in helping project authorities respond to changing conditions. The supervision process is an important way of identifying country-wide or sectoral problems that need to be taken up in the broader country dialogue. Moreover, the findings of supervision are used systematically to improve the design of new projects . IDA management has therefore recently taken a number of steps to improve the supervision process and thereby the implementation of projects.

Performance of the IDA Portfolio

4. IDA routinely monitors the performance of its portfolio. Every project under implementation is regularly rated by IDA supervision missions who visit the project site. The project's overall status, along with several individual performance indicators , is rated on a scale of 1 to 4. Projects

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Page 2

whose overall status is 3 or 4 are considered problem projects. 1 The ratings are aggregated periodically for regions, sectors and the entire portfolio. At the end of FY91, 76 per cent of the 930 IDA and IDA/IBRD blend projects were proceeding satisfactorily, with the remainder considered problem projects. As Table 1 shows, the portfolio as a whole had a rating of 2.1, with the Asia and Latin America and Caribbean regions performing significantly better than the other two regions.

that is encountered in all regions. With too few qualified staff, along with low wages that engender low morale, many government agencies face serious constraints in designing, implementing and monitoring projects. Moreover, technical agencies are sometimes subject to bureaucratic interference or unnecessary regulation, making it difficult for them to maintain rational staffing levels and efficient services. Even when there are no such fundamental obstacles to success, virtually every project encounters unexpected snags: floods,

Table 1: Status of IDA and IDA/IBRD Blend Projects in FY9l

Africa

Asia

EMENA ..

. :•:LAC••>•:•••••••• ••••• •· ... ................. . .. . ...... .............. .... ... ... . . . . . . . . . . . . . . . . . . . . . . . . . . . ..... ... ... .. .............. .

·.• TOTAL ........ .

Number of IDA and Blend Projects

531

267

86

46

930

5. All projects in developing countries risk encountering problems of varying kinds. In the lowest income countries, fundamental economic and administrative weaknesses inevitably increase these risks, and this is reflected in the generally poorer performance of IDA's projects in Africa. The most common cause of problem projects is the borrowing government's failure to provide its share of project financing, called counterpart funds. In some cases, the government's revenues have not kept pace with its commitments, and in others the procedures for budgeting and releasing funds are disorganized and cumbersome. The weakness of borrowers' public sector agencies is a second frequent implementation problem--one

Overall Status Rating

2.2

1.9

2.2

Percentage of Problem Projects

29

14

28

LL.

24

droughts, engineering difficulties, personnel changes, contractors' mistakes, and delayed deliveries occur routinely. These events need to be handled promptly if they are to be kept from generating major delays that push a project into the "problem" category.

6. As Table 2 shows, the proportion of problem projects has increased during the past three years. This trend is largely accounted for by the sluggish world economy and a generalized decline in the price of primary tropical commodities like coffee, tea, most edible oils and fats and rubber. For projects aimed at increasing the production of these commodities, a sudden or

!/ The ratings are defined as follows : 1 =no significant problems; 2=moderate problems; 3 =major problems but appropriate actions are being taken to address them; 4=major problems not being adequately addressed by the borrower.

Page3

Table 2: Trend in Performance of IDA and Blend Projects

Fiscal Year

89 <>·· ·.·.·.··.· 9(}

91

IDA & Blend Projects

918

929. ·.··

930

sustained drop in world prices can spell serious difficulty or even failure; if the price drops below production costs, farmers lose the incentive to expand production. Moreover, the temporary oil price increases caused by the Gulf crisis led to serious economic problems that exacerbated the budgetary weaknesses described in paragraph 5 for some 18 low and middle income countries. Finally, civil strife has affected an unusual number of borrowers during the last eighteen months. The disruptions in Cameroon, Congo, Ethiopia, Liberia, Sierra Leone, Somalia, Sudan and Zaire have caused severe delays or suspension in the implementation of IDA projects.

7. The impact of these recent events has been felt particularly in the performance of projects in Africa, which accounted for most of the deterioration in the overall portfolio during the past three years. In view of Africa's especially difficult environment for project implementation and the recent downward trend in performance, IDA's Africa region has introduced many special initiatives to improve project implementation. These initiatives supplement the Bank-wide procedures and practices that are described in the remainder of this note.

Improving Project Implementation

8. IDA uses three complementary approaches to improving implementation. First, it ensures that the supervision process does everything possible to identify and handle problems in

Percentage of Problem Projects

12.2 ...

19;5

24.o ··

ongoing projects. Second, it helps the borrower deal with underlying economic and institutional weaknesses that cause implementation problems. This is done through the macroeconomic policy dialogue, public expenditure reviews, support for structural adjustment and direct technical assistance. Finally, IDA staff use the findings of supervision missions and project completion reports to design future projects so as to ensure smoother implementation.

a) Strengthening Supervision

9. The supervision process is IDA's principal means of identifying and helping borrowers solve implementation problems. Supervision is effective at overcoming technical and design problems that crop up during implementation, because IDA staff help borrowers decide how to change--for instance--engineering designs, equipment specifications, cropping patterns or program targets. IDA procedures can be clarified and disagreements among project agencies resolved. While supervision alone cannot usually solve macroeconomic, political or sector-wide constraints on project implementation, it helps project authorities cope with the impact of these problems.

10. IDA management places high priority on using supervisiOn to improve project implementation. In early 1992, Mr. Preston observed that "Successful implementation ofBank­financed operations far outweighs new commitments as the principal indicator of the

Page 4 ;:;:;:;:::::;:;:;:;:;:;:;::::::=!:::;:;:;:;:::::::::::::::::::::::::::::::::::::::::::;:;:;:;:;:;:;:::::;:;:;:;:;:;:;:;:;:::;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:::::::::;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:; :;:;:;: ;:; :::;:::;:;: ;:;:;:;:;:;:;:;:;:;:;:;:::::::;:;:;:;:::::::::::::::;:;:;:;:;:::::::::;:;:;:;:;:;:;:;:;:;:;:;:;::::::.~:;:;:;:;:;:;:;:;:;:;:::::::::::;:;:::;:;:;:;:;:;:::::::;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:;:::::::;.)":!;:;:;."::O.::~-:::.-::;:$;::>.::

Bank's development effectiveness ... Our efforts to strengthen project supervision have begun to pay off, but more must be done." Accordingly, he has established a task force under the chairmanship of a former senior vice-president to take stock of the system of portfolio management and evaluation. The task force will review the role of various units within the Bank, the methods used to assess success and failure, and the means of disseminating implementation experience. Its full terms of reference are attached. It is expected to report to the President by I ul y 1, 1992, and its findings will be presented to the Executive Directors.

11. Supervision involves many kinds of activities: reviewing the Government's progress on policy reforms; checking the quality of physical works and the execution of contracts; monitoring the progress of studies and research and the implications of their findings, and checking compliance with legal covenants and rules on accounts, reports and procurement procedures. Projects are typically visited twice a year by a full mission from headquarters. Between missions, staff of IDA's field offices follow up with project authorities and headquarters staff communicate frequently by facsimile , telephone and electronic mail.

12. In the last two years, IDA has allocated more resources to supervision. The average time spent on supervision for all projects increased from about 12 staff weeks per project in FY90 to about 13 in FY91. The average for the Africa region was even higher: 14.4 staff weeks . Most of the increase was concentrated on investment, as opposed to adjustment operations, and particularly on problem projects. Managers give more frequent or intensive attention to projects in their early stages, or whose implementing agencies are weak, or that are experiencing difficulties. IDA's field offices are handling an increasing share of supervision and have hired more local professional staff for this purpose. In the Asia region, nearly one-quarter of the supervision in FY91 was done by local staff and consultants.

13. Several steps have been taken to improve the quality of supervision. Project appraisal missions now have to agree with the borrower on a supervision plan that includes the timing and objectives for the first few supervision missions and the expertise required. To correct the perception that work on new lending is valued more highly than supervision work, management has been instructed to give more emphasis to supervision effectiveness in staffs annual performance evaluations and promotion decisions. In the Africa region, a checklist of 'best practices' in supervision has been prepared and widely disseminated to staff. Box 1 lists several other new approaches developed in the Africa region to improve the effectiveness of supervision.

14. For many projects, IDA organizes a project launch workshop for borrowers' staff. Soon after the project is approved, an IDA team visits the country to train project staff and help them set up forms and systems. The workshops usually focus heavily on procurement procedures, because mistakes and delays can hold up the arrival of key equipment or consultants, or the construction of facilities. Many workshops also cover accounting, disbursement, auditing, reporting, annual budgeting and planning. The Africa region is implementing a new Action Program for Improvements in Procurement to upgrade the attention to procurement issues during project preparation. Many country departments have worked with their borrowers to prepare standard bidding documents, to avoid having to write new documents for each tender.

15. To supplement regular supervision, IDA organizes Country Implementation Reviews (CIRs) . These are meetings involving senior IDA managers--usually department directors--and borrowers' officials--usually ministers--to identify and solve implementation problems like procurement delays, shortage of counterpart funds, donor coordination and performance of technical assistance. That is, they focus on selected generic issues rather than the performance of individual projects. They are held in the borrowing country

Page 5

Box 1: The Africa Region's Innovations in Supervision

IDA's Africa Region has introduced new procedures to enhance the effectiveness of supervision; They include:

·· ·· Thematic supervision: A specialized staff member; like an expert in agricultu@l exterisipij;

. ··········••••••·su~ryises ·similar components ·of seyeral •. p~oJe<:ts··· ()n••.the •• sa.m:~•·• rlli~~iOilf •········•·•·•••••·•···• > •.•. ·.···•·•·• : ••••••<•••••·•·•·•·•·• ····-:.::::···.::··":>·:::·:-. .:.:::-:-:·-::·· -·-··· . ::-:·.:.::·- ·:···· _··· ·. ·--:-::·-:-::-.. _-... ··· ··:::::->:::::::::::::·-:-:::::-:::::::-::·::·:::::-:.:::.···:·:·_·:·::.·::-:·:. .. ··::·-::::·::-::·--.-::-:·--:):::::·:::::::::::::::::::::::_:

• Mid~year management reviews: The .. Regio~al Vice-President m~ts at ~id~y~ 'With ea.6~ }

de(m.rtment· to review implementatioll···issues"··.··· •••························ •·••••••••·••··· •·•·•••·•••••••••••••••••·•••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••••• • Workshop on Country Implementation Review~ ~ A Regionalwork~hop < \"\;~ held fol' st!f.fi to share experiences on the organization and focus.<()fCIRs; .

. :... · ... : .. - .. . ... ·: .. . ....... .:::::::_:::-: .. :>:. . .. · _· _. . ... _ ....... ::::::::::::::::::::::::

• Review of project ratings: The members of the country team review the performance tilting~ : that task managers assign to projects. This helps ensure that ratings are consistent and t~a( < appropriate weight is given to country economic •. factors.

. .. , ........

• Monthly workshops on implementation: Regional staff are invited to hear speakers and < exchange information on implementation issues and good supervision practices. ·· · · ··

at intervals varying from once every few years to once a year. In FY91, for example, CIRs were held in China, Guinea, Madagascar, Kenya, Ghana, Yemen and Pakistan. A recent internal assessment of CIRs concluded that they are a very effective way to handle implementation issues, provided both sides' representatives are sufficiently senior and that the CIR's timing and objectives are carefully tailored to the country's situation.

16. A full-scale review of a project midway through its implementation allows IDA and the borrower to assess progress, reflect on changes in the external environment, and change the project design if needed. They may decide to reallocate funds among activities, to cancel components or even cancel the whole project. Moreover, the review can serve as a good springboard for discussion of future operations in the same sector. These mid-term reviews were occasionally used in the past for large or complex projects but are now routinely used for many types of projects. In the Africa region, they are now mandatory for all new investment projects, and detailed guidelines have been issued on how to conduct them.

17. The Bank's management has recently mounted a special effort to encourage borrowers to submit in a more timely manner the annual audit reports on project accounts that are required by all project legal agreements. The system for monitoring the receipt of audit reports has been improved, and each region has designated a lead audit adviser to look into problems with compliance. If a project audit report is significantly overdue, IDA applies sanctions such as suspending disbursement on the project, suspending use of its revolving fund, or delaying an extension of the closing date until the report is received . In Africa, where many government auditing agencies are weak, IDA provides long­term institutional development assistance to several borrowers, and is also encouraging borrowers to use qualified auditors from the private sector.

18. Each year the regions assess the performance of their portfolios. These reports are the basis of the Annual Report on Implementation and Supervision (ARIS), which is an important management tool for systematic stock-taking of the portfolio. The ARIS for FY91,

Page 6

which was discussed by Executive Directors in February 1992, includes an action plan for FY92-93 that is intended to forestall further deterioration in performance. The plan envisages increased use of CIRs; greater recourse to cancellation and disbursement suspensions when needed; more staff training on supervision and a moderate increase in the resources allocated to supervision. Each region has also prepared its own action plan. During the coming year, the Bank will undertake two special reviews--on the implementation of financial covenants and on the skill mix used in supervision--for inclusion in next year's ARIS, which the Directors will review in the spring of 1993.

19. The President's Task Force on portfolio management described in paragraph 11 above will set the stage for improving IDA's contribution to project implementation over the long term. It will have the benefit of the findings of a study just completed by the Operations Evaluation Department, Bank Experience in Project Supervision. The study found that further efforts may still be needed to enhance the chances for successful implementation of Bank-supported projects. Among its recommendations are:

• Management should issue a directive that clearly indicates the importance the Bank and IDA place on the successful implementation of projects.

• The Annual Report on Implementation and Supervision should provide better analysis of portfolio performance by sector and region, and describe how implementation problems are being addressed.

• Supervision requirements should be built up from a realistic assessment of individual project requirements. Sufficient staff resources should be budgeted and allocated to permit all the requirements to be met.

• The leader of each supervision mission should specify the expertise needed for the next mission.

b) Addressing Underlying Problems

20. The common implementation problems of inadequate government funding and institutional weaknesses cannot be adequately handled on a project-by-project basis. Country Implementation Reviews are a good forum for discussing with governments the scope and dimensions of these generic problems. But to help the borrower solve them often requires actions outside the framework of supervision. IDA has several tools to deal with a country's macroeconomic framework including analytical work and policy recommendations and adjustment lending. A structural adjustment program can ease the shortage of counterpart funds because it obliges the government to prioritize its expenditures better, eliminate subsidies and rationalize the prices of publicly provided goods. The Annual Report on Implementation and Supervision for FY91 found that low income countries with adjustment programs under way had about half as many problem projects as non­adjusting countries.

21. A government's failure to allocate enough counterpart funds sometimes signals that it is not committed to the project objectives, or that its spending does not correspond to its stated priorities. IDA's increasing emphasis on reviews of public expenditure and public investment programs helps address these problems, as do the other measures outlined in Box 2 below. In Bangladesh, IDA's assistance goes even farther: a Public Resources Management Adjustment Credit is under preparation that focuses on investment prioritization and funding and current expenditure allocations.

22. Dealing with borrowers' institutional weaknesses also requires action both inside and outside the project framework. In the past, in order to bypass ineffectual government agencies, many IDA projects created free-standing implementation units bolstered by expatriate technical advisers. This approach is now being discouraged because it leaves little benefit after the project ends, and creates conflicts and inequities between the project unit and the main agencies.

Page 7

Box 2: Steps to Improve the Allocation ofCounterp::J.rt funds ...... ······. ·::: .... : .. . ·.·.::::::::

· - -·. .. ······· . . .. . .. . . ...... .

Among the steps IDA has taken to ensure that governments supply their share of projeCt < firiancing are the following:

. . .. . ... .. .. . . . ........ . . . . . . . .... . . .. ... . . ..... .

• • Helping . ~trengthen economic management.to redll~ .\\'astei:O.. Uie .l:>lidg¢t <and ()v~rhaijf .· C()Jripl~ly the budgetary• process i inclucfiitg millistlies~ ~C¥<>untiri.8frilfuew~J:l(.< .. ·.·.· .............. .

. ....... .. .... . .... ... ... ···· ·· ················-········· ············ · ...................... .... ... ····- ., .... ---- ... .... ···········-·············· ·· ············-······ ···· ·· ··· · · · ········ ................... . ...................... .. . ....... . ......................... . .... -· . . . -- ··········· ............. . .................... . ............................... . . .......................... . . . . .. . . ........ .......... .. ..... . .. ........... . ............................................................................. ...... . .... .... .. ......... ................ ............... .. .................................... ..

• Revie~g public expenditure and investinenrpt<>grams t6 ~rik\1&- < ihat spt!nding t6f'I~ { .·.· <agteciJ priorities.

·: . ··.:::::::::::·.·· ·:.:·::·:.:-:.-.. ··:.:::. . ......... ·-:::::::::::::::::::::::::::::::

• Using the macroeconomic dialogue, structural adjtistmentopgratirit1s and project pfep~r~ti6ri to seek government's commitment to allocate adequate resoutc& to the sector. · ······ ···

. . . · ........ :.............. . .. :.:.-::: ......... :.:::: ....... ::::: .... :.::::.·· ..... ::::::::::::::::::::::::-.· . . . :-::-:·:·:·. ...... .::::::::::::::::::::::::::::::,.

• ·•· Ouring preparation, analyzing the project's budgetat.Jrequirembrit~. Evaluating theadeq\lll6y << offunding at the Mid-Term Project Review, andagreeing .on corrective actions. For revenu~~ < earning entities, ensuring tariffs are increased before approval and adjusted regularly . . ··.· ..

Instead, most projects now finance training and short term technical assistance for staff of the implementing agency. If a reorganization is needed, it may be a condition of project approval. Projects are structured so that they can be implemented by existing agencies, and not require long term technical assistance. Non-governmental entities may be used to do work once done by government departments (see Box 3). IDA also helps governments improve their management of public expenditures, reform civil services, and restructure or privatize public enterprises. These activities are described in IDA Technical Note No. 5, "Strengthening Institutional Capacity".

c) Design of New Projects

23. Efforts are continually under way to improve and simplify project designs so as to smooth the path of implementation. The findings of supervision missions and annual implementation reviews continuously feed into the preparation of new projects. Staff of the regions' technical departments play a special role; by participating in supervision and appraisal missions they can disseminate lessons from ongoing projects. Steps have also been taken to increase the usefulness of Project Completion Reports (PCRs) and the related audit reports issued by the Operations

Evaluation Department. A new computerized data base allows staff to identify PCRs and other evaluation reports that deal with particular sectors and issues. In recent years, many PCRs had become overdue, but in FY91 much of the backlog was cleared. In order to make PCRs more relevant, the staff guidelines for preparing them are being revised for issue this year. They require staff to prepare an issues paper before the completion mission and send it to the borrower, to help focus the mission's attention on key issues instead of merely recording data. All PCRs will contain a performance rating based on the criteria for project success that were defined at appraisal.

24. Besides evaluating individual projects, the Operations Evaluation Department (OED) reviews performance in entire sectors: for instance, it has recently reported on projects to support small and medium industry, and on the Bank's experience in lending for forestry development. In addition, OED assesses particular implementation activities, like procurement and monitoring and evaluation, that take place within projects of all types. OED's studies are widely used by staff, and project appraisal reports describe how relevant findings of OED have been taken into account in the project design.

Page 8

.......... .

Box 3: Improving Project Execution in Senegal Using a Private Firm

In Senegal, AGETIP, a private non-profit company, bas emerged as a poweif\lldevelopm~nt instrument. It acts as a general contractor for urban infrastructure works financed by IDA and otli¢r sources; It prepares bidding documents, evaluates and adjudicates the bidS, ~nd ·signs a contract \villi the winrung bidder; It then supervises, evaluates and pays .the< small .contracting firmsJh~t4o the wtirie;<

··•• firihlly,it hands (jyer the· worksto< the.·mu~d@ity for opetatidtiali4 Jiiilin.ttl:lan~>J~j~, AG~{> • ······· un.aertook·t•tg•>pl'ojects ·averaging •·· us$73 ;ooo ••each· using . 7s ·• IJBvatt{~ecffiffirms >and 4t:eati.ng•. llJm9~f>

2,000 person~years of employment. . . . . . .. . .··· ... <···>.·•··• • <> < > >

.. ·:::. . .. >::- ·: -::· :-:::·:-: ·: ·:. . . . . . . . ::. . . . :-. : . ... ·::: ·.: : . . . . . . . . :' . _::-_:::::. . . . ·:::\.:-.-.'.·_:::::·::-:-: _::::::::-:-_}:::: .. ::::::::::::::::::::::::::::::::

. ·.·.····AGETIP prepares · regular progress ·· reports • rutd ... ·. ~~dits and• sehd~ them .· td IDA al'ld tl'i~ ? GovernmeriL The firm brings private sector mimagement to public works~ It contracts< on iln as-ri~~ · basis, avoiding public sector bloat, operating with • private sector agility; . and maintaixliog high bi.isill.~W standards . . fn this way, IDA projects are effiCiently built and the domestic private sector ~s ~tiJ:r:nli~##· }

The idea is spreading to other African countries. AGETIP itself is planning to expancll:>()tJ1 > geographically and in the types of projects it undertakes; . · ··· ··· ·········

Conclusion

25. The management of the Bank and IDA has many procedures and methods for monitoring the performance of projects and helping borrowers deal with implementation problems. The supervision process is the central mechanism for identifying and addressing these problems, as well as feed directly into the design of new projects. The Africa region, which faces the most serious difficulties in implementation, has led the

... •.·:::::··

institution in strengthening IDA's ability to diagnose and help find solutions to both routine problems and underlying constraints to good project performance. The effectiveness of the supervision process is regularly assessed by the Annual Review of Implementation and Supervision; during FY93 the recommendations of the President's Task Force on Portfolio Management are expected to give renewed impetus to management's continuing emphasis on project implementation.

LEW IS T. PR ESTO N Pr~siden t

TH E WORLD BANK Wash ington, D.C. 20433

U.S.A.

To : Vice Presidents and Department Directors

Subject: Portfolio Management

February 7, 1992

The Bank and IDA have currently under implementation USD 138 billion in lending commitments, representing about USD 360 billion in projects/programs. The effective implementation of this portfolio is vital to the growth prospects of our member countries. The advice the Bank provides through its supervision of these operations is one of the most important forms of development assistance we provide. Successful implementation of approved operations outweighs new annual commitments as an indicator of the Bank's development effectiveness. It, therefore, deserves commensurate management attention.

The Bank's ability to provide effective assistance in the course of supervision depends on its capacity to draw timely lessons from on-going and completed lending operations and to disseminate this information expeditiously. The learning cycle starts with the negotiation for a project/program and continues through the PCR to OED impact studies. Its major components are the regular supervision missions, Country Implementation Reviews, the regional management of supervision follow-up, the Annual Report on Implementation and Supervision, OSP analysis on development effectiveness, and the PCRs. The OED project evaluations, sectoral reviews and impact studies complete the cycle.

It is timely for us to review the management of this cycle; the respective responsibilities of the owner, the guarantor and the lender, and the role assigned to the various unit in the Bank; the methodologies employed to assess success and failure ; and the system for extracting and disseminating implementation experience. Our implementation and evaluation systems should enable us to have a regular and realistic assessment of the development impact of the Bank's lending activities and serve as a basis for practical recommendations to improve operating policies and practices.

To conduct the review of, and to make recommendations for improving, the Bank's basic portfolio management and evaluation process, I have asked Mr. Wapenhans to chair a Task Force staffed from the Regions, OSP and OED. The areas to be included in the review are described in the attachment. The recommendations of the Task Force will, of course, take fully into account OED's current mandate and its special reporting relationship to the Executive Directors. In addition to being supported by OSP and OED, Mr. Wapenhans will consult closely with Messrs. Rovani and Picciotto, the current and the prospective Director-General for Operations Evaluation, and with Mr. Rajagopalan, VP, Sector and Operations Policy.

Once the Task Force has been organized, Mr. Wapenhans will advise me of its work program. The complexity and scope of the subject requires that we devote adequate time and resources to it. The work will be facilitated by the considerable amount of prior analysis in the Bank, by work now in progress on some elements of portfolio management, by the experience of other organizations, and by the extensive literature on evaluation and assessment of development effectiveness. I expect that the Task Force should be able to report by July 1, 1992.

cc: Messrs. Wapenhans, Rovani

Attachment

Review of Portfolio Management

The Task Force will make use of the considerable analytical work available on the supervision and evaluation functions in the Bank and elsewhere, consult with borrowers, EDs, staff and other organizations, as appropriate; and engage consultants, as necessary, to supplement internal capabilities. The Task Force review should include:

• Objectives of supervision; implementation responsibilities and mandates; measurement of performance and its consistency with the objectives.

• Projectlprogram design; articulation of goals and commitments; policy environment and consensus of major participants; consistency of covenants negotiated in a country/sector context.

• The technical quality and scope of supervision, including specifically the staffing of missions, the allocation of time between review of financial, procurement, institutional and technical issues; the allocation of time between site visits and agency offices; supervision work at Headquarters, including procurement; non-regional (Legal, OSP) support for supervision.

• The regional management structure and practice for supervision, including the roles of TD, SOD and country teams ; the internal review mechanisms; the use of field office staff (headquarters and local) in supervision and the efficiency of the current balance in countries with different size lending programs; the role of the SOAs and Project Advisors; relevance and accuracy of the rating system; effectiveness of compliance supervision; allocation of staff to supervision in response to technical complexity and institutional needs; procedures and practices for projectlprogram reformulation during implementation; internal processes for collating and disseminating experiences.

• Handling of that part of the portfolio which is consistently demonstrating unsatisfactory performance.

• The learning and dissemination process; the adequacy of regional/country/sectoral analysis of experience and its dissemination; and the efficacy of training provided to staff in supervision objectives; preparation of project managements for transition to operational phase. Possibilities of consolidating project supervision into supervision reports; consolidation of PCRs.

• The quality and utility of the Annual Review of Implementation and Supervision (ARIS); proposals for regular and explicit treatment of development effectiveness and its measurement; the PCR system and its management.

• The post-completion evaluation concepts and methodology, audit coverages and practices; the respective accountabilities of borrowers and the Bank in project evaluation; the policy and practice of OED project, country sector evaluations; the link between OED findings, staff training and project/program design.

• Proposals for longer-term impact studies and the evaluation of borrower institutional capacity and service levels beyond the confines of the project/program objectives.

'lJ7/92