PERFORMANCE AUDIT REPORT - World Bank · 2016. 8. 5. · The ASSP, cofinanced by a DFID US$14.7...

Document of The World Bank

Report No

PROJECT PERFORMANCE ASSESSMENT REPORT

BANGLADESH

AGRICULTURAL SUPPORT SERVICES PROJECT (CREDIT 2233-BD)

AND AGRICULTURAL RESEARCH MANAGEMENT PROJECT

(CREDIT 2815-BD)

May 6, 2003

Sector and Thematic Evaluation Group Operations Evaluation Department

This document will be disclosed to the public following its presentation to the Board of Executive Directors of the World Bank. Its contents may not be disclosed before that time without World Bank authorization.

Currency Equivalents (annual averages) Currency Unit = Taka (Tk) 1990 US$1.00 Tk34.9 1992/93 US$1.00 Tk39.3 1993/94 US$1.00 Tk39.3 1994/95 US$1.00 Tk40.3

1995/96 US$1.00 Tk41.0 1996/97 US$1.00 Tk42.7 1998 US$1.00 Tk45.0 2001 US$1.00 Tk57.0

Abbreviations and Acronyms ADB Asian Development Bank ADP Annual Development Program (Development Budget) ARI Agricultural Research Institute ARMP Agricultural Research Management Project ASIRP Agricultural Services Innovation and Reform Project ASSP Agricultural Support Services Project ATI Agricultural Training Institute BADC Bangladesh Agricultural Development BARC Bangladesh Agricultural Research Council BARI Bangladesh Agricultural Research Institute BAU Bangladesh Agricultural University BB Bangladesh Bank BFRI Bangladesh Forest Research Institute BJRI Bangladesh Jute Research Institute BLRI Bangladesh Livestock Research Institute BRAC Bangladesh Rural Advancement Committee BRRI Bangladesh Rice Research Institute BSRI Bangladesh Sugarcane Research Institute BSU Breeders Seed Unit CERDI Central Extension Resources Development Institute CGIAR Consultative Group on International Agricultural Research CRP Contract Research Program DAE Department of Agricultural Extension DFID Department for International Development DTW Deep Tubewells ERR Economic rate of return

FAO/CP Food and Agriculture Organization/ World Bank Cooperative Programme FINA Farmer Information Needs Assessment Project FRI Fisheries Research Institute FRR Financial rate of return FSR Farming Systems Research GB Governing Board GDP Gross Domestic Product GOB Government of Bangladesh Hortex Horticultural Export Development Foundation ICR Implementation Completion Report IDA International Development Association IPM Integrated Pest Management IRR Internal rate of return ISNAR International Service for National Agricultural Research MOA Ministry of Agriculture MTR Mid-Term Review NAEP New Agricultural Extension Policy NARS National Agricultural Research System NGO Nongovernmental organization NPV Net present value PETRRA Poverty Elimination Through Rice Research Assistance PIU Project Implementation Unit PP Project Performance REA Revised Extension Approach SAR Staff Appraisal Report SDR Special Drawing Rights T&V Training and visit TA Technical assistance TAT Technical Assistance Team USAID U.S. Agency for International Development

Fiscal Year Government: July 1 to June 30

Director-General, Operations Evaluation : Mr. Gregory Ingram Acting Director, Operations Evaluation Department : Mr. Nils Fostvedt Manager, Sector and Thematic Evaluation : Mr. Alain Barbu Task Manager : Mr. Ridley Nelson

The World Bank Washington, D.C. 20433

U.S.A.

GREGORY K. INGRAM Office of the Director-General Operations Evaluation

May 6, 2003

MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT

SUBJECT: Project Performance Assessment Report on the Bangladesh Agricultural Support

Services Project (Credit 2233-BD) and the Bangladesh Agricultural Research Management Project (Credit 2815-BD) Attached is the Project Performance Assessment Report (PPAR) for the Bangladesh Agricultural

Support Services Project (ASSP; Credit 2233-BD) and the Bangladesh Agricultural Research Management Project (ARMP; Credit 2815-BD). The ASSP, cofinanced by a DFID US$14.7 million grant, was approved June 17, 1991, for SDR 24.4 million (US$35 million equivalent) and closed June 30, 1998, on schedule with US$11.2 cancelled. The ARMP was approved February 8, 1996, for SDR 33.6 million (US$50 million equivalent) and closed on schedule in December 31, 2001with $1.9 million canceled.

The main objective of the Agriculture Support Services Project (ASSP) was to increase agricultural production, especially in food grains, and initiate a diversification process aimed at the introduction of high-value export crops. This was to be achieved through support for institutional reform, upgrading extension training, enhancing the availability of improved seeds, improving the efficiency of public services, and promoting crop diversification and agribusiness. The main components were technology transfer (including support for extension staff), training, seed industry improvement, homestead production, agribusiness, and technical assistance. The main objective of the Agricultural Research Management Project (ARMP) was to increase the efficiency of the national agricultural research system and thereby generate profitable, relevant, and sustainable agricultural technologies by strengthening the capacities of the Bangladesh Agricultural Research Council (BARC) in research organization and management, including better research resource allocation; supporting priority research; and strengthening research linkages to extension through participatory farming systems research. The main components were: research management and organization, priority research (on crops, fisheries, livestock, and forestry) and participatory farming systems research.

Overlapping for about two years, but with research coming later, the two projects were only partially linked temporally but, within the series of projects, substantially linked thematically.

In ASSP, outcome is rated moderately satisfactory. An important change of extension direction was undertaken and the objectives of increased productivity and diversification appear to have been partly achieved, although the data are weak. Relevance is rated high. Efficacy is rated modest. A new and quite promising group and demand-based extension system based on demonstrations was put in place but there were a number of weaknesses. Efficiency is rated modest. There are a number of uncertainties related to allocation of overhead costs and limited attributable impact data. Institutional development is rated substantial. Shifting from the training and visit extension system to the new approach was an important change. Bank performance was satisfactory. The shift in approach was partly attributable to Bank experience. While initial supervision was weak, the Mid-Term Review resulted in a number of sound adjustments. Borrower performance is rated, on balance, satisfactory, but there were significant weaknesses particularly in the early years. Sustainability is rated non-evaluable due to a number of

2

uncertainties about the future. Operating funds have fallen since the project closed. However, extension, calling mainly for salaries and training funds and being more flexible, is somewhat less vulnerable than research. Currently, the follow-on Agricultural Services Innovation and Reform Project (ASIRP) has consolidated a number of activities initiated under ASSP, but the future after that is uncertain.

In ARMP, outcome is rated moderately unsatisfactory largely because there were substantial shortcomings in the achievement of the main objective of improving efficiency, much of it in the overdue area of institutional reform. Relevance is rated high. Efficacy is rated modest due to concerns about the extent to which the main stated objective of improved efficiency was achieved. Efficiency is rated modest for similar reasons. Institutional development in the ARMP is rated, at best, modest, in agreement with the ICR. The “deep institutional reform” believed at preparation to be needed did not occur. Process changes had to partly make up for failure to put in place more fundamental reforms. Bank performance is rated, on balance, unsatisfactory, although there were a number of satisfactory aspects related to technical research and improved processes. Through preparation, the files show a story of progressive retreat from very sound earlier analyses. Necessary, well understood, and originally sought deep reform was given up too readily. Borrower performance is rated, on balance, satisfactory, but there were significant weaknesses. Sustainability is rated unlikely for ARMP against the ICR rating of likely. With the closing of the project, research is now starved for funds making it difficult to make full use of project-trained staff. The recurring issue from earlier projects of financial sustainability was not adequately addressed during appraisal.

The main findings and lessons, which are largely consistent with findings on research and extension in other countries, are, first, that the lessons of previous projects and the diagnosis of constraints emerging during preparation should lead to a set of reforms identifying the minimum for an effective and efficient project which should be clearly articulated in the evolving Bank management documentation. Second, adoption of new unified national extension approaches should be based on evaluated cost-effectiveness data, but it pays to maintain experimentation with alternative approaches to further pursue cost-effectiveness. Third, cutting research costs by institutional rationalization—generally closing of stations—is politically difficult but too seldom faced by either the borrower or the Bank. Fourth, Bank Project Appraisal Documents should include a thorough analysis of projected financial sustainability in the post-project period; too frequently the focus is only on funding during the project. Fifth, startup workshops and intensive early supervision with a focus on procurement can reduce implementation delays. Borrower institutions that have implemented Bank projects before still may have inadequate procurement experience due to high staff turnover. Sixth, the level and rate of introduction of computerized management systems needs to be carefully matched to the current and realistically projected computer skills and technical support capacity.

Attachment

i

OED Mission: Enhancing development effectiveness through excellence and independence in evaluation.

About this Report

The Operations Evaluation Department assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, OED annually assesses about 25 percent of the Bank’s lending operations. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. The projects, topics, and analytical approaches selected for assessment support larger evaluation studies.

A Project Performance Assessment Report (PPAR) is based on a review of the Implementation Completion Report (a self-evaluation by the responsible Bank department) and fieldwork conducted by OED. To prepare PPARs, OED staff examine project files and other documents, interview operational staff, and in most cases visit the borrowing country for onsite discussions with project staff and beneficiaries. The PPAR thereby seeks to validate and augment the information provided in the ICR, as well as examine issues of special interest to broader OED studies.

Each PPAR is subject to a peer review process and OED management approval. Once cleared internally, the PPAR is reviewed by the responsible Bank department and amended as necessary. The completed PPAR is then sent to the borrower for review; the borrowers’ comments are attached to the document that is sent to the Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public.

About the OED Rating System

The time-tested evaluation methods used by OED are suited to the broad range of the World Bank’s work. The methods offer both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. OED evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (more information is available on the OED website: http://worldbank.org/oed/eta-mainpage.html).

Relevance of Objectives: The extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Possible ratings: High, Substantial, Modest, Negligible.

Efficacy: The extent to which the project’s objectives were achieved, or expected to be achieved, taking into account their relative importance. Possible ratings: High, Substantial, Modest, Negligible.

Efficiency: The extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. Possible ratings: High, Substantial, Modest, Negligible. This rating is not generally applied to adjustment operations.

Sustainability: The resilience to risk of net benefits flows over time. Possible ratings: Highly Likely, Likely, Unlikely, Highly Unlikely, Not Evaluable.

Institutional Development Impact: The extent to which a project improves the ability of a country or region to make more efficient, equitable and sustainable use of its human, financial, and natural resources through: (a) better definition, stability, transparency, enforceability, and predictability of institutional arrangements and/or (b) better alignment of the mission and capacity of an organization with its mandate, which derives from these institutional arrangements. Institutional Development Impact includes both intended and unintended effects of a project. Possible ratings: High, Substantial, Modest, Negligible.

Outcome: The extent to which the project’s major relevant objectives were achieved, or are expected to be achieved, efficiently. Possible ratings: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory.

Bank Performance: The extent to which services provided by the Bank ensured quality at entry and supported implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of the project). Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory.

Borrower Performance: The extent to which the borrower assumed ownership and responsibility to ensure quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development objectives and sustainability. Possible ratings: Highly Satisfactory, Satisfactory, Unsatisfactory, Highly Unsatisfactory.

iii

Contents Fiscal Year ...................................................................................................................................... 2

Principal Ratings............................................................................................................................ v

Key Staff Responsible .................................................................................................................... v

Preface...........................................................................................................................................vii

1. Background.............................................................................................................................. 1

Project Design........................................................................................................ 1 Project Objectives and Components ................................................................................... 2

2. Analysis..................................................................................................................................... 3

Relevance............................................................................................................... 3 Outcome of the Two Projects ................................................................................ 3 Institutional Development.................................................................................... 12 Bank Performance................................................................................................ 15 Borrower Performance......................................................................................... 16 Sustainability ....................................................................................................... 16 Lessons................................................................................................................. 19 Future Directions ................................................................................................. 19

Annex A. Basic Data Sheet .......................................................................................................... 21

Annex B: Brief Summary of Extension Impact Study Findings.............................................. 25

Annex C. Comments from the Borrower................................................................................... 27

Annex D: Comments from the Ministry of Fisheries and Livestock, Planning and Evaluation Wing................................................................................................................................................. 31

Annex E: Comments from the Ministry of Agriculture .......................................................... 35

This report was prepared by Ridley Nelson (Task Manager). The report was peer reviewed by G. T. Keith Pitman and edited by William Hurlbut. Helen Phillip provided administrative support.

v

Principal Ratings

AGRICULTURAL RESEARCH MANAGEMENT PROJECT (CREDIT 2815-BD)

ICR ES Assessment Outcome Satisfactory Satisfactory Moderately Unsatisfactory Institutional Development Impact

Modest Modest Modest

Sustainability Likely Unlikely Unlikely Borrower Performance Satisfactory Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Unsatisfactory


ICR ES Assessment Outcome Satisfactory Satisfactory Moderately Satisfactory Institutional Development Impact

Modest Substantial Modest

Sustainability Likely Likely Non-Evaluable Borrower Performance Satisfactory Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Satisfactory

* The Implementation Completion Report (ICR) is a self-evaluation by the responsible operational division of the Bank. The Evaluation Summary (ES) is an intermediate OED product that seeks to independently verify the findings of the ICR. Key Staff Responsible

AGRICULTURAL RESEARCH MANAGEMENT PROJECT (CREDIT 2815-BD)

Task Manager Division Chief Country Director

Appraisal John McIntire Ridwan Ali Pierre Landell-Mills Midterm Robert Epworth Ridwan Ali Frederick Temple Completion Robert Epworth Ridwan Ali Frederick Temple


Task Manager Division Chief Country Director

Appraisal Ross Wallace Chaim Helman Groves (Acting) Midterm Robert Epworth Ridwan Ali Pierre Landell-Mills Completion Abdul Ghani Ridwan Ali Frederick Temple

vii

Preface This is a Project Performance Assessment Report (PPAR) for the Bangladesh Agriculture Support

Services Project (Credit 2233-BD), for which a credit in the amount of SDR 24.4 million (US$35 million equivalent) was approved on June 17, 1991, and the Bangladesh Agricultural Research Management Project (Credit 2815–BD), for which a credit in the amount of SDR33.6 million (US$50 million equivalent) was approved on February 8, 1996. US$14.7 million cofinancing was provided in the former project by ODA (now DFID). An Implementation Completion Report (ICR) was submitted for the Agriculture Support Services Project on December 30, 1998 (Report no. 18757) and for the Agricultural Research Management Project on June 24, 2002 (Report no. 24349).

The PPAR was prepared by the Operations Evaluation Department (OED) based on the Implementation Completion Reports, the appraisal reports, the Development Credit Agreements, review of Bank files, and an assessment mission to Bangladesh in October 2002. The project was also discussed with Bank staff, beneficiaries, government staff at all levels, nongovernmental organizations (NGOs), and other donors. The cooperation and assistance of all stakeholders and government officials, especially DAE and BARC, is gratefully acknowledged, as is the support of the staff of the World Bank Country Office in Dhaka.

The main rationale for selecting these two projects for evaluation was to assess the performance of research and technology dissemination projects as a complementary pair. The assessment provided an opportunity to look at both together, to assess them against the lessons of a series of projects supporting agricultural technology and to investigate the sustainability aspects. It was considered particularly important to assess the Agriculture Support Services Project because it was reported to have been innovative and because it appeared to be a redirection following disappointment with the earlier training and visit extension approach and therefore to have lessons for extension elsewhere.

Following standard OED procedures, the draft PPAR was sent to the borrower for comments before being finalized. Borrower comments have been taken into account, and are included as Annex C.

1

1. Background 1.1 About half the Bangladesh population of 130 million is poor. Agriculture accounts for about one-third of GDP, about two-thirds of employment, and 15 to 20 percent of commodity export earnings. The major agricultural challenge is to increase productivity from a land resource already very intensively used. Arable land covers about 55 percent of the national land area and, because Bangladesh is one of the most densely populated countries in the world, there is almost no possibility to increase cultivated area. Intensification is primarily through irrigation and flood control and almost all available potential is fully utilized. There are about 14 million farm households of which about half are landless. Of great significance to agricultural production is the extreme of seasonal monsoon flooding, which may cover 50 percent of land, followed by a nine-month dry season. With land use intensification over recent years the marginal returns to fertilizer have fallen from the levels of the mid-1980s. The heterogeneity of agriculture is a major challenge for agricultural technology development and dissemination. There arwith different soil types, water management needs, and different hot

1.2 Agriculture output value grew at 1.7 percent in 1990 to 1995capita food grain supply rose through the 1980s and 1990s and real ppoverty alleviation. Rice is still the most important crop, with about rice, but diversification has increased. Fisheries contribute about 15 livestock about 11 percent, and forestry about 10 percent. With partiDevelopment Goals, about two-thirds of children under the age of fiv

1.3 The government’s agricultural policy aims to stimulate growpoverty, and improve food security. This is to be achieved through cclimate for private investment, targeted interventions for poverty alleand infrastructure, and investment in the generation and disseminatioproductivity.

Project Design

1.4 The Agricultural Support Services Project (ASSP) and AgricProject (ARMP) both presented challenging institutional issues arisinhistory of project interventions in the sector. This included the ExtenBD), approved in 1977, establishing the training and visit (T&V) sysResearch I Project (AR I; Cr. 828-BD), approved in 1978; the SeconResearch Project (AER II; Cr. 1215-BD), approved in 1982; and theII; Cr. 1455-BD) in 1984. OED evaluated the latter three, finding ouunsatisfactory in AR II, and marginally satisfactory again in AER II.institutional development and uncertain for sustainability.

1.5 Issues identified by previous OED evaluations included pooAgricultural Research Council (BARC) functions, concerns about qudisseminated, financial sustainability, and efficiency in research. Anabout the scale of both the extension and research service. Many of tidentified for ASSP during project preparation were that T&V had facommunications linkage due to weak technical messages, skill limitasufficient contact farmers, and that there was a need to expand exten

The major agricultural challenge in Bangladesh is to increase productivity from a land resource already very intensively used.

e 30 identified agro-ecological zones , warm, and cold seasons.

and 5.1 percent in 1996 to 2000. Per rices fell with significant impacts on

42 percent of farms growing only percent of agricultural GDP, cular relevance to the Millennium e are malnourished.

th to increase rural incomes, reduce ontinued liberalization to improve the viation, investment in human capital n of improved technology to raise

ultural Research Management g partly from the past checkered sion and Research Project I (Cr. 729-tem of extension; Agriculture d Agriculture Extension and Agriculture Research II Project (AR tcome marginally satisfactory in ARI, All three were rated modest on

r development of the Bangladesh ality of technology developed and

d, related to sustainability, concerns hese issues remain. Main lessons iled to develop fully a tions, and difficulties in maintaining sion options by using more

2

demonstrations, working with groups, and increasing media campaigns. Main lessons identified for ARMP during project preparation were that BARC had no influence on resource allocation outside the Ministry of Agriculture (MOA) and that, even within MOA, BARC had little power to influence the two largest research institutes. Also, BARC’s organizational structure had proved to be functionally unproductive. Moreover, there had been a negative attitude toward using commercial crop varieties and recommending technology generated in other countries with similar agro-ecological conditions. The two projects being assessed followed the above series of projects but with only partial overlap between ASSP and ARMP, with the ASSP leading ARMP by five years and overlapping for three years (1996 – 1999).

PROJECT OBJECTIVES AND COMPONENTS

1.6 The main objective of the Agriculture Support Services Project (ASSP) (1992 – 1999) was to increase agricultural production, especially in food grains, and initiate a diversification process aimed at the introduction of high-value export crops. This was to be achieved through support for institutional reform aimed at making technology transfer more responsive to the needs of farmers and more widely available (especially to women), upgrading extension training, improving the availability of improved seeds through the participation of private seed companies, removing restrictive seed regulations, improving the efficiency of public services, and promoting crop diversification and agribusiness. There were six components:

• Technology Transfer aimed at improving the efficiency of outreach and increasing coverage (US$16.0 million - 32 percent of project costs)

• Training of Department of Agricultural Extension (DAE) staff and farmers (US$12.0 million - 24 percent of project costs)

• Seed Industry Improvement aimed at improving the production of breeder’s seed of rice, wheat, and jute (US$2.2 million - 4 percent of project costs)

• Homestead Production to demonstrate on-farm and homestead activities for women (US$4,3 million - 9 percent of project costs)

• Agribusiness Promotion to stimulate private sector interest in the export of high-value horticultural crops through support to Hortex1(US$10.1 million - 20 percent of project costs)

• Technical Assistance Team to assist DEA with strategic planning and provide technical expertise (US$5.1 million - 10 percent of project costs).

1.7 The main objective of the Agricultural Research Management Project (ARMP) (1996 – 2000) was to increase the efficiency of the national agricultural research system and thereby generate profitable, relevant, and sustainable agricultural technologies. This was to be achieved through (a) strengthening the capacities of the Bangladesh Agricultural Research Council (BARC) in research organization and management, including better research resource allocation; (b) supporting priority research; and (c) strengthening research linkages to extension through participatory farming systems research. There were three components:

• Research Management and Organization to strengthen research planning and managerial capabilities of BARC through institutional reform, improved economic analysis, technical assistance, civil works, training, contract research, and supervision of master plans for participating agricultural research institutions (ARIs); (US$14.0 million at appraisal - 24 percent of project costs)

1. Hortex is the Horticultural Export Development Foundation set up in 1993 as a not for profit foundation for development, promotion and marketing of exportable horticultural produce, particularly high-value, non-traditional crops to high-priced non-conventional markets.

3

• Priority Research to carry out research at ARI’s on crops, fisheries, livestock, and forestry (US$40.6 million at appraisal - 68 percent of project costs)

• Participatory Farming Systems Research to strengthen farming systems research programs through support to research-extension linkages and testing and dissemination of appropriate technology (US$4.6 million at appraisal - 8 percent of project costs).

2. Analysis

Relevance

2.1 ASSP. Increasing agricultural production, especially in food grains, and initiating a diversification process aimed at the introduction of high-value export crops, are central to the country’s current development priorities, as outlined in the government-endorsed New Agricultural Extension Policy and reflected in the Bank CAS, and are fully consistent with the Bank’s current rural strategy. The project objectives were therefore highly relevant.

2.2 ARMP. The project objectives were highly relevant. Increasing productivity and contributing to diversification were consistent with the development priorities of Bangladesh and with Bank strategies and goals. Indeed, along with infrastructure, enhanced technology in agriculture arguably is the highest priority for the alleviation of poverty.

Outcome of the Two Projects

2.3 Outcome of ASSP is rated moderately satisfactory and of ARMP moderately unsatisfactory2. Overall, project design only partially addressed the main lessons, with more progress in extension than in research. In extension, progress was made in shifting away from the rigid, high-cost, highly staff-intensive T&V system, reversing much of what had been introduced with Bank support except for a core element of work planning. Improved feedback and decision-making processes were developed at the local level. The Department of Agricultural Extension (DAE) has been somewhat reduced in size. But the mechanism for reduction has been inefficient, through the freezing of recruitment, distorting the age structure of the extension service. DAE is now somewhat better organized and exhibits greater decentralization of decision making and a number of processes have been put in place to link research and extension at the local level. Nevertheless, most donor and NGO observers believe that, overall, there has been only modest progress on research extension linkages and farmer feedback. In research, the problem of the role of BARC with coordinating responsibilities but little power, no responsibility for channeling core funding, and limited skills, was appreciated during project design but largely remains. The plethora of institutes and substations—some rural centers have five or more institutes and substations—has scarcely been tackled except with “Band-aid” process adjustments. The number of staff has been reduced in research also, partly again through the recruitment freeze, but also partly through loss of skills overseas due to lack of incentives.

Project design only partially addressed the main lessons, with more progress in extension than in research.

2 The borrower in Annex D says this rating is too sweeping and inconsistent with the ICR.

4

Agricultural Support Services Project

2.4 Overall, efficacy is rated modest due to modest farmer contact rates and uncertainties about the production impact of those contacts. However, with the important shift away from the old T&V system towards a group approach, achievement of objectives in this project was relatively better than in the Research Project, but still insufficient to lift to a substantial rating. Increased agricultural production and the initiation of diversification was to be through working with groups using demonstrations. Plans at Block and Sub-District level were drawn from farmers’ needs assessments, thus setting priorities at a decentralized level. This promising approach was insufficiently tested and adapted early in the project partly because of limited skills and incentives. Little progress was made for the first three years, but actions taken following the Mid-Term Review turned project performance around and the important New Agricultural Extension Policy was developed.

2.5 The extent to which the objective of increasing productivity was met and is attributable to the project is uncertain due to limited outcome data. The increase in overall productivity in Bangladesh agriculture was low during the first part of the project period at about 1.7 percent per annum—the same as the population growth rate—but it accelerated to 5.1 percent from 1996 to 2000.3 During that period, the highest growth rate was in fisheries—a subsector largely outside DAE’s responsibility and serviced by other Bank-supported projects. One circumstantial piece of evidence is that the OED mission found substantial increases in land values over the past 10 years in all locations visited, suggesting a capitalization of substantial productivity gains. However, attributing this to the project is difficult. A substantial portion is certainly due to improved infrastructure. In particular, the new Jamuna River bridge appears to have had an enormous impact. Searching further back in the logical chain for indicators at the input level, there are three pieces of evidence. First, there was some improvement in the rate of farmer contact. Based on a 1999 impact study done by consultants for DFID, the percentage contacted by DAE rose from 2.8 percent in the baseline year to 7 percent and then briefly to 9 percent in 1997–98 but then dropped back to 5 percent in 1999. However, the study argues that this underestimates contacts because there are other DAE-originated activities not picked up under the survey. Second, an important target input indicator—the number of on-farm demonstrations—was more than met, achieving double the target at 433,000 demonstrations. Quality of demonstrations was probably satisfactory, although there were some concerns about the treatment “controls” that were lacking or inappropriately designed. Third, field days also came very close to achieving the target indicator at 846,000 against a target of 902,000.

Little progress was made for the first three years, but actions taken following the Mid-Term Review significantly improved project performance.

2.6 With respect to the objective of initiating diversification, this was almost certainly achieved, although the term “initiating” is open to a wide range of interpretation. In the area of extension diversification, the mission encountered a clear focus on crop diversification by many Block Supervisors at the farm/group level, often aided by NGOs. Technologies transferred appear to cover a wide spread including, particularly, homestead vegetables, which, with NGO support, was a major focus of the project—a component mostly involving women and very good for poverty targeting and diversification. As one of the more successful vegetable-growing farmer groups told the mission, “Before we got this assistance, other people did not want to marry into our families, now they come seeking us out.”

2.7 The other main prong in the diversification focus, the agribusiness (Hortex) component, was cut back in scale but it did contribute to getting one main exporter quite well established and about another four or five close to some significant export activity. Moreover, the field enterprises in this component

3. Total factor productivity in rice improved from the mid-1970s/mid-1980s (0.79) to the mid-1980s/mid-1990s (1.37), but whether that improvement was sustained through the later project period is uncertain.

5

have been quite well poverty targeted through export crop employment intensity. While there are questions about how long the subsidy to Hortex is justified, it appears to have provided a modest kick-start to high-value, high-employment, agricultural exporting. However, there are some questions about the efficacy of this component. While supporting the initiation of horticulture exports was appropriate, the actual achievement has been modest in volume. So far, barely 200 tons per year of exports are being achieved, largely by one organization (the Bangladesh Rural Advancement Committee, BRAC), which no longer needs the support of Hortex. The task of initiating horticulture exports is clearly not finished and competitors to the dominant exporter now need to be encouraged. However, given the unlikelihood that Hortex will be allowed to compulsorily collect service fees from former clients based on export volume to ensure its sustainability,4 a decision is needed soon on how long Hortex should be sustained with public funds and what the criteria would be for exit from this initially-justified public support. The government should draw lessons from related experiences in other countries.

2.8 On the process and planning side there have been some positive developments: new planning and management systems incorporating work program prioritization were introduced and appear to have been largely sustained—the mission sat in on one quite lively and productive District meeting. However, several activities have been less successful. Farmer Needs Assessments are not working very well. Skills are still weak and, in the absence of substantial project funding, there are serious doubts about the capacity of the system to maintain adequate technical skills.5 There is also excessive bureaucratic form-filling. The mission counted about 200 to 250 forms per year that have to be completed by a Block Supervisor. This may be a record globally.

2.9 Links to research remain weak. Poverty targeting has some weaknesses, a situation that is not helped by the limited range of low-cost technology packages emanating from research.6 DAE extension operates separately from forestry, livestock, and fisheries with limited coordination at the Block level. Extreme “projectization” in the budget means virtually no base level of operating or training funds and, therefore, extension direction at the local level is pulled toward whatever has been negotiated with the particular donor of the time. While there is much to be said for the motivation and management efficiency of a “projectized” extension campaign approach (it is often the approach adopted by the sales forces of commercial rural companies), it does seem to have been carried to extremes in Bangladesh due to budget exigencies. One important elements of a broader agricultural sector support package has been missing and should have been addressed within the overall agricultural lending program. There has been a gaping hole in credit supply between the landless—handled by such agencies as the Grameen Bank—and the quite well off, who can get credit from commercial banks. Development practitioners even have a name for this neglected in-between cohort: “the sandwich people.”

2.10 The contact rate with farmers was found to be about 7 to 9 percent, only just within the range of achievements in other countries (Annex B) and this after prolonged Bank support . As noted above, the contact rate was found to have declined in 1999. In the impact studies, there were high levels of positive responses—in the region of 90 percent—to questions about whether, after receiving information from demonstrations or other events, farmers planned to adopt the featured technologies, and similarly high 4. Hortex was expecting to raise a voluntary cess per kilogram of produce exported as a reward for the training and services supplied initially. This is not happening and cannot be expected to happen unless it is legislated or unless it is agreed collectively by a very strong exporter organization, which is unlikely to be in place for some time.

5. There are, in fact, differences of view among local observers about whether the greatest weakness lies in extension methods or in technical skills. The evaluator is inclined to believe that strong technical skills can override much weakness elsewhere whereas the reverse is not true. For example, one observer noted that, in the relevant area of the south, few, if any, Block Supervisors seemed aware of the new saline-tolerant varieties of rice.

6. More research teams should be answering pro-poor, systems-oriented, questions like, “If I have only Tk2000 to spend on this crop or system on this land area what combination of inputs and practices should I spend it on to maximize my income increase?”

6

percentages who found worthwhile benefits to the adopted technologies. However, both these questions present interpretation problems. There is no evidence linking measured incremental incomes to attributable extension effort, but such evidence is rarely obtainable anywhere.

2.11 Debate is ongoing about extension staff numbers. Appropriately given the financial sustainability concerns, there was a Bank-imposed condition on reducing staff numbers. The ratio is now about 1 Block Supervisor to 1,200 farm households. In the absence of the broad early retirement retrenchment sought earlier by the Bank and rejected by government, the use of the economy-wide recruitment freeze to achieve the reduction has resulted in a distorted age structure (nearly all Block Supervisors are now above the age of 37) that cannot be sustained much longer. DAE officials would like to improve the age structure, but believe that they may be legally constrained by the Bank’s earlier staff ceiling conditionality. Until financial sustainability at the present scale is more assured, increasing numbers would be counterproductive.

At the peak about 7 to 9 percent of farmers were contacted at least once a year by DAE staff. Most tested technologies offered and found them worthwhile, but productivity impact is unclear.

2.12 Quite apart from the overall government budgetary constraints, which leave serious questions about the capacity to sustain a larger extension service, it is by no means clear what the optimal pattern of expenditure in extension should be, not only with respect to numbers of staff but also to supporting expenditure on Agricultural Training Institutes, audiovisual media, and different levels of expenditure on support such as transport (for example, motorbikes versus bicycles). Before any decisions are made about changing farmer/extension staff ratios, cost effectiveness evidence needs to be assembled from monitored and evaluated alternatives for extension intensity. DAE and the Bank are to be commended for using the follow-on Agricultural Services Innovation and Reform Project (ASIRP) to test alternative approaches at different cost levels. One view expressed to the mission was that the ASSP-adopted group approach was anointed as the chosen approach too early. This may well be correct. In some districts, a Farmer Field School approach for integrated pest management (IPM) is now being supported. As an extension approach this also warrants the same cost effectiveness scrutiny.7

2.13 The project supported eight breeder seed units, covering rice, wheat, lentils, and jute, which are now largely functional. The increased supply of improved seed through the farmer exchange program8 has clearly had some benefit. Surveyed farmers who had participated in the program estimated about 110 percent to 130 percent productivity increases without additional complementary inputs and about 200 percent to 250 percent with additional inputs. However, inadequate and impermanent staffing remains a serious concern and seed subsidies were not lifted, raising questions about sustainability.

2.14 The efficacy of the training component was mixed. Civil works were completed and training modules developed with assistance from the ASSP Technical Assistance Team. However, the Block Supervisor training largely came before the New Agricultural Extension Policy (NAEP), and the “cascade” training—the training of trainers, who then trained at lower levels, as noted in the ICR, did not work very well.

2.15 The homestead component, which supported intensive homestead production for women and covered crops, livestock, and horticulture in partnership with about 50 selected NGOs, appears to have progressed 7. The need for experimentation and careful analysis of cost effectiveness is even more important given the increasing possibility that decentralization will, in due course, hand over extension to local governments perhaps with budgetary allocation guidelines or controls. Decision makers at such decentralized levels presumably will be particularly interested in performance evidence to guide their allocation of resources.

8. Every participating farmer returns twice the amount of seed received, which is then redistributed—the result being a snowball effect, provided quality is satisfactory.

7

quite well. A 1998 donor-funded independent evaluation, while not offering incremental productivity data, suggests that the programs were generally effective and beneficial to resource-poor women. The mission also observed some promising results in the field. The relative success of this component raises the question of comparative advantage of public agencies and whether public support for extension should not now quite rapidly shift further toward mainly supporting other providers, such as NGOs. This is not a new debate in Bangladesh or globally.

2.16 Technical assistance, largely financed by DFID, has made, and continues to make, a substantial contribution to strategy, implementation, and training. The only caveat is that the management information systems developed by some consultants may have been somewhat too sophisticated for the available skills—particularly computer skills.

2.17 The linkage between the ARMP and ASSP via the liaison committees at both central and local levels and the Farming Systems Research component, while improving feedback in both directions, still has weaknesses. Farmer Needs Assessments require improved skills and the Farming Systems work is close to a state of collapse for lack of funds. The follow-on ASIRP has sustained the research-extension linkages in some respects, but overall there is much unfinished business.

Agricultural Research Management Project

2.18 Overall efficacy is rated modest partly because quality at entry was weak in the critical area of institutional reform. The main objective stated in the SAR was to “increase the efficiency (emphasis added) of the national agricultural research system and thereby generate profitable, relevant, and sustainable agricultural technologies through strengthening BARC…., supporting priority research, and strengthening research linkages….” As is often the case, the objectives were not well translated into actionable and monitorable implementation indicators with a focus on outputs.

2.19 With respect to improved efficiency of the system, there were no indicators that might enable tracking of efficiency changes. Improved efficiency would call particularly for improvements in the overall allocation of financial and human resources toward areas of highest economic return both between main commodity or thematic areas and within those areas and for reduced costs per unit of research output. Although data is limited, results are mixed. The systemic problem from the past remains. Separate ministries still retain control of their own pieces of the sector, making coordination difficult9.

While useful research was done, efficacy of ARMP was modest in improving efficiency. Notwithstanding process improvements, an inefficient institutional structure remained with limited influence of BARC on ARI programs, insufficient capacity to shift resources, little improvement in staff incentives and no rationalization of the plethora of institutes and substations.

2.20 On the negative side, BARC still has no control over the allocation of resources between crops, livestock, fisheries, and forestry since these come under different ministries. Even within those subsectors, while BARC staff assured the mission that they have influenced allocations, the mission found it difficult at the institute level to elicit examples where this was acknowledged to have happened. In any case, resource allocations can be and are subsequently altered without reference to BARC. Also, the sheer number of independent institutes and substations in the same location, each with its separate local management and support structure and facilities, each starved for operating funds (for example, eight in Rajshahi and at least five in or near many other towns), cannot be efficient.

9 In relation to rationalization, the borrower notes in Annex C that this is now approved and underway.

8

2.21 On the positive side, at the institute level, the increased use of prioritizing procedures, including some use of ex-ante economic analysis, suggests the potential for improved resource allocations within institute programs, although there are still a number of programs where many research proposals offer simply one or two sentences of economic justification. The ICR noted that there was “some evidence of research teams and assessment criteria being modified in response to feedback from farmers by the Farming System Research programs.” The contract research program applied a rigorous selection process, but still 90 percent went to government institutions probably at least partly substituting for core programs starved of government funds rather than being truly incremental. While it could be argued that there were too many small contracts, overall the contract component was managed satisfactorily.

2.22 With respect to technologies, while it is still mostly too early to use adoption level of released varieties or husbandry recommendations as a measure of performance of this particular project, what limited evidence there is on efficiency is somewhat mixed. Agriculture growth did pick up from a low rate (negative for rice) in the early 1990s to 5.1 percent after 1996, but given research/adoption lags little of the latter can yet be attributed to the project. Much of the recent growth achieved in agriculture is attributable to an approximately 8 percent growth in fertilizer use. The incentive for this does come partly from responsive varieties, and also from increased irrigation due to increased groundwater use. Impact studies, as is often the case, suggest high rates of return to selected individual research programs—mostly in the area of plant breeding. However, there are some troubling features. First, stagnant growth of HYV rice yield in the aus and aman seasons is a growing concern. Second, across the three research projects—AR I, AR II, and ARMP—there has been a sustained stream of new technologies with modest adoption but, as BARC noted in its own impact study, some question about whether the adoption rate on improved varieties has been declining for some crops., The average age of the most widely adopted varieties is now very high.10 This may suggest relatively higher economic returns to research investments before the mid- 1980s, both by program and in aggregate, with declining returns subsequently. The project in question, of course, falls in the latter period. More evidence should be sought by BARC on this issue.11 In addition, research has been slow to shift toward meeting the more recent commercialization and quality marketing needs and many observers in the rural NGO community see research/extension links as still weak.

It is too soon to assess impact. Studies suggest high rates of return to some selected programs—mostly in plant breeding which usually gives high returns. The series of research projects has produced a stream of new technologies, but with modest percentages of area under the newer varieties, and with stagnant HYV rice yields, there may be a slowing of benefit streams more recently. Research has also been slow to shift toward meeting newer needs such as commercial product quality.

2.23 With respect to BARC capacity, this was somewhat improved in skills and processes, but the new ordinance did more to clarify BARC’s already assigned role than to enhance its capacity or resource 10. For example, about 45 percent of T Aman season rice is sown to BR11, a variety released in 1981, about 80 percent of wheat is Kanchan, a variety released in 1983, and the dominant mill-zone sugar was released in 1967.

11. Some decline in economic returns to research investments should be expected as one shifts from the easy initial high-yielding varieties (HYVs), which more than doubled yields over traditional varieties to second-generation improved HYVs, which must offer significant increments over the first-generation HYVs. Nevertheless, the apparent modest levels of adoption of newer HYVs should be of concern.

12. For example, about 45 percent of T Aman season rice is sown to BR11, a variety released in 1981, about 80 percent of wheat is Kanchan, a variety released in 1983, and the dominant mill-zone sugar was released in 1967.

13. Some decline in economic returns to research investments should be expected as one shifts from the easy initial high-yielding varieties (HYVs), which more than doubled yields over traditional varieties to second-generation improved HYVs, which must offer significant increments over the first-generation HYVs. Nevertheless, the apparent modest levels of adoption of newer HYVs should be of concern.

9

allocation authority. Therefore, the extent to which BARC really contributed much to improved resource allocation is questionable. There remain many staff vacancies and the staff of BARC, after many years, are still on the development budget rather than the revenue budget, a debilitating signal of impermanence also affecting the extent of their influence on the system.

2.24 With respect to support for priority research, incremental support was provided in a number of priority research areas both through the core program support and the grant program. External reviews of this were generally positive, suggesting that there had been significant improvements in the quality of core research activities. At the institute level selected ex-post economic analyses suggested good rates of return, but given the lag in research benefits these still rely largely on projected rather than actual benefit streams.

2.25 With respect to research-extension linkages, most donors and NGOs who have observed the process closely see rather modest gains in this direction. There appears to have been better progress in research-extension linkages through the limited-scale Farming Systems Research mechanism—sustainability of which is doubtful—than through direct linkages to the broader range of core programs.14 Chronic shortages of operating funds make it very difficult for researchers to reach out much to the field.

2.26 Linkages with the Consultative Group on International Agricultural Research (CGIAR) system are in place in a number of research areas and the involvement of CGIAR institutes, particularly the International Service for National Agricultural Research (ISNAR), during the project contributed to improving performance. While the mission did not have the resources to probe this question fully, based on limited observation and discussion the linkage with international centers appears satisfactory.

Efficiency


2.27 On balance, efficiency is rated modest. There are a number of uncertainties due to the limited attributable impact measurement. Available evidence is more on farmer contact, poverty targeting, and adoption intent than on incremental production or income from actual adoption. There are questions about what costs to attribute.

2.28 Economic Rate of Return. There are two sources of estimates for the ERR, the 1999 impact assessment and the ICR. These estimates are based on adoption estimates which mainly arise from the Technology Transfer component. It is not clear in these analyses how much these benefit streams also cover benefits from the Training, Seed Improvement, Homestead Production and Agri-business components and thus how much of those component costs should be attributable. The 1999 impact assessment economic analysis using representative budgets for selected typical crop enterprises found an ERR of 48 percent to ASSP costs alone. However, there is a concern about the sensitivity analysis because adding only 20 percent of DAE costs at the district level lowers the ERR to 8 percent.

14. There is some risk that a designated Farming Systems Research program, as opposed to FSR that permeates the whole program, will be seen by researchers in the core programs as exonerating them from any need to maintain links with farmers or extension, i.e., the opportunity to say, “someone else is doing it.”

10

Surprisingly, this sensitivity, while tabulated, is not discussed in that impact assessment.15

2.29 The ICR finds an ERR of 26 percent with a reduced adoption area in the sensitivity test giving an ERR of 17 percent. Methodologically, this is a very simplified economic analysis of limited value. It is usually possible to show satisfactory rates of return for relatively modest adoption levels of improved technology in a large agricultural sector. The question is always whether even such modest levels are actually being achieved and are attributable to the extension input. Overall, therefore, in both analyses, there are considerable difficulties with attribution.

2.30 The Efficiency of the Agricultural Training Institutes and the Central Extension Resources Development Institute. The efficiency of ATIs and CERDI, both supported under the project and earlier projects, is questionable. The ATIs are now largely teachers training colleges16. About 70 percent of the 5,000 students who leave each year go on to teach science. While this ensures quite good agriculture elements in school science, it is doubtful whether this is cost-effective for direct and immediate agricultural impact. DAE appears to be going along with this budgetary sleight-of-hand to ensure a surviving training capacity for when the freeze on Block Supervisors recruitment comes to an end or funds somehow become available. With about 10,000 Block Supervisors in the extension service, an annual capacity to produce 5,000 new ones appears way beyond DAE’s need, even allowing for some going to the private sector and NGOs—one tenth of that number would probably be sufficient. What is needed to make sound decisions on ATIs is a projection of Block Supervisor demand and a decision on whether the next step would be toward using graduate extension staff. The use of quality-controlled private ATI’s (there are several) as a source for Block Supervisors should be considered. CERDI also needs a decision. Currently, it operates simply as a facility. Yet it has a complement of training staff, which is wasteful. There appears to be a case for an active center that could develop both training and materials—as was the original intent supported by earlier Bank funding. But if funds are not available (another area where extreme projectization of funding has had an impact), then it should simply be administered as a set of buildings or be sold.

Although impact data are limited, the estimated ASSP ERR is above the Opportunity Cost of Capital. However, it is very sensitive to the share of DAE costs included and there are questions about attribution. There are questions about agriculture returns to investments in ATIs because 70 percent of those trained become science teachers.

2.31 DAE Resource Allocation Efficiency. There are questions about the efficiency of overall DAE extension expenditure. One might expect some relationship between total development and revenue budget allocation per Block and current or projected agricultural output per Block. In fact, each Block receives about the same allocation. While the size of each Block (these are not national administrative units; they are determined by DAE) is adjusted by DAE to approximately match the number of farmers, it is not clear that the resulting resource allocations are broadly optimal since neither growth nor poverty is necessarily correlated with density or number of farmers. While there are political parameters, better analysis of spatial resource allocation is warranted, leading, perhaps, to greater differentiation depending on potential extension impact.

15. The mission attempted to identify and add up costs assigned to extension in the research analysis and cost assigned to extension in the extension analysis to assess the extent of under-counting or double counting, but found the data too partial to complete that task. 16 This is disputed by the borrower in Annex E but alternative data on the percentage who become teachers is not provided.

11


2.32 Efficiency is rated modest overall. The overall economic rate of return was probably adequate—although there is insufficient data to recalculate an aggregate—but it could have been much higher with greater attention to rationalization, a system with improved allocative efficiency across sub-sectors, and improved staff incentives. Since improved efficiency was the main objective, efficiency has been largely covered above under the efficacy heading. This section provides some elaboration.

2.33 Improved Efficiency in Management Processes. The project achieved a number of improvements in management processes including: two Standing Committees on Budget and Research Programming to review proposals submitted to the BARC Governing Body, staff promotion criteria, criteria for the selection of candidates for higher training, the introduction of a research prioritization mechanism, procedures for selecting grant proposals, and software and supporting manuals for personnel management, budget, accounting, and auditing. Mostly these have been sustained. However, there are weaknesses. First, while promotion criteria exist, impact on performance incentives is negligible because of a limitation in promotion positions; a number of staff, such as Farm Systems Research staff, are still not on the revenue budget and therefore are temporary; difficulties of transfer between institutions; and very small salary increments. Second, the new management systems are being used only to a very limited extent due to complexity, lack of computer skills, lack of management understanding of what such systems offer, and incomplete implementation. The gains in this area are, therefore, probably quite modest. Expectations were, on the other hand, excessive

2.34 Economic Rate of Return. During the project, numerous attempts were made at economic analysis of individual programs, all using a widely accepted economic surplus model and incorporating a share of extension and BARC overhead costs. Indeed, the use of such analysis is one of the benefits to come from the training component. As noted in the ICR, IRRs17 for six major crops (T Aman rice, Boro/Aus rice, wheat, potatoes, jute, sugarcane) in ex-post analyses ranged from 15 percent (jute) to 59 percent (T Aman rice). In addition the following IRRs for other crops/systems were found: agro-forestry, 70 percent to 112 percent; brinjal, 44 percent; guava 50 percent.

2.35 Such high rates of return are common in agricultural research. While there is little doubt that economic rates of return to even a suboptimally designed agricultural research project such as this may be adequate to cover the opportunity cost of capital, there remain doubts about economic performance on four grounds. First, the impressive individual commodity IRRs inevitably cover programs that researchers believe have had a significant impact. Failed or weaker lines of research do not surface. Second, as noted earlier, the high average age of adopted varieties in rice, wheat, jute, sugarcane, and potatoes, raises the possibility that returns, at least to breeding work, were very high before the early 1980s but may have fallen substantially since then. Third, the aggregate impact analysis does not adequately accommodate a “without project” productivity scenario—a composite of such things as earlier research benefits streams, the impact of improved infrastructure such as the Jamuna bridge, and indigenous technical knowledge. Fourth, rationalization of the NARS could have greatly reduced costs for the same outcome.

IRRs to selected research are high. Were aggregate analysis possible the IRR would still probably be satisfactory. But, given excessive age of varieties, the on/off nature of research, difficulty of applying new skills with project closure, weakness in research-extension links, and lack of institute rationalization, IRRs are well below what they could have been and they may be falling.

17. We use the term IRR (Internal Rate of Return) since most estimates seen do not include full economic adjustment of financial values into economic.

12

Institutional Development


2.36 Institutional development impact is rated substantial overall. Substantial weight is given here to the shift from T&V to the group-based approach, to the introduction of promising management processes, and to the NAEP. Skills were improved, but due to the late emergence of the NAEP, less than hoped for. Unfortunately, the substantial unfinished business in the improved computerized management information systems was not picked up for completion under ASIRP, an element that would have offered high returns given the sunk costs. The technical assistance made a significant institutional impact, bringing in mostly strong skills from global experience. The Bank’s conditionality on holding down the numbers of Block Supervisor staff was sound with respect to financial sustainability but should have been done through retrenchment not the expedient of the hiring freeze, which has left the legacy of an increasingly unhealthy (in the institutional sense) age profile.

The Bank used the lessons of two previous projects showing the need for “deep institutional reforms”—to develop, with local scientists, a sound set of minimum institutional reforms for ARMP. Most were abandoned to accommodate borrower concerns. They should not have been.


2.37 Institutional development impact was, at best, modest, a rating also given by the ICR.. This assessment is based on five main elements of institutional development—changes in the overall enabling environment, including regulatory changes, changes in organizational structures, changes in management processes, changes in staff incentives, and skills development.

2.38 Enabling Environment. The main shifts were in new legal ordinances for BARC and the ARIs. The impact of this was modest. The BARC ordinance largely restated the already established role of BARC as a coordinating body with no budget allocation responsibility beyond control over any assigned grant programs. The ARI ordinances did achieve something of significance by establishing a more formal requirement that the ARIs pass their research plans through BARC for review, but it did not require the adoption of review recommendations; moreover, ARIs were expected to do this even before the ordinance.

2.39 Organizational Structures. The three separate ministries retained their hold over their own ARIs, and BARC’s place and remit in the organizational structure remained essentially unchanged.18 There was no reduction in the number of institutes or substations. Anomalies such as the overlap of the Bangladesh Institute of Nuclear Agriculture (BINA) with the Bangladesh Rice Research Institute (BRRI) on rice breeding, and the lack of common service rules across institutes, were not addressed. The ICR argues, “Quality at Entry was enhanced by the decision to delay appraisal until agreement was reached with the government on the future role of BARC, solidifying it as an apex organization for NARS (the national agricultural research system) responsible for research organization and, ultimately, research funding.” It is not entirely clear what “solidifying” means in this context, although it is correct that the BARC role was clarified somewhat, but with few real changes. The reference to “and, ultimately, research funding” appears to be wishful thinking. Was it efficient to design a project that largely accepted the

18. The legal relationship between the Boards of Management of the independent institutes and the Executive Council and Governing Body of BARC is not entirely clear. Some ARIs have boards headed by a minister and are therefore politically stronger than the BARC Executive Council. In the absence of legal sanction, the hope was that intellectual authority in redirecting research would make legal authority superfluous. The reality, however, is that BARC has been unable to recruit sufficient high-level scientists to maintain intellectual authority partly due to the predominance of uncertain revenue budget positions.

13

institutional status quo that the previous two research projects had shown to be deficient? In this respect the project files are revealing, as the box below shows.

Box. The Anatomy of a Rationalization

Bank files show that, during preparation, there was a determination that this third research project would not be “more of the same’ with respect to institutional reform. As it turned out, while some gains were made, it largely was.

The Identification Mission Aide Memoire of September 1991 said, “There are a number of fundamental changes that are needed in the…structure of the NARS…. There is a need for a virtual sea change in the way in which agricultural research is viewed, organized…and held accountable…. Research institutions cannot be treated simply as another government department…. If these general parameters are not in place it is likely that additional resources in agricultural research will not be very productive.” A subsequent Bank mission indicated that the key characteristics of a revised NARS would be “a unitary system with considerable autonomy,…(have) no direct links between ministries and institutes…. funding for the NARS should be passed from the government to the governing body…thus research funds would no longer feature in a line ministry budgets,…the governing body should have substantial representation from clients,…rationalization of facilities of regional and substations is required.” Over the subsequent months, following resistance from GOB, the files show a gradual rationalization. Although acceptance of the reforms had originally been a condition for preparation, a mission was still fielded. An internal memo in December 1992 rationalizes that, “we should not be dogmatic on the exact shape of the research system…. We should go back to first principles and look at why research is not working at present.” This after two major studies, a workshop, and two projects had already established why research was not working and produced carefully considered institutional reform proposals. An internal memo then requests Bank staff to, “define again clearly the problems that would be addressed by the reorganization proposed earlier…. With a clear definition of the problems, I am confident that management will not object to a sensible solution even if this represents a major deviation from the proposal which has been promoted so vigorously up until recently.” The files show an inexorable rationalization toward what was termed a “Modified Unitary System.” In fact, the result was not unitary and really constituted a relatively minor adjustment to the institutional status quo—essentially, changes to BARC and ARI Councils/Governing Bodies and ordinances, a unified personnel system (subsequently dropped), and improved management systems/processes that one would have expected anyway under any institutional structure. This sequence of events seems consistent with a general trend of very marginal reform. At this point, when the ARMP final design was largely consolidated, a consultant notes in a Back-To-Office Report of March 1993, “The progress made between the appraisal of the first research project in 1977 and now seems to have been minimal.” The Revised Executive Project Summary of May 1993 hid more than it revealed. Having correctly summarized the earlier recommendations, it reported simply, “With the exception of discontinuing the administrative control of the ARIs by their respective ministries, GOB has agreed to the salient features of the proposed reforms.” The reality was that GOB had largely disagreed with the fundamental elements. GOB would later, at appraisal, announce that it could not implement the main staffing reforms due to civil service rules. Also, the autonomy of the ARIs was extended. There is no evidence of assessment of the impact of this greater autonomy on the relative strengths of BARC and ARIs or on resource allocation decisions.

2.40 In the end, three of the four big institutional issues— the weak position of BARC vis-à-vis the ARIs, the lack of a single decision locus and channel for budgetary allocations, and the inefficiency of the excessive number of institutes and substations—were not confronted in project design. The fourth—the management process improvements—were tackled with some success but to some extent these represent procedural devices to minimize the failure to achieve the initially intended “deep institutional reform.” The underlying problem is, and always has been, the placing of ministry sectoral interests—agriculture, forestry, livestock—above national research coordination interests, and the inability of government at any level—either BARC, or the Ministry of Finance, or Planning, to manage funds to get around this political problem and to finally make substantive progress on research management and efficiency. It is questionable whether climbing down on what had been accepted by Bank management as fundamental

14

institutional reform principles was the right Bank strategy for a third project in a series. It is impossible to characterize the project institutional design as “deep institutional reforms.”

2.41 To be fair, one element in the Bank’s rationalizations with substance was the concern that BARC was too weak to manage the funding of a unified NARS. This was a valid concern but could have been addressed in a phased transition and, alone, was not enough to abandon an overall sound and carefully thought through institutional reform strategy.19

2.42 The Counterfactual. What would have happened had the Bank held to its earlier positions? Almost certainly, there would have been a standoff between the Bank and GOB. Depending on the position of other donors this might have been relatively short-lived with some window of opportunity emerging. It might also have been longer. Would this have mattered? Given the substantial gap between research yields and farmer’s yields (about 70 percent fairly consistently across commodities) and the persistent financial sustainability issue, it is argued that the short-term negative impact would have been relatively modest and the potential positive medium to longer-term impact on research efficiency would have been substantial. The Bank could have waited for ownership while maintaining, within a reasonable band of flexibility, a minimum bottom-line, perhaps in association with some low scenario “holding” lending.

2.43 Management Processes. Gains in this area are one of the main reasons for rating Institutional Development modest rather than negligible. There were important improvements in processes, although the systems established are by no means fully used and some are not yet functional. In particular, the system of prioritization and the establishment of selection procedures for grants were important improvements. The ARI Master Plans have been of some value. However, in the post-project period, the increasing gap between these plans and operational funds availability raises serious questions about the realism of those plans. In one institute, it was estimated that they were currently receiving about one-third of the full plan estimated funding.

2.44 Staff Incentives. While, in theory, performance is now a factor considered in promotions, in practice, with the long-standing national recruitment freeze, constraints on movement between institutes, low salaries,20 and wide fluctuations in availability of operating funds, there are few incentives for performance that can be offered. Moreover, there are still no common service and promotion criteria across institutions.

2.45 Skills Development. This is another area with probable positive impact although there are questions about utilization of training. There were undoubtedly major achievements in arranging and completing training; this accounted for about half the credit amount. Training was financed for 114 doctoral degrees, 17 masters degrees, and 400 short courses. Given the late start due to program management problems, it was, in the end, an impressive achievement in terms of completion of a project input. However, even here there are questions about outcome, impact, and sustainability. First, while the intended skills were largely obtained (with some exceptions due to staff hastily taking whatever course was available to get the funds before project closing), the lack of operating funds on return following project closing raises questions about the newly trained staff’s ability to apply their skills. As one ARI director commented, “It seems a waste to spend money on training and then have no operating funds for the returning trainee to use for research.” Nevertheless, quite a number of those trained do appear to be engaged in significant research.

19. An alternative, although second-best, BARC option would have been dissolving it, shifting the current role back into MOA, and applying the released funds to research.

20. Even staff with a doctoral degree, at the lower levels and with a family and no other source of income, are not far above the global poverty level of $1 per day per capita. Not surprisingly, many of those with the best skills have left.

15

Bank Performance

2.46 Agricultural Support Services Project. Bank performance in ASSP was mixed, but overall is rated satisfactory. The shift to a group approach was appropriate, timely, and, as suggested by the ICR, courageous, although the particular extension methodology was arguably consolidated somewhat prematurely. This has been corrected in the follow-on ASIRP, a Learning and Innovation Loan (LIL), which reverts to a more pluralistic experimental approach. Appraisal drew appropriately on the lessons of experience. Supervision, while modest in intensity at the start, with no apparent start-up workshop and only one supervision in 1993,21 improved subsequently. The result of the Mid-Term Review was a number of sound decisions and adjustments, and given the slower than projected uptake of horticulture activity, appropriately reduced the scale of the agribusiness (Hortex) component.

2.47 Procurement skills and procedures presented some problems. A common Bank misperception is that, because the borrower agency has implemented Bank projects before, it will have the skills to do so again. Due to the extremely high turnover of civil service staff this is rarely the case and project implementation skills need to be built up almost from scratch for each project, especially with respect to procurement, unless there is a permanent project management unit embedded in the agency. There was a conflict between Bank procurement rules or practice and preferences of extension staff taking bikes on hire purchase terms. There are strong brand associations with bicycles in Asia, which procurement requirements ignore at their peril: 5,000 “unacceptable” bicycles were procured.

2.48 Agricultural Research Management Project. Bank performance was mixed, but on balance unsatisfactory. Preparation and appraisal was thorough with respect to technical research aspects and establishment of improved research processes but were very weak in the pursuit of the deep reforms clearly argued for and well justified in preparation documents. The extent of accommodation to a sub-optimal structure was excessive and probably counterproductive to longer-term research efficiency and impact.22 Supervision was generally well done but could not make up for the institutional design flaws and broader civil service constraints. The weighting applied for the rating in this mixed performance rests largely on three arguments. First, after two earlier projects strongly suggesting more fundamental reforms, the failure to achieve fundamental reform yet again warrants a higher weighting than usual and than supervision performance - generally satisfactory supervision which had to muddle through with the institutional structure given. Second, the argument that having more BARC autonomy and control over funding was problematic at the time due to BARC’s weakness would have some validity if it were not for the fact that avoiding reform due to BARC’s weakness suggests a treadmill of not reforming because BARC is too weak to reform, resulting again in BARC being too weak to reform, and so on. Again, a less sustainable argument the third time around than the first. Third, the borrower’s reasons for not reforming more fundamentally were another aspect of the needed reform, more related to the “turf” of participating ministries than to BARC skills. Overall, the Bank appears to have remained a step behind the reform needs of the time through the series of projects, for example, by the MTR in this last project, deciding to settle for trying to get BARC management and skills improved within the largely unreformed structure as a launching pad for a fourth project, a stage that was really reached by the end of the previous project.

21. For some reason, the ICR text says there were no supervisions in 1992 but it lists two in the Annex which do, indeed, seem to have taken place.

22. The experience of the earlier projects was outlined in OED Performance Audit Report No. 13218 dated June 27, 1994, and although this date is later than the ASSP appraisal, the lessons were already largely evident.

16

Borrower Performance

2.49 Agricultural Support Services Project. Overall, borrower performance is rated satisfactory, although a number of fundamental civil service problems outside the immediate project design had an impact on this, as on most, projects. MOA and DAE made a commendable effort to redirect extension toward a demand-based, efficient system backed by improved skills. As indicated in the ICR, performance in the early years was disappointing, mainly due to the lack of an agreed extension policy and some internal DAE resistance to change23. Later, as ownership developed, performance improved. Performance with respect to Hortex was less satisfactory, but also improved later.

2.50 Agricultural Research Management Project. This assessment agrees with the ICR that borrower performance was satisfactory overall, but there are very substantial qualifications here. First, the borrower bears some responsibility for not pursuing more fundamental institutional reforms. Second, as noted in the ICR, bureaucratic procedures caused numerous delays. Third, service rules were never amended as planned. Fourth, staffing issues related to the recruitment freeze and the failure to achieve the revenue budget shift constrained performance. However, there were improvements in borrower performance during the later part of implementation.

Sustainability


2.51 Overall, the sustainability of ASSP is rated non-evaluable due to significant uncertainties about the future. The follow-on ASIRP has assured some resilience and consolidation and further experimentation with extension methods, but after that the future is uncertain. The pluralistic approach outlined in the NAEP, involving an increasingly wide range of significant players and funding, including greater partnership with NGOs, gives some hope for greater stability of support. The decline in Block Supervisor numbers makes financial sustainability more achievable. Extension with bicycle transport can largely be sustained by salaries with modest operating costs and can be leveraged through groups and NGOs and can gradually shift toward some degree of cost recovery in one or two higher-value crops. These factors make sustainability of extension more likely than sustainability of research.

2.52 It was observed that the core processes established are largely being sustained four years after project closing. Those individual processes that have been abandoned, such as the excessively complex Seasonal Extension Monitoring System, appear to have been abandoned for sound reasons. The main sustainability uncertainty is whether, after ASIRP, extension will be sustained at the current rather modest achievement level associated with the lack of operating costs, and difficulties in maintaining staff morale with low salaries, or if it will now, on the basis of lessons learned, encourage government to increase resources to allow the next steps in the NAEP transition to be taken. Another uncertainty is the extent to which future decentralization may place extension funding at lower levels with less central ministry and, presumably, greater client control.

2.53 The agribusiness (Hortex) component is not sustainable in terms of cost recovery. It is largely dependent on government subsidy and donor support, both of which are fragile. It was not expected to be self-financing initially and it would be more realistic to assume that it will never be substantially self-financing. The question is whether it is a high priority for continued subsidy and to what level. Given the soil, climatic, and seasonal timing advantages and low labor costs, there is no reason why Bangladesh should not eventually compete quite well with other horticultural exporting countries, such as Kenya,

23 The borrower in Annex E notes inadequate supervision by IDA with which OED agrees, as suggested in para 2.46.

17

once skills are developed and air freight volume increases. This promising potential demand, along with the employment potential, is an argument for continued infant-industry subsidy for a limited period (another five years perhaps). But criteria for exit need to be established now so that there is policy clarity and milestones.

2.54 The sustainability of the Seeds Component is unlikely in the absence of both the removal of subsidies and a revolving fund arrangement.


2.55 Sustainability in ARMP is rated unlikely. The history of research funding (see figure) shows substantial funding available during periods of significant donor projects, for example 1990 to 1992 for the end of the Agricultural Research II Project and 1997 to 2001 for the peak of ARMP, falling to very low levels after projects close. In constant terms, total GOB expenditure for the fiscal year commencing 2002 will be about the same as in 1990/91 despite significant agricultural GDP growth over that period. The total expenditure for agricultural research in real terms including donors is now about 50 percent of the 2000 peak, which, as can be seen in the chart, was a blip. The Revenue Budget allocation has declined

in real terms since about 1993. This is not surprising since, essentially, the Revenue Budget is simply a reflection of numbers of staff, and there has been a recruitment freeze. Operating costs come almost entirely from “projects,” either government or donor projects, so there is never a reliable operational expenditure platform for sustaining ongoing core work and travel capacity—for example, to remain in touch with farmers and extension staff to be responsive to new potential lines of enquiry.

Research Expenditure in Constant 1996 Taka

0

200

400

600

800

1000

1200

1400

1600

1800

1990 1992 1994 1996 1998 2000 2002Year (Start of FY)

Mill

ion

Taka

ADP Budget

Revenue Budget

Total GOB

Donor

Total GOB and Donor

2.56 Financial sustainability was inadequately addressed in the Staff Appraisal Report. After two earlier projects that had faced issues of financial sustainability, the SAR should have contained a very thorough analysis of the history of operating and capital financial allocation by GOB, the prospects for increases, and, depending on the outcome of that analysis, measures to contain expenditure, for example, through rationalization of institutes and substations. While allocation of adequate research funds is listed

18

as one of four main risks,24 this refers only to funding during the project period, which has never been the biggest issue. The appraisal report estimate of annual operation and maintenance costs related to project investments would call for about Tk135 million annually in the post-project period. It is difficult to see where those funds are now provided for since the total government revenue and development allocation for the year following ARMP closure actually fell by about Tk190 million over the previous year.

2.57 And there are other manifestations of problems with sustainability: the partial collapse of the Farming Systems Research program following the closing of ARMP with a number of staff either not being paid or being laid off, widespread problems with research program operating costs, and difficulties with infrastructure and equipment maintenance.

2.58 The ICR, while raising concerns about sustainability, argues, “ultimately…sustainability… involves using (the) new skills and procedures to manage the research system within any future budget envelope…” (emphasis added). While certainly the skills and procedures developed under the project are of value, sustainability cannot be completely divorced from the issue of scale. The evidence that has accumulated over the span of three projects suggests that the scale is simply too big to sustain, at least until private research funding reaches a greater share. The two main arguments justifying higher levels of expenditure are, first, that the percentage of agriculture GDP spent on agricultural research is below the average globally and for South Asia, and, second, that there are quite good economic rates of return. However, after about two decades of problems with funding operating costs, it may be time to face the reality that a substantial cutback in the number of institutions and substations and research staff may be unavoidable, but only within a framework that ensures an improved ratio of operating costs to salary.25

ARMP sustainability is unlikely. Resources peak with donor projects and crash afterwards. After three projects, the argument that what matters for sustainability are really processes not the scale of the NARS is difficult to sustain—the scale seems fiscally unmanageable. Substantial pruning is overdue.

2.59 Indicative of the scale of the sustainability problem, the BARC Vision for Agricultural Research document projects a need for the year 2003 of about Tk2,350 million, whereas the 2002/03 financial allocation (still probably considerably higher than the actual), including the donor component, is about 40 percent of this, which raises questions about realism.

2.60 With respect to environmental sustainability, no major problems were observed. Bangladesh has experienced increasing problems with pesticide impact on fisheries, and research has tried to address this problem. However, in this case, the solutions lie as much with community-based approaches as with regulation. On private land the increase in rice-fish systems augurs well for reduced insecticide use. There are a number of actual and potential problems with shrimp culture, which also have been the subject of some research. There has been research on integrated pest management and application through farmer field schools under a DANIDA-supported project. While fertilizer application is rising and needs monitoring, the levels of application are still relatively modest by South Asia standards. Unbalanced

24. The SAR simply says: “These risks are addressed as follows:….A satisfactory arrangement has been made to transfer adequate allocations to the revenue budget for the project. The government agreed at negotiations that adequate funds would be made available for the project.” There is no analysis of the post-project scenario.

25. The dilemma for the line ministries and for the research service vis-à-vis Finance and Planning in staff and station reductions is that in a system where operating costs allocation is almost wholly correlated with staff numbers, cutbacks in institutes and staff numbers are likely to be matched almost exactly by cutbacks in budget allocation. There is no management incentive. What is needed is a commitment to maintain a budgetary allocation at a certain percentage of agriculture GDP, leaving BARC and the institutes free to shift those funds between salaries and operating costs. This would require more financial autonomy.

19

nutrient use is widely reported, although in the one district where the mission looked at the aggregate district fertilizer purchase figures they did not seem to be excessively unbalanced.

Lessons

The following are the main lessons from the two projects:

2.61 The lessons of previous projects and the diagnosis of constraints emerging during preparation should lead to a set of reforms identifying the minimum for an effective and efficient project which should be clearly articulated in the evolving Bank management documentation. Any changes, whether the result of new knowledge or borrower concerns, should be justified with further analysis and be clearly documented. (In ARMP, expediency seems to have overtaken analysis, and rationales in the evolution of the design during preparation and appraisal were not adequately recorded towards the end as the pressure to lend built.)

2.62 Adoption of new unified national extension approaches should be based on evaluated cost-effectiveness data. It pays to maintain experimentation with a number of promising alternative approaches at different cost levels for a considerable period, if not permanently, since the target is cost-effectiveness and since resources and technical challenges can be expected to change over time.

2.63 Cutting costs by institutional rationalization—generally calling for the closing of research stations and substations—is politically difficult, but too seldom faced by either the borrower or the Bank. It may hold as much potential for improving efficiency as increasing benefits and is a quicker route to financial sustainability.

2.64 Bank Project Appraisal Documents (PADs) should include: a thorough analysis of past capital and operating cost allocation and donor support to the sector or subsector; a projection of a reasonable expectation of future resources available from borrower and donors; leading to a realistic assessment of the expected financial sustainability in the post-project period. Too frequently the focus of appraisal documents is on counterpart funding during the project but not afterwards.

2.65 Startup workshops and intensive early supervision with a particular focus on procurement can reduce implementation delays, which can seriously damage the logical coherence and sequencing of a project. A common misconception is that borrower institutions that have implemented Bank projects before have adequate procurement experience. With the high staff turnover of most government departments this is rarely the case.

2.66 The level and rate of introduction of computerized management systems needs to be carefully matched to the current and realistically projected computer skills and technical support capacity.

Future Directions

2.67 The following selected issues, a number of them long-standing ones from the past, are high priorities for improved research and extension efficiency in Bangladesh:

• The vision for agricultural research in the future should include in particular: greater attention to the enabling framework and less to the financing of particular research; a central body with strong skills and authority over funding allocation (as for the University Grants Commission); incentives for research performance either through substantial independence from civil service rules or substantial civil service reform; a major rationalization including closure of institutes and

20

substations; improved incentives for research/extension/farmer feedback coordination; and resolution of the financial sustainability issue.26

• A review of the adoption level and adoption rate of improved varieties of different release age to better understand the adoption lag and possible reasons, and to better understand the economic returns to breeding research.

• Quick action on the outstanding issues of seeds policy and subsidies with the aim of sustainability. • A review of, and decisions on, the future of CERDI. • A review of the economic impact of ATI expenditure now and over the next 10 years to assess the

extent to which that investment is impacting on agricultural production. • A continuation of experiments in extension methods with strong monitoring of cost-effectiveness

(particularly farmer field schools if these are to be used more widely) with a focus on lowering costs perhaps through increased audiovisual and information technology means.

26. Probably through some form of government commitment to fund at a certain percentage of agriculture GDP associated with a phased program to increase the share of private, NGO, user fee, and commodity cess funding.

21 Annex A

Annex A. Basic Data Sheet AGRICULTURAL RESEARCH MANAGEMENT PROJECT (CREDIT 2815-BD)

Key Project Data (Amounts in US$ million)

Appraisal Estimate

Actual or current estimate

Actual as percent of Appraisal estimate

Total project costs 59.09 49.49 84 Credit amount 50.00 42.86 86

Project Dates

Original Actual Initiating memorandum 08/19/91 Negotiations 04/95 Board Approval 02/08/96 Signing Effectiveness 07/31/96 07/31/96 Closing date 12/31/01 12/31/01

Staff Inputs (staff weeks)

Actual Weeks Actual US$000 Preappraisal 139.7 349.9 Appraisal/Negotiations 44.8 98.1 Supervision* 260.0 489.6

Completion** 7.5 37.2 Total 452.0 974.8

Mission Data Performance rating Date

(month/ year)

No. of Persons

Specializations represented Implementation

Status Development

objectives Identification/ Preparation

09/1991 2 2 AG

Appraisal/ Negotiation

04/1995 7 4 AG, 1 IE, 1 AE, 1 EC

Supervision 1 Supervision 2 Supervision 3 Supervision 4 Supervision 5 Supervision 6 Supervision 7 Supervision 8 Supervision 9 Supervision 10 Supervision 11

05/1996 02/1997 09/1997 03/1998 08/1998 02/1999 06/1999 10/1999 06/2000 12/2000 07/2001

2 4 4 7 8 8 2 7 6 7 8

1 AE, 1 AG 1 AG, 1 IE, 1 DO, 1 PS

1 AG, 1 DO, 1 OO, 1 FMS 1 AG, 1 DO, 1 OO, 1 FMS, 1 TA

2 AG, 1 OO, 1 FMS, 1 DO, 1 PS, 1 SAES, 1 TA 3 AG, 1 SAES, 1 FS, 1 DO, 1 PS, 1 IS (FAO-CP), 2

AG 2 AG, 1 AE, 1 FS, 1 DO, 1 PS, 1 TA

2 AG, 1 PS, 1 DO, 1 FM, 1 TA 2 AG, 1 AE, 1 PS, 1 DO, 1 FMS, 1 TA

2 AG, 1 EC, 1 DO, 1 PS, 1 FMS, 1 TA, 1 RMS (FAO-CP)

S S S U U U U S S S S

S S S U U U U S S S S

ICR 01/2002 2 1 RMS (FAO/CP), 1 AE (FAO/CP) AE – Agricultural Economist, AG – Agriculturalist, IE – Irrigation Engineer DO – Disbursement Specialist, PS – Procurement Specialist, OP – Operations Officer, FMS – Financial Management Specialist, TA – Team Assistant, SAES – Senior Agricultural Extension Specialist, EC – Economist, RMS – Research Management Specialist, IS –Institutional Specialist, RMS – Research Management Specialist, FA – Financial Analyst

23 Annex A


Key Project Data (Amounts in US$ million)

Appraisal Estimate

Actual or current estimate

Actual as percent of Appraisal estimate

Total project costs 59.4 55.2 93 Credit amount 35.0 22.7 65

Cumulative Estimated and Actual Disbursements

FY92 FY93 FY94 FY95 FY96 FY97 FY98 FY99 Appraisal estimate 0.10 7.70 18.90 25.00 30.40 33.00 35.00 35.00 Actual (US$M) 1.44 2.40 5.99 8.22 11.44 15.85 21.95 22.70 Actual as percent of appraisal 1440 31 32 33 38 48 63 65

Project Dates

Original Actual Initiating memorandum 06/90 Negotiations 02/22/91 Board Approval 05/02/91 Signing 06/17/91 Effectiveness 09/17/91 12/30/91 Closing date 06/30/98 06/30/98

Staff Inputs (staff weeks)

Actual Weeks Actual US$000 Preappraisal 96.6 119.1 Appraisal 46.5 107.1 Negotiations thru Board 12.1 36.0 Supervision* 324.0 275.0

Completion** 31.1 119.5 Total 510.3 656.7

24 Annex A

Mission Data Performance rating Date

(month/year) No. of

persons Staff days

in field Specializations

represented Implementation Status

Development objectives

Through Appraisal

6/89 9 A, PO, Ps

Appraisal through Board Approval

6/90 10 A, Ps, P, F, I

Post Appraisal 6/89 9 Appraisal through Board Approval

6/90 10

Supervision 1 Supervision 2 Supervision 3 Supervision 4 Supervision 5 Supervision 6 Supervision 7 Supervision 8 Supervision 9 Supervision 10 Supervision 11 Supervision 12 Supervision 13

7/91 3/92

11/92 4/93 4/94 1/94

11/94 5/95

12/95 5/96

12/96 8/97 7/98

3 2 4 5 4 4 5 3 3 4 3 5 4

0 0 0 0 0 0 0 0 0 0

11.0 6.0 8.0

A, I, Ps A, D

A, DA, P, Ps A, DA, P, Ps A, DA, P, Ps

A, D, P, A, I, P, PA

A, I, P D, PO, SPO DS, PO, PS

A, I, P A, DS, F, I, P

D, F, I

S HS S S S S S S S S S S S

S S S S S S S S S S S S S

Completion* S S A=Sr. Agriculturalist D = Sr. Disbursement Officer DA = Assistant Disbursement Officer DS = Disbursement Specialist F = Financial Management I = Irrigation Engineer

P = Program officer PA = Principal Agriculturalist PO = Project Officer Ps = Sr. Program Officer POS = Sr. Project Officer PS = Procurement Specialist

Other Project Data Borrower/Executing Agency: FOLLOW-ON OPERATIONS Operation Credit no. Amount

(US$million) Board date

Agricultural Services Innovation and Reform Project Cr.32840(a LIL)

5.0 09/14/1999

25 Annex B

Annex B: Brief Summary of Extension Impact Study Findings Two extension impact studies offer somewhat different findings on impact. One has remained in draft because Department of Agricultural Extension (DAE) officials have doubts about the methodology and findings, the other has been formally released. However, neither offers data on actual observed adoption or incremental incomes from it.

The Extension Services Survey 2000 (ESS2000) Initial and Additional Analysis Draft Report of December 2001, based on a survey of 2,787 randomly selected respondents, found that DAE staff contacted at least once within the last year 7.7 percent of male respondents and 1.7 percent of female respondents. They may have been visited more than one time. Other department contact rates (fisheries, livestock, forestry) were lower, between 3.3 percent and 4.5 percent for male respondents and again much lower for female. The DAE contact rate was substantially higher than NGOs—not surprisingly since DAE have more staff in the field—but NGOs did better in contacting women. Of those households that received advice, the percentage that said they will use it next time was high, about 90 percent, depending on subsector/commodity type, but there are obvious problems with this type of question. Of those households that had already followed advice, about 90 percent said they achieved good results. This finding of about a 7.5 percent contact rate is referred to in DAE’s Strategic Plan for 1999 to 2002, which notes that the contact rate had been as high as 15 percent during ASSP but declined due to the end of World Bank funding for core of extension work. Globally, in developing countries, contact rates of around 7 to 12 percent have been common.

The Impact Assessment Program Final Report, February 1999, also reports a 7.5 percent participation in “field extension events.” It is not entirely clear what the relationship is between these two sets of findings showing a 7.5 percent contact rate but following somewhat different definitions. The study found a participation rate by small and marginal farmer categories and the landless of 61 percent, noting that this did not reach the target of 70 percent because “larger farmers also attend events.” It estimated an economic rate of return of 48 percent. It found inadequate progress toward the objective of ensuring that district extension plans reflected farmers’ identified needs. The report also notes a number of problems with which this assessment largely concurs. It says, “Staff morale and performance is low and the financial function suffers an acute shortage of senior staff because of inadequate succession planning. Training management has been ineffective and as a result much of the staff training has not resulted in improved delivery of relevant messages to farmers; field staff still have difficulty undertaking a truly farmer oriented, participatory method of needs diagnosis. Targeting criteria and methods of farmer information needs assessment require overhaul.… Partnerships with other extension providers have not progressed beyond the supply of limited technical services to NGOs, and DAE staff await directives rather than take initiatives. These residual attitudes threaten many of the achievements in recent years and planning, monitoring and work programming systems, which are at the heart of DAE’s implementation of the New Agricultural Extension Policy, are in danger of slipping back into ritual form filling.… ASSP can take credit for making considerable breakthroughs which have started to change the course of agricultural extension in Bangladesh. However the process is at a delicate stage and few, if any, of the changes are currently sustainable. It therefore now falls to DAE to develop the process further under the forthcoming Agricultural Services Innovation and Reform Project (ASIRP).”

With respect to farm-level impacts, the same study followed up participants at 71 demonstrations in four districts in early 1999 and related this to a baseline survey initiated in 1996. (At the peak of the project an impressive 91,000 demonstrations were held, but this has fallen by at least 50 percent since project close.) The findings of this study involve quite a number of debatable assumptions and the study followed only a small sample of those people who were recorded as having actually attended a field extension event. By extrapolating from the numbers of demonstrations and using assumptions about numbers of farm

26 Annex B

households per block, the study concluded that the proportion of farm households contacted through these events rose from 2.8 percent at the baseline, reached 9 percent in 1997/98, and then dropped back to 5 percent. However, the report argues that there are many other projects operated by DAE outside the demonstration program and, therefore, that the number of households contacted should be doubled. Therefore, the 15 percent target contact rate in 1998 was achieved but was not sustained and has now dropped back to 7.5 percent, which is consistent with the other study. The study finds the proportion of demonstrations that targeted small, marginal, or landless farmers was 71 percent. The study found that 53 percent of participants in these events had a reasonable memory of, and claimed to have tested, the technology. Of those who tested a technology about 95 percent said they would use it again the following season. Answers to questions such as this are difficult to interpret. The study concluded that district extension plans do not yet properly reflect the needs of farmers because of concerns over the quality of Farmer Information Needs Assessments. The economic analysis carried out as a part of this study is covered in the section on Efficiency.

27 Annex C

Annex C. Comments from the Borrower

28 Annex D

29 Annex D

30 Annex D

31 Annex D

Annex D: Comments from the Ministry of Fisheries and Livestock, Planning and Evaluation Wing

32 Annex D

33 Annex D

35 Annex E

Annex E: Comments from the Ministry of Agriculture

36 Annex E

37 Annex E

38 Annex E

39 Annex E

40 Annex E

41 Annex E

Helen Phillip O:\RURAL\Nelson\BangResExt\BangPPAR4.DOC May 14, 2003 4:37 PM

PERFORMANCE AUDIT REPORT - World Bank · 2016. 8. 5. · The ASSP, cofinanced by a DFID US$14.7...

Documents

Transcript of PERFORMANCE AUDIT REPORT - World Bank · 2016. 8. 5. · The ASSP, cofinanced by a DFID US$14.7...