The World Bank · the world bank report no: icr0000713 implementation completion and results report...
51
Document of The World Bank Report No: ICR0000713 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-37820, IDA-3782A, IDA-H0450) ON A CREDIT IN THE AMOUNT OF SDR 24.10 MILLION (US$ 32.80 MILLION EQUIVALENT) AND GRANT IN THE AMOUNT OF SDR 20.00 MILLION (US$ 27.20 MILLION EQUIVALENT) TO THE REPUBLIC OF MALAWI FOR A THIRD SOCIAL ACTION FUND IN SUPPORT OF THE FIRST PHASE OF THE COMMUNITY EMPOWERMENT AND DEVELOPMENT PROGRAM December 23, 2008 Human Development 1 Southern Africa 2 Africa Region Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized Public Disclosure Authorized
Transcript of The World Bank · the world bank report no: icr0000713 implementation completion and results report...
Document of The World Bank
Report No: ICR0000713
IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-37820, IDA-3782A, IDA-H0450)
ON A
CREDIT
IN THE AMOUNT OF SDR 24.10 MILLION (US$ 32.80 MILLION EQUIVALENT)
AND GRANT
IN THE AMOUNT OF SDR 20.00 MILLION (US$ 27.20 MILLION EQUIVALENT)
TO THE
REPUBLIC OF MALAWI
FOR A
THIRD SOCIAL ACTION FUND
IN SUPPORT OF THE FIRST PHASE OF THE COMMUNITY
EMPOWERMENT AND DEVELOPMENT PROGRAM
December 23, 2008
Human Development 1 Southern Africa 2 Africa Region
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
Pub
lic D
iscl
osur
e A
utho
rized
CURRENCY EQUIVALENTS
(Exchange Rate Effective July 31, 2008)
Currency Unit = Malawi Kwacha (MK) 1.00 = US$ 0.007143 US$ 1.00 = MK 140
FISCAL YEAR July 1 – June 30
ABBREVIATIONS AND ACRONYMS
APL Adaptable Program Loan MASAF I First Malawi Social Action Fund CAS Country Assistance Strategy MASAF II Second Malawi Social Action Fund CBOs Community Based Organizations MDGs Millennium Development Goals CDD CDPs
Community-Driven Development Community Development Projects
M&E MFIs
Monitoring and Evaluation Micro Finance Institutions
CMPs COMSIC COMSIG COMSIP DDPF DEC DFID EPAs EU FIMTAP GOM ICB IEC IFAC IPF KISS LA LAMP LAMIS
Community Managed Projects Community Savings and Investment Club Community Savings and Investment Group Community Savings and Investment Promotion District Development Planning Framework District Executive Committee Department for International Development Extension Planning Areas European Union Financial Management, Transparency and Accountability Project Government of Malawi International Competitive Bidding Information, Education and Communication International Federation of Accounts Indicative Planning Figure Knowledge Information Sharing System Local Authority Local Authority Managed Project Local Assembly Management Information System
MOF MPP MU NACCEA NCB NGOs OPC PRA PWP-CCT SSP TAP UND VAM WB ZO
Ministry of Finance Microprojects Management Unit National Advocacy Committee for Community Empowerment and Accountability National Competitive Bidding Non-Governmental Organizations Office of the President and Cabinet Participatory Rural Appraisal Public Works Programme-Conditional Cash Transfer Social Support Program Transparency and Accountability Promotion United Nations Development Vulnerability Assessment Mapping World Bank Zone Office
Vice President: Obiageli K. Ezekwesili Country Director: Michael Baxter
Sector Manager: Christopher J. Thomas Project Team Leader: Nginya Mungai Lenneiye
ICR Team Leader: Hardwick Tchale
MALAWI THIRD SOCIAL ACTION FUND
CONTENTS
Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph
1. Project Context, Development Objectives and Design.............................................. 12. Key Factors Affecting Implementation and Outcomes ............................................. 63. Assessment of Outcomes ......................................................................................... 104. Assessment of Risk to Development Outcome........................................................ 185. Assessment of Bank and Borrower Performance .................................................... 206. Lessons Learned ...................................................................................................... 217. Comments on Issues Raised by Borrower ............................................................... 23Annex 1. Project Costs and Financing......................................................................... 24Annex 2(a): Outputs by Component ............................................................................ 25Annex 2(b): Baseline and MASAF contributions towards Millennium Development Goals ............................................................................................................................. 26Annex 3. Economic and Financial Analysis ................................................................ 28Annex 4. Bank Lending and Implementation Support ................................................ 30Annex 5. Beneficiary Survey Results .......................................................................... 32Annex 6. Stakeholder Workshop Report and Results.................................................. 35Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR.................... 38Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders...................... 39Annex 9. List of Supporting Documents ..................................................................... 40Annex 10. List of Technical Research Studies Commissioned by MASAF under .... 41Annex 11. Allocation of MASAF III resources to LAs.............................................. 42MAP. IBRD 31130 ...................................................................................................... 44
A. Basic Information Country: Malawi Project Name:
Third Social Action Fund (MASAF III)
Project ID: P075911 L/C/TF Number(s): IDA-37820,IDA-3782A,IDA-H0450
ICR Date: 12/23/2008 ICR Type: Core ICR
Lending Instrument: APL Borrower: GOVERNMENT OF THE REPUBLIC OF MALAWI
Original Total Commitment:
XDR 44.1M Disbursed Amount: XDR 44.0M
Environmental Category: B Implementing Agencies: Malawi Social Action Fund Management Unit, Office of the President and Cabinet Cofinanciers and Other External Partners: B. Key Dates
Process Date Process Original Date Revised / Actual Date(s)
Concept Review: 01/24/2002 Effectiveness: 04/08/2003 04/08/2003 Appraisal: 03/25/2002 Restructuring(s): Approval: 05/30/2002 Mid-term Review: 02/01/2006 01/30/2006 Closing: 12/31/2006 12/31/2007 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Moderate Bank Performance: Satisfactory Borrower Performance: Moderately Satisfactory
C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings
Quality at Entry: Satisfactory Government: Moderately Satisfactory
Quality of Supervision: Satisfactory Implementing Agency/Agencies: Satisfactory
Overall Bank Performance: Satisfactory Overall Borrower
Performance: Moderately Satisfactory
C.3 Quality at Entry and Implementation Performance Indicators Implementation
Performance Indicators QAG Assessments (if any) Rating
Potential Problem Project at any time (Yes/No):
Yes Quality at Entry (QEA):
None
Problem Project at any time (Yes/No):
No Quality of Supervision (QSA):
Moderately Satisfactory
DO rating before Closing/Inactive status:
Satisfactory
D. Sector and Theme Codes
Original Actual Sector Code (as % of total Bank financing) General agriculture, fishing and forestry sector 10 10 General water, sanitation and flood protection sector 10 10 Micro- and SME finance 10 10 Other social services 60 60 Roads and highways 10 10
Theme Code (Primary/Secondary) Gender Secondary Secondary Other social protection and risk management Primary Primary Participation and civic engagement Primary Primary Social analysis and monitoring Secondary Secondary Social safety nets Secondary Secondary E. Bank Staff
Positions At ICR At Approval Vice President: Obiageli Katryn Ezekwesili Callisto E. Madavo Country Director: Michael Baxter Hartwig Schafer Sector Manager: Christopher J. Thomas Dzingai B. Mutumbuka Project Team Leader: Nginya Mungai Lenneiye Norbert O. Mugwagwa ICR Team Leader: Nginya Mungai Lenneiye ICR Primary Author: Hardwick Tchale F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) To empower individuals, households, communities, and their development partners in the implementation of measures which can assist them in better managing risks associated with health, education, sanitation, water, transportation, energy and food
insecurity, and to provide support to the critically vulnerable through a variety of sustainable interventions. Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s)
Indicator Baseline Value
Original Target Values (from
approval documents)
Formally Revised Target Values
Actual Value Achieved at
Completion or Target Years
Indicator 1 : (1) Proportion of individuals living on less than $1 a day. Value quantitative or Qualitative)
55% 44.2%
Date achieved 01/01/2000 12/31/2007
Comments (incl. % achievement)
This was not determined, but the current Welfare Monitoring Survey indicates that the poverty headcount has reduced from 52% to 40%. Such reduction in poverty cannot all be attributed to MASAF interventions under the APL 1. However, under the APL1, a total of 815,557 persons were employed and received US$14.6 million in wages. Over 1.5 million people had access to socio-services.
Indicator 2 : Grade 1 children (5) reaching grade 5. Value quantitative or Qualitative)
20% 48% 90% (for 2015) 73%
Date achieved 01/01/2000 12/31/2007
Comments (incl. % achievement)
238,438 pupils in primary school every term; Primary school facilities tracked by: 471 classroom blocks, 462 toilets, 10 offices, 80 water boreholes, 65 woodlots, 118 teacher houses and 273 functioning school committees and Parent Teacher Associations.
(b) Intermediate Outcome Indicator(s)
Indicator Baseline Value
Original Target Values (from
approval documents)
Formally Revised
Target Values
Actual Value Achieved at
Completion or Target Years
Indicator 1 : Population with improved access to social services (water, health, education, etc) Value (quantitative or Qualitative)
0 494,000 N/A 1,594,857
Date achieved 01/01/2000 12/31/2007 Comments (incl. % achievement)
The number of people that have directly been reached under the various interventions implemented under MASAF 3 APL 1.
Indicator 2 : Persons with employment from PWP-LAMPs Value (quantitative or Qualitative)
0 72,000 N/A 141,975
Date achieved 01/01/2000 12/31/2007
Comments (incl. % achievement)
Achievement on regular LAMP only.
Indicator 3 : Persons with employment from PWP-CCT Value (quantitative or Qualitative)
0 314,361 N/A 673,582
Date achieved 01/01/2000 12/31/2007 Comments (incl. % achievement)
Achievement on PWP-CCT beneficiaries only (for both 2005 and 2007 operations).
Indicator 4 : Households reached with assistance under SSP and PWP Value (quantitative or Qualitative)
0 90,000 72,000 75,336
Date achieved 01/01/2000 12/31/2007 Comments (incl. % achievement)
Benefits from outputs in completed SSPs.
Indicator 5 : Persons participating in savings groups Value (quantitative or Qualitative)
0 96,000 8,000 savings groups 8,392 groups
Date achieved 01/01/2000 12/31/2007 Comments (incl. % achievement)
These are savings groups comprised of 10-15 members each.
G. Ratings of Project Performance in ISRs
No. Date ISR Archived DO IP
Actual Disbursements (USD millions)
1 10/23/2003 Satisfactory Satisfactory 0.00 2 04/28/2004 Satisfactory Satisfactory 6.00 3 10/27/2004 Satisfactory Satisfactory 6.92 4 05/05/2005 Satisfactory Satisfactory 16.31 5 12/13/2005 Moderately Satisfactory Moderately Satisfactory 48.73 6 05/08/2006 Moderately Satisfactory Moderately Satisfactory 61.90 7 11/16/2006 Satisfactory Satisfactory 61.90 8 06/28/2007 Satisfactory Satisfactory 64.80 9 12/28/2007 Satisfactory Satisfactory 64.80
H. Restructuring (if any) Not Applicable
I. Disbursement Profile
1
1. Project Context, Development Objectives and Design
1.1 Context at Appraisal 1. At the time of appraisal, Malawi was rated one of the poorest countries in the world and was ranked fourth from the bottom by the United Nations (UN Human Development Report 2000). An estimated 65 percent of the population was living below the poverty line, and poverty was seen to be endemic in rural and peri-urban areas, according to the Malawi Integrated Household Survey conducted in 19981. Social welfare indicators were poor: high illiteracy (about 41 percent; 36.2 percent for men and 54.7 percent for women), life expectancy estimated at 37 years while infant and child mortality rates were estimated at 104 and 189 per 1000 live-births, respectively. Maternal mortality had more than doubled from 620 to 1,200 per 100,000 between 1992 and 2000. An estimated 43 percent of the population lacked access to safe and portable water (Malawi Demographic and Health Survey 2000). Malawi also faced high rate of HIV/AIDS incidence, estimated at 15 percent and this resulted in loss of life of people in the productive age group of 15 to 49 years, consequently contributing to an increase in orphans (National AIDS Commission 2002)2. 2. The Malawi Government, in a bid to address these social welfare and development challenges launched the Community Empowerment and Development Programme (Third Social Action Fund) in 2003 as an instrument of achieving sustainable poverty reduction. The project was designed within the context of the Malawi Poverty Reduction Strategy adopted in 2002 and was therefore in line with the Government service delivery reforms aimed at achieving sustainable poverty reduction. 1.2 The Historical Context of Social Fund Projects in Malawi 3. Malawi started implementing the first generation social fund project (MASAF I) in 1995 with a US$ 56 million IDA credit. Government envisaged to use MASAF as an instrument of achieving broad-based poverty reduction as stipulated in the Poverty Alleviation Plan unveiled in the same year. MASAF I was implemented between 1995-2000, but when funds were fully committed, a successor project, MASAF II, was initiated with another IDA credit worth US$ 66 million, which ran from 1998-2002. As such, between 1995-2002, the Government invested a total of US$ 122 million in community infrastructure in education, health, water supply and sanitation, postal services, markets, roads, bridges and other rural access infrastructure. It is estimated that,
1 The 2004/05 Integrated Household Survey estimated the poverty level at 52 percent and more recently, the National Statistical Office, based on results from the Welfare Monitoring Survey has reported that overall poverty level has reduced to 40 percent.
2 HIV/AIDS prevalence is also reported to have declined to 12 percent.
2
as a result of the community investments, a total of 5.8 million people had increased access to social services through MASAF I (1995-2000) and MASAF II (1998-2002). (see Figure 1).
Source: Malawi: An Atlas of Social Statistics, National Statistical Office, 2002. 4. In order to consolidate achievements of MASAF I and II, the Government of Malawi decided to implement the Community Empowerment and Development Program as an operational framework for the 2002 Malawi Poverty Reduction Strategy. The aim was to continue with interventions to assist the Government to incrementally attain sustainable poverty reduction at the community level.
Figure 1: Map of Malawi showing locations of projects and concentration of beneficiaries of MASAF I and II.
3
5. MASAF III was therefore designed as a three-phased program to be implemented over a 12-year period. Each phase, called an Adaptable Programme Loan (APL), was aimed at addressing specific challenges undermining effective service delivery, so that communities could progressively move towards the attainment of the Millennium Development Goals (MDGs). The design of APL 1 was synchronized with a Medium Term Expenditure Framework and a Community Demand-Driven (CDD) approach being the key instruments for achieving sustainable poverty reduction within a decentralized governance environment.
1.3 Original Project Development Objectives (PDO) and Key Indicators (as approved) 6. The development objective of MASAF III APL I was: “to empower individuals, households, communities, and their development partners in the implementation of measures which can assist them in better managing risks associated with health, education, sanitation, water, transportation, energy and food insecurity, and to provide support to the critically vulnerable through a variety of sustainable interventions”. 7. Specifically, the project was aimed at integrating the community empowerment processes into district planning and deliver community service packages to better align communities, CBOs/NGOs and LAs with the District Development Planning Framework (DDPF). 8. The key performance indicators included: (a) the number of households with improved service access and status (health, water, sanitation, education); (b) number of LAs and CBOs/NGOs implementing interventions compliant with the CDD approach; and (c) number of households with measurable cash incomes from Local Authority Managed Projects (LAMPS), Social Support Program (SSP), and Community Savings and Investment Promotion (COMSIP); all towards meeting 12 MDG indicator targets.
1.4 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 9. The PDO and key performance indicators were not revised.
1.5 Main Beneficiaries
10. The main project beneficiaries were: (a) poor communities with inadequate access to social services such as schools, clinics, water, and effective transport; (b) poor households experiencing food insecurity; (c) vulnerable individuals unable to organize interventions for themselves and needing the mediation of CBOs/NGOs; (d) communities, who after participating in MASAF activities, are able to participate in group savings; (e) implementing agencies such as PMCs, COMSICs, NGOs/CBOs, micro-finance institutions, and LAs. The project was meant to strengthen the capacity of communities, community leadership, and CBOs to effectively implement activities aimed at addressing their development needs through improved access to social services. The project was
4
targeted to reach nearly two-thirds of Malawi’s poor population whose work would contribute to the achievement of the 12 MDG targets for Malawi.
1.6 Original Components (as approved) 11. MASAF III APL I had five components described in the Project Appraisal Document as follows:
(i) Community Development Projects (CDPs): This comprised financing windows for Community Managed Projects (CMPs) and LAMPs. The component supported demand-driven and community-based socio-economic infrastructure. This was meant to build the capacity of communities to articulate their priority development concerns within the social fund criteria and the DDPF. For CMPs and LAMPs, the average support from IDA was US$20,000 and US$40,000, respectively. Community contributions were normally required in kind in form of local materials and labour. At appraisal, the estimated cost for this component was US$ 28.48 million, but the actual cost was US$ 36.83 million (see Annex 1(a)).
(ii) The Social Support Program (SSP): This component directly targeted
vulnerable groups3 through NGOs/CBOs to stimulate and support traditional coping systems. Activities that were funded through this component varied depending on the specific needs of the vulnerable groups and the nature of the risks affecting them. The average cost of IDA’s contributions to SSPs was US$10,000, and could not exceed US$30,000. At appraisal, the estimated cost of this component was US$ 24 million, but the actual cost was US$ 19.94 million.
(iii) The Community Savings and Investment Promotion (COMSIP): This
component was used to finance community savings mobilization through provision of business skills training and provision of 80% of the costs towards the construction of infrastructure required for Savings Clubs. The component built on the experiences from the SSP component of MASAF II where groups had increased their capacity to generate incomes which they used to provide assistance to orphans and other vulnerable groups. COMSIP also facilitated the formation of Community Savings and Investment Groups (COMSIGs) and 10-15 such groups would form COMSICs. This facilitated the sustainable income generation among vulnerable members of the community. At appraisal, the estimated cost for this component was US$ 11.87 million, but the actual cost was US$ 2.12 million.
3 Including, but limited to, widows/widowers, aged, orphans and foster parents, destitute, disabled, persons affected by HIV/AIDS and malnourished under-five children.
5
(iv) Transparency and Accountability Promotion: This was a cross-cutting
component aimed at enhancing the capacity of implementing agencies to assist Malawi in achieving the MDG targets in a transparent and sustainable manner. The component financed training of MASAF staff, LA staff, NGO/CBO staff and members of Project Management Committees (PMCs) in the use of procedures, systems and criteria for improved project implementation. The component was also aimed at establishing the Technical Support Services Framework and provided financial support to the Secretariat for the National Advocacy Committee for Community Empowerment and Accountability (NACCEA). The component was also aimed at developing an effective management information system (MIS). The MIS would feed into the overall monitoring and evaluation system where the output from various project components would form a knowledge and information sharing system (KISS). It was envisaged that through the KISS, MASAF could be a learning organization and a provider of training and capacity building on CDD approach to development stakeholders in Malawi. At appraisal, the estimated cost for this component was US$ 6.67 million, but the actual cost was US$ 6.65 million.
(v) Institutional Development: This component was aimed at addressing the
capacity requirements of MASAF to deliver community development and related services in a transparent and accountable way. The component supported the provision of technically competent staff to MASAF MU and strengthened LAs capacity to implement community sub-projects using financial accountability and community procurement procedures put in place under MASAF II. The initial cost estimate for this component was US$ 7.06 million, and the actual cost was US$ 11.31 million.
1.7 Revised Components
(vi) Project components were not revised, but the CDP, in particular the LAMPS component, was broadened as a result of the scaling-up of the Public Works Programme – Conditional Cash Transfer (PWP-CCT).
1.8 Other significant changes
(vii) In response to the nation-wide food shortages as a result of the 2004/05 drought, Government implemented an emergency response, the PWP-CCT within the LAMPS component of MASAF III APL I. The total resources made available for this emergency operation were US$12.1 million and most of these resources were reallocated from the COMSIP component.
6
2. Key Factors Affecting Implementation and Outcomes 12. Project design and implementation were affected by a number of factors which included (a) slow start of the project after effectiveness due to delays in disbursements and (b) low initial levels of justification for resources disbursed at the local level. Other factors that emerged during the implementation process included, but perhaps not limited to: (a) the political campaign of 2003 taking up most of the nation’s attention; (b) the 2004/2005 drought; (c) high staff attrition levels especially at the project MU after 2006; (d) lack of standardized procedures for project implementation at the LA level; and (e) the slow pace of decentralization. 2.1 Project Preparation, Design and Quality at Entry 13. The Quality of Supervision Assessment indicated that the project design at entry was moderately satisfactory. Although the project design was largely based on the success of MASAF I and II, it was noted that it underestimated the difficulty of handing operations over to the LAs. However, MASAF MU increased its follow-up and supervision of LAs to address this challenge. As such, project implementation was not affected as LAs were able to pick up their responsibilities. 14. The project started on a slow note. Some of the early delays were due to the steep learning curve given that stakeholders needed adequate time to internalize the CDD design and operational principles and instruments. This was also exacerbated by the 10-months delay in disbursements that coincided with the run-up to the Presidential and Parliamentary elections; followed by some political uncertainty after the elections. As such, MASAF MU had to start using its own resources when it proved difficult to mobilize project resources to start project implementation. It is not clear from the project supervision documents whether the delay in the flow of funds was due to the Bank or lapses in the Government. However, given that project effectiveness coincided with an election year, political issues must have taken precedence. 15. The other factor that affected project progress in its early stages was the low justification of community disbursed resources. This affected the pace of disbursement of community funds. Following recommendations from implementation support missions, the project intervened with a contingent strategy of deploying Justification Assistants and Auditors at various points to speed up justification rates. This resulted in improving justification rates from 24 percent in mid 2006 to 80 percent by the end of March 2007 and subsequently, the rate increased to 84 percent by September 2007. Such turn-around was attributed to the pro-activeness of the implementation support missions by both the Government and Bank teams.
2.2 Implementation 16. Prior to 2005, overall implementation was rated Satisfactory by Quality Assurance Group. However, a number of challenges emerged thereafter. First, the
7
project had to confront the need for emergency drought response in 2005 which required the scaling-up of the public works program at short notice. Government responded by rolling-out the PWP-CCT which provided a wage which communities used primarily to buy food and agricultural inputs. Due to the provision of consumption smoothing during the lean period, the PWP-CCT acted as timely incentive for communities to participate in community development work. However, although the rolling out of the PWP-CCT was a timely intervention, its downside effect was that it placed enormous pressure on the project to meet huge demand in terms of capacity and coordination that was not originally envisioned at design. Some re-prioritization had to be done to accommodate the scaling-up of the PWP-CCT. In spite of these challenges, the PWP-CCT program stimulated LA capacities and the demonstrated abilities paved the way for MASAF III APL II design as a mainstreamed operation through LA with new approach to Inter-Governmental Fiscal Transfer Systems. The rolling out of the PWP-CCT was seen by many stakeholders, including donors, as having been implemented in an effective manner and the Government was hailed for the role it played, through MASAF 3 APL I, in averting imminent hunger in a well coordinated and effective way. 17. Secondly, implementation was affected by significant staff turn-over towards the end of the APL I. This occurred due to uncertainties regarding the continuity of the project into a new phase (APL II). The MASAF MU was reduced to less than half its normal size and all Justification Assistants, except four had left by December 2006. This affected smooth completion of the final activities under the APL I. The down-sizing of staff towards the end was consistent with the design of the project. As part of scaling-down, the project was designed to cut down the number of zones from 12 to 3 (with a total of 9 field offices). The implementation of the strategy towards scaling down was however not well planned to avoid creating unnecessary pressure on the remaining staff. Government and the Bank could have strategized early enough on how to retain some of the critical capacity to avoid disrupting smooth completion of the APL 1 and the continuity of MASAF into the next APL. 18. Thirdly, implementation was affected by the lack of standardized procedures for project implementation and coordination with other projects at the LA level. The supervision of MASAF funded sub-projects was somehow affected because LA staff were more responsive to high daily subsistence allowances provided by projects supported by other development partners. The multiplicity of different project implementation and reporting procedures also strained the already low levels of capacity in some LAs. This is a key issue requiring the streamlining of procedures and requirements related to the overall management of development aid in Malawi. Government is already, through the Debt and Aid Coordination of the Ministry of Finance taking positive steps to address this systemic challenge. 19. Fourthly, project implementation was affected by the delay in the provision of counterpart funds. In spite of several reminders by various supervision missions, a greater proportion of the counterpart funds were made available towards the end of the project. This has led to considerable delays in the implementation of some project
8
activities. However, the flexibility in the design and exchange rate gains allowed the project to meet its targets without difficulty.
2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 20. The M&E system design was technically sound. All the required systems for a functioning M&E framework were in place such as data capture, analysis and feedback mechanisms. The system also allowed for a vibrant linkage between the LAs and the MASAF MU where information was analyzed and reports were produced and disseminated. 21. However, the functionality of the system was in some cases constrained by systemic technical capacity gaps at the LA level. APL I inherited a relatively weaker M&E capacity from the previous MASAF II. As such there were delays to fully operationalize the framework especially at the LA level. In some cases, LAs had a lot of challenges to institutionalize the Local Assembly Management Information System (LAMIS) and ensure regular capturing of data. In most LAs, the LAMIS was normally updated by the Justification Assistants. Although training on LAMIS was provided to other relevant staff, it was difficult to catch up with the backlog of data. This problem was somehow addressed when Government hired M&E Specialists that were deployed to all the LAs. However, due to their many commitments, it still proved difficult for the LAs to ensure regular update of the LAMIS. In some cases the LAMIS was not instituted into the LA system, as it was treated as a tool for managing MASAF projects. As such, continuity of data capture and linkage to the MASAF MU became a challenge after most of the Justification Assistants had left the project. 22. As such, due to limited capacity and institutionalization of the M&E system at the LA level, the sustainability of utilizing the system for effective management of projects at the LA level remains a challenge.
2.4 Safeguard and Fiduciary Compliance 23. There were no major challenges that affected the project’s compliance to fiduciary issues. The financial management system worked very well and there were no major disbursement lapses that were experienced. The only challenge relates to some lags in the justification of LA and community funds especially towards the end of the project when most Justification Assistants were winding up their activities. However, arrangements were put in place to sort out this issue and by September 2007, 84 percent to the funds disbursed to LAs and communities had been accounted for. Intermingling of project funds and some sub-project cost over-runs were identified in a few LAs (as reported at Mid Term Review). This could result in distorting fund accountability and create room for fraud. However, punitive corrective measures were put in place to check such practices. 24. Auditing arrangements for projects funds worked very well, involving both internal and external auditors, in collaboration with the National Local Government
9
Finance Committee. MASAF also had a working arrangement with the Anti-Corruption Bureau to curb corruption through the use of appropriately packaged Information, Education and Communication (IEC) campaigns. 25. There were also no procurement challenges that were experienced especially at the MASAF MU. However, there were some challenges at the LA and community level. For example, some PMCs faced a number of challenges that included inflated material prices and supply of wrong materials by unscrupulous suppliers and service providers. It was also noted that in some cases, such problems arose due to poor facilitation by some officials especially when it came to procurement of goods and services with complex technical specifications. In such cases, functionality and sustainability of some projects has been a key challenge. 26. Compliance with environmental safeguards was noted as a systemic challenge during project implementation. There was low priority given towards the implementation of the Environmental Management Plan (EMP) at both the LA and community level. This was due to lack of follow-up by LA staff because of staffing constraints and difficulties in accessing resources. This resulted in low appreciation by communities of the importance of implementing mitigation measures. This issue was noted by both the Bank and Government during supervision, and appropriate steps were taken to address it through aggressive awareness campaigns throughout the sub-project cycle coupled with close follow-up by relevant sector staff. However, the problem of compliance persisted till the end of this phase of the program. This poses a risk in terms of the sustainability of the project’s development outcomes. However, in the follow-up APL II, adequate attention has been given towards the design of simple and appropriate CDD environmental procedures with clear criteria and standards for screening sub-projects and putting in place workable plans for implementing mitigation measures. 27. There were no cases of down-side social risks such as elite capture, rent seeking and adverse behavior related to community accountability and transparency. This may have been the result of proper sub-project governance systems at the local level through transparent choice and clear terms of reference for the Project Management Committees. Furthermore, there was adequate capacity building on community accountability.
2.5 Post-completion Operation/Next Phase 28. The design of MASAF envisaged that it will be implemented within a decentralized governance framework, where LAs would be comprised of elected officials who would be supported by Government staff. However, although some form of devolution has occurred in a few sectors, the Decentralization Policy itself has not fully and practically been rolled out. As such the existence of less than fully devolved entities appears to impede effective functioning of the LAs. Although the Project has built capacity at the LA among the employed officials, lack of progress on decentralization poses significant institutional challenges in terms of sustaining the capacity that has been built through the project mainly in terms of the maintaining efficiency and accountability of service delivery at the LA level. The risk arises from the lack of continuity given staff
10
changes, dual reporting of sector staff between the LA and central authorities and the associated challenges in transferring skills to the elected officials once they are in place. Furthermore, the LAs are yet to transform into solid management entities that can effectively take on the responsibilities associated with full-scale decentralization (political, fiscal and administrative). 29. The slow pace of decentralization has however not adversely affected project implementation and the achievement of the PDO because the MASAF MU continued to provide support to LAs to ensure that the sub-projects were delivered and that the capacities were imparted even in the absence of elected LAs. However, considering the systemic capacity challenges at the LA, it is likely that the delivery of community sub-projects will be affected in terms of quality and meeting completion targets, in the absence of continued support. 30. The experiences in the implementation of MASAF III APL 1 have been instrumental in informing the design of the next phase of the project which has since been approved by the Bank’s Board and is only awaiting effectiveness.
3. Assessment of Outcomes
3.1 Relevance of Objectives, Design and Implementation 31. The project objective and design of the components were not changed throughout the implementation of the project largely because of their relevance to the development challenges of the country and that the project was conceptualized within the country’s own poverty reduction strategy. Furthermore, there were thorough and intensive preparations and consultations that also benefited from the experience and knowledge gained in previous phases of the project. The project gained momentum and good-will among the development partners and was also in line with, and strengthened the mechanisms, for the country’s Decentralization Policy and implementation framework.
3.2 Achievement of Project Development Objectives 32. Achievement of the MASAF III APL I is rated Satisfactory, as measured through the set of outcome/impact indicators. The detailed project performance on these indicators is highlighted in Annex 2(a). The impact of the APL I on the achievement of the 12 selected MDG indicators could not be clearly determined. It is important to note that the project’s impact on the selected MDG indicators relates to the entire 12-year APL program. The APL I has contributed towards some intermediate outcomes required to reduce poverty such as: community empowerment; increased social capital and expanded social service delivery; and improved livelihoods. Each of these is elaborated in more detail in the next sub-sections.
11
Community Empowerment 33. One of the strongest achievements of the project was its dynamic interaction with communities in capacity building and fostering ownership and transparency. This resulted in an increased number of sub-projects implemented at the community and district levels. The project managed to integrate community empowerment processes into district planning and delivery of community service packages. Tapping on experiences from MASAF I and II, the APL I contributed towards the alignment of communities, NGOs/CBOs and LAs with the DDPF in a number of ways. First, as reported by the final project evaluation, the project has had a positive influence on the performance of Local Assemblies in social service delivery. The introduction of new project delivery systems, including the community-based procurement and accountability systems, such as Extended-PRAs and community score cards, helped to develop a cadre of community-based project managers, accountants, procurement specialists and artisans. Secondly, and more importantly, the application of sector norms and standards to community sub-projects through the “community service package” mechanism helps to ensure an effective synchronization of the top-down and bottom-up approaches to development. All these have assisted to enhance the capacity of the LAs and communities to delivery social services and development projects in a more efficient and cost-effective manner. Access and Use of Social Services 34. MASAF III APL I has made an important contribution towards the provision of social and economic infrastructure. Beneficiaries attribute a number of social welfare improvements directly to the project, largely through the CMPs, LAMPs and SSPs. For instance, through better health infrastructure such as construction of clinics and better opening up of tertiary and feeder roads, including bridges, communities are able to access health services all year round. As a result, beneficiaries attribute reduced levels of maternal mortality partly to the project interventions. Some of the notable project contributions, as mentioned by communities, include the reduction in the distances to nearest pick-up points for agricultural products destined for the market, improved access to school especially among young children, many of whom used to fail to attend classes during the rainy season for fear of drowning in streams that did not have bridges across them. The project has also contributed to community empowerment especially that of women, according to the beneficiary feedback generated through the community score card process undertaken in 2005. This has long-term human capital development implications. Food Security and Incomes 35. On incomes and food security of the poorest segments of the communities, through the CCT, the project managed to secure the livelihoods of over 670,000 vulnerable community members. An analysis of the expenditure patterns show that the earnings from the CCT wages were mainly used for purchasing fertilizer and seed (68% and five percent, respectively), maize for immediate consumption (25 percent) and other seed for pulses, mostly groundnuts (one percent) and beans (one percent). Most of the
12
beneficiaries of the CCT reported a three – four fold increase in their crop production as a result of applying the inputs bought with the cash transfers. This has obvious long-term benefits on the nutritional status of both the parents and children. Through COMSIP, over 8,300 groups were formed and provided capacity in generating savings and investment towards the betterment of their livelihoods. This has created an opportunity to open up the communities to micro-finance institutions thereby bringing rural communities to directly participate in mainstream economic activities. Project contribution towards achievement of MDGs 36. MASAF III APL I was also aimed at assisting communities and LAs in the achievement of the 12 MDG indicator targets for Malawi. It should be understood that this long-term goal relates to the entire 12 year APL which was expected to coincide with the target of attaining the MDGs by 2015, by which time it was expected that LAs would have completely taken over the implementation of CDD investments. Within the APL I, the goal was to attain at least 70% of the LAs with baseline data and targets for the 12 MDG indicators. In addition, at least 70% of the LAs were supposed to have developed five fully costed service packages and that LAs would develop by-laws to register CBOs that would help in the implementation of activities towards the achievement of the MDGs at the community level. All of the 40 LAs had fully satisfied these triggers by the end of the APL I’s implementation. As such, on the basis of the key impact indicators (the triggers), MASAF III APL I contribution towards the achievement of the MDGs within the targets set for APL I has been satisfactory. However, the direct impact of the APL I and the improvements in the 12 selected MDG targets has not been clearly measured. 37. Overall, the project has managed to assist communities to achieve intermediate outcomes that will contribute towards closing the service gaps related to education, water, sanitation, transport, communication and food security. In total over 1.5 million people have directly been reached under various interventions implemented during the course of APL I’s implementation. If Government and other stakeholders, including the newly approved APL II, continue to build upon and consolidate the achievements of the APL I, it is highly likely that the country will make remarkable progress towards the achievement of the MDGs. The progress on the MDGs, and the project’s contribution are presented as an addendum to Annex 2(b).
3.3 Efficiency 38. The efficiency of project delivery under MASAF III APL I was assessed using a rather detailed cost-effectiveness analysis that was undertaken as part of the impact evaluation. Conventional measures of economic viability or efficiency such as net present value and internal rate of return were not computed at appraisal due to the challenges associated with the identification and valuation of project activities ex-ante. 39. The assessment of cost-effectiveness was undertaken by comparing the unit costs of MASAF infrastructure such as classrooms, roads and other community level structure,
13
against those of similar projects implemented by other development stakeholders such as for example, Department for International Development (DFID) and European Union (EU). In general, the results indicated that MASAF sub-projects were delivered more cost-effectively compared with projects supported by other stakeholders. Moreover, the overheads/administrative costs of MASAF were also much lower compared to those of other projects that implemented similar sub-projects. Some fairly detailed aspects of the cost-effectiveness analysis are highlighted in Annex 3.
3.4 Justification of Overall Outcome Rating 40. The achievement of the development objective is Satisfactory. The development objective was seen to be clear, realistic and was inline with both the Country Assistance Strategy of 2003 (Report No. 25906 MAI) and the Government of Malawi’s Poverty Alleviation Program. Given the prevailing high levels of poverty, the project played a pivotal role to improve communities’ access to socio-economic services. Through the income generating activities implemented under SSP, and the savings and investment promotion under COMSIP, the project achieved remarkable economic empowerment. Moreover, the project objective was well aligned to the decentralization policy through its support to local government capacity building. As such, strengthening the capacities of LAs and communities for effective local development management remains relevant in facilitating the achievement of efficient service delivery. 41. During the internal review meeting for the draft ICR, reviewers felt that the revised ratings on safeguards compliance, risk to development outcomes and borrower performance should be reflected in the down-grading of the PDO as well. However, upon further consideration by the team, it was decided that since the PDO and implementation do not seem to have greatly been affected, the team felt there was no justification to down-grade the PDO. Moreover, since various phases of the APL build upon each other, APL II and III will probably make up for the deficiencies identified in APL I. 42. The implementation support missions of December 2005 and May 2006 down-graded the ratings for both the DO and IP to Moderately Satisfactory primarily due to the problems related to project management and the continued delays in the release of counterpart funds. The ratings were brought back to Satisfactory during the November 2006 mission when agreements were reached on the release of counterpart funding and project management issues. The ratings have been maintained at Satisfactory until the closure of the project. While we agree that both DO and IP are Satisfactory, the decision to upgrade as early as November 2006 could not have been based on Government’s full payment of the counterpart funding because there were still substantial arrears (amounting to MK501 million) up until May 2007. As such, the ICR team felt that the up-grading of the rating should have been done much later.
3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development
14
43. The poverty, gender and social development impacts were achieved through intermediate outcomes attributed to project activities implemented under the CDP, SSP and COMSIP components. 44. Poverty Impacts: Under the LAMPs, a total of 815,557 people, about 83 percent, benefited from the wages received under the PWP-CCT and about 17 percent benefited from the regular operations. The total amount of money disbursed in wages amounted to US$ 14.6 million. This component resulted in rehabilitating 3,392 km of rural roads and the outcome was the increased accessibility of socio-economic services from very remote areas that would otherwise remain detached from socio-economic activities. Other outcomes include an improvement in food security as a result of agricultural inputs purchased out of the wages received. According to the beneficiary assessment, over 80 percent of the beneficiaries of the PWP-CCT reported that the program had a positive impact on their agricultural performance. Other added benefits that may not have been conceived at design included the development of small-scale irrigation schemes, some of which have attracted nation-wide recognition such as Manthimba in Thyolo district and Bua-Dwangwa drought mitigation in Kasungu and Nkhotakota districts. Together, these projects have benefited about 12,000 drought-affected poor rural households that would have experienced high levels of poverty and vulnerability without the project intervention. 45. One other intermediate outcome that has a direct bearing on reducing poverty is the organization of the communities into business cooperatives. Through COMSIP, 8,392 COMSIGs were established with a total savings of MK28.8 million. This opened up an opportunity for communities to participate in business initiatives, thereby creating gainful sources of employment in the rural areas. In line with the Cooperative Act (1998), a COMSIP Foundation was established to coordinate the provision of capacity building to community savings groups. The groups are encouraged to register themselves into cooperatives so that they could benefit from low cost, less risky financial products and services meant to improve the scope of their businesses. In this way, the project managed to effectively transform social capital into economic capital. However, as a result of the reallocation of US$ 12.1 million from the COMSIP component to support the up-scaling of the PWP-CCT, the project could not support the establishment of club houses as originally envisaged at design. Moreover, the fostering of linkages between community savings groups and commercial micro-finance institutions such as the Malawi Savings Bank (MSB) could not materialize. As a result, some community groups and individuals ended up demoralized because although they received training, they could not access capital for micro-investments. 46. MASAF III APL I also delivered in terms of attending to the needs of vulnerable groups. Under the SSP, 75,336 vulnerable people such as widows/widowers, the aged, orphans and foster parents, the destitute, disabled and person affected with HIV/AIDS as well as malnourished children were directly targeted through interventions channeled through NGOs/CBOs. The target under this component was 72,000 beneficiaries and this surpassed by nearly five percent. Interventions under this component have contributed
15
towards improvement in the livelihoods of these vulnerable groups. For example, the early Childhood Development Centers have directly improved child development through improved nutrition as a result of the feeding programmes. Other benefits include economic empowerment of the orphans and the physically challenged through life skills acquired in vocational training provided through the project’s interventions. In some circumstances, the project has supported orphans with fees for secondary education, and income transfers for foster households. The design of this component led to strengthening of the social support mechanisms through the development of community level specialized structures that have promoted the productivity of the vulnerable groups and the mitigation of social risks. Furthermore, this component has also provided a conduit through which other development partners such as UNICEF could channel their support to the vulnerable children, thereby providing a mechanism for continuity of such support. Many of the social protection related instruments implemented in Malawi by so many stakeholders including the Government have modeled their interventions on the design principles instituted under the APL I. 47. Gender Impacts: One of the key goals of MASAF was to contribute towards the achievement of the MDG of promoting gender equality. As such, the project endeavored to promote fairness in the inclusion of women and men in sub-project activities. For example, in the implementation of LAMPs, the involvement of women (50.9 percent) was in fact slightly higher than that of men, because in many cases it is women and/or female headed households that were likely to face a higher level of deprivation and vulnerability as a result of the drought. Furthermore, according to the citizen feedback through the community scorecard process, it was indicated that the project contributed to the empowerment of women through their involvement in income generating activities and membership in the project management committees. 48. Social Development Impacts: The key outcome of the sub-projects implemented under the CDPs was the improvement in the access to social amenities such as health, education, water and sanitation. Under the CMPs, 571 of the targeted 591 projects were completed by September 2007, representing about 97 percent performance. The social amenities provided under this include new classrooms which have since been providing pupils with improved learning environment thereby improving enrollment and retention rates in primary school. In addition, the provision of new teachers’ houses has attracted and retained teachers in the rural schools thereby improving the pupil: teacher ratio. Furthermore, Traditional Birth Attendants (TBAs) have been trained thereby increasing the number of births delivered by trained health personnel. This has resulted in reducing the maternal mortality rates in the project areas. The project also implemented a Malaria Sub-project and distributed mosquito nets to 400 households, as well as reaching 11,113 people with a drug revolving fund. This resulted in the reduction in the incidence of common killer diseases such as malaria which often detract communities from engaging in their productive activities. This component also provided improved water and sanitation services through drilling of boreholes and piped water supply kiosks. Admittedly, many beneficiaries expressed satisfaction with the water supply provided under the project which substantially reduced the incidence of water-
16
borne diseases and also reduced the burden on women in terms of the effort to walk long distances to fetch water. (b) Institutional Change/Strengthening 49. APL I managed to empower the Local Governments with the requisite skills to manage community development projects. With the exception of the National AIDS Commission, stakeholders at both the national and LA levels have indicated that MASAF III APL I is the only programme that has given full autonomy and authority to communities and LAs to manage development funds. During the implementation of MASAF, significant resources were provided to the LAs (see Annex 11)4. As such, MASAF also raised the predictability of funding to the LAs and this led to positive transformation of attitudes and responsiveness towards community development work. The learning by doing strategy has helped LAs to develop and refine the systems for effective development planning. While many development practitioners have avoided directly involving communities in managing development funds fearing the associated risks, MASAF has successfully tested and clearly demonstrated that with proper systems and motivation, communities are able to plan and execute their own development projects. This not only provided full ownership to communities of their own development, but also deepened their understanding that they can improve their own situation. This in itself is a key achievement that will have long-lasting implications on community development in Malawi. 50. In addition, through the Transparency and Accountability Promotion (TAP), the project has created an enabling environment to develop and nurture the capacities of communities, LAs, and CBOs/NGOs towards effective development planning and service delivery. The supportive framework for effective community development included development communication interventions through which a number of IEC messages were formulated and disseminated. This created awareness among the key stakeholders about the project’s organizational and institutional set-up. To the beneficiaries, the sensitization enabled them to effectively demand development projects. A broad range of stakeholders were also reached with various capacity development activities related to the project cycle in wide-ranging aspects such as design and operational principles of CDD type projects, sub-project standards and control within sectors norms and standards, participatory monitoring and community accountability, knowledge management, monitoring and evaluation of sub-projects. A total of 48,165 people, among them traditional leaders, ward councilors, LA and sector staff were trained in these aspects through the Leadership Challenge Facility (LCF). 51. Other aspects of institutional capacity building include the establishment of the LAMIS which was used as an accountability tool for public resources at the LA level. The system allows for tracking utilization of resources and achievement of results,
4 These were planning figures at the beginning of the APL 1, as such they may be slightly different from those that were disbursed by the end of the project.
17
thereby building transparency and accountability in the use of resources. As a resource management tool, LAMIS has enforced price discipline through the building of cost database that has widely been adopted for LA project management. As a monitoring and evaluation tool, it has been very instrumental in providing centralized reporting on project inputs and outputs. The system has already been adopted by other donor supported projects to ensure accountability in the utilization of resources. 52. The project also developed organizational skills and knowledge management capacity through its research and operational framework. The project emphasized on the use of knowledge generated through research to build capacity among LAs and communities to identify service gaps and how to address them (using approaches such as the Simba Mindset)5. To enhance this capacity, MASAF commissioned a number of studies and established a knowledge resource and repository, where local experiences on CDD implementation are housed and will continue to be consulted by stakeholders for a long time to come. Already, MASAF has been instrumental in providing these experiences to other countries in the region and beyond. Finally, under this component, MASAF has also developed and refined social accountability tools such as community score cards, which have improved community effectiveness and oversight thereby enhancing management and performance of community development projects. 53. Notwithstanding these achievements, the transfer of capacity from MASAF to LAs was somehow constrained by the limited progress in the implementation of the National Decentralization Policy. Throughout the implementation of the APL I, the LA continued to operate without effective devolution of fiscal, administrative and political responsibilities. As such the LAs could not receive sufficient direct fiscal transfers to finance their local development needs, lack authority to manage their human resources (due to dual reporting) and have very limited political decision making power. This poses significant institutional challenges in terms of sustaining the capacity that has been built through the project. (c) Other Unintended Outcomes and Impacts (positive or negative) 54. The project has had a number of unintended outcomes, many of which have been positive. Three of the notable positive unintended outcomes include: first, the shift in Government and other stakeholders’ thinking from the use of Starter Pack to conditional cash transfers as an effective way to address food insecurity and vulnerability. Related to this is the refining of the targeting mechanisms for such type of community interventions. Today many cash transfer interventions are using the targeting mechanism developed under MASAF III APL I; secondly, MASAF’s support towards knowledge generation for CDD type interventions has created a resource which will be used in future interventions
5 Simba Mindset is an approach to capacity enhancement whose main features are rapidity in action, adaptability of strategies, demonstration of competencies, efficiency in the use of resources and a clear focus on results.
18
within and beyond the boarders of Malawi. Thirdly, MASAF has also been managing the funds for the community demand-driven components of two Government projects: the Community Based Rural Land Development Project and the Irrigation Rural Livelihoods and Agricultural Development Project. Both projects are financed by the Bank and their success in terms of delivering the expected outcomes and accounting for community funds is attributed solely to the fund management services provided by MASAF. This is an interesting feature which helps IDA funded projects implemented at the community level to consolidate and build on each other’s achievements, thereby ensuring internal coordination and consistency within IDA supported rural development investment portfolio in the country. 55. Although none of the project documents highlighted any possible negative outcomes, it is quite likely that such a project created massive demand for community development which could not fully be met resulting in some community polarization. In some cases, community infrastructure established by the project could not be used effectively or ended up being put to unintended uses due to lack of complementary support from LAs.
3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 56. Direct project beneficiaries reported improvement in access to social and economic services with women anecdotally confirming reduced mortality among pregnant women in areas where MASAF III implemented road projects and where vehicular traffic increased on the respective roads. Access to health facilities, schools, markets and other socio-economic amenities has generally improved in the project areas. This is largely attributed to the opening-up of tertiary and feeder roads, leading to reduction in the distances to the nearest trunk or all-weather roads. Increased frequency in the use of the rural roads has improved the business climate in most rural areas where the project was implemented. Local capacity to manage community development projects has also greatly improved. 57. Beneficiaries, however, noted the functionality challenges of the facilities provided due for example to inadequate health personnel and the declining quality of the roads. This finding highlights the importance of maintenance, both in terms of funding and institutional setups.
4. Assessment of Risk to Development Outcome 58. Assessment of the sustainability of the development outcomes focuses on three key dimensions: (a) sustainability of livelihood impacts; (b) sustainability of physical infrastructure and (c) sustainability of the social capital created through the CDD approach. 59. On the sustainability of the livelihood impacts, there are indications from the impact evaluation that most beneficiary households had improved and sustained livelihoods especially as a result of the re-investment of the payments received through
19
the CCT. The majority of the households that participated in the CCT used 73 percent of the money received to buy fertilizer and improved seed. As a result, many were able to achieve a significant increase in yield (i.e. three-fold increase on average). Such levels have been maintained during the subsequent seasons as a result of the complementary Input Subsidy Program. Some of the beneficiaries have also been able to diversify into small-scale businesses through the training and support provided under COMSIP. Indications of improved livelihoods among the project beneficiaries include a consistent decline in the number of people with missing food entitlements and improvements in some nutrition indicators such as decline in global acute malnutrition. Some of these reflect general improvements in the economy and may not directly be attributable to the project. However, the project managed to contribute to these positive outcomes. 60. On the sustainability of physical infrastructure, it was noted that most of the infrastructure such as school blocks, clinics and roads were built according to specified norms and standards. Many of such infrastructures are of acceptable quality. However, sustainability of such infrastructure is likely to be affected by two key factors: (a) functionality problems in cases where no provision were not made for other complementary aspects that could make such infrastructure to be fully functional such as chairs, boards, teacher-houses in the case of schools, culverts, bridges and drainage systems in the case of feeder roads; and (b) lack of adequate operations and maintenance resources. For each of these challenges, the project had made arrangements for complementary support from the LA, through for example, the matching grant for operations and maintenance, however, given the resource constraints faced by many LAs, such arrangements have not been workable. 61. On the sustainability of the social capital and community development capacity created at the LA and community levels, there is adequate evidence that this has been achieved. LA and community level institutional arrangements to ensure functionality and sustainability of project outcomes have been put in place and are largely functional. For example, there are various committees that are responsible for the sustainability of each of the completed sub-projects in terms of routine care and maintenance. The involvement of local NGOs/CBOs with proven history of working with communities is likely to enhance sub-project continuity. Furthermore, the establishment of community groups (for management of water points, forestry, parent-teacher associations) including the registration of COMSIP groups into member-based organizations and formation of cooperative unions are all indications of institutional sustainability built by the project at the local level. 62. Overall, although the borrower ICR has rated sustainability as being Satisfactory, the assessment of the Bank team is that there is moderate risk that development outcomes may not be sustainable given the low capacity and resources for routine operations and maintenance of some infrastructure sub-projects. This will continue to be the case even with the APL II if the project resources are not fully integrated into the LA budgets.
20
5. Assessment of Bank and Borrower Performance
5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory (b) Quality of Supervision Rating: Satisfactory (c) Justification of Rating for Overall Bank Performance 63. The Bank’s performance in the implementation of the APL I has been very supportive to the success of the project. The project was prepared well in time and except for the few challenges at the beginning, there was no disruption in the flow of funds. Moreover, throughout the implementation of the APL I, Bank staff continued to provide technical, managerial support and fiduciary oversight that kept the implementing unit always in line with the project implementation plan. The three key attributes of successfully implemented projects such as timeliness, quality and staying within the budgetary allocations were largely made possible by the pro-active and timely Bank advice. Moreover, this agrees with the findings of the Quality of Supervision Assessment undertaken by the Bank in 2006 which stated that the supervision did quite well to increase the momentum of the project and maintained the dedication and commitment of the team even in the face of some challenges.
5.2 Borrower Performance (a) Performance of the Borrower6 Rating: Moderately Satisfactory (b) Justification of Rating for Borrower Performance 64. The Government of Malawi persistently supported the project through provision of technical staff that constituted the various technical committees that appraised and recommended sub-projects: such committees include the NACCEA, the Joint Technical Committee and the National Technical Advisory Committee. Strategic policy oversight and direction was provided by the MASAF Board which was constituted fully by the Government. Various disbursements arms and mechanisms such the Reserve Bank of Malawi, the Accountant General and Ministry of Finance, including the Auditor General
6 The borrower comprises Government and its local project implementation arms such as LAs and PMCs.
21
and the Office of the Director of Public Procurement all acted diligently to ensure smooth project implementation. Overall, the project enjoyed widespread support at both the technical and political levels, and this explains the very high profile that MASAF has had since its inception. 65. However, as noted by the Quality Assurance Group, the Government made some abrupt changes in top level staff at the MASAF Management Unit and was unwilling to discuss the issue with the Bank. Secondly, the project experienced slow and late release of counterpart funds which flouted one of the covenants in the legal agreement. Thirdly, the abolition of elected local councils called into question the overall decentralization strategy assumption that underpinned the APL program. As such the performance of the implementing agency as a whole was Moderately Satisfactory. (c) Performance of the Implementing Agency7 Rating: Satisfactory (d) Justification of rating for Implementing Agency 66. Throughout the implementation of the APL 1, the Project Management Unit demonstrated diligence, commitment and professionalism towards successful delivery of the project. In particular, the Project Management Unit managed to maintain a clean fiduciary record at a time when other implementation units were struggling with procurement, financial management and disbursement challenges. Moreover, even with the slow pace of decentralization and later staff changes at a senior level, the Unit managed to provide adequate support to the LAs to effectively deliver community development. Even though the implementation agency is part of the Borrower whose rating is moderately satisfactory for the reasons provided above, the ICR team felt that the MASAF Management Unit demonstrated diligence that was very instrumental to the success of the APL1 (including the previous two phases), and therefore deserved a Satisfactory rating.
6. Lessons Learned 67. The following are the key lessons and experiences learnt during the implementation of MASAF III APL I mainly through observations by staff and Bank teams in the course of project supervision, studies and direct feedback from communities and other stakeholders. These lessons and experiences have been considered in the design of the follow-on APL II and will also be invariably important in the design of future programs.
7 The implementing agency refers to the Malawi Social Action Fund Management Unit (MASAF).
22
6.1 Improved accountability in community development needs to continue 68. Improved accountability system for community development is critical in order to enhance positive perceptions among beneficiaries, and other stakeholders including donors. This provides an incentive for their support and involvement. For this to be achieved there is need to strengthen the IEC so as to enhance civic engagement and community accountability. The APL 1 has managed to put in place proper accountability systems. Throughout project implementation, the role of IEC continued to be instrumental to the success of the project. Through the IEC program, experiences from communities, local authorities and NGOs were synthesized and packaged into useful knowledge and information that enhanced community participation in the sub-project implementation process. Furthermore, the IEC also enhanced knowledge and information sharing with other stakeholders beyond the beneficiary communities through the Statistics Days. This enhanced positive perception and support for the project. As such, the role of development communication needs to be retained and perhaps strengthened in future operations to ensure effective community mobilization and participation in community development. 69. Communities are now capable of ensuring that local PMCs and District Executive Committees (DECs) are accountable for the project resources entrusted on them. Moreover, the project has instilled a sense of community cooperation which enables communities to create socio-economic capital. This has been achieved through such processes as participatory community monitoring and accountability through the community score-card system introduced under the project. The project also noted the emerging community-level information networks that provided technical advice to new PMCs on the sub-project cycle and other implementation challenges. There is need to ensure that future interventions consolidate and build on community development institutions that will be adaptable and remain valuable for sustainable community development.
6.2 Results measurement systems need to be strengthened 70. Through the APL 1, several innovations were made with regard to results monitoring and measurement. Such systems need to be maintained at each level of project implementation. For example, at the community level, there is need to promote the use of community monitoring and feedback through the community statistics days as an integral part of the monitoring and evaluation framework. At the LA and program level, there is need to promote participatory development of delivery benchmarks related to the Community Sub-Project Cycle so that beneficiaries and stakeholders are aware of what they have to be accountable. There is also the need to enhance a performance management culture at the LA level by ensuring the harmonization of systems and procedures (i.e. LAMIS, financial management, procurement and reporting procedures) including building the capacity for well coordinated development planning. Furthermore, the tracking of outcome and impact indicators and the linkages with mainstream national monitoring and evaluation system needs to be improved. Under the
23
APL 1, the assessment of MASAF’s contribution to poverty reduction was seen to be quite challenging due to this particular aspect. Such linkages are important and have to be established to ensure easy tracking of the project impacts to higher level goals such as poverty reduction. The development of LAMIS is a positive move in this respect. Capacity to update the LAMIS and ensure that it feeds into national M&E systems needs to be strengthened and sustained in future interventions.
6.3 Community-level sustainability mechanisms need to be institutionalized 71. The sustainable mechanisms to secure continuity and functionality of community assets and institutions established through the APL I and other similar interventions are important achievements for this project, and need to be institutionalized. This entails: (a) provision of adequate operations and maintenance resources and institutional arrangements for sustainable functionality of social service infrastructure; (b) systematic mainstreaming of capacity and resources to ensure compliance with environmental and social safeguards at all levels of sub-project implementation; and (c) ensure that sub-projects continue to be reviewed, screened and approved within the DDPF. Such sustainable mechanisms should also ensure a better match between the top-down and bottom-up in terms of technical specifications (e.g. procedures for establishing Bill of Quantities (BOQs), material costs, etc.) including sector norms and standards. MASAF III APL II design has integrated sector norms-based planning at the community, LA, and national levels while institutionalizing community involvement in service delivery through LAs.
7. Comments on Issues Raised by Borrower (a) Borrower 72. The Borrower was consulted at all stages during the preparation of the ICR. Most of the Borrower’s comments focused on the factual consistency, clarity of the analysis and justification of the ratings and completeness to ensure that important details are not left out. All the comments were incorporated and the Borrower, through its Implementing Agency, expressed satisfaction with the current draft ICR. This draft was therefore cleared by the Borrower. (b) Cofinanciers: N/A (c) Other partners and stakeholders: N/A
24
Annex 1. Project Costs and Financing
(a) Project Cost by Component (in USD Million equivalent)
Components Appraisal Estimate)
Actual/Latest Estimate
Percentage of Appraisal
1. COMMUNITY DEVELOPMENT PROJECTS (CDP)
28.48 36.83 129%
2. SOCIAL SUPPORT PROJECTS (SSP) 24.00 19.94 83%
3. COMMUNITY SAVINGS AND INVESTMENT PROMOTION (COMSIP)
11.87 2.12 18%
4. TRANSPARENCY AND ACCOUNTABILITY PROMOTION (TAP)
6.67 6.65 100%
5. INSTITUTIONAL DEVELOPMENT (ID) 7.06 11.31 160%
Total Baseline Cost 78.08 76.85 98%
Physical Contingencies 0.00
0.00
0.00
Price Contingencies 0.00
0.00
0.00
Total Project Costs 0.00 76.85 Front-end fee PPF 0.00 0.00 .00 Front-end fee IBRD 0.00 0.00 .00
Total Financing Required 0.00 76.85
(b) Financing (in USD Million Equivalent)
Source of Funds Type of Cofinancing
Appraisal Estimate
Actual/Latest Estimate
(USD millions)
Percentage of Appraisal
Borrower 7.68 0.00 .00 Local Communities 5.60 0.00 .00 International Development Association (IDA) 32.80 0.00 .00
IDA Grant for Debt Vulnerable 27.20 0.00 .00 Non-Government Organization (NGO) of Borrowing Country 4.80 0.00 .00
25
Annex 2(a): Outputs by Component
#
Indicator
Target
Achieved by September 2007
% achievement
on target
Note 1
CMPs completed and operational
591
571 projects
96.6
On course towards achieving target
2
LAMPs completed
385
377 projects
97.9
On course towards achieving target
3
H/H reached with assistance under SSP
18,000
16,029
89.1
Based on benefits from outputs in completed SSPs – HH in nutrition projects; Under-fives in nutrition projects; chronically ill reached with home based care.
4
Vulnerable reached with assistance under SSP
72,000
75,336 individuals
104.6
Benefits from outputs in completed SSPs
5
Individuals in receipt of LAMP wages (regular LAMP)
72,000
141,975
197
Achievement on regular LAMP only
6
Individuals in receipt of LAMP wages (PWP-CCT)
N/A
673,582 individuals
N/A
Achievement on PWP-CCT beneficiaries only (both 2005 and 2007 operation)
7
Sub-projects with O&M arrangements (%)
90%
100% achieved
111.1
All sub-projects have O&M arrangements at approval
8
Communities trained in Operation & Maintenance (%)
95%
22% achieved
23.2
Low achievement due to lack of training for school committees. Committees already trained by Ministry of Education.
9
Women representation in PMCs (%)
40%
46.3%
115.75
Target surpassed
10
Savings groups formed and operational
8,000
8,392 COMSIGs
120.6
Target surpassed
11
Savings clubs formed and investing
800
32 COMSICs
4.0
Low achievement on club formation due to reallocation of funds to PWP-CCT
12 Field Offices wound up. 2 9 offices 450 All field and zone offices wound up as of December 31, 2006 as the APL Phase one closed.. Effectively, LAs are now in charge of all aspects related to project implementation and accountability of resources.
13
LAs meeting sub-project benchmarks
8
N/A
N/A
No formal assessment done. However, sub-project cost and time overruns continued to be challenges
26
Annex 2(b): Baseline and MASAF contributions towards Millennium Development Goals
MDG Targets MDG Indicators Malawi 2000
Baseline
2007 Status8
Malawi 2015
Target
MASAF’s contribution based on LA Indicators
1. Eradicate extreme poverty and hunger
(1) Poor households receiving daily transfer or assistance of US$0.30 or more.
55% 52% 27% - 815,557 household in receipt of PWP wages for two months
(306,840 men and 318,717 women -excluding 2007 operation) - 16,029 households in receipt of SSP incomes over two years. 8,392 COMSIGs formed and working. 100,704 households in COMSIGS and; – US$ 96,794 – based on produce from IGA related SSP subprojects.
2. Achieve universal primary education
(2) Grade 1 children (%) reaching Grade 5.
20% 73% 90% 238,483 pupils in primary school, every term. Primary school facilities tracked by: - 471 classroom block - 462 toilets - 10 offices - 80 water boreholes - 65 wood lot - 118 teacher house 273 Functioning School Committees and PTAs.
3. Promote gender equality and empower women
(3) Girls in primary schools as % of total.
48% 50% 50.4 %
4. Reduce child mortality
(4) Under-fives malnutrition (%) using weight for age method.
30% 22% 15% 6,629 households participating in nutrition projects. 6,456 Under-fives in nutrition projects.
8 Updating based on the MDG Report for 2007 produced by the Malawi Government.
27
MDG Targets MDG Indicators Malawi
2000 Baseline
2005 Status9
Malawi 2015
Target
MASAF-LA Indicators
6. Combat HIV/AIDS, malaria and other diseases
(6) Orphans given training and tools for production.
(7) Chronically ill reached with home based care.
(8) Households in an anti-Malaria program.
N/A N/A N/A
12,849 orphans supported. 720 Production tools (carpentry, tinsmith, tailoring, knitting) distributed. 2, 944 chronically ill persons covered by Home-Based Care projects. 400 households using impregnated bed nets.
7. Ensure environmental sustainability
(9) Forest cover for non- agricultural land (%)
(10) Households with sanplats for sanitation.
(11) Households with improved water source
27.6% 77% 37%
27.2% 66%
84% 68%
3,649 Hectares planted with seedlings 8,710,823 seedlings planted. 20 households given sanplats. 111,750 households with improved water source.
8. Develop a global partnership for development
(12) Households participating in functioning Drug Revolving Funds (stocked with a specified minimum list of drugs)
N/A 80% 9 Drug Revolving Funds fully stocked and in use, benefiting 11,113 individuals
9 Updating based on the MDG Report for 2007 produced by the Malawi Government.
28
Annex 3. Economic and Financial Analysis The economic and financial viability of the project is based on an analysis of cost-effectiveness of MASAF interventions, more especially infrastructure sub-projects10. The analysis compares the costs of delivering specific infrastructure sub-projects compared to the costs of similar infrastructure delivered through projects supported by other development partners11. The analysis assessed whether project outputs were achieved at the least possible cost, in terms of financial outlays made, the time taken to complete the sub-projects and their quality compared to sector norms and standards. On the basis of this analysis, MASAF III APL 1 costs were seen to be on average 15 – 40 percent lower than those of DFID and EU. These cost differences mainly result from the different approaches that are adopted in the delivery of the sub-projects. For instance, MASAF has a higher matching contribution from the community whereas DFID and EU European Union funded Microprojects (MPP) provide 100 percent grant financing. However, even if these community contributions are factored into the sub-project costs, there is still higher cost-effectiveness under MASAF compared to the others. The other difference could be attributed to the quality differences. It was however noted that quality differences were not so wide as to explain the significant cost differences because all these sub-projects were guided by the same sector norms and standards. The calculation of the cost effectiveness ratios is shown in Table A3. Other measures of cost-effectiveness included the costs per beneficiary for education infrastructure, unit costs for roads as well as the overhead or administrative costs of sub-project delivery. On the cost per beneficiary for education infrastructure, MASAF classrooms cost about US$167 per pupil compared to US$238 for classrooms constructed with support from the DFID Education Programme. Considering continuous enrollment throughout the 40-year design lifespan of the classrooms, the average costs become US$4.19 to 4.51 and US$5.96 per pupil, respectively. In terms of the roads sub-projects, for MASAF the earthworks cost between US$1,116 – 1,200 per kilometer excluding culverts compared to US$1,286 per kilometer for EU-MPP funded roads. On the overheads or administrative costs, MASAF’s CCT Programme had a much lower overhead cost estimated at US$1.15 per dollar transferred compared to I-Life (US$8.21) and EU Public Works Programme (US$ 6.08).
10 Conventional measures of economic viability such as net present value and economic rate of return were not calculated at appraisal due to valuation challenges associated with CDD-type projects because of the difficulty in defining project activities or sub-projects ex-ante.
11 The comparison of unit costs of infrastructure sub-projects included MASAF 3 APL 1, Department for International Development and the European Union. Data on costs was obtained from LAMP and CMP records in case of MASAF, the Education Programme in the case of DFID and the Micro-project Programme in the case of the EU.
29
All these indicators support the fact, notwithstanding the comparative challenges, MASAF projects were delivered more cost-effectively compared other similar sub-projects implemented by other donors. Moreover, sustainability of many sub-projects under MASAF is higher because of the greater involvement of the beneficiaries in the entire sub-project cycle which enhances their ownership.
Table A3: Comparison of direct costs of MASAF infrastructure sub-projects with other donors (costs in US$)
Cost-effectiveness ratio (excl. LA overheads)
Item Units Descripti
on
MASAF, (excl LA
overhead)
MASAF, (incl. LA overhead)
DFID Education
Programme
EU Micro projects
PE10
Compared with DFID
Projects
Compared with EU Micro-
projects Earth Road Km
Reshaping/ drains 1,116.10 1,199.81 n/a 1,285.71 n/a 15.2
Bridge No 10m span 4,810.33 5,171.11 n/a 64,285.71 n/a n/a Culvert line Line
600mm x 6m 561.93 604.07 n/a 1,107.14 n/a n/a
Classroom block No Double 14,418.62 15,500.01 20,500.00 20,000.00 42.2 38.7 Teachers House No
Single quarters 13,132.06 14,116.96 21,525.00 20,000.00 63.9 52.3
VIP Latrine No Double 1,975.95 2,124.15 1,845.00 2,500.00 -6.6 26.5 VIP Latrine No Single 1,188.48 1,277.61 1,435.00 n/a 20.7 n/a
Source: MASAF III APL I Impact Evaluation (2008)
30
Annex 4. Bank Lending and Implementation Support
(a) Task Team members
Names Title Unit Responsibility/ Specialty
Lending
Norbert O. Mugwagwa Operations Adviser AFTHD Lending Task Team Leader
Nginya Mungai Lenneiye Sr. Social Protection Specialist AFTH1 Yisgedullish Amde Operations Office AFTH1 Prasad C. Mohan Lead IEC specialist AFTKL Victoria L. Fofanah Program Assistant AFTH1 Tesfaalem Gebreiyesus Sr. Procurement Specialist AFTPC Valerie J. Kozel Sr. Economist AFTH1 Enid Kyomugisha Program Assistant AFMUG Hope C. Phillips Volker Sr. Operations Officer AFTH1 Donald Herrings Mphande Sr. Financial Mgmt. Specialist AFTFM Krishna Pidatala Sr. Operations Officer CITPO
Supervision/ICR Nginya Lenneiye Mungai Sr. Social Protection Specialist AFTH1 Task Team Leader Norbert O. Mugwagwa Operations Adviser AFTHD Suleiman Namara Sr. Social Protection Specialist AFTH1 Simon Chirwa Procurement Specialist EAPCO Janet Chido Bvumbe Program Assistant AFMZW Alfred Sambirani Chirwa Population & Health Spec. AFTH1 Lori A. Geurts Senior Program Assistant AFTH1 Hardwick Tchale Agricultural Economist AFTAR Sylvester Kofi Awanyo Sr. Procurement Specialist EAPCO Wedex Ilunga Procurement Spec. AFTPC Chrissie Kamwendo Operations Officer AFMMW Donald Mphande Sr. Financial Mgmt. Specialist AFTFM Francis Mkandawire Financial Management Specialist AFTFM Michael N. Mambo Sr. Education Specialist AFTH1 Ramson M. Mbetu E T Consultant (SP) AFTH1 Prasad C. Mohan Lead IEC Specialist WBGSA Gert Johannes Alwyn Van Der Linde
Sr. Financial Management Specialist AFTFM
Eva K. Teh Program Assistant AFTH1 Maggie M. Mshanga Team Assistant AFMMW Rose D. Ndalama Administrative Assistant AFMMW Krishna Pidatala Senior Operations Officer CITPO Pascal Tegwa Sr.Procurement Spec. AFTPC Anju Vajja E T Consultant IEGSG
31
(b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)
Stage of Project Cycle No. of staff weeks USD Thousands (including
travel and consultant costs)Lending
FY02 14 80.07 FY03 69 370.29 FY04 1.94
Total: 83 452.30 Supervision/ICR
FY04 26 104.47 FY05 30 116.24 FY06 53 184.03 FY07 15 84.20 FY08 2 10.05
Total: 126 498.99
32
Annex 5. Beneficiary Survey Results This is a shortened version of the executive summary of the beneficiary assessment that was undertaken as part of the Impact Evaluation. The final Impact Evaluation report is available and can be consulted for detailed findings, conclusions and recommendations. This impact evaluation assessed only the first of the three programme phases (i.e. APL1, from 2003 to 2006). The objective of this impact evaluation was to conduct a retrospective assessment across five specific areas: (1) Access to services, including changes in access, utilization and beneficiary satisfaction with services and facilities; (2) Conditional cash transfers, assessing both the targeting benefits to the poor and the resultant mitigating effects on households; (3) Cost effectiveness of various sub-project investments; (4) Institutional assessment, evaluating the institutional impact of MASAF III-APL I, including influence on District Development Planning (DDP) system and LA capacity to manage projects; and, (5) Sustainability, to measure the technical, financial and physical sustainability of facilities and services implemented under MASAF III-APL I. The purpose of the impact study is to feed lessons and recommendations into the follow-up to MASAF III-APL I or similar programmes. Given the demand-driven nature of MASAF III and that there was no basis to determine ex-ante the type of projects communities were to demand, there was no baseline survey conducted for MASAF III. The evaluation team therefore explored the possibility of reconstructing a baseline using secondary data from the National Statistics Office (NSO), such as the Integrated Household Survey (IHS, 2004). The team however concluded that there was an insufficient overlap between IHS/ MASAF villages necessary to obtain a sample that would sufficiently represent the diversity of MASAF III-APL I’s projects. The primary data collection for this study was based on the household survey. The sampling strategy was based on a detailed analysis of the typology and sectoral focus of projects funded under MASAF III-APL I. The final sample size of control and treatment areas was 1590. To triangulate the results from the household survey, the evaluation team undertook a qualitative survey of project committee members (n=83), focus group discussions (n=60), key informant interviews, and conducted visits to five Local Authorities (LAs) where specialists undertook assessments of sustainability, cost-effectiveness, institutional development and the MIS, M&E and IEC systems.
Key findings on impact based on the beneficiary assessment Incomes and food security of the poorest: The Conditional Cash Transfers (CCT) programme has added a valuable complement to both the regular public works programme and the fertilizer subsidy programme. By providing higher wage rates (MK200/day) the CCT programme provided a viable means for households to purchase some food for immediate consumption and some agricultural inputs. In addition, since the CCT projects were mostly short lived, the potential negative impact of the higher wage on local labour markets was minimised. In terms of impact, an analysis of
33
expenditure patterns shows that earnings were mainly used to purchase fertilizer (about 68%), maize grain for consumption (25%), maize seed (5%), bean seed (1%) and groundnut seed (1%) across the four districts surveyed. Earnings allocated to other food items and clothing was low (below 1%). Of those households that purchased fertilizer, most purchased between 50kg and 100 kg of fertilizer. For maize seed, the vast majority of households bought less than 20kg, with over a third of households buying less than 5kg. Most households interviewed confirmed a three to four -fold increase in their maize harvest due to higher levels of fertilizer application and use of high yielding maize seed, coupled with good rainfall. For maize grain for consumption, the purchases of households varied widely, but are estimated to be equivalent to about two calendar months of household food consumption requirements. Although significant, beneficiaries would have required supplementary support to meet annual requirements in a drought season. Overall, expenditure patterns indicated that the programme made an important contribution to addressing vulnerability and the food security, and that households utilized the cash primarily to enhance their agricultural productivity. In terms of targeting, the gender allocation of tasks and the setting of wage labour rates, the CCT programme performed generally well. In most cases the selection of beneficiaries was appropriately conducted, with local leaders generally attempting to target the most vulnerable. Furthermore, local adaptations to shorten the duration of the contracted time so as to maximize the number of beneficiaries helped to further reinforce the participation of larger sections of the vulnerable population. Women were targeted more than men. This focus on women was especially important, as the targeting of women is generally believed to enhance food security and reduce poverty more than that of men. This is because of increasing evidence that women are more likely to spend cash on food and other basic commodities. Men and female household members were generally allocated different tasks, and for most women (76%) participation in the scheme was said to have elevated their position. In addition, since the CCT projects were mostly short lived (5-10 days of contract duration), the potential negative impact on the local labour markets was minimal. In terms of national impact, the results showed that the CCT assisted households to access at least 15% of the total quantity of subsidized fertilizer supplied by the government in 2005. On national food security, the CCT programme together with the subsidy on fertilizer and a favorable rainfall season contributed to the national surplus in staple food production realized in 2005/6 agricultural season of 487,000 metric tones. Access, use & satisfaction with services/facilities: MASAF III-APL1 has made an important contribution at national level in terms of social and economic infrastructure provision in both rural and urban areas. Project beneficiaries reported improvement in access to social and economic services with women confirming reduced mortality among pregnant women in areas where MASAF III implemented road projects and where vehicular traffic increased on the respective roads. MASAF APL I had made key contributions in opening-up tertiary and feeder roads, leading to reduction in the distances to social services such as health, education and to markets. According to the household survey, the main drawback with the road improvements has been the declining
34
quality given the inadequate provisions for operations and maintenance. This finding highlights the importance of maintenance, both in terms of funding and institutional setup. Key findings on sustainability from the perspective of the beneficiaries For CMP education projects (classroom blocks, staff houses, boreholes, VIP latrines), the quality of construction was found to be generally acceptable; although it could have been improved with more financial resources, correct workmanship and better technical supervision during critical stages of construction. In terms of overall durability of the buildings, this is an aspect that has yet to be studied in real time, although the differences in approach and technology between various agencies (MASAF, EU MPP, and DFID) may be expected to result in different levels of durability. This is especially important, as maintenance remains a concern, with only 22% of education projects having mobilized community funding to finance routine maintenance – a fact that is compounded by insufficient allocation of funds by government or local authorities towards maintenance. For road infrastructure projects, MASAF III-APL1 has made an important contribution to the opening up of new rural feeder roads where none existed previously. Under PWP-CCTs the primary objective of transferring cash to the poor has largely been achieved. There is a need to pursue more strongly the secondary objective (i.e. to create sustainable community assets) by reducing defects in construction and omissions at field appraisal. While it is accepted that a balance needs to be struck between the efficient distribution of cash and the construction of roads, the analysis of cash transfers showed there being relatively low overhead costs. It is therefore argued that some increase in costs could result in a much more effective programme, delivering assets that have a more far-reaching and sustained impact for all community members. Small additions such as one-off investments in basic equipment (e.g. hand rollers) and improved technical supervision would enhance the quality of the outputs, and yet the programme could remain an efficient channel for transferring cash to the poor. For Social Support Projects (SSP), the viability of projects varied considerably between different LAs. In general, success was more often dependent on the quality of the initial design and the capacity to implement the project. The capacity of CBOs in particular appears to have had a direct bearing on the performance of the project. In some projects (especially paraffin lamps and goat raising), the evaluation found that flawed design and budgeting resulted in a disproportionate amount of funds as working capital, compared to the total investment in the fixed assets (goat kraals, paraffin pump houses, etc). Only 23% of the SSP project committees interviewed had a community fund to finance routine maintenance and repair work for the infrastructure constructed. Community Savings and Investment Promotion (COMSIP) projects were one of the most promising in terms of sustainability. In the early stages however, most groups that were formed did not last long due to delays in training delivery and unmet expectations (misinformed expectations) related to the provision of loans from MASAF was not met.
35
Annex 6. Stakeholder Workshop Report and Results The summary provides only the major issues that were discussed during the implementation completion review workshop. Some of these have been highlighted in the main text of the ICR. A complete workshop report is available. APL I Design Issues 1. The extent to which the design of MASAF III APL I took into account lessons from the MASAF II: The design of MASAF III took into account lessons generated under the I and II. However, the overall design of the MASAF programme was based on the community needs assessment that was carried out by the Malawi Government in collaboration with the World Bank – where, major issues pointed to the need to have a direct community financing mechanism for improved service delivery.
2. The synergies among the project components: MASAF III components were designed in such a way that they complemented each other. Examples below illustrate the existence of such synergies:
(a) The Local Assembly Managed Projects – improve accessibility of basic social economic services (i.e. schools, health centres - CMP).
(b) Social capital from LAMP and SSP investments were being transformed onto economic capital under the COMSIP component.
(c) Assistance provided to vulnerable individuals such as orphans enables those orphans to attend schools; assistance to the critically ill eased the burden of foster households. In turn, foster households were able to participate in LAMP subprojects and earn cash income for themselves but also participate effectively in the implementation of CMP subprojects.
3. The Project’s contribution towards the achievement of the MDGs: Stakeholders noted the challenges related to the project’s contribution to the MDGs given the CDD framework, since LAs have no control on the choice of subprojects that may be need to be implemented e.g. malarial control subprojects as communities are largely used to infrastructural type of projects. Future programmes may need to consider supply driven approach for specific subprojects earmarking specific contribution towards the MDGs. Operational Issues 4. Financial management safeguards for community funds: Stakeholders pointed out some inadequacies in safeguards on revenue from SSP IGA related subprojects, which may not benefit intended beneficiaries. There was a suggestion to strengthen current mechanisms to ensure that proceeds from these projects reach the intended beneficiaries. Other operational issues included:
• Poor relationship between PMC and CBOs particularly after project completion existed in a number of subprojects and that future programmes should provide mechanisms for ensuring that the two committees work together.
36
• Consistency between community priorities and District Development Plans
(DDP). To ensure such consistency, LAs indicated that they monitor community requests, which feed into the DDP.
• Some LAs proposed that some elements of community procurement should be
vetted at the LA to minimise cases of misprocurement. It was noted that there were some isolated cases of misprocurement more especially where the PMC did not seek advice from the LA.
• Justification assistants were not fully integrated into the LA system. Justification
Assistants were viewed as MASAF staff and that LAs could not fully involve them in other LA activities.
• There were few instances where community demands for sub-projects were not
approved. It was clarified that the CDD approach allows communities to prioritise their needs, which are funded by the Project. However, community priorities may not be funded if they were not in line with sector norms e.g. request for a health centre where there is another one nearby.
• Weak relationship between PMC and other existing local level structures
(VDC/ADC). It was observed that Village Development Committee/Area Development Committee participated in pre-cycle processes such as subproject appraisal. Once subproject implementation commenced, VDCs /ADCs were not actively involved in the implementation.
• LAs indicated that functions of other community level structures such as Area
Executive Committees (AECs) were not fully recognised by the Project, particularly during subproject implementation.
Policy Issues
• Need for standardisation of appraisal systems and other operational procedures. • Need to improve on lunch allowances versus adopting traditional motivational
system. • Unclear communication channels among the key players – Communities, LAs and
MASAF MU, consequently leading to contradictory messages at times. • The need for donors to coordinate their engagement with the LA system – the LA
is flooded with different procedures from various donors which put a strain on their already constrained systems.
37
CDD and Social funds: Progress and Next steps • Major accomplishments
o Significant impact among poor populations and communities o Systems to plan, finance and manage local investment o Methods for community empowerment
• Current concerns
o Sustainability of investment and services o Sustainability of institutions o Overcoming fragmentation of instruments and interventions
• The need for broader approach to local development o How to institutionalise processes which empower communities o How to engage governance to improve front line service delivery o How to build operational links among local public sector, civil society and
private sector development efforts o How to sustain programmes supported from above through local level
institutional arrangements
38
Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Highlights of the Borrower’s ICR are contained in the main text and a complete report is available as per list of Supporting Document (see Annex 9).
39
Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N/A
40
Annex 9. List of Supporting Documents Implementation Completion Report (ICR) for Malawi Social Action Fund 3 APL 1 (MASAF III APL I: 2003-2007) Government of Malawi, February 2008. Malawi Social Action Fund 3 APL 1 Completion Review Workshop Report. MASAF Management Unit, Lilongwe. October 2007 Malawi Social Action Fund 3 APL 1 Impact Evaluation Final Report. JIMAT Development Consultants. March 2008. Malawi Social Action Fund 3 APL 1 Project Appraisal Document (PAD). World Bank Report No. 25562 – MW. April 2003. Malawi Government. 2007. MDG Report for 2007. Lilongwe, Malawi. Enhancing the Implementation of Environmental Mitigation Measures in MASAF III Sub-projects. December 2005. Report on Quality of Supervision Assessment (QSA7) for MASAF III APL I. World Bank. August 2006. Project Mid-Term Review for MASAF III APL I, MASAF Management Unit. February 2006.
Multi-dimensional results measurement in CDD projects. World Bank Social Development Findings Brief No. 285. December 2007. Integrating MDG’s into the Malawi Social Action Fund. World Bank Social Development Findings Brief No. 233. December 2003.
41
Annex 10. List of Technical Research Studies Commissioned by MASAF under APL I
1. Evaluation of the effectiveness of the SSP Project cycle, 2004 2. Cost Effectiveness of SSP subprojects, 2004-2005 3. District Baseline Surveys on MDGs, 2004-2005 4. Assessment of Financial Management Capacity of LA accounting staff, 2004-05 5. Development of CBO Bye-laws Framework, 2005-06 6. Development of Transformation Strategy for MSB, 2006 7. Evaluation of PMC Training, 2004-05 8. Lessons & Experiences of Ten Years of MASAF, 2005-06 9. Production of National and LA Base Maps, 2006 10. Development of Project Mapping System (GIS), 2006 11. Bi-Annual Audit Consultancy, 2006 12. Market Studies for COMSIP, 2006-07 13. SSP Tracking Studies, 2006-07 14. 1995-2005 MASAF Monograph, 2005-07 15. IEC Impact Assessment, 2006-07 16. MASAF III APL I Impact Evaluation, 2007 17. Development of Indigenous Knowledge Resources, 2007 18. Review of the Intergovernmental Fiscal Transfer System in Malawi, 2007 19. Partial Enhancement of the Local Assembly Management Information System,
2007 20. Preparation of the COMSIP Business Plan Note: This excludes financial audits, media production, news materials design and printing as well as internal studies conducted through the MASAF Research Unit
42
Annex 11. Allocation of MASAF III resources to LAs.
Name of Local Social Community Local Authority Total Per Local Authority Support Managed Managed Allocated Capita Authority (LA) Population Projects Projects Projects to each LA MK'million MK'million MK'million MK'million MK
1 Chitipa 126,799 20.47 23.55 22.21 66.23 522
2 Karonga 166,761 23.85 26.32 18.13 68.30 4103 Karonga Town 27,811 3.98 3.08 6.01 13.07 4704 Rumphi 128,360 26.23 19.76 19.78 65.77 5125 Nkhata Bay 164,761 28.44 30.45 15.89 74.78 4546 Likoma 8,074 1.39 2.87 2.38 6.64 8227 M'mbelwa 524,014 88.97 54.73 59.52 203.23 3888 Mzuzu City 86,980 14.77 13.95 10.17 38.89 4479 Kasungu 452,905 83.65 58.45 53.35 195.45 432
10 Kasungu Town 27,754 5.13 3.24 16.51 24.87 89611 Nkhotakota 229,460 37.12 37.37 29.22 103.71 45212 Ntchisi 167,880 22.68 32.73 22.72 78.12 46513 Dowa 411,387 64.24 48.46 47.88 160.59 39014 Salima 227,859 55.87 32.37 34.85 123.08 54015 Salima Town 20,355 4.99 2.81 15.56 23.36 1,14816 Lilongwe Rural 905,889 152.76 80.65 93.92 327.33 36117 Lilongwe City 440,471 74.28 37.85 43.73 155.86 35418 Mchinji 324,941 73.88 40.59 43.59 158.05 48619 Dedza 471,274 99.25 50.34 58.75 208.34 44220 Dedza Town 15,408 3.24 2.38 12.45 18.08 1,17321 Ntcheu 370,757 77.02 38.43 47.25 162.70 43922 Mangochi 583,669 92.18 59.41 64.22 215.81 37023 Mangochi Town 26,570 4.20 2.69 9.95 16.84 63424 Machinga 353,913 87.78 46.14 40.83 174.75 49425 Liwonde Town 15,701 3.89 1.90 8.98 14.77 94126 Balaka Town 14,298 3.08 1.85 9.69 14.62 1,02327 Balaka 238,800 51.39 32.56 30.85 114.80 48128 Zomba Rural 480,746 77.51 49.25 53.04 179.80 37429 Zomba Municipality 65,915 10.63 11.32 12.55 34.49 52330 Chiradzulu 236,050 36.66 28.16 30.12 94.94 40231 Blantyre Rural 307,344 40.55 33.12 33.36 107.03 34832 Blantyre City 502,053 66.24 42.63 50.65 159.52 31833 Neno 74,678 24.23 17.66 13.46 55.35 741
43
34 Mwanza 63,337 20.55 22.23 12.57 55.35 87435 Thyolo 450,134 88.75 50.46 54.48 193.69 43036 Luchenza Town 8,842 1.25 1.47 7.52 10.24 1,15837 Mulanje 428,322 60.41 45.20 47.76 153.37 35838 Phalombe 231,990 33.18 31.86 30.84 95.89 41339 Chikwawa 356,682 70.91 41.74 41.81 154.46 43340 Nsanje 194,924 38.32 22.56 23.60 84.49 433
Total Allocated 9,933,868 1773.90 1182.60 1250.18 4206.68 423 Average per LA 44.35 29.57 31.25 105.17 Minimum LA 1.25 1.47 2.38 6.64 Maximum LA 152.76 80.65 93.92 327.33 US$'million 16.20 10.80 11.42 38.42
