Document of The World Bank

Report No: 26216

IMPLEMENTATION COMPLETION REPORT(TF-20383; TF-21206; TF-20716; TF-21746; IDA-30500)

ON A

CREDIT

IN THE AMOUNT OF US$250 MILLION

TO THE

ISLAMIC REPUBLIC OF PAKISTAN

FOR A

SECOND SOCIAL ACTION PROGRAM PROJECT

June 25, 2003

HUMAN DEVELOPMENT SECTOR UNITSOUTH ASIA REGION

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective January 1998)

Currency Unit = PK Rupees (Rs) Rs. 1.00 = US$ 0.022US$ 1.00 = Rs. 44.0

At completion (June 30, 2002) US$ 1.00 = Rs. 62.54FISCAL YEARJuly 1 June 30

ABBREVIATIONS AND ACRONYMSADB Asian Development BankAG Auditor GeneralAJK Azad Jammu and KashmirAOPs Annual Operational PlansBHUs Basic Health UnitsCAS Country Assistance StrategyCBOs Community Based OrganizationsCFAA Country Financial Accountability AssessmentDACs Departmental Account CommitteesDAOs District Account OfficersDCA Development Credit AgreementDDOs Drawing and Disbursement OfficersDERA Drought Emergency Relief AssistanceDfID Department for International DevelopmentDHQs District Headquarter HospitalsDO Development ObjectivesDOTs Directly Observed Treatment – Short-CourseEC European CommissionEMIS Education Management Information SystemEPI Expanded Program of ImmunizationESR Education Sector ReformFATA Federally Administered Tribal AreasFDs Finance DepartmentsFM Financial ManagementFY Fiscal YearGDP Gross Domestic ProductGER Gross Enrollment RateGoP Government of PakistanGoS Government of SindhHMIS Health Management Information SystemICR Implementation Completion ReportICT Islamabad Capital TerritoryIDA International Development AssociationIMF International Monetary FundIMR Infant Mortality RateI-PRSP Interim Poverty Reduction Strategy PaperIUDs Intrauterine Device

KPP Khushal Pakistan ProgramLACI Loan Administration Change InitiativeLGO Local Government OrdinanceLHVs Lady Health VisitorsLHW Lady Health WorkersM&E Monitoring and EvaluationMCH Maternal and Child HealthMMR Maternal Mortality RateMOUs Memorandum of UnderstandingMSU Multi-Donor Support UnitMTR Mid-Term ReviewNA Northern AreasNAB National Accountability BureauNCB National Competitive BiddingNER Net Enrollment RateNFC National Finance CommissionNGO Non-Governmental OrganizationNLG Netherlands GuildersNWFP North West Frontier ProvinceO&M Operations and MaintenanceORS Oral Rehydration SaltP&D Planning and DevelopmentPA Project AgreementPAD Project Appraisal DocumentPC1 Planning Commission Proforma No. 1 (Project Document)PDP Participatory Development ProgramPFAAs Provincial Financial Accountability AssessmentsPFC Provincial Finance CommissionPHED Public Health and Engineering DepartmentPIFRA Project to Improve Financial Reporting and AuditingPIHS Pakistan Integrated Household SurveyPSR Project Status ReportPSU Primary Sampling UnitPTA Parent Teacher AssociationPWP Population Welfare ProgramQMRs Quarterly Monitoring ReportsRHCs Rural Health Centers RWSS Rural Water Supply & SanitationSACs Structural Adjustment CreditsSAP Social Action ProgramSAPP 1 Social Action Program Project 1SAPP II Second Social Action Program ProjectSBP State Bank of PakistanSC School CouncilSMC School Management CommitteeSOEs Statement of ExpendituresSWAPS Sector Wide ApproachesTA Technical AssistanceTA/DA Travel Allowance/Daily Allowance

TB Tuberculosis TGM Technical Group MeetingTHQ Tehsil Headquarter HospitalsTMA Tehsil Municipal AdministrationTPV Third Party ValidationTT Tetanus ToxoideVBFPWs Village Based Family Planning WorkersWUA Water User Associations

Vice President: Mieko NishimizuCountry Manager/Director: John W. Wall

Sector Manager/Director: Michelle Riboud Task Team Leader/Task Manager: Tahseen Sayed

PAKISTANSECOND SOCIAL ACTION PROGRAM PROJECT

CONTENTS

Page No.1. Project Data 12. Principal Performance Ratings 13. Assessment of Development Objective and Design, and of Quality at Entry 24. Achievement of Objective and Outputs 65. Major Factors Affecting Implementation and Outcome 166. Sustainability 217. Bank and Borrower Performance 228. Lessons Learned 279. Partner Comments 3010. Additional Information 33Annex 1. Key Performance Indicators/Log Frame Matrix 34Annex 2. Project Costs and Financing 42Annex 3. Economic Costs and Benefits 44Annex 4. Bank Inputs 45Annex 5. Ratings for Achievement of Objectives/Outputs of Components 49Annex 6. Ratings of Bank and Borrower Performance 50Annex 7. List of Supporting Documents 51Annex 8. Beneficiary Survey Results 53Annex 9. Stakeholder Workshop Results 54

Project ID: P037835 Project Name: SOCIAL ACTION PRG IITeam Leader: Tahseen Sayed TL Unit: SASHDICR Type: Intensive Learning Model (ILM) of ICR Report Date: June 25, 2003

1. Project Data

Name: SOCIAL ACTION PRG II L/C/TF Number: TF-20383; TF-21206; TF-20716; TF-21746; IDA-30500

Country/Department: PAKISTAN Region: South Asia Regional Office

Sector/subsector: Primary education (59%); Health (19%); General water/sanitation/flood protection sector (9%); Sub-national government administration (8%); Central government administration (5%)

Theme: Gender (P); Education for all (P); Health system performance (P); Other social protection and risk management (P)

KEY DATESOriginal Revised/Actual

PCD: 11/17/1995 Effective: 05/28/1998Appraisal: 08/29/1997 MTR: 10/01/2000Approval: 03/24/1998 Closing: 06/30/2003 06/30/2002

Borrower/Implementing Agency: GOP/Planning & Development Division; Provincial Line DepartmentsOther Partners: Asian Development Bank (ADB); Department for International Development

(DfID); European Commission (EC); and Netherlands Government

STAFF Current At AppraisalVice President: Mieko Nishimizu Mieko NishimizuCountry Director: John W. Wall Sadiq AhmedSector Manager: Michelle Riboud Ralph HarbisonTeam Leader at ICR: Tahseen Sayed Ian P. MorrisICR Primary Author: Shahid Hafeez Kardar and Task

Team

2. Principal Performance Ratings

(HS=Highly Satisfactory, S=Satisfactory, U=Unsatisfactory, HL=Highly Likely, L=Likely, UN=Unlikely, HUN=Highly Unlikely, HU=Highly Unsatisfactory, H=High, SU=Substantial, M=Modest, N=Negligible)

Outcome: U

Sustainability: L

Institutional Development Impact: M

Bank Performance: U

Borrower Performance: U

QAG (if available) ICRQuality at Entry: U

Project at Risk at Any Time: Yes

3. Assessment of Development Objective and Design, and of Quality at Entry

3.1 Original Objective:The Second Social Action Program (SAPP-II) was designed to continue support to the Government of Pakistan’s (GoP) Social Action Program (SAP) which was started in 1992. It also built upon the successes and lessons learnt during the multi-donor supported SAPP-I (1993 – 1997). Under SAPP-II, IDA contributed $213 million, while other donors contributed US$40 million, against the GoP contribution of US$5 billion. SAPP-II was designed truly as a project to support the wider government program for social sectors. At SAPP-I’s inception, Pakistan’s level of social development was low because of uncertain political commitment to improve human development indicators, inadequate levels and composition of funding, problems of gender disparities of access to basic social services, poor quality of service delivery, weak institutional and management capacity, and issues of governance that influenced the siting of schools and staff recruitment. To overcome these factors, the GoP initiated a broad-based social sector reform program through the Social Action Program in 1992, supported by donors through the Social Action Program Project (SAPP) I.

The justification for IDA assistance was to support GoP's reform program, to harmonize donors and reduce the burden of multiple projects on Government. The Bank's current support for Sector Wide Approaches (SWAPs) is based on the same justification. A learning ICR was conducted as the lessons of SAPP-II should be instructive for the design of SWAPs in other countries.

The specific objectives of the multi-donor supported SAPP-II were to:

a) build government capacity for planning, monitoring and implementing social service programsb) increase transparency and improve governance c) increase government expenditures on basic social servicesd) promote sustainabilitye) encourage NGO and private sector participation.

Assessment of Objectives:The development objectives of the project were clear, important and appropriate to address the key constraints to delivery of social services and to improve outcomes in the social sectors. The reforms focused on institutional deficiencies that held back improvements in human development outcomes, and on the policy framework required to enhance efficiency and effectiveness in the delivery of services. These objectives were also in line with the declared priorities of Government and consistent with IDA’s Country Assistance Strategy (CAS), which emphasized the need to improve social indicators as a means to achieve poverty reduction. Moreover, the human development strategy articulated in GoP’s Interim-Poverty Reduction Strategy Paper (I-PRSP) incorporates lessons learned from the SAP experience.

These objectives were pursued under the backdrop of a very complex program that aimed to bring about change in the culture and way of thinking in the entire public service delivery system. SAP sought to bring about institutional reform in a system where internal and external accountability was poor, and which included over half a million demotivated and poorly trained teachers and basic health workers, around ten to fifteen thousand demotivated and inadequately trained education and health department officials and managers, and about thirty thousand poorly trained accounting clerks, financial officials and managers handling several million SAP-related transactions manually. By contributing about 10% of the social sector expenditures, the Project was not only attempting to influence the quality, but also the fiduciary

- 2 -

integrity of the remaining 90% expenditures. All of these objectives were aimed at improving service delivery for almost several million families and their children across the four provinces of Pakistan. The task of bringing about change and institutional reforms was gigantic to start with, and was made more challenging in an overall environment which lacked capacity and accountability mechanisms. It was being implemented during Pakistan's 'lost decade' of the 1990s.

3.2 Revised Objective: Based on the inadequate progress in achieving project objectives as noted during the Mid-Term Review (MTR), and based on a shift in Government's social service delivery strategy and decentralization of responsibilities from provincial to newly-formed local governments, agreement was reached with Government to restructure the project. With the shift in service delivery strategy to local governments, the original project design was no longer suitable for provincial level program components. After project restructuring, while the overall Development Objectives (DOs) remained the same, project support was restricted to federal health, education, and population welfare programs, which had showed relatively better outcomes and were not affected by the decentralization reform measures. The objectives were revised to include Drought Emergency Relief Assistance (DERA). A portion of the undisbursed Credit was reallocated to finance essential drought-related imports with the objective of restoring and improving the productive capacity and incomes of drought-affected households, and to strengthen Government's capacity to manage emergencies.

3.3 Original Components:The original components of the $250 million SAPP-II project included:

Component Category Cost US$M (Including Contingencies)

% of Total

Program Support (27 Subprograms)

Policy, Physical, Institution Building

$218.7 m 88%

Participatory Development Program

Policy and Other $15.9 m 6%

Coordination, M&E Project Management $9.7 m 4%TA & Institutional Capacity Development

Institution Building $5.7 m 2%

Under the Program Support Component, a sub-program was defined as a SAP activity in a Province or Federally Administered Area. Since there were three SAP sectors and four provinces – there were a total of 12 provincial sub-programs. Similarly, there were 12 sub-programs in Federal Areas and three sub-programs at the federal level, giving a total of 27 sub-programs that were to be funded under SAPP-II.

The objective of the Participatory Development Program (PDP) was to strengthen the participation of NGOs and community-based organizations (CBOs) in the delivery of social services.

The Coordination and M&E component was designed to promote monitoring and evaluation (M&E) capacity at the federal and provincial levels through the strengthening of the line departments, and to support program monitoring through the Planning & Development (P&D) Departments, independent third party monitoring (through the Auditor-General of Pakistan), and impact monitoring through the Pakistan Integrated Household Survey (PIHS) undertaken by the Federal Bureau of Statistics.

The Technical Assistance and Institutional Capacity Development component was designed to support

- 3 -

capacity development in addition to those associated with the M&E component, particularly to provide technical assistance (TA) and training to federal and provincial implementing line departments. A Multi-Donor Support Unit (MSU) was established under SAPP-I to provide technical assistance (TA) to Government. MSU, funded by several donors, was continued during SAPP-II. Additional bilateral donor funds were also available to support the TA and capacity development needs of Government.

3.4 Revised Components:Following project restructuring in June 2001, the Program Support component was restricted to the federal education, health and population welfare programs, while assistance to the provincial and federal area sector programs was withdrawn. PDP was also discontinued owing to the ineffective implementation of this component. Due to the flexibility of the disbursement framework laid out in the Development Credit Agreement (DCA), requiring annual notification of allocations, the PDP and provincial sub-program components were not deleted in the DCA. Based on the restructuring agreement with Government, these components were not financed after project restructuring. The M&E and technical assistance components were kept intact after restructuring, although with reduced allocations. At GoP's request, a new component for Drought Emergency Relief Assistance (DERA) was added. With this reallocation, the balance in the IDA Credit and the trust funds was available to finance the restructured project in FY02. At the request of Government, IDA agreed to close the project in June 2002.

3.5 Quality at Entry:Given the lessons learned from SAPP-I, the policy and institutional reform agenda and initiatives proposed to address the issues were appropriate, well-directed and consistent with the objectives of the project. However, the breadth of the program, the number of actions, innovations and new mechanisms and institutional arrangements proposed for implementation during the project period made the project design complex and formidable. From a standard project mode, the shift to implementing a program was challenging. To this extent, it was unfortunate that the concerns expressed by the provincial governments at the conclusion of SAPP-I about the complexity of a multi-sector, multi-province project were not adequately factored in the design of SAPP-II.

One of the lessons learned from SAPP-I was that ownership was tenuously held at the provincial level because provincial governments were not active participants in the project design and negotiation processes. To address this during preparation of SAPP-II, workshops at the provincial and national levels were held involving a large number civil society representatives, implementation staff and government officials. However, generating political commitment and ownership of a reform program requires consistent and continuing engagement with stakeholders in program preparation, design and implementation. Subsequent events showed that more follow-up actions were required to create awareness and build ownership to consolidate gains from the consultations and dialogue that took place prior to finalization of the design of SAPP-II.

The project was over-designed from the point of view of provincial resources, and did not sufficiently account for limited provincial capacities and time required to bring about far-reaching institutional reform and behavioral change for large-scale public institutions (i.e. in over 100,000 schools and for over half a million teachers). Given the time frame for project execution, the scope, geographic dispersion, proposed interventions, targets for operationalizing initiatives and expected outcomes with regard to institutional and governance issues were unrealistic. This was largely because of the over-estimation of the financial and institutional capacity of provincial governments to implement the reform program and the expectation that the project would provide the required capacity building support. The reforms had to be implemented through existing government institutions and procedures, and were dependent on the capabilities and jurisdictions of the various executing agencies, which possessed limited capacity. Moreover, the fiscal

- 4 -

stress experienced by the provincial governments following implementation of the 1997 National Finance Commission (NFC) Award compounded their difficulties in meeting expenditure targets under SAPP-II. The NFC is a constituted institution established every five years to determine the pool of tax and non-tax revenues, and to determine the federal and provincial shares from this divisible pool.

It was also unrealistic to expect to reverse decades of neglect of indicators of social development, re-prioritize allocations in budgets characterized by rigidities and induce institutional changes within the time-frame of the project. The large number of sub-programs did not inevitably make the project unwieldy. It was the additional requirements of five-year rolling expenditure plans; financing and procurement plans for each sub-program; process and institutional changes; new policies, instruments and innovations for service delivery improvements; and expenditure reimbursement mechanisms that made the task daunting. As a consequence, the project was unnecessarily complicated for the provincial governments with hardly any experience or capacity to meet these challenges within a tight implementation schedule. Although SAPP-II was propagating and hoping to institutionalize a bottom-up approach, the mechanism of an umbrella project tended to centralize decision-making.

Governance issues that undermined the achievement of objectives related to difficulties encountered in introducing accountability mechanisms in isolation for a few sectors without modifying the overall service delivery structure. Similarly, decentralization of powers could not be instituted in the social sectors alone in an otherwise highly centralized system of administration, in which lower level government functionaries were reluctant to exercise powers already delegated to them.

The priority emphasis placed on SAP sub-sectors meant that for provinces which were already resource constrained additional budgetary resources for these sectors could only be made available at the expense of other sub-sectors. This created intra-sectoral imbalances, e.g., between elementary and higher levels of education, which would accentuate the issue of access at the secondary level.

The performance indicators for assessing accomplishment of objectives were reasonable and related to program goals. However, there was an overload of indicators to evaluate performance and to chart the progress of the mechanisms designed to achieve the project objectives. Consequently, the linkage between project inputs and performance indicators and between objectives and outcomes was most unclear.

The design required preparation of a multitude of financial reports (including Quarterly Monitoring Reports [QMRs], Statements of Expenditures [SOEs], and quarterly financial summaries a semi-annual financial forecasts) and program financial statements. Of these, only the QMRs and the SOEs eventually were prepared. Government manually maintained the accounting system and had limited human capacity for generating such statements within the time-frame for submission. Financial statements could not be prepared as envisaged in the Memorandum of Understanding (MoU) on Financial Management and, in the end, SOEs were considered as financial statements. The manual tracking and documenting of several million manual transactions to IDA standards was humanly impossible.

SAPP-II was appraised and became effective prior to the introduction of the Bank's Loan Administration Change Initiative (LACI) guidelines. Hence, no formal financial management assessment was conducted at the appraisal stage. While an MoU on financial management was agreed, which provided a schedule and format for the dissemination of regular financial information, it could not be followed due to a lack of client ownership and capacity at the provincial levels.

A reliable system for generating accurate financial information was not in place throughout SAPP-II, largely because the design of the expenditure accounting and reporting system had not factored in the need

- 5 -

for separate SAP and non-SAP account heads. Consequently, field staff found it difficult, both conceptually and procedurally, to segregate eligible SAP expenditures. As expenditure heads for development and non-development/recurrent budgets are not the same, it made financial reporting less intelligible and consistent. As a result, the accounting system failed to provide the needed reliable data to ensure timely reimbursements and to facilitate efficient service delivery and effective monitoring and evaluation. It unduly constrained the ways and means positions of provincial governments.

Whereas difficulties were encountered by the provinces in the functioning of the expenditure disbursement mechanism, additional complications were introduced, including the requirement that SOEs be verified by the Office of the Accountant General, a condition that was out of tune with institutional capacities at the provincial and lower levels. Moreover, the incentive for line departments to prepare SOEs was weak in light of the fact that allocations had already been made, and reimbursements against eligible SOEs would flow to provincial exchequers.

Differing GoP and IDA procurement procedures led to a host of unresolved issues. The design was weak in that it was not possible to have a SAP-specific financial management and procurement system, but rather it would be an integral part of the overall system. Although the vast majority of procurement contracts were below the threshold of US$200,000 for prior review, as mentioned in the DCA, ex-post reviews carried out on a sample basis were still required. These requirements and all relevant information on IDA procurement procedures could not feasibly have been communicated to all staff dealing with government elementary schools and basic health units, as well as to the large number of executing agencies implementing the education and health programs at the district level and below. Government procedures, as specifically applicable to the program, along with other specific requirements of donors (e.g. preparation of annual financial statements, statement of expenditures and Third Party Validation exercises), should have been collated in the form of a manual/standard operating procedures, agreed upon by all the stakeholders and properly disseminated at the start of the program to all levels of staff responsible for implementation of the program.

Procurement training workshops were held, although late in project implementation. At the design stage, IDA should have conducted a detailed dialogue on NCB procedures to reach agreement on a set of NCB procedures acceptable to all. When SAPP-II was appraised, these references were not included in the legal documents. Due to this lack of prior agreement, the Auditor General's Third Party Validation (TPV) reports continued to cite non-compliance with IDA procedures and, in certain cases, with those of Government. While procurement reviews were included in the third party review of governance issues, as a part of the Auditor General's annual TPV reports, it was also important to include specific procurement ex-post reviews in the project design. This was necessary in view of the large numbers of small procurement actions below IDA's prior review threshold. These ex-post reviews should have been agreed to, incorporated in the project design and funded by the project itself.

4. Achievement of Objective and Outputs

4.1 Outcome/achievement of objective:The achievement of the objectives of the project, including an assessment of the current status, are described and rated below:

(i) Building government capacity for planning, implementation and monitoring of social service programs The achievement of this objective is rated Unsatisfactory for reasons given below:

- 6 -

SAPP-II aimed to enhance government capacity for planning and monitoring by requiring provincial line departments to prepare Annual Operational Plans (AOPs), Quarterly Monitoring Reports (QMRs) and estimated associated budgetary requirements. Planning sections of the line departments prepared AOPs and the QMRs. These institutional arrangements for planning and monitoring were supported by instruments such as the Education Management Information System (EMIS) and the Health Management Information System (HMIS).

Planning capacity showed some improvement during the SAP years, as evidenced by the ability of the line departments to prepare AOPs with limited support from outside consultants. Whereas SAPP-II introduced discipline through the QMRs, accountability structures tend to become more diffused in a program approach unlike a project in which targets are clearly defined and responsibilities can be fixed. Moreover, little headway was made in the organizational restructuring and streamlining of management systems which were required for service delivery improvements. The experience of the monitoring activities was mixed because: (a) monitoring was largely reduced to a reporting exercise, rather than an action-oriented one for the purposes of identifying timely corrective measures; and (b) the monitoring systems mostly generated quantitative data and were not sensitive to the qualitative aspects of service delivery, which was the key issue under SAPP-II.

Generally, audit reports were not submitted six months after closure of the financial year as required by the Development Credit Agreement (DCA), and these were mostly submitted during the grace period. This started to improve later in the project. The understaffed system was unable to manually prepare and submit audit reports, which had been reviewed by the line department's Developmental Account Committees [DACs], within six months of the end of each financial year.

Finally, Government implementation and monitoring capacity at all levels was constrained by critical gaps in leadership and continuity of leadership, particularly in the line departments. The absence of strong and stable departmental teams made it more difficult to implement the sector strategies and to deliver improved services. However, during the last two years of the project, effort was made to provide more stable leadership to line departments. Given such progress during the SAP years, the troubling situation over the last two years in the area of government capacity has only worsened, now exacerbated by the enormous capacity building needs at the district level.

(ii) Increasing transparency and improving governance:The achievement of this objective is rated Satisfactory for reasons given below:

There was some progress on governance issues. While some areas were not adequately addressed during the course of SAP, some elements of governance improvement measures were undertaken, were adopted widely, and have become institutionalized at various levels of government.

Government-donor coordination was instrumental in the distinct improvement of site selection for facilities and in merit-based recruitments of staff, these areas continued to be vulnerable to intervention. This was further compounded by frequent staff transfers.

The Third Party Validation (TPV) exercises conducted during SAPP-II were useful to track progress of governance reforms, particularly in the line ministries. While there was mixed acceptance of the TPV survey results by departmental officials, TPV as a concept was appreciated and third party validation began to be accepted as an appropriate instrument for certifying certain outputs. Line departments are now often using this method to verify inputs and progress. Further, in the last couple of years provincial governments have put in place measures to improve staff accountability, including systematic efforts to

- 7 -

monitor staff absenteeism. As a result, some provinces have been successful in reducing absenteeism in the education sector. The effectiveness of supervision has remained weak owing to: (i) inadequate allocations for travel allowances and per diems, and (ii) lack of administrative control of supervisors over the field staff whose performance they were required to supervise. Overall, however, measures to improve governance and transparency have been widely adopted.

(iii) Increasing government expenditures on basic social services and promoting fiscal sustainabilityThe achievement of this objective is rated Unsatisfactory for reasons given below:

GoP agreed to increase public expenditures on the social sectors from 2% of GDP to at least 2.4% during the SAPP-II period, with a government contribution of between 1.8% to 1.9% of GDP per year, with the donors providing the remaining 0.5% to 0.6%. Additionally, the protection of spending for SAP sectors was a structural benchmark under the IMF Enhanced Structural Adjustment Facility and the Extended Fund Facility for 1997-2000.

As noted in Section 5.1, during SAPP-II implementation, Pakistan experienced severe fiscal stresses which required expenditure compression to stay within IMF-guided limits for budget deficits. Nonetheless, the social sectors fared better than what might have been expected in light of the squeeze on expenditures. Defence spending declined from 5.2% of GDP in 1996-97 to 4.6% in 2000-01, compared with a fall in education-related expenditures from 1.8% of GDP to 1.7% of GDP over the same period and a leveling off of health and population spending at 0.6% of GDP. Health and population spending could be maintained throughout this period, in part, because a significant proportion of expenditure was funded by federally-run programs which were less susceptible to resource flow volatility than provincial governments following the 1997 NFC Award. As provincial fiscal crises worsened, allocations and expenditures on SAP sectors declined during the last two to three years of SAPP-II. As a result, total allocations for SAP sectors declined from 2.3% of GDP in 1997/98 to only 1.8% of GDP, with a smaller decline in expenditure from 1.7% of GDP to 1.5%.

(Percent of GDP)

1997/98 1998/99 1999/00 2000/01 2001/02Allocations Total 2.3 2.3 2.1 1.9 1.8 Foreign Aid 0.3 0.4 0.3 0.1 0.1 Govt. Contribution 2.0 1.9 1.9 1.8 1.7Expenditure Total 1.7 1.6 1.4 1.4 1.5 Foreign Aid 0.1 0.1 0.1 0.1 0.1 Govt. Contribution 1.6 1.5 1.3 1.3 1.4

SAP Allocations and Expenditures

Funds for recurrent spending (both salary and non-salary) were not commensurate with the number of facilities, particularly schools, that had been established during SAPP-I. For example, very few teachers were recruited or redeployed to make schools operational and line departments were unable to take on these recurrent obligations to ensure the financial sustainability of quality-related interventions. Although allocations for non-salary items were increased during SAPP-II, releases were usually delayed and often not in line with the allocations because of resource constraints. In fact, according to an agreement with donors, funds for approved sub-program activities were to be released to implementation agencies no later than the fifteenth day of the beginning of each quarter. Despite this, success in securing timely releases

- 8 -

continued to be poor. To achieve expenditure level targets agreed with donors, governments often hastened to spend funds released late in the fiscal year, yielding numerous audit objections.

In retrospect, projections to determine provincial capacities to sustain recurrent costs were either missing or overly optimistic. Several large health and education investment projects came to a close during SAPP-II. Provinces had to find a greater share of recurrent costs than originally envisaged, making the transition from project financing to the regular recurrent budget more difficult. Overall, social sector budgets were protected during SAP, despite tight fiscal constraints. In the last couple of years, allocations for education and health have increased with pro-poor programs under PRSP and with provincial government reform programs.

(iv) Encouraging NGO and private sector participation in the delivery of social servicesThe achievement of this objective is rated Marginally Satisfactory for reasons given below:

While the Participatory Development Program (PDP) was the major vehicle for promoting the role of civil society for improving the delivery and quality of social services, other processes were undertaken to encourage private sector and community participation in social service delivery. Over 100,000 School Management Committees (SMCs)/Parent-Teacher Associations (PTAs) were established for primary schools and Water User Associations (WUAs) were formed, indicating a trend toward increased community involvement at the facility level.

The PDP component proved to be a non-viable mechanism to foster community and NGO participation, mainly due to design problems. PDP was an initiative imposed by donors with little support at the provincial level, stemming largely from concerns about providing grants to NGOs from borrowed funds. Implementation of PDP was also hampered by delays in the selection process. In most cases, the selection process was delayed for long periods of time at the provincial level, often reaching the national level two years following the initial grant request. Complaints about the lack of transparency in the selection process added to delays at the national level, leading to a virtual stalemate in implementation of this component.

Although SMCs/PTAs and WUAs were established, progress in achieving their stated objectives was limited. This was partly because communities were not seen as key partners in the process, and partly because of an under-estimation of the time required to foster, consolidate and institutionalize community participation and to scale-up successful strategies and approaches for community mobilization. Despite the lack of progress in institutionalizing NGO and community partnerships within line departments, some expansion of partnerships with beneficiary communities did take place. These included training of PTAs, the formation of community water users’ groups and recruitment of community mobilization staff in the RWSS sector.

As a result of the intense focus of SAP on emphasizing community involvement at the grassroots level, the importance of community participation for improving service delivery is now gaining institutional acceptance. Continuing support to SMCs by provincial governments is evidence of a realization of the need for community involvement in the delivery of education services. Many government policies and programs have emerged that emphasize public-private partnership and community involvement, including the Education Sector Reform (ESR), Tawana Pakistan (a national girls nutrition program), the Khushal Pakistan Program (KPP), and micro-finance sector work involving community organizations. SAP's emphasis on NGO and community involvement has been widely accepted by Government over the last couple of years as the standard practice for most development programs. While much remains to be done to strengthen and institutionalize outreach programs through NGOs and communities, acceptance of partnership with NGOs and communities has grown. It was unfortunate, however, that donors had begun

- 9 -

to withdraw from the program by the time provincial governments began to accept the importance of community involvement for managing assets from public sector investments. While PDP did not work as planned, SAP resulted in a policy shift towards community participation and NGOs.

Overall Summary:For ordinary citizens expanded investments and active donor involvement did not fully translate into more and better quality service delivery. An important reason underlying these weak outcomes was a failure to functionalize all the facilities established during SAPP-I as a result of continued recruitment bans (resulting in around 20% of health outlets being without a female health worker, and hundreds of closed schools due to lack of teacher appointments), thus defeating the purpose of creating additional service outlets.

Outcomes were also poor owing to underutilization of service delivery outlets because of inappropriately selected sites, poor maintenance of installed infrastructure, and an inadequate number and skill mix of staff at the service centres. This was compounded by: (i) compression of expenditures to adjust for resource shortfalls, (ii) lack of timely provision of complementary non-salary inputs, as well as equipment and consumables critical to service delivery, and (iii) continued weaknesses in accountability mechanisms. Nonetheless, the upgrading and repair of physical infrastructure in the education sector improved the overall teaching and learning environment and contributed to better access.

Other factors that affected progress on project objectives included frequent transfers of key personnel and poor progress on governance related issues and persistent absenteeism of service providers. Limited financial and human resource capabilities in provincial governments also slowed reform progress, and did not attract the degree of attention required to attain project outcomes, particularly for a program this large and complex.

While there was mixed progress with some satisfactory ratings for the achievement of objectives, as outcomes did not improve to a satisfactory level, overall performance of achievement of development objectives is rated Unsatisfactory.

Despite the uneven progress on outcomes, for the first time in Pakistan’s history SAP provided a framework that brought together policy makers, managers and implementation staff of the social sectors and pushed to the forefront of public discourse the need to accelerate social development. To that end, SAPP-II had notable success in changing mindsets and in creating awareness about past neglect of social indicators, especially in the sub-sectors of elementary education and primary healthcare. Expenditures on the social sectors were accorded some degree of protection, even during periods of severe financial crisis, with a focus on the operationalization of facilities rather than merely the establishment of new infrastructure.

Other than SAPP-II’s accomplishment in changing attitudes and focusing attention on the social sectors, its positive impacts also include enhanced emphasis on improving service delivery through staff training and checks on service provider absenteeism. Heightened emphasis on the provision of quality services created an environment in which the structural framework for the local governments installed in 2001 was designed. SAP's emphasis on the need to improve service delivery at the implementation level created a rationale for the devolution of powers to local governments.

A realization also developed that the provision of additional funds is not a sufficient alone for ensuring the implementation of a policy reform agenda. Institutional restructuring and strengthening are required to buttress it. In particular, during the second year of the project, the findings of the Annual Audit Report and identification of ghost schools and teachers in a survey conducted by the Government of Punjab alerted

- 10 -

both decision makers and donors to the continuing failure to enhance expenditure efficiencies and ensure sustainable improvements in matters related to governance, accountability and transparency of operations. To address this, SAPP-II initiated contract-based, facility-specific recruitment of teachers and health sector personnel, as well as policy to induct female teachers for primary schooling.

In addition, initiatives like scholarships for girls and the provision of free textbooks for primary classes as incentives for parents to send their children to school are now being funded by provincial governments from their own resources. Another important contribution of SAPP-II was the establishment of PTAs and SMCs through which communities were involved in the functioning of government-run primary/elementary schools. Although the role of SMCs in the management of schools is somewhat peripheral in nature, their formation and wider acceptance of a community role in the running of schools appears to be a sustainable intervention under SAPP-II. Provincial governments have both continued to support these committees financially as well as empowered them to utilize these funds.

4.2 Outputs by components:Performance in outcomes by sector is discussed below.

SAP/SAPP is generally perceived to have failed in achieving its objectives in view of the poor outcomes and continued poor delivery of publicly provided social services.

Education:Outcomes for Education are rated Unsatisfactory for reasons given below:

Primary enrollment levels based on PIHS data remained relatively flat from 1998-99 to 2001-02, rising by only one percentage point, from 71% to 72%, with the gross enrollment rate (GER) for girls constant at 61%, while for boys increasing from 80% to 83%. There was, however, a narrowing of the urban-rural gap, with rural GER growing from 63% to 66%, and urban GER declining from 94% to 91%.

The net enrollment rate (NER) at the primary level for both rural areas and girls increased by one percentage point, but fell by the same margin for urban areas and boys, thus keeping the national NER at its 1998-99 level of 42%. Similarly, NER for the middle level of education remained constant at 16%, the same as in 1998/99. Subsequent analysis done by GoP shows NER to be 52%. For the purpose of this report, original data provided by the Federal Bureau of Statistics is being used.

SAPP-II introduced a number of good initiatives to improve the quality of and access to elementary education, especially for the most disadvantaged. Some important measures that were undertaken to achieve these objectives included the introduction of contract-based, facility-specific hiring of teachers, adoption of a merit-based system of recruitment to improve teacher accountability, and teacher training to promote quality education. Policy reform to adopt co-education and induct female teachers for primary schooling during this period aimed to expand female access to education. Incentives for parents to send their children to school, including free textbooks for primary students and scholarships for girls enrolled in grades six to eight were also available in some areas to address demand side aspects of low participation, poverty, and high opportunity costs.

The concepts and strategies launched by SAP have become a part of the policy debate on education, with at least two provinces continuing to fund free textbooks and stipends to students from their own resources. These interventions are increasingly accepted as promising instruments for enhancing enrollment and retention of children, especially girls. While the potential of these initiatives is increasingly recognized, provincial governments are grappling with the mechanisms and institutional arrangements for their effective

- 11 -

implementation.

Health: Outcomes for Health are rated Marginally Satisfactory for reasons given below:

There was a marked improvement in core health indicators during the SAPP-II period. The infant mortality rate (IMR) declined from 89 per 1000 births in 1998-99 to 82 per 1000 births in 2001-02, with a large urban-rural gap of 23 percentage points, reduced from 27 points in 1995/96. The percentage of children 12-23 months who had been fully immunized improved slowly, increasing from 49% to 53% with a small male/female gap. Immunization coverage of children increased in all provinces, except Balochistan, where coverage fell from 34% in 1998/99 to 24% in 2000/01.

Antenatal care utilization grew modestly, increasing from 31% in 1998/99 to 35% in 2001/02, with a persistent urban-rural differential (63%/26%). An increase in the use of antenatal care were observed across all provinces. However, there was no change in post-natal care consultations. The proportion of women who received at least one Tetanus Toxoid (TT) vaccination increased from 39% in 1998/99 to 46% in 2001/02. However, only 38% of pregnant women received two doses of TT vaccine during their last pregnancy. The proportion of women assisted by skilled birth attendants during their last pregnancy increased modestly from 19% in 1998/99 to 24% in 2001/02. The 2003 Health Management Information System (HMIS) report indicates a modest increase in the utilization of first level care facilities from an average of 21 patients to 28 patients per facility per day. The 2001/02 Pakistan Integrated Household Survey (PIHS) assessment of first level care health facilities reported all Rural Health Centers (RHCs) (n=17), and 92% of the (87 out of 94) Basic Health Units (BHUs) had female health staff.

As part of the programmatic policies, there was an increase in resource availability for the Lady Health Workers (LHWs) program, immunization, HIV/AIDS control and TB Control. The increased allocations were used to expand coverage for the LHWs program, ensuring regular availability of vaccines, expanded health education activities of the HIV/AIDS program, and initiation of the TB control program using the Directly Observed Treatment Short-Course (DOTS) strategy. There was also expansion of family planning services through the health sector, although this was constrained by a lack of regular availability of contraceptives. Improvements in the areas of malaria control and nutrition were limited.

Programmatic expansion resulted in improved program outputs. LHWs program coverage was expanded from 25% to 50% of the population, with 62% of rural household having access to basic primary health care and family planning services through LHWs. Evaluation of the LHWs program indicates that it has had a significant impact in areas with improved access to basic services. Health outcomes, such as the proportion of mothers who receive Tetanus Toxoide (TT) vaccinations, births attended by skilled birth attendants and contraceptive use were considerably better in areas served by LHWs. Impact assessment of the information education campaign of the HIV/AIDS program in 2001, in comparison to 1996 data, indicates a major improvement in people’s knowledge of HIV transmission and prevention measures, with and an increase from 4% to 75%. Although the TB DOTS program was launched late in SAPP-II, population coverage rose to 45% from 8%.

These advances indicate a shift away from tertiary health care in favor of preventive, primary and secondary health care. The policy shift towards preventive health care is still being pursued in the new GOP’s 2001 Health Policy, which continues to focus on strengthening preventive health services, including immunization, communicable diseases, such as malaria, TB, and HIV/AIDS, maternal and child health,

- 12 -

nutrition and family planning, and improving the quality of health care at the tehsil and district headquarter hospitals.

Health outcomes and health care utilization continue to improve slowly. Improvement in indicators for maternal health care was slow because change in the quality of related services was, at best, marginal. Whatever positive results there have been for maternal care, can be partly attributed to an improved availability of female health care providers, including LHWs. Despite higher investments, however improvement in the availability of services by skilled birth attendants was nominal. It is misleading to attribute the entire improvement in health indicators to the effectiveness of sub-program interventions, as these indicators are also strongly influenced by factors such as educational levels, fertility rates, availability of clean drinking water, etc.

Rural Water Supply and Sanitation (RWSS):Outcomes for RWSS are rated as Unsatisfactory for reasons given below:

The rural water supply sector did not undergo any significant change over the period 1998-99 to 2001-02. There was no increase in the percentage of households relying on piped water inside the house, although there was a decrease in reliance on open sources of drinking water, with the use of hand and motor pumps having increased from 65% to 70%.

There was hardly any change in sanitation practices in rural areas. The percentage of households relying on covered and underground drains remained at a low 1%, while there was only modest improvement in access to open drains, from 31% in 1998-99 to 33% in 2001-02. The availability of toilet facilities also improved marginally. This facility was available to 37% of the rural population in 1998-99, which increased to 41% in 2001-02.

SAPP-II improved consistency in the policy environment for the RWSS sector. The adoption of a uniform policy at the outset ensured that all investments in rural water supply and sanitation, regardless of the source of financing, were based on community organized schemes -- a fundamental departure from the supply-driven approach prior to SAP. A moratorium continued to apply to the installation of new schemes not designed using these principles, with the exception of the KPP schemes started in 2000. The effective translation of a uniform policy into better access to clean potable water in rural areas was unclear.

There have been prevalent concerns that many water supply and drainage schemes ostensibly transferred to communities, particularly mechanized drainage schemes, would eventually be abandoned and fall into disrepair. The inability of provincial governments to rehabilitate schemes to deliver service levels acceptable to communities impeded their transfer. In some cases of successful transfers, WUAs were formed on the basis of official directives in the absence of substantive preparation and genuine participation by communities. Neither situation bodes well for the future sustainability of functioning water supply schemes.

Population Welfare:Outcomes for Population Welfare are rated as Satisfactory for reasons given below:

The downward trend in fertility rates noted after SAPP-I was maintained during SAPP-II, with a decline from 5.4 in 1996/97 to 4.8 in 2000/01. There was a gradual increase in awareness and use of family planning methods during SAPP-II. Contraceptive use among married women increased from 93% to 96%. The number of women currently using any family planning method increased from 24% in 1996/97 to 28% in 2000/01. Contraceptive use has been increasing, but it still remains very low compared to other

- 13 -

countries in the region. Knowledge of family planning also improved in rural areas, from 91% to 95%. Availability and access was improved through diversification of family planning sources through health facilities (about 50% of health facilities provide family planning services), expansion of community-based workers, and expansion of social marketing thorough private outlets and NGOs. The availability of contraceptives in the population welfare centers was maintained. However, availability of adequate supply levels in health facilities deteriorated as the Ministries of Population Welfare and Health failed to resolve policy differences on the pricing of contraceptives and procedural requirements to issuing sales receipts to replenish supplies.

There was little progress in implementing a broader reproductive health approach, despite development of a National Reproductive Health agenda, through a joint effort by the Ministry of Health and Population Welfare.

The most effective interventions were arguably those which brought about improvements in knowledge dissemination to promote behavior change, with endorsement to permit method-specific messages through mass media (except for those pertaining to condoms use). Increased availability and access to family planning services through the LHW and Village Based Family Planning Workers (VBFPWs) programs, and expanding social marketing interventions made services and supplies more accessible overall.

Participatory Development ProgramWhile NGO and community participation in social sector service delivery was not increased effectively through PDP implementation, the emphasis on increasing the involvement of NGOs and communities has been accepted as standard in the policy frameworks (ESR, PRSP, etc.) of most major social sector programs (see 4.1 above).

Coordination and Monitoring & EvaluationThe strengthening of line departments and P&D departments, independent third party monitoring through the Office of the Auditor General, and impact monitoring through PIHS were key elements of the monitoring & evaluation component. The TPV as an instrument to monitor governance issues was found to be extremely valuable. The PIHS was very useful in monitoring of output and outcome data, and also served as a major source of information and evaluation by donors and government at MTR. Line departments' capacity for monitoring and evaluation was not developed extensively, although planning tools (Annual Operational Plans) were introduced and made a regular part of the SAPP-II requirement. Monitoring through quarterly monitoring reports was carried out regularly during SAPP-II. These mechanisms were not institutionalized, however, and the building of capacities to plan, implement and monitor remains as a major challenge for most line departments.

TA & Institutional Capacity DevelopmentTechnical assistance was designed to build the capacity of line and P&D departments, mainly through the Imprest Accounts mechanism. This mechanism did not work, owing to delayed approvals of internal government procedures for utilization of Imprest Accounts. These accounts became operational towards the end of the Project. As a result, funds for TA went largely unutilized. Additional TA was provided through the Multi-Donor Support Unit (MSU), which managed to provide assistance to line departments and P&D departments on specific sectoral issues. The MSU was found by line and P&D departments to be quite useful, although new capacities were not always successfully institutionalized.

4.3 Net Present Value/Economic rate of return:Not applicable

- 14 -

4.4 Financial rate of return:Not applicable

4.5 Institutional development impact:The federal and provincial SAP Secretariats were responsible for the coordination of SAP activities linked to planning missions, handling administration, financing issues and monitoring. With the help of the MSU, the personnel of these units were trained to carry out their duties, although capacity building gains were frequently lost to staff transfers. The monitoring of actual program implementation by various line departments was weak.

The capacity of the new organizations such as teaching institutions, district health management teams, SMCs and WUAs, as well as the human capital for developing policy and for planning new instruments and program interventions (particularly in health sector) were strengthened through a variety of parallel initiatives funded through sector investment projects. However, some of the benefits of training to realize project objectives were lost with the re-deployment and transfer of trained staff.

Limited progress was made in strengthening capacity and creating minimum critical mass in terms of the number of personnel, their competence and skill composition, and in supporting the infrastructure required to manage systems. The necessary financial management staff were recruited only after significant delays. At MTR, there was realization of the need to review procedures for the use of Imprest Accounts to enable timely and flexible use of funds for capacity building and monitoring and evaluation. While planning, monitoring and evaluation capacities at the provincial level improved somewhat (as demonstrated in the preparation of AOPs and, in some cases, the formulation of five-year rolling expenditure plans), they remained particularly weak at the district level.

The continued lack of institutional capacity, which was a major handicap in the implementation of SAPP-II, was partly because projects for capacity building and technical assistance under SAPP-II were not processed by the provinces in time, and only a fraction were approved. This unduly delayed the capacity development initiative under SAPP-II, which accounts for the fact that the majority of capacity building activities took place towards the end of the project. This affected, in particular, the availability of trained staff at the district level. For example, training for Drawing and Disbursing Officers (DDOs) in financial management was carried out in the concluding phase of the project rather than at the beginning, as it should have been, resulting in a less-than-anticipated impact on improving staff's skill and experience. Part of the problem in building capacity was that, other than policy reforms to improve service delivery, some management and institutional changes also required additional funding which provincial governments could not provide.

Building ownership for reforms, and dissemination of SAP objectives and strategies to a wider government and non-government audience should have been an important part of the program. As these were not part of the mainstream program, there were negligible efforts to communicate the program's objectives and strategies. The design of the program might have benefited from the incorporation of a communications strategy, particularly for building ownership of the reforms prior to implementation. Such a communications strategy would have helped to raise awareness and reduce the effects of institutional memory loss stemming from constant staff turn over in key SAP-related departments. One donor agreed to support development of a SAP communications strategy and to support awareness campaigns. However, this was both a late and abortive entry which never took off as it could not be integrated into the mainstream program.

The rating for institutional development impact is Modest. While there were problems in building

- 15 -

institutional capacities, particularly for planning and monitoring, SAPP-II gave rise to the foundations for policy reforms and ownership of them. Community and NGO participation, devolved social service delivery, and improvements in governance and transparency are three critical areas for social sector programs which have become increasingly institutionalized over the past couple of years.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:A major constraint to the achievement of SAPP-II objectives, particularly with respect to governance and program management issues was that the environment and the overall political and macro economic situation that existed at the program design stage changed continuously during implementation. This period was characterized by political instability, economic crisis, a severe drought, austerity measures stemming from a resource squeeze and accountability drives launched in 1999/2000. The project became effective shortly after Pakistan’s nuclear test in May 1998, which resulted in sanctions against the country. Provinces were faced with a severe financial crunch owing to a shortfall in revenue receipts (70% of the difference between actual and projected revenues due to a decline in the tax ratio, and 30% due to a lower than expected GDP) and lower than expected fiscal transfers from the divisible pool. Under the 1997 NFC Award, resource flows to the provinces during the SAPP-II period were Rs.192 billion less than what they would have been had the 1990 NFC formula been retained. The provincial governments also encountered difficulties in financing expenditures upfront due to the GoP's fiscal deficit targets.

In addition, the proportion of expenditures eligible for reimbursement were agreed annually, thus creating problems for provincial finance departments to understand the financing arrangements. They were unable to factor in estimates of potential inflows on the recurrent side of the budget from donor reimbursements as the ratios were agreed and finalized much later in the year. However, provincial line departments admitted that the project had served as a useful instrument for protecting expenditures on the social sectors, even in such adverse circumstances, and that without donor support allocations for SAP sectors may have been significantly lower.

There was a funding hiatus between the last disbursement under SAPP-I and the first disbursement under SAPP-II. Credit flows from social sector projects were tapering off and the volume of retroactive financing that was available as a replacement was inadequate. This gap, and accompanying uncertainty, demotivated the finance departments, especially since they were hard pressed to provide upfront funding for on-going activities, and meet the fiscal deficit targets set by GoP. One of the reasons for a lack of provincial interest in SAPP-II was provinces' belief that donor funding was inadequate to address the real resource requirements. The donor reimbursement ratios, ranging between 6% to 8% were viewed as modest, especially in view of the conditionalities that were attached to the funding. However, while the level of external financing was perceived to have fallen short of expectation, provincial governments never spent what they had committed to, particularly because they were in a fiscal bind and were unable to finance expected levels of social sector expenditures.

Progress in achieving the desired outcomes in the social sectors was adversely affected by socio-cultural factors whose influence cannot be reversed, but rather requires continuous and sustainable interventions over a long period of time. For instance, although awareness of contraceptives among married women in the age group 15 to 49 increased sharply from 38% in 1991-92 to 96% in 2001-02, there was a lower proportional increase in actual use of such methods and instruments, from 24% to only 28% (Pakistan Reproductive Health and Family Planning Survey). The gap between knowledge and practice/use continues to be wide because of traditional socio-cultural resistance to family planning, and inadequate number of outlets and female workers, and lack of female education. However, the utilization of family planning

- 16 -

services improved in urban centres because of the demand created through social marketing efforts.

5.2 Factors generally subject to government control:Government ownership of SAPP-II and its commitment to the objectives was built slowly, but remained uncertain and generally weak throughout the duration of the project. In addition, formal donor supervision, oversight and coordination arrangements contributed to the development of a perception that SAPP-II was donor-driven, which further undermined efforts directed at creating domestic ownership of the program. Resultantly, SAPP-II was often perceived as being donor driven with uneven awareness of its objectives and proposed prescriptions at the provincial and lower tiers of government.

Whereas the importance of sensitizing the political leadership was apparent, there was a lack of a concerted effort to strengthen and expand this awareness. Although policy dialogue played an instrumental part in raising awareness of the need to attend to the low social indicators, it was largely confined to the bureaucracy. Similarly, awareness of the program at the field level was restricted to the provision of funding, with implementation staff having virtually no knowledge of the details of the program and its objectives. Nevertheless, ownership improved in the last two years of the project with the establishment of the Social Sector Committee for Cabinet under the stewardship of the Federal Minister for Education, with the participation of provincial social sector ministers. This resulted in social sector issues being brought to the forefront of policy discussions, although commitment to the reform agenda remained stronger at the federal level. The ESR was framed under this context of ownership of the reforms.

5.3 Factors generally subject to implementing agency control:The key factors that had a substantial adverse impact on implementation of SAPP-II and the functionalization of service outlets constructed during SAPP-I were not sector-specific, but persistent and common across sectors and geographical divisions of the country. Although the Government agreed that recruitment bans would not extend to SAP sectors, in practice these restrictions were applied indiscriminately, with the social sectors not exempt from their reach.

A lack of continuity of key staff negatively affected the level of commitment, the quality of leadership, institutional memory and understanding of the program, making it difficult to hold anyone accountable, thus disrupting reform efforts and compromising project implementation. The inability to ensure meaningful community participation in service delivery, continuing governance issues, such as lack of credible and effective arrangements to check staff absenteeism, and ineffective supervision and monitoring systems impeded sustainable improvements in service delivery.

5.4 Costs and financing:The project appraisal document estimated the five-year cost of the program at over US$10 billion. Out of this US$8 billion was estimated as Government financing and US$610 million (including ADB) was committed through SAPP-II donor financing. In addition, approximately US$1.4 billion was forecast as additional donor support, including disbursements from ongoing donor assisted projects, new project assistance, and additional mobilization of donor funds. By the end of the program period, Government expenditure on SAP was about 61% of appraisal estimates, and SAPP donor financing amounted to approximately 66% of the committed amount. The financing gap was also due to a lack of new large donor project assistance, estimated in the total program costs during appraisal, compounded by the fact that several large ongoing donor assisted projects had closed by 2000.

SAPP-II project financing mechanism-SOEs, financial management issues, disbursements and procurement:

There were two modes of funding, the SOE based reimbursement mechanism and Imprest Accounts for

- 17 -

capacity building and monitoring activities.

Donors provided finances retroactively through reimbursement of expenditures on activities that complied with approved operational plans (there were hundreds of thousand transactions annually). Funding mechanisms were designed to provide additional resources and ease such disbursements, thereby avoiding problems of lack of counterpart funds encountered under project funding.

Financial management of the program was weak due to a lack of proper understanding, non-compliance and delays in the implementation of the program arrangements for financial management, and the organization of the financial management system around weak internal control mechanisms. The design envisioned the use of adequately skilled government staff and was based on Governments’ mainstream systems for accounting, reporting and auditing. However, staff were either appointed late (and then subjected to frequent transfers) or not appointed at all and often lacked an appropriate skill set. A lack of proper accounting staff at the Provincial levels resulted in the absence of any analysis of financial management issues, or the development and implementation of corrective actions. Consequently, the requirements of the program such as preparation of annual financial statements, and SOEs were not properly disseminated.

The set of sub-programs that would qualify for reimbursement were agreed each year, and the understanding reached could not be fully communicated to lower levels on a timely basis. Rules, procedures and instructions for using Imprest Accounts had not been prepared in advance and therefore these accounts become functional only in the latter part of the project.

Additional specific points relating to financial management include the following:

Without the core issue of improved and timely recording of expenditures having been addressed, procedural mechanisms for claiming reimbursements through SOEs complicated by a requirement that only the Accountant General’s Office verify expenditures and expectation that departmental staff, with limited exposure to financial systems, decompose them into salary and non-salary components.

The main problems with SOEs related to: (a) the requirement that line departments reconcile SOEs with the records of the Accountant General’s office; (b) frequent revisions in their formats; and (c) difficulties in obtaining monthly expenditure statements from hundreds of District Account Officers (DAOs). This resulted in staff responsible for project implementation and technical managers spending a disproportionate amount of time preparing SOEs, ensuring timely reconciliations and settling audit queries. Financial management and capacity building through the recruitment of staff was severely delayed and took place only toward the end of the program.

An unfortunate outcome of SAPP-II was that it became a victim of the audit procedures built into the expenditure reimbursement mechanism. Part of the problem was that the audits were conducted in a traditional manner. Staff were bound by prescribed classifications for the categorization of audit observations. Resultantly, procedural lapses and misclassifications of expenditures were, at times, categorized as irregularities and misappropriations. These audit report issues made administrators extremely cautious in incurring expenditures, especially on new initiatives and on non-salary components.

The audit report submitted by the Auditor General (AG) of Pakistan for FY 1997/98 and FY 1998/99 identified a very large number of audit observations pointing towards major irregularities and misappropriations. In the view of Government, these reports were hastily submitted to donors without conducting proper Departmental Account Committee (DAC) meetings. The audit procedures are that draft

- 18 -

audit reports are discussed with line departments, and finalized on the basis of the review. Draft reports were sent to donors. Due to the findings of these reports, and as a result of private complaints received by the Bank, the Bank’s Department of Institutional Integrity conducted an Integrity Assurance Mission to Pakistan during April/May 2001. There was an apparent disconnect between the AG's office estimation of ineligible expenditures, and the Mission's estimation. The Mission placed these observations well above the 1% estimated by the Government, the Mission’s findings and recommendations were communicated to the Government. The Government responded to the findings at two levels. First, it started system-wide governance and financial management reforms, and second, has taken the necessary legal, disciplinary and other actions to address specific identified cases. The Government has indicated that the quality of the audit reports was affected by a lack of required audit follow-up with Departmental Account Committees (DACs), and inability of the audit staff to differentiate between a departure from rules and procedures and proven fraud cases.

SAPP was the first Bank project in Pakistan which financed a percentage of all government program expenditures in the basic social services. Therefore, the issues which emerged during implementation had wide raging impact, and actions have been initiated at various levels by both Bank and GOP to address the systemic financial management issues which emerged during SAPP-II implementation.

Actions by the Bank:From 1999 onwards, the Bank included a full time financial management specialist in the SAPP team, and a separate mission on financial management issues was conducted in September 2000 which presented specific recommendations to line departments and GoP for addressing systemic issues as well as those related to expenditure reporting and verification.

The Bank also twice withheld disbursements (in 2001 and 2002) pending satisfactory and credible resolution of financial management and audit issues. Disbursements were subsequently authorized based on submission of verified Statements of Expenditure by Provincial Accountant Generals, submission of SOE audit opinions and unqualified audit reports. Disbursements were made based on expenditures reported by line departments and the Accountant General, and where there was a discrepancy between the two figures, the Bank used the lower expenditure number for disbursements. To ensure that disbursements were not financing ineligible expenditures, the Bank initiated recoveries by deducting the identified ineligible expenditures from the overall expenditures submitted for reimbursements, and recoveries were made for the full program period.

As weak public financial management is endemic throughout the public sector in Pakistan. The Bank supported GOP’s reforms under the Project to Improve Financial Reporting and Auditing (PIFRA) to strengthen accounting and audit functions, and build public sector capacity. In addition, a Country Financial Accountability Assessment (CFAA) was conducted, and two provincial PFAAs (Provincial Financial Accountability Assessments) are underway. The Bank has also negotiated major financial management reforms as a part of the two Provincial Structural Adjustment Credits (SACs) in Sindh and NWFP, and two SACs for the GOP.

Actions by Borrower:The Ministry of Finance initiated an intensive effort with the AG’s office and all concerned SAP agencies to conduct the pending DAC meetings to discuss the audit reports and take remedial actions. The Accountant General’s office was specifically asked to verify all expenditures submitted through SOEs, with Auditor General confirming the authenticity of expenditures through submission of SOE audit opinion. The Auditor General established two task forces for reviewing audit observations pertaining to SAP and other foreign-aided-projects. Detailed reviews were conducted on each audit observation along with

- 19 -

supporting documentation. The subsequent annual audit reports contained less significant audit observations pertaining to misappropriation and embezzlement. The findings of the audit reports were also brought to the attention of the external accountability agency, the National Accountability Bureau (NAB), which was established by the military government soon after it assumed power in October 1999. Army monitoring teams were established to specifically take action again misuse of funds and authority. NAB conducted detailed investigations based on the information provided by the AG. Specific examples of action taken include initiation of disciplinary cases against more than 4000 teachers, launching investigations against at least fifty high profile cases, and conviction of three ministers who were disqualified from holding public offices due to cases of illegal teacher appointments and misappropriation of resources under a donor-assisted program.

More recently, a National Anti-Corruption Strategy has been developed with the assistance of the Department for International Development (DfID). This strategy has been approved by the President and Prime Minister. While these actions have been taken, significant work is still needed to build implementing line departments’ capacities, and to institute strong internal controls at all levels of government, and especially in the district governments which are now directly responsible for social service delivery.

Some systemic financial management improvements which have come about after the SAPP-II experience include: (i) regular reconciliation, consolidation and timely reporting of expenditure data, to expedite appropriate financial reconciliations and setting of audit queries; (ii) timely follow up on audit observations; (iii) setting up of internal audit units in each of the departments, as recommended by the AG, as well as the Bank; (iv) appointment of properly skilled accounting staff in the line departments; and (v) development of an FM manual to help provide important guidelines and procedures, including rules, procedures and instructions for using Imprest Accounts.

Procurement:Inadequate procurement capacity of implementing agencies to diligently follow IDA’s procurement procedures affected the overall quality and timeliness of procurement under the project. The Bank conducted numerous procurement training workshops in which many project staff participated. Nonetheless, the dispersed nature of the project and the large number of implementing agencies and staff made wider interactions difficult. Additionally, the numbers involved several hundreds of procurements. Capacity building through engagement of procurement consultants in each province was conceived in the Project Appraisal Document (PAD), but did not materialize due to administrative delays and bottlenecks. Most procurement, including all civil works, involved small contracts over widely dispersed geographical areas, and was below the Bank's prior review threshold. Large procurement operations were confined to pharmaceuticals and contraceptives which were procured through UN agencies, and large consultant contracts were to be based on the Bank's policy for selection of consultants.

In the procurement of civil works, in particular, generic deviations were observed between Government’s procurement procedures and practices, as followed by the implementing agencies, and those of IDA. These included the Government’s procedure to restrict bidding only to contractors pre-registered with government departments, and the use of a composite schedule of rates instead of item rates for inviting bids and awarding contracts. While improvements for National Competitive Bidding (NCB) procedures were specified in the PAD, these were not explicitly stated in the Development Credit Agreement (DCA) as, at the time of SAPP-II appraisal, it was not the practice to make such references in IDA legal documents. These improvements were legally incorporated at a later stage in the project, during Project restructuring in 2001. In more recent IDA-financed operations, procurement capacity strengthening measures, including the hiring of procurement consultants where required, are being explicitly agreed upfront during project

- 20 -

preparation. The required NCB improvements are now being clearly specified in the legal agreements to eliminate any ambiguities, and to facilitate remedial actions by the IDA in event of serious deviations.

5. Major Factors Affecting Implementation and Outcome

5.1 Factors outside the control of government or implementing agency:A major constraint to the achievement of SAPP-II objectives, particularly with respect to governance and program management issues was that the environment and the overall political and macro economic situation that existed at the program design stage changed continuously during implementation. This period was characterized by political instability, economic crisis, a severe drought, austerity measures stemming from a resource squeeze and accountability drives launched in 1999/2000. The project became effective shortly after Pakistan’s nuclear test in May 1998, which resulted in sanctions against the country. Provinces were faced with a severe financial crunch owing to a shortfall in revenue receipts (70% of the difference between actual and projected revenues due to a decline in the tax ratio, and 30% due to a lower than expected GDP) and lower than expected fiscal transfers from the divisible pool. Under the 1997 NFC Award, resource flows to the provinces during the SAPP-II period were Rs.192 billion less than what they would have been had the 1990 NFC formula been retained. The provincial governments also encountered difficulties in financing expenditures upfront due to the GoP's fiscal deficit targets.

In addition, the proportion of expenditures eligible for reimbursement were agreed annually, thus creating problems for provincial finance departments to understand the financing arrangements. They were unable to factor in estimates of potential inflows on the recurrent side of the budget from donor reimbursements as the ratios were agreed and finalized much later in the year. However, provincial line departments admitted that the project had served as a useful instrument for protecting expenditures on the social sectors, even in such adverse circumstances, and that without donor support allocations for SAP sectors may have been significantly lower.

There was a funding hiatus between the last disbursement under SAPP-I and the first disbursement under SAPP-II. Credit flows from social sector projects were tapering off and the volume of retroactive financing that was available as a replacement was inadequate. This gap, and accompanying uncertainty, demotivated the finance departments, especially since they were hard pressed to provide upfront funding for on-going activities, and meet the fiscal deficit targets set by GOP. One of the reasons for a lack of provincial interest in SAPP-II was provinces' belief that donor funding was inadequate to address the real resource requirements. The donor reimbursement ratios, ranging between 6% to 8% were viewed as modest, especially in view of the conditionalities that were attached to the funding. However, while the level of external financing was perceived to have fallen short of expectation, provincial governments never spent what they had committed to, particularly because they were in a fiscal bind and were unable to finance expected levels of social sector expenditures.

Progress in achieving the desired outcomes in the social sectors was adversely affected by socio-cultural factors whose influence cannot be reversed, but rather requires continuous and sustainable interventions over a long period of time. For instance, although awareness of contraceptives among married women in the age group 15 to 49 increased sharply from 38% in 1991-92 to 96% in 2001-02, there was a lower proportional increase in actual use of such methods and instruments, from 24% to only 28% (Pakistan Reproductive Health and Family Planning Survey). The gap between knowledge and practice/use continues to be wide because of traditional socio-cultural resistance to family planning, and inadequate number of outlets and female workers, and lack of female education. However, the utilization of family planning services improved in urban centres because of the demand created through social marketing efforts.

- 21 -

5.2 Factors generally subject to government control:Government ownership of SAPP-II and its commitment to the objectives was built slowly, but remained uncertain and generally weak throughout the duration of the project. In addition, formal donor supervision, oversight and coordination arrangements contributed to the development of a perception that SAPP-II was donor-driven, which further undermined efforts directed at creating domestic ownership of the program. Resultantly, SAPP-II was often perceived as being donor driven with uneven awareness of its objectives and proposed prescriptions at the provincial and lower tiers of government.

Whereas the importance of sensitizing the political leadership was apparent, there was a lack of a concerted effort to strengthen and expand this awareness. Although policy dialogue played an instrumental part in raising awareness of the need to attend to the low social indicators, it was largely confined to the bureaucracy. Similarly, awareness of the program at the field level was restricted to the provision of funding, with implementation staff having virtually no knowledge of the details of the program and its objectives. Nevertheless, ownership improved in the last two years of the project with the establishment of the Social Sector Committee for Cabinet under the stewardship of the Federal Minister for Education, with the participation of provincial social sector ministers. This resulted in social sector issues being brought to the forefront of policy discussions, although commitment to the reform agenda remained stronger at the federal level. The ESR was framed under this context of ownership of the reforms.

5.3 Factors generally subject to implementing agency control:The key factors that had a substantial adverse impact on implementation of SAPP-II and the functionalization of service outlets constructed during SAPP-I were not sector-specific, but persistent and common across sectors and geographical divisions of the country. Although the Government agreed that recruitment bans would not extend to SAP sectors, in practice these restrictions were applied indiscriminately, with the social sectors not exempt from their reach.

A lack of continuity of key staff negatively affected the level of commitment, the quality of leadership, institutional memory and understanding of the program, making it difficult to hold anyone accountable, thus disrupting reform efforts and compromising project implementation. The inability to ensure meaningful community participation in service delivery, continuing governance issues, such as lack of credible and effective arrangements to check staff absenteeism, and ineffective supervision and monitoring systems impeded sustainable improvements in service delivery.

5.4 Costs and financing:The project appraisal document estimated the five-year cost of the program at over US$10 billion. Out of this US$8 billion was estimated as Government financing and US$610 million (including ADB) was committed through SAPP-II donor financing.