Implementation Completion and Results Report (ICR)...

Document of

The World Bank

FOR OFFICIAL USE ONLY

Report No: ICR00004541

IMPLEMENTATION COMPLETION AND RESULTS REPORT

TF Number: TF-13825

ON A

LOAN/CREDIT/GRANT: P130878

IN THE AMOUNT OF SDR 13.7 MILLION

(US$19 MILLION EQUIVALENT)

TO THE

REPUBLIC OF MALAWI

FOR THE

FINANCIAL REPORTING AND OVERSIGHT IMPROVEMENT PROJECT

November 30, 2018

Governance Global Practice

Africa Region

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

Pub

lic D

iscl

osur

e A

utho

rized

CURRENCY EQUIVALENTS

(Exchange Rate Effective November 14, 2018)

Currency Unit = Malawian Kwacha (MWK)

MWK 727 = US$1

US$1.38 = SDR 1

FISCAL YEAR

July 1 – June 30

ABBREVIATIONS AND ACRONYMS

AGD Accountant General Department

BPR Business Process Reengineering

CABS Common Approach to Budget Support

CIAU Central Internal Audit Unit

CISA Certified Information System Auditor

CoA Chart of Accounts

CPS Country Partnership Strategy

DFID U.K. Department for International Development

EPICOR Brand Name of the Software used for Malawi IFMIS System

ERM Enterprise Risk Management

EU European Union

FIMTAP Financial Management Transparency and Accountability Project

FROIP Financial Reporting and Oversight Improvement Project

GDP Gross Domestic Product

GFEM Group on Financial and Economic Management

GIZ German Agency for International Cooperation Development (Deutsche Gesellschaft für Internationale Zusammenarbeit)

GoM Government of Malawi

HRMIS Human Resource Management Information System

ICR Implementation Completion and Results Report

ICT Information and Communication Technology

IFMIS Integrated Financial Management Information System

IFR Interim Financial Report

ISR Implementation Status and Results Report

ISSAI International Standards of Supreme Audit Institutions

IT Information Technology

LAN Local Area Network

M&E Monitoring and Evaluation

MDAs Ministries, Departments, and Agencies

MDTF Multi-Donor Trust Fund

MGDS Malawi Growth and Development Strategy

MoF Ministry of Finance

NAO National Audit Office

PAD Project Appraisal Document

PDO Project Development Objective

PEFA Public Expenditure and Financial Accountability

PFEM Public Finance and Economic Management

PFEMRP Public Finance and Economic Management Reform Program

PMF Public Financial Management

SN Serenic Navigator

TA Technical Assistance

TTL Task Team Leader

Regional Vice President: Hafez M. H. Ghanem

Country Director: Bella Bird

Senior Global Practice Director: Deborah L. Wetzel

Practice Manager: Nicola J. Smithers

Task Team Leader(s): Saidu Dani Goje, Hugues Agossou

ICR Main Contributor: Kirk Schmidt, Dolele Sylla

TABLE OF CONTENTS

DATA SHEET .......................................................................................................................... 1

EXECUTIVE SUMMARY ........................................................................................................... 5

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ........................................................... 7

A. CONTEXT AT APPRAISAL .........................................................................................................7

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 11

II. OUTCOME ....................................................................................................................... 12

A. RELEVANCE OF PDOs ............................................................................................................ 12

B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 13

C. EFFICIENCY ........................................................................................................................... 16

D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 17

E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 18

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME .................................. 19

A. KEY FACTORS DURING PREPARATION ................................................................................... 19

B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 20

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .... 22

A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 22

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 24

C. BANK PERFORMANCE ........................................................................................................... 24

D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 25

V. LESSONS AND RECOMMENDATIONS ................................................................................ 26

VI. COMMENTS ON ISSUES RAISED BY BENEFICIARY/DEVELOPMENT PARTNERS ................... 27

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS .......................................................... 29

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 36

ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 38

ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 39

ANNEX 5. BORROWER, CO-FINANCIER, AND OTHER PARTNER/STAKEHOLDER COMMENTS .. 40

ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 62

The World Bank Financial Reporting and Oversight Improvement Project (P130878)

Page 1 of 62

DATA SHEET

BASIC INFORMATION

Product Information

Project ID Project Name

P130878 Financial Reporting and Oversight Improvement Project

Country Financing Instrument

Malawi Investment Project Financing

Original EA Category Revised EA Category

Not Required (C) Not Required (C)

Organizations

Borrower Implementing Agency

Republic of Malawi Ministry of Finance

Project Development Objective (PDO) Original PDO

The project development objective (PDO) is to improve the internal controls, accounting, reporting and oversight of government finances at the central and decentralized levels in Ministries, Departments and Agencies (MDAs) in Malawi.


Page 2 of 62

FINANCING

Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$)

World Bank Financing TF-13825

19,000,000 19,000,000 10,900,341

Total 19,000,000 19,000,000 10,900,341

Non-World Bank Financing

Borrower 0 0 0

Total 0 0 0

Total Project Cost 19,000,000 19,000,000 10,900,341

KEY DATES

Approval Effectiveness MTR Review Original Closing Actual Closing

07-Mar-2013 28-Mar-2013 31-Aug-2015 30-Jun-2016 30-Apr-2018

RESTRUCTURING AND/OR ADDITIONAL FINANCING

Date(s) Amount Disbursed (US$,

millions) Key Revisions

17-May-2016 8.94 Change in Results Framework Change in Loan Closing Date(s)

KEY RATINGS

Outcome Bank Performance M&E Quality

RATINGS OF PROJECT PERFORMANCE IN ISRs

No. Date ISR Archived DO Rating IP Rating Actual

Disbursements (US$, millions)

01 05-Aug-2013 Satisfactory Moderately Satisfactory 2.00

02 17-May-2014 Moderately Satisfactory Moderately Satisfactory 3.45


Page 3 of 62

03 10-Oct-2014 Moderately Satisfactory Moderately Satisfactory 6.04

04 21-May-2015 Moderately

Unsatisfactory Moderately Unsatisfactory 7.49

05 31-Dec-2015 Unsatisfactory Unsatisfactory 8.94

06 15-Apr-2016 Moderately Satisfactory Moderately Satisfactory 8.94

07 02-Dec-2016 Moderately Satisfactory Moderately Satisfactory 9.18

08 11-Jul-2017 Moderately Satisfactory Moderately Satisfactory 9.53

09 04-Apr-2018 Moderately

Unsatisfactory Moderately Unsatisfactory 10.16

SECTORS AND THEMES

Sectors

Major Sector/Sector (%)

Public Administration 100

Central Government (Central Agencies) 88

Sub-National Government 12

Themes

Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 101

Public Finance Management 31

Public Expenditure Management 31

Public Administration 70

Administrative and Civil Service Reform 10

Transparency, Accountability and Good Governance

31

E-Government, incl. e-services 20

Municipal Institution Building 9

ADM STAFF

Role At Approval At ICR

Regional Vice President: Makhtar Diop Hafez M. H. Ghanem


Page 4 of 62

Country Director: Kundhavi Kadiresan Bella Bird

Senior Global Practice Director: — Deborah L. Wetzel

Practice Manager: Patricia McKenzie Nicola J. Smithers

Task Team Leader(s): Pazhayannur K. Subramanian Saidu Dani Goje, Hugues Agossou

ICR Contributing Author: — Kirk Schmidt, Dolele Sylla


Page 5 of 62

EXECUTIVE SUMMARY

1. The Financial Reporting and Oversight Improvement Project (FROIP) was a US$19 million investment financing project which was approved on March 7, 2013, and closed on April 30, 2018, after a 22-month extension from the original closing date of June 30, 2016. Project implementation was affected by the 2013 ‘cashgate’ financial scandal which shifted the project focus toward the procurement and implementation of a new Integrated Financial Management Information System (IFMIS). The ‘cashgate’ also necessitated the project extension. However, the restructuring that followed due to ‘cashgate’ did not include a change in the Project Development Objectives (PDOs) as improving financial controls was already a component of FROIP. The project was funded through a multidonor trust fund (MDTF) which was jointly funded by the U.K. Department for International Development (DFID), European Union (EU) Commission, Ireland (IrishAid), Norway, and German Agency for International Cooperation Development (Deutsche Gesellschaft fur International Zusammenarbeit, GIZ). The World Bank administered the MDTF but was not a contributor.

2. ‘Cashgate’ had an impact on FROIP but it is not responsible for not fully achieving the development objectives. The Malawi ‘cashgate’ in which about US$32 million in government funds was misappropriated between April and September 2013, threw into stark relief the fundamental and deep-rooted governance problems facing the Malawian Government.1 The ‘cashgate’ scandal revealed the immediate need to address the long-lasting public financial management (PFM) problems, such as delays or the inability of the Government to (a) prepare bank reconciliation statements; (b) prepare and publish quarterly in-year financial statements according to the PFM Act; (c) prepare and publish audited annual financial statements in accordance with the PFM Act; (d) communicate budget ceilings to ministries, departments, and agencies (MDAs) and enforce commitments control; (e) prevent misapplication of funds due to nonadherence with rules regarding budget virements; (f) ensure adequate segregation of duties between end users and system administrators of the IFMIS; and (g) enforce managerial accountability to ensure compliance with rules and regulations. ‘Cashgate’ facilitated a shift in resources between components to allow for more funding to go toward the procurement of a new IFMIS; however, it was the way resources were reallocated that undermined the achievement of the development objectives. First, the decision to move forward with the procurement of a new system was slow to develop and, second, once decided, the changes in the reallocation of resources or expectations of achievement was never clearly communicated to implementing units.

3. The PDO was to improve the internal controls, accounting, reporting, and oversight of the recipient’s finances at the central and decentralized levels in its ministries, departments, and agencies (MDAs). Project ratings are summarized as follows:

Relevance Efficacy Efficiency Overall

Outcome Bank

Performance Monitoring and Evaluation

(M&E) Quality

Substantial Modest Modest Moderately Unsatisfactory

Unsatisfactory Negligible

4. FROIP’s development objective was highly relevant for Malawi. From project preparation to completion, the project objectives aligned well with the World Bank’s and counterpart’s development

1 Bridges, Kate and Michael Woolcock. 2017. How (Not) to Fix Problems That Matter: Assessing and Responding to Malawi’s History of Institutional Reform.


Page 6 of 62

priorities. The PDO was aligned with Theme 3: Mainstreaming Governance for Enhanced Development Effectiveness, of the World Banks’s Country Partnership Strategy (CPS) for 2013–2016, which remains the most current CPS in Malawi. The project objectives directly supported the Government’s Public Finance and Economic Management Reform Program (PFEMRP) three-year plan by targeting its components and subcomponents.

5. FROIP did not fully achieve its objectives. FROIP failed to make substantive progress in strengthening internal controls mostly due to the shift in focus to procure a new IFMIS and the subsequent failure to procure the new system. It was agreed that the Government would proceed with the procurement of the new IFMIS with its own resources. The project did achieve the objective of improved oversight of MDAs and partially achieved improvements to accounting and reporting by MDAs. The most successful activities under FROIP involved the support to the internal and external audit units.

6. The efficiency of FROIP was Modest. Gains were made in several of the components, but the project failed to achieve, in a sustainable manner, major improvements to the overall PFM environment in Malawi. The ‘cashgate’ scandal had an impact on FROIP; however, given that it happened early in the project implementation phase and a 22-month extension was granted, FROIP could have been much more efficient in its implementation. Aside from the unsuccessful IFMIS procurement, implementing other activities such as strengthening the establishment of an interface between district and central government systems or fully implementing the continuous automated audit of the IFMIS would have served to make the gains in the PFM environment more sustainable. FROIP did make substantial improvements to information technology (IT) equipment, resulting in improved connectivity at the district level and faster processing speed of the IFMIS and Human Resource Information Management System (HRMIS).

7. M&E quality is rated Negligible. There were severe shortcomings in both the M&E implementation and utilization. During the project restructuring in 2016, the opportunity to introduce qualitative-based equivalent of the Public Expenditure and Financial Accountability (PEFA) indicators was not taken, given the fact that no PEFA assessment had been conducted since 2011 nor was one scheduled. Instead, during the restructuring, the project introduced additional PEFA indicators as immediate results indicators. The lack of the qualitative or quantitative PEFA equivalent had significant impact in hindering the ability to monitor and assess progress toward the final outcomes. Additionally, the Public Finance and Economic Management (PFEM) unit planned to hire an M&E specialist but was unable to identify a suitable candidate, leaving the responsibility of M&E with the unit. Despite a series of inconsistent ratings within the Implementation Status and Results Reports (ISRs) and multiple efforts by the World Bank to support the PFEM unit in identifying a suitable consultant to conduct the M&E, the PFEM unit was unable to recruit an M&E consultant.

8. This Implementation Completion and Results Report (ICR) highlights five recommendations. First, complex information and communication technology (ICT) procurement activities, such as a new IFMIS, require adequate support to be effective. Second, teams should avoid using composite indicators like PEFA for M&E purposes and should introduce intermediate indicators at the beginning of the project. Third, changes in the direction or emphasis of a project need to be clearly communicated to all stakeholders after any significant events that affect the project. Fourth, in projects with multiple activities that need proper sequencing and include a capacity-building component, implementing units should be encouraged to hire an expert as part of technical assistance (TA) to support implementation. Fifth, M&E must be the primary focus of any project and its application should be continually reviewed.


Page 7 of 62

I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES

A. CONTEXT AT APPRAISAL

Context

1. At the time of the preparation of FROIP in 2013, the gross domestic product (GDP) growth of Malawi had entered a period of a slowdown. From 2004 to 2010, Malawi experienced solid growth through prudent macroeconomic policies and a supportive donor environment. This performance was built on strong stabilization policies and debt relief from the Heavily Indebted Poor Countries initiative which helped improve public expenditure management and created the fiscal space needed to generate the momentum for growth. By early 2012, it was estimated that the 2011 real GDP had slowed to about 1.4 percent.

2. After the election of Joyce Banda as President in 2012, the Government took decisive policy measures to halt the slowdown in the economy and began implementing a comprehensive package of economic reforms. These reforms aimed to address the current external imbalances with plans to cushion the vulnerable poor against the impact through social protection programs and facilitate a growth rebound in the short term. In this regard, some decisive and credible measures were taken by the authorities to strengthen economic governance while at the same time signaling to development partners and the private sector the Government’s commitment to create a favorable environment for a return to strong GDP growth.

3. The Second Malawi Growth and Development Strategy (MGDS II) 2011–2016, which was the country’s second medium-term plan, was approved by the Cabinet in April 2012. The MGDS II was a medium-term strategy designed to attain Malawi’s long-term aspirations as spelled out in its Vision 2020 and strived to foster a more inclusive, job-creating growth to address the unemployment problem as well as reduce poverty. The strategy reflected a general consensus on the country's broad goals for growth, social equity, and governance.

4. The Government of Malawi (GoM) has been reforming its PFM systems since 2002. While this yielded significant improvement in the legal framework, IT systems, and budget procedures, the full benefits of these reforms were not yet being felt in terms of aggregate fiscal discipline, strategic allocation of resources, and effective service delivery. The authorities and development partners recognized the need to move to a new phase of reforms focused on greater implementation of the new rules and regulations, tighter internal controls, greater attention to the benefits of PFM reforms for MDAs, and sectors and capacity development in the various PFM institutions. The management of the PFM system is mainly concentrated within the Ministry of Finance (MoF). Within the ministry, the secretary to the Treasury has a key responsibility for budget planning and execution, while the accountant general has responsibility for producing timely and appropriate management of financial accounts. Within MDAs, it is the controlling officers who have overall responsibility for effective PFM.

5. The PFM reforms were being supported previously by donors through the Common Approach to Budget Support (CABS) Group which had been providing assistance to the GoM since 1997. With support from the CABS Group suspended, many development partners other than the CABS Group were also providing support to the reforms and many more were expected to join in these efforts. The reform efforts were coordinated between the GoM and donors through the Group on Financial and Economic


Page 8 of 62

Management (GFEM) meetings that were jointly chaired by the secretary to the Treasury and a representative from the donors. Through the Financial Management Transparency and Accountability Project (FIMTAP) financed by the World Bank and the EU, capacity development assistance was provided to internal audit, external audit, and the Office of the Director of Public Procurement. The project also supported the acquisition, installation, and operationalization of the IFMIS, the Government Wide Area Network, and general governance improvement among others. Through the Malawi Social Action Fund (known as ‘MASAF’), the World Bank helped in building capacity including financial management capacity at the district level. There were also significant contributions to the PFEM reform from DFID, Norway, GIZ, United Nations Development Programme, and the EU. The EU Capacity-Building Project for Economic Management and Policy Coordination was instrumental in developing the PFEMRP and strengthening capacity in both the MoF and the Ministry of Economic Development and Planning.

6. Though progress was registered in several areas, it was not comprehensive. Progress was made on many of the activities in the PFEM Action Plan during 2006–2011 but many activities were well behind initial target dates and insufficient attention and resources had been given to identifying and resolving bottlenecks. Several reforms and improvements had been introduced but were uncoordinated and suffered as implementation lagged. A more comprehensive approach was established to provide strategic direction through a sector-wide approach to PFEM reforms. Additionally, four PEFA assessments were conducted in Malawi in 2005, 2006, 2008, and 2011. The 2011 PEFA assessment was based on an analysis of performance for the years from 2007–2008 to 2009–2010. The main findings were related to budget execution, accounting and financial reporting, and internal and external oversight.

7. The Government introduced a PFEMRP which is supported using a common basket funding mechanism through the PFEMRP MDTF administered by the World Bank. The PFEMRP was aimed at improving the GoM’s macro-fiscal management, accountability and transparency in PFM, and public oversight. Interventions and support happening outside the MDTF were tracked using a PFM support matrix maintained by the PFEM unit and updated from time to time through discussions during the GFEM meetings. The PFEMRP covered 10 reform areas (planning and policy analysis, resource mobilization, budgeting, procurement, accounting and financial management including internal audit and cash and debt management, parastatal financing, monitoring and reporting, external auditing, and program management). FROIP covered three main areas out of the PFEMRP: accounting and financial reporting, internal and external audit, and program management. These had been decided as the immediate focus areas by the Government and the MDTF partners based on priority, resources availability, and component readiness.

Theory of Change (Results Chain)

8. The project had a Results Framework that could be connected to the key elements of the PDO. The Project Appraisal Document (PAD) did not describe in detail the results chain or the logic behind the operation. This was not required by the World Bank procedures at the time of project preparation. The links between the activities, results indicators, and development outcomes were not always an easy connection to follow. With the number of activities and subcomponents listed in the PAD, additional non-PEFA intermediate indicators could have made the results chain easier to follow.

9. The PDO is broken down into four subobjectives to establish connection with indicators and activities. Figure 1 links subobjectives with PDO indicators, intermediate results indicators, and key activities under different subcomponents.


Page 9 of 62

Figure 1. Theory of Change (Project PDO, Indicators, and Activities)


Page 10 of 62

Project Development Objectives (PDOs)

10. The objective of the project was to improve the internal controls, accounting, reporting and oversight of the Government’s finances at the central and decentralized levels in its ministries, departments and agencies (MDAs) in Malawi.

Key Expected Outcomes and Outcome Indicators

11. The PDO is measured by the following key outcome indicators:

(a) IFMIS and payroll management:

• PI-23.4 (formerly PI-18) Improved effectiveness of payroll controls

• PI-25 (formerly PI-20) Improved effectiveness of internal controls on non-salary expenditures

(b) Internal audit:

• PI-26 (formerly PI-21) Improved effectiveness of internal audit

(c) External audit:

• PI-30 (formerly PI-26) Scope nature and follow-up of external audit

• PI-30 (formerly PI-28) Legislative scrutiny of external audit reports

• PI-29 (formerly PI-25) Improved quality of annual financial statements

Components

12. The project had four components:

13. Component 1: Accounting and Financial Management (US$9.2 million) aimed to improve the systems and controls for accounting and financial management through (a) improved accounting and IFMIS efficiency and comprehensiveness in MDAs; (b) improved payroll management through business process reengineering (BPR), expansion of payroll system to regions, and increased user training; and (c) improved coverage of the IFMIS to cover all districts and establishing local area network (LAN), connectivity within districts.

14. Component 2: Internal Audit (US$2.8 million) was designed to support the Central Internal Audit Unit (CIAU) by improving the size and capacity of staff through trainings and improvements to the governance and legal framework.

15. Component 3: External Audit (US$4.7 million) aimed to improve the operation capacity of the National Audit Office (NAO) through (a) revising audit manuals and procedures; and (b) increasing independence and accountability by meeting the International Standards for Supreme Audit Institutions (ISSAI).


Page 11 of 62

16. Component 4: PFRMRP Management (US$1.6 million) was designed to manage FROIP by providing procurement and financial management support to the implementing departments and to monitor the objectives of the performance against the indicators. The unit will be staffed either through transfers of people from other units or the hiring of new staff.

B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE)

Revised PDOs and Outcome Targets

17. The PDO remained unchanged for the duration of the project.

Revised PDO Indicators

18. Multiple changes to the PDO indicators were made during the project restructuring in May 2016. This was the only time the project was restructured and revisions to the PDO indicators were made. The changes are listed in table 1. Each of the indicators changed due to the change in PEFA, intermediate results indicators were added, and one indicator was dropped.

Table 1. Revised PDO Indicators

Status Indicator Name Baseline End Target

Revised PEFA PI-23.4 (formerly PI-18) Effectiveness of payroll controls

B+ B+

Original PEFA PI-18 Improved effectiveness of payroll controls B+ A

Revised PEFA PI-25 (formerly PI-20) Internal controls on non-salary expenditure

C+ B

Original PEFA PI-20 Improved effectiveness of internal controls for non-salary expenditures

C+ B

Revised PEFA PI-26 (formerly PI-21) Effectives of Internal audit D+ C

Original PEFA PI-21 Improved effectiveness of internal audit D+ C

Marked for Deletion

PEFA PI-23 Improved information on resources received by service delivery units

D D+

Revised PEFA PI-29 (formerly PI-25) Accounting, recording and reporting

C+ B

Original PEFA PI-25 Improved quality and timeliness of annual financial statements

C+ B

Revised PEFA PI-30 (formerly 26) Characteristics of External Audit D+ C

Original PEFA PI-26 Scope nature and follow-up of external audit; and PI-28 Legislative scrutiny of external audit reports

D+ C

New Audit coverage and standards D+ B

New Payroll audits C B+

New Supreme Audit Institution Independence – Implement 4 out of 5 International Standards

None implemented

Four implemented

New Compliance with payment rules and procedures C B

New Submission of reports for external audit 48 months 3 months

Revised Components

19. There were no changes made to the design of the components during implementation.


Page 12 of 62

Other Changes

20. A restructuring was undertaken on May 17, 2016, to extend the closing date and intermediate results indicators. The original closing date was June 30, 2016, and the Government requested the extension of 22 months to April 30, 2018. The reasons provided for the extension request were (a) the remaining time was insufficient to implement the rest of the activities and also achieve the project objectives; (b) the project implementation was interrupted for more than one year by the ‘cashgate’ scandal, during which the entire Government, the Project Implementation Unit, and development partners were focused on the scandal, reducing the effective implementation time by 15 months; (c) one of the key components of the project, oversight responsibility of the NAO, stalled for more than a year due to the delay in appointing an auditor general, which was a disbursement condition; (d) the project was critical in supporting the GoM to improve its PFM systems and enable it to gain some credibility and the confidence of the development partners to resume budget support; and (e) the Joint Government Donor Committee approved the funding of the procurement and implementation of the new IFMIS software from FROIP, and the World Bank provided no-objection to the procurement. The procurement process was about to be completed, which was followed by implementation and contract management. There was the need to extend the closing date, so that the World Bank could continue providing the implementation support to ensure proper contract administration and implementation of the software. A second request for restructuring, including an extension of the closing date was made in 2018 but was not approved.

Rationale for Changes and their Implication on the Original Theory of Change

21. The changes had minimal impact on the original theory of change. The PDO remained unchanged, and the changes to the indicators allowed for the transition to the new PEFA and added intermediate results indicators that were not previously included. Because the intermediate indicators that were added were also the PEFA scores, it was difficult to use them in tracking success because no PEFA assessment was undertaken during the life of the project.2

II. OUTCOME

A. RELEVANCE OF PDOs

Assessment of Relevance of PDOs and Rating

Rating: Substantial

22. From project preparation to completion, the project’s objectives aligned well with the World Bank’s and counterpart’s development priorities. The PDO was aligned with Theme 3: Mainstreaming Governance for Enhanced Development Effectiveness, of the World Bank’s CPS for 2013–2016, which remains the most current CPS in Malawi. The project objectives directly supported the Government’s PFRMRP three-year plan by targeting components and/or subcomponents 5.1: GoM Accounting and

2 A PEFA assessment was conducted in 2018, after the ICR conducted their mission. So, the report was not an available resource to be used during the writing of this ICR. However, a draft of the report has been made available to the team. The results are consistent with the findings in the ICR and a summary of the PEFA ratings from the 2018 draft assessment are presented in annex 6.


Page 13 of 62

IFMIS; 5.2: Local Government Financial Management; 5.3: GoM Payroll Management; 9: External Auditing; and 10: PFEM Management.

23. The GoM and the World Bank are continuing to build on the objectives and priorities of FROIP under a new operation called ‘Malawi Governance to Enable Service Delivery’. Although the IFMIS-related parts of FROIP is excluded, the new project includes components for the development of capacity building and strengthening the system of PFM, both at the national and subnational levels. This project, with several overlapping objectives with FROIP, provides additional support to the relevance of the PDOs of FROIP.

24. The objectives were clear but the connection between the components, activities, and the PDO was not. The PAD did not align the activities with indicators nor did it align the indicators with objectives. Additionally, the intermediate results indicators were not introduced until the project restructuring in 2016, which was three years after the project began, and there was an overreliance on the PEFA scores as PDO indicators contributed to ambiguity in the results chain. A lack of specificity left the project open to different interpretations by the World Bank and Government agencies regarding the sequencing and link between activities, indicators, and objectives.

B. ACHIEVEMENT OF PDOs (EFFICACY)

Assessment of Achievement of Each Objective/Outcome

25. The PDO is broken down into four subobjectives, which are assessed individually based on the progress achieved in the respective PFM areas and indicators and targets in the Results Framework. Due to the absence of a new PEFA in Malawi during this project, the ICR team independently rated the achievement of the identified activities and outcomes within each PDO to determine the overall rating.

Table 2. Summary of PDO Ratings

PDOs Rating Overall Rating

PDO 1: Improved internal controls of the MDAs Modest

Modest PDO 2: Improved accounting at the MDAs Modest

PDO 3: Improved reporting of the MDAs Modest

PDO 4: Improved oversight of the MDAs Substantial

26. PDO 1. To improve internal controls of the MDA - partially achieved (rating: Modest). Despite having upgraded EPICOR or the new IFMIS as the main activity and single largest planned expenditure within FRIOP, the Government is still using the same EPICOR system that was already in need of replacement in 2013. A full advertisement, bidding, and selection of the most responsive bidder took place. However, the award could not happen under FROIP as the Government was unable to amicably resolve, satisfactory to the World Bank, complaints by one of the bidders. It was therefore agreed that the Government would proceed and procure the new IFMIS using Government-owned resources. The EPICOR system remains in place and despite hardware upgrades undertaken under FROIP to strengthen the control environment, the system has reached its end of life and needs replacement.

27. The interfaces between the IFMIS and other Government systems developed under FROIP are not being utilized. The physical interface between the HRMIS and the central IFMIS (EPICOR) and between the district-level IFMIS (Serenic Navigator [SN]) and EPICOR was completed. However, this interface is not


Page 14 of 62

fully operational due to some discrepancies that remain in the Chart of Accounts (CoA) used by the two systems. The interface between the Central Bank and the IFMIS is not fully operational; as a result, the automatic bank reconciliation cannot be executed through the system and consolidation of financial reports cannot yet be done automatically; it only functions in one direction from the Central Bank to the IFMIS, resulting in the IFMIS operators at the MoFEPD continuing to send data manually to the Central Bank.

28. The control environment around the current IFMIS was strengthened. SoftTech, the firm which implemented and supports the EPICOR system, dispatched two consultants on site to improve the EPICOR control environment in 2013. The consultants are still on the ground supporting EPICOR daily. Important security controls have been implemented including better access control methods—restricting user rights and eliminating ‘super users’ and the implementation of an audit trail system that captures all the changes made within the system.

29. The budget module and a fixed assets register intended to be implemented under FROIP were not completed. Both activities were put on hold to focus on the procurement of the new IFMIS. The delays in the IFMIS procurement resulted in neither activity being implemented despite funding being available.

30. The NAO and the auditor general undertook a full headcount and payroll audit for the public service in 2015 and 2016. The auditor general probed the HRMIS to identify all employees on the Government payroll and to assess whether data provided can be relied upon. This data was matched with the employees on site and related personnel records maintained in the MDAs. The results showed that the public service had 17,669 ghost workers.3

31. Payroll processing times and frequency of arrears have been reduced due to achievements under FROIP. County-level payroll hubs were implemented in six counties allowing payroll administrative functions to be decentralized. At each of the six counties, four computers were set up allowing staff from nearby counties to go to one of the hubs each month and enter payroll information. Teachers, making up almost half of the civil servants at the county level, had payroll delays averaging one to two weeks. Those delays have been eliminated since the system was put into place under FROIP; however, some of the equipment is beginning to fail raising sustainability concerns.

32. PDO 2. To improve accounting at the MDAs - partially achieved4 (rating: Modest). Audit knowledge coverage has improved across government ministries and systems. Actions such as embedding ACL continuous auditing software in the IFMIS and increasing the number of CIAU staff who have received additional training have already started. However, the rollout of the continuous auditing software of the IFMIS was only partially completed and the CIAU was unable to establish a training lab to convert the formal training knowledge into practice.

33. PDO 3. To improve reporting of the MDAs - partially achieved (rating: Modest). The timeliness of financial reports has improved. The time lag to produce financial reports was reduced to 3 months from approximately 48 months when the project began. However, the quality of the financial reports still needs

3 The information on the payroll audit has been provided through the 2018 draft PEFA assessment. 4 PDOs 2 and 3, although rated separately, have significant overlaps and are grouped together for the Efficacy discussion.


Page 15 of 62

improvement. Although the reports are produced more quickly, they are often still incomplete, including bank reconciliations.

34. The institutional setup for internal audit has been strengthened. Before FROIP, internal audit regulations were only identified in the Internal Audit Charter which was not legally enforceable. Through FROIP, the internal audit policy and bill were drafted and submitted to the Cabinet for approval. The policy and bill will have the effect of strengthening the effectiveness of internal auditing and improving reporting and coordination with other key stakeholders. As part of the initiative, the audit committees were revived, and their composition was changed from ministerial staff to nonpublic officers to enhance their credibility.

35. Risk-based internal audit approach was implemented throughout the Government. Risk-based internal audit approach has modernized the internal audit process in Malawi and it now conforms to international standards. This is allowing the CIAU to focus on where there is the greatest potential for impact rather than choosing audits in an ad hoc way.

36. Enterprise risk management (ERM) was discussed with MDAs but never put into place. Although the CIAU has laid the groundwork for moving forward with ERM by promoting its use and benefits for all of government, it has not yet been implemented. Guidelines for implementing ERM were not developed. This is a key initiative that was planned under FROIP that will allow ministries to develop a better understanding of potential risks and how to better avoid or address them if they arise.

37. The majority of internal audit recommendations are not being implemented. According to the 2018 draft PEFA assessment, management response to internal audit recommendations is weak. Reasons for not taking action include lack of funding to a lax attitude of MDAs toward the recommendations.

38. PDO 4. To improve oversight of the MDAs - partially achieved (rating Substantial). Due to the elimination of external audit report backlogs at the national level, Parliament is now able to make funding decisions based on what is currently happening, not what happened years ago. Before FROIP began, Parliament was looking at reports, and subsequently issues and problems, that were from four to five years prior. The reduction in the backlog of audit reports was a result of multiple efforts and programs, of which FROIP was one.

39. The oversight component of FROIP worked closely with and was supported by multiple programs and development partners. There were several ongoing programs targeting improved oversight of MDAs when FROIP was implemented and each of the programs, along with FROIP, complemented each other’s activities rather than competing or duplicating efforts. The World Bank had an ongoing Development Policy Operation to strengthen PFM and improve transparency. Other development partners, mainly Irish Aid and KfW, provided substantial support to bring in external auditors and assist the Government in reducing the report backlog.

40. Performance of the external and internal audit offices has improved but the resources required to sustain the improvements are lacking. Additional staffing, training, hardware, and software have improved staff ability and overall performance of internal and external audit offices but most of the trainings were onetime events. Ongoing need for training programs have not been funded by FROIP or by the GoM.


Page 16 of 62

Justification of Overall Efficacy Rating

Rating: Modest

41. FROIP substantially achieved the objective of improved oversight of MDAs and partially improved controls, accounting, and reporting of MDAs. The most successful activities under FROIP involved the support to the internal and external audit units. FROIP helped modernize both units by introducing new software, hardware, and techniques to allow the units to be more efficient in their time and approach. Additionally, FROIP supported the development of new policies to strengthen each unit and help establish the independence of the NAO. The biggest issue with the failure to make substantive progress in strengthening internal controls was the failed procurement of a new IFMIS. Despite the addition of updates to control the environment of the EPICOR system, it has reached its end of life and needs replacement. A second issue with improving the controls is that the CoA was not harmonized and remains an impediment to the different Government systems interfacing with each other.

C. EFFICIENCY

Assessment of Efficiency and Rating

Rating: Modest

42. The PAD did not provide a cost-benefit analysis of the investments that were going to be made under FROIP due to the difficulty in quantifying the value of PFM reforms. The nature of project activities does not lend itself to standard economic or financial analyses to determine value for money. As a result, the efficiency assessment of FROIP is based on a combination of qualitative assessment and a description of the project’s implementation efficiency such as time overruns, underspending, and operating costs. Based on this analysis and the limited availability of data, FROIP’s efficiency is rated Modest.

43. A summary of original approved cost and actual expenditure is presented in table 3. The project was extended by 22 months from the original three years to nearly five years; the extension did not include a change to the financing and was implemented to allow additional time for the procurement and implementation of a new IFMIS. When the project ended, only 55 percent of the total funding had been disbursed.

Table 3. Project Cost by Component

Components Allocation (US$) Actual (US$) Percentage of Allocation

Internal Audit 2,800,000 1,432,554 51

External Audit 4,700,000 1,376,234 29

PFEMRP Management 1,600,000 2,214,111 138

Accounting and Financial Management

9,200,000 5,877,441 64

Contingency 700,000 0 0

Total 19,000,000 10,900,340 57


Page 17 of 62

44. The percentage of spending against the original allocation, for each component, indicates poor management of resources. The PFEM unit was responsible for overseeing implementation of each component. The result was the PFEM unit overspending their allocation by US$200,000 while each of the other components underspent their allocation between 38 percent and 70 percent. Within each of the components, there are activities that were either partially or not completed that would have improved outcomes under FROIP. Implementation and utilization of FROIP funds, throughout the life of the project, were not efficiently managed.

45. During the project restructuring in May 2016, the project failed to adjust the component allocations or indicators despite the knowledge that the cost of a new IFMIS would reduce the amount of funding available for other components and activities. The PFEM unit has stated it was necessary to reduce the allocation for each of the components to support the higher anticipated cost of the new IFMIS under the Accounting and Financial Management component. The amount of reduction under each component or which activities should be eliminated was not clearly communicated to implementing units. The result was that the implementing agencies were unsure how much funding they would receive or which activities within their work plan would be implemented.

46. When comparing the activities implemented within FROIP with the planned activities and intended results of these activities, the ICR found that many of the gains made due to FROIP are at risk of not being sustained. The lack of clear adjustment to activities and expected outcomes from the implementation of a new IFMIS allowed units to move forward with hardware and software purchases but reduced the amount of training staff received on how to effectively utilize the new tools. Other reductions left equipment without a maintenance agreement or disaster recovery system in place, both of which were originally intended under FROIP.

47. Gains were made in several of the components, but the project failed to achieve, in a sustainable manner, major improvements to the overall PFM environment in Malawi. The ‘cashgate’ scandal had an impact upon FROIP; however, given that it happened early in project implementation and a 22-month extension was granted, FROIP could have been much more efficient in its implementation. Aside from the unsuccessful IFMIS procurement, implementing other activities such as strengthening the establishment of an interface between district and central government systems, or fully implementing the continuous automated audit of the IFMIS would have served to make the gains in the PFM environment more sustainable. FROIP did make substantial improvements to the IT equipment, resulting in improved connectivity at the district level and faster processing speed of the IFMIS and HRMIS.

D. JUSTIFICATION OF OVERALL OUTCOME RATING

Rating: Moderately Unsatisfactory

48. The ICR rates the overall outcome of FROIP as Moderately Unsatisfactory. The reforms and improvements to the control environment that were targeted under FROIP were relevant and aligned with both the Government and World Bank strategy for the duration of the project. The internal audit and external audit components were substantially successful, and within the accounting and reporting components, there were some successes in the areas of improving the IFMIS (EPICOR) environment and infrastructure, the rollout of IFMIS to remaining districts, the audit coverage of payroll, the rollout of payroll system to districts including hardware and connectivity support, the building of the HRMIS interface with IFMIS, and so on. However, most of the components’ objectives were only partially


Page 18 of 62

completed which raises concerns over sustainability of all gains made under the project. While ‘cashgate’ and the subsequent planned procurement of a new IFMIS absorbed a considerable amount of time and resources that was above the initial plan, the biggest contributor to the underwhelming performance of FROIP was the decision not to formally realign resources and reduce the activities and expectations to match the reprioritization for the new IFMIS under FROIP. The impact of not doing this, compounded by failed procurement of the new IFMIS, resulted in a project that left about 45 percent of funds undisbursed and without the new IFMIS in place or the full achievement of the stated development objectives.

E. OTHER OUTCOMES AND IMPACTS (IF ANY)

Gender

49. FROIP did not have a gender component.

Institutional Strengthening

50. FROIP was critical in supporting the GoM to strengthen its PFM systems and enable it to regain the confidence of development partners to resume budget support. Reducing the time lag for the submission of reports for external audit, improving the independence of the NAO, and increasing support to the Parliament, specifically the Parliamentary Accounts Committee, has improved credibility of systems within the Government and the development partners.

Mobilizing Private Sector Financing

51. FROIP did not mobilize private sector financing.

Poverty Reduction and Shared Prosperity

52. FROIP did not explicitly target poverty reduction. However, it can be expected that increased capacity of the Government to improve expenditure reporting, payroll management, audits, MDAs, and improving performance of key functions of the Government oversight institutions, indirectly contributes to poverty reduction and shared prosperity. Also, the improvement in the PFM system will enable better resource management for service delivery and access to services by citizens, which contributes to poverty reduction and shared prosperity.

Other Unintended Outcomes and Impact

53. Support to the NAO had the unintended outcome of reducing the backlog of audit reports at the local government level from five years to two years. The activities were designed to reduce the backlog of audit reports at the national level only, but the additional capacity provided through TA by Irish Aid, KfW, and through FROIP, freed up staff time to also support reducing the local government backlog. This has allowed Parliament to better appreciate and make decisions based on more current realities for local governments.


Page 19 of 62

III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME

A. KEY FACTORS DURING PREPARATION

54. Alignment with Government’s priorities. FROIP was prepared to support the GoM’s efforts to sustain and advance the PFM systems reforms that started 10 years back. While the FIMTAP, funded by the World Bank and the EU, yielded significant improvements in the legal framework, IT systems, and budget processes, the full benefits of the reforms were yet to materialize on the aggregate fiscal discipline, allocative efficiency of resources, and service delivery. The new phase of the reforms focused on strengthening the implementation of the new rules and regulations, internal controls, and greater attention to the benefits of PFM for MDAs.

55. Harmonization of donor support. The PFM reforms in Malawi were supported previously by donors through the CABS Group since 1997. After the CABS suspension until 2005, there were multiple initiatives of development partners which made it difficult to coordinate donor support but also added an undue burden on the Government. At the time of preparation, the reform efforts were coordinated between the GoM and donors through GFEM meetings that are jointly chaired by the secretary to the Treasury and a representative from the donors. The mixed results achieved between 2006 and 2011 in implementing the PFEM Action Plan5 has led the GoM to establish a more comprehensive approach to provide strategic direction through a sector-wide approach to the PFEM reforms—PFEMRP. The implementation of the program was provided using a common basket funding mechanism through the PFEMRP MDTF administered by the World Bank (TF13825). Donors participated in the various technical working group meetings and implementation supervision missions for the project.

56. Government’s capacity. At the preparation stage, capacity of the Government to implement reforms was very low. Various reform reviews have shown that PFM reform requires strong political commitment, implementation designs tailored to the country context, and Government-led coordination arrangements. The MoF recognized the importance of developing a facilitating unit for PFEM reforms and acting as a secretariat, and thus set up a PFEM unit in 2008. The FROIP Trust Fund was thus managed by a high-level unit in the MoF. This unit oversaw implementing all PFM project activities including fiduciary activities (financial management, procurement) as well as M&E. The project was also designed to address capacity constraints through training in financial management, procurement, and M&E.

57. Approach to IT investments. The IFMIS and payroll management system expansion including interfacing with the Central Bank and other PFM systems and rollout to regions were supported in a phased approach by the project (following a yet-to-be-completed review of regulations and businesses processes funded separately by the EU). The existing IFMIS (based on EPICOR) was not being leveraged to its full potential due to several control weaknesses and incomplete coverage. Key aspects of the Government’s strategy involved implementing urgent fixes and reevaluating EPICOR and SN (subnational IFMIS) as suitable technology platforms, BPR and subsequent revision of its accounting manual and IFMIS upgrade/re-implementation. This reevaluation was supported by GIZ. All the following three options were considered at the time of preparation: (a) upgrade of the existing IFMIS - EPICOR to version 9.0, which will provide an opportunity to leverage existing IT investments; (b) adopt SN as a unique technology architecture for supporting the Government’s financial operations at both the central and subnational

5 The PFMRP was developed with the support of the EU Capacity-Building Project for Economic Management and Policy Coordination in Public Finance.


Page 20 of 62

levels; (c) consider a third option of acquiring and implementing a new enterprise resource planning system, should EPICOR and SN not meet the Government’s requirements. The team has acknowledged the risks and delays inherent to the last option and agreed that this will be used only as a last resort option. A provisional amount was allocated in the project cost to cater for this eventuality.

58. Results Framework. The Results Framework mirrored complex design of the project, was based on PEFA indicators, and had somewhat ambitious targets. At the time of preparation, Malawi has a track record of carrying out PEFA assessments which have made an important contribution to the shaping and implementing of reforms and improvements to the PFM system. The results of the 2011 PEFA assessment were used as a baseline for the indicators within the Results Framework. FROIP had six PDO-level indicators, all based on PEFA indicators which complicated monitoring and distorted focus of the project as no PEFA assessment has been conducted since 2011. The team has also recognized that recruiting specialists for M&E and other critical areas such as IT, financial management, and procurement in the implementing agency at the beginning of the project was very important to avoid implementation delays.

B. KEY FACTORS DURING IMPLEMENTATION

Factors Subject to Government’s Control

59. Coordination and engagement of stakeholders. The individual components of FROIP were managed by senior officers who were heads of the beneficiary’s relevant units—accountant general, auditor general, internal audit director, and head of the PFEM unit. Project implementation arrangements have been derived from the governance arrangements set up for the PFEMRP (PFEM Steering Committee, PFEM Technical Committee, technical working groups, and the PFEM unit. In addition to this, there is also a Joint Government Donor Committee. The PFEM Steering Committee is chaired by the secretary to the Treasury with the PFEM unit providing secretariat services. The accountability of the component teams for achieving the milestones within the agreed dates was rigorously monitored through regular meetings of the PFEM Technical Committee and the PFEM Steering Committee. The Joint Government Donor Committee which is supposed to review the implementation progress of the PFEMRP MDTF was not convening on a regular basis. With the advent of the ‘cashgate’ during implementation, there were several issues that showed that the coordination of stakeholders did not work well. After the ‘cashgate’, the Government, with the approval of the Joint Government Donor Committee, decided to upgrade or replace the current IFMIS through a competitive process. This leaves a financing gap of US$5.6 million which the Government committed to provide. To address this, the GoM undertook a reprioritization to eliminate or defer some of the planned activities to a later stage of the project, or find alternative sources of funding them. This negatively affected many activities in the other components.

60. Political commitment. The commitment of the Government and implementing units to the overall project remained high throughout the life of FROIP and in particular, after the ‘cashgate’ scandal highlighted the immediate need to address long-lasting PFM problems. However, the Government’s commitment to ensuring the desired outcomes to be fully achieved can be questioned. While implementation of some components lagged for reasons discussed earlier, the Government’s inability to recruit an M&E specialist over the entire life of the project or ability to resolve the IFMIS procurement issues with the World Bank stand out as the areas where senior decision makers should have intervened to help the project meet its objectives.


Page 21 of 62

61. Project management. The PFEM program management and monitoring arrangements under FROIP had been unsatisfactory until the midterm review. There were continued slippages of agreed dates for completion of various project activities for several reasons. The meetings of the various committees especially the technical working groups and reporting on the implementation were not enforced and monitored. After the midterm review, the project management and monitoring arrangements were back on track because of the ability of the project management team to organize two Steering Committee meetings in each of the quarters ending December 2015 and March 2016. The strength of the PFEM Division which is responsible for project management has been strengthened by four additional staff. However, resources for expertise in project management and M&E were still not available.

62. Sustainability. FROIP targeted to increase capacity building by utilizing Government systems, through the training programs, and by supporting the infrastructure acquisition/upgrade of various IT systems (IFMIS, payroll, internal audit system, NAO IT system, and so on). A comprehensive training program was implemented during project implementation: (a) at the internal audit unit training focused on IT audit techniques (one certified information system auditor [CISA] trained, establishment of an IT audit unit with two IT auditors posted at the Office of the Accountant General Department, financial management and procurement; (b) at the NAO, staff have been trained on forensic computing, performance auditing, and IT auditing. At the district level, FROIP has supported achievements in decentralizing payroll processing, and the rollout of the IFMIS into several districts. However, many additional activities were not completed—mainly due to the reprioritization of the resources toward the procurement of a new IFMIS that could have cemented the sustainability of many of FROIP’s activities. As was discussed earlier, some of the IT infrastructure acquired by FROIP have reached their end of life with no clarity on whether their replacement will be taken care of by the GoM and it is still unclear if the Government has enough trained auditors to prevent arrears in external audits from accumulating again.

63. Fiduciary. The project has been using Microsoft Excel to process its financial transactions since project effectiveness. Apart from exposing the transactions to errors, it also places a burden on report preparation. The first audited financial statements for the financial year ended on June 30, 2014, was completed and a copy of the report and the Management Letter was submitted to the World Bank within the six-months deadline. The project procured and installed a new accounting software—SAGE enterprise resource planning system only after three years of implementation. The project incurred ineligible expenses amounting to US$519,142.32 during implementation. This was invoiced, and the Government was able to reimburse the World Bank in 2017. The internal audit unit at the MoF is supposed to audit the project at least twice in a year. Despite several reminders, the project management unit has not arranged for the internal auditing of the project. It was agreed that the first audit by the internal audit was to be completed not later than March 31, 2017, however, no audit was ever conducted.

64. Procurement management. There were delays in almost all procurement activities of the project. The procurement of the audit management software, the computer-assisted auditing tools for the NAO, and the outsourcing of audits to private audit firms are a few examples. The issues and delays encountered during the procurement of the new IFMIS are also another example of poor procurement activities by the project management unit.

Factors Subject to World Bank Control

65. Project design. The overall design of FROIP was sufficiently aligned with Government and World Bank priorities, was flexible, and had a mix of activities to introduce new reforms and provide training to


Page 22 of 62

allow them to be sustainable. The scope was neither too broad nor too ambitious when FROIP began. However, once ‘cashgate’ happened, and the realization of how the focus of the project would change was understood, the initial design became no longer adequate. After it was decided to procure a new IFMIS, the components should have been rescoped to put adequate focus on the upcoming procurement and reduce the number of activities within each of the other areas. This type of rescoping was proposed for the second restructuring proposal and extension in 2018, but the extension was denied.

66. Supervision. The project was closely supervised with progress and issues reflected in the Aide Memoires and ISRs. Action plans for issues that were pending resolutions were prepared and follow-up actions were well documented. However, frequent changes in task team leaders (TTLs) over the project implementation has negatively affected implementation strategies, specifically on the dialogue with the Government on the IFMIS upgrade or acquisition.

Factors Outside the Control of the Government

67. The ‘cashgate’ scandal. The Malawi ‘cashgate’, in which about US$32 million in Government funds was misappropriated between April and September 2013, threw into stark relief the fundamental and deep-rooted governance problems facing the Malawian Government.6 The ‘cashgate’ scandal revealed the immediate need to address long-lasting PFM problems, such as delays or the inability of the Government to (a) prepare bank reconciliation statements, (b) prepare and publish quarterly in-year financial statements according to PFM Act, (c) prepare and publish audited annual financial statements in accordance with the PFM Act, (d) communicate budget ceilings to MDAs and enforce commitments control, (e) prevent misapplication of funds due to nonadherence with rules regarding budget virements, (f) ensure adequate segregation of duties between end users and system administrators of the IFMIS, and (g) enforce managerial accountability to ensure compliance with rules and regulations. More broadly, ‘cashgate’ has presented the opportunity to revisit the support to the Government’s PFM reforms specially to strengthen the internal controls. The project implementation was interrupted for more than one year by the ‘cashgate’ scandal—the entire Government and specially the Project Implementation Unit and development partners were focused on the scandal, in effect, reducing the effective implementation period to about two years instead of three years and three months.

IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME

A. QUALITY OF MONITORING AND EVALUATION (M&E)

M&E Design

68. The PFEM unit did not have manpower nor enough attention to carrying out project’s M&E. The original plan was to hire an M&E specialist, but the unit was unable to identify a suitable candidate, so the responsibility remained with the unit. However, there is little indication that the PFEM unit made M&E a priority. FROIP outcome indicators were based on PEFA methodologies and baselines were taken from the 2011 PEFA assessment. Based on history in Malawi, the project anticipated that at least one additional PEFA would be carried out before the end of the project. This was a reasonable assumption because PEFA assessments had been undertaken in Malawi in 2005, 2006, 2008, and 2011; however, no additional PEFA

6 Bridges, Kate and Michael Woolcock. 2017. How (Not) to Fix Problems That Matter: Assessing and Responding to Malawi’s History of Institutional Reform.


Page 23 of 62

was conducted after 2011. During the 2016 FROIP restructuring, because no PEFA had yet been undertaken, text-based PEFA equivalent (qualitative) indicators could have been introduced in the event that another PEFA assessment would not be undertaken before the close of the project.

69. The project Results Framework, based on the PEFA composite indicators, was not suitable for assessing the advancement of the project’s achievement toward its development objectives on a regular basis. The World Bank’s project supervision requires a review of project implementation as well as the achievement toward the PDO twice a year at least. Without updating indicators, it was almost impossible to provide evidence for the achievements of the PDO. Furthermore, it was odd that the project started without intermediate results indicators. The six intermediate results indicators were added during the project restructuring; however, there was no clear link as to how these intermediate results indicators would contribute to achieving the PDO. The additional intermediate results indicators were another set of PEFA composite indicators and did not provide a clear bridge between the components and activities of FROIP.

70. The original target values of indicators (improvements in PEFA scores by half or one full grade improvement) were not too ambitious but should have been revised after ‘cashgate’. The financial scandal exposed that some of the controls may not be as strong as their original baseline indicated. After ‘cashgate’ but before the restructuring, several of the PDOs were being rated below their original baseline based on the counterpart’s self-assessment. Given the indication that the original PEFA indicator grades were too high, the restructuring period was a missed opportunity to reexamine both target grades and the use of PEFA indicators overall. The only changes made to the PDO indicators during the restructuring was to remap them to the new PEFA equivalents.

M&E Implementation

71. The reporting on the M&E framework throughout the project was done on time but the quality was inconsistent. The ISR ratings on PDO progress and implementation progress were generally ‘Moderately Satisfactory’ even as problems arose and implementation started to lag. It was stated in the PAD that the Government, along with the MDTF partners would develop lower results indicators to track results in service delivery. The intermediate results indicators that were introduced were not detailed or specific enough to monitor progress toward service delivery.

M&E Utilization

72. There is little evidence that the M&E framework was utilized as a management tool to guide project activities, resource allocation, or supervision. Deficiencies in the design of the Results Framework, the inconsistent reporting of ratings, and a lack of any significant revision to the framework, despite obvious challenges to implementation, demonstrate that M&E did not receive the attention it required.

Justification of Overall Rating of Quality of M&E

Rating: Negligible

73. There were severe shortcomings in both the M&E implementation and utilization. During the project restructuring in 2016, the opportunity to introduce text-based equivalent of the PEFA indicators


Page 24 of 62

was not taken, despite no PEFA having been conducted since 2011 and not having one scheduled either. Rather, the project introduced additional PEFA indicators for use as immediate results indicators. This had a significant impact, hindering the ability to monitor and assess progress toward the final outcomes.

B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE

Environmental and Social

74. The project did not trigger social and environmental safeguards. The project was an environmental assessment Category ‘C’ TA project. The project did not trigger any social safeguards policies. However, it was expected to have positive social impacts through improved public confidence in the Government in the management of public finances in a more transparent and accountable manner. The related outcomes have been covered in this report.

Fiduciary

75. The project was able to effectively comply with the Financing Agreement covenants. The quarterly interim financial reports (IFRs) and the annual audited financial statements were submitted to the World Bank before due dates most of the time, and the reports were acceptable to the World Bank in terms of form and contents. The procurement procedures were generally followed except in the second year of the project when substandard computer items were procured leading to declaration of ineligible expenses amounting US$519,142. The project stipulated that new computers were to be purchased. But it was determined that the computers supposed to be purchased with FROIP funds were instead refurbished . The said ineligible amount was invoiced by the World Bank. The Government has reimbursed the amount to the World Bank in 2017.

C. BANK PERFORMANCE

Quality at Entry

76. Project preparation was carried out with an adequate number of specialists who provided the technical skill mix necessary to address sector and fiduciary concerns. The World Bank provided adequate human and financial resources to ensure quality preparation and appraisal work. The project was consistent with the Country Partnership Framework at the time of approval and with the Government priorities in the sector. Most remaining issues of the proceeding project, the PFEMRP, were well considered in the design and scope of FROIP. The World Bank had a close working relationship with the borrower during preparation and appraisal.

Quality of Supervision

77. The World Bank team conducted routine missions and reporting was generally timely but support for the PFEM unit was too little. The team remained engaged through missions and provided support, particularly during the early part of the project. The project team failed to increase their procurement support even after the issues resulting from the procurement of computers came to light and despite the procurement of the new IFMIS became more complex than originally envisioned. The World Bank’s responsibilities were rather focused on providing the funding available for procuring items


Page 25 of 62

and training, but they lacked the continuous presence of the TTL to provide the hands-on implementation support necessary for operations in a low-capacity environment.

78. The ISRs of the project were not utilized effectively as a tool to monitor and measure the state of project implementation. The appropriateness of ratings is questionable: among a total of nine ISRs, six of them rated implementation progress as Moderately Satisfactory. Nonetheless, in multiple instances, the PDO indicators fell below the original baseline, yet progress was rated satisfactory. Additionally, progress was rated Moderately Satisfactory throughout 2016 and 2017 even as the disbursement of funds was minimal. The ratings in the final ISR were more aligned with the actual project performance but at that point there was little time for improvement.

79. The World Bank team did not optimize the project restructuring as an opportunity to strengthen the Results Framework or address how the new IFMIS procurement would impact implementation of other activities. The decision not to formally realign resources and reduce the activities and expectations to match the shift toward an IFMIS focus had a negative impact on the supervision and progression of FROIP activities. After the restructuring, it was clear that funds would have to shift toward the IFMIS procurement, but it was not explicitly stated how this would happen, increasing the ambiguity in the implementation and sequencing of other activities.

Justification of Overall Rating of Bank Performance

80. The overall rating for World Bank performance is Unsatisfactory. The World Bank worked closely with the GoM to prepare the project to make sure that it would fulfill the needs of the Government by partnering with appropriate units and have proper fiduciary safeguards in place. However, there were major shortcomings in the M&E framework and in the supervision of the project, specifically in the lack of adjustments made during the project restructuring to support the increased focus on the procurement of a new IFMIS. These shortcomings made it difficult for FROIP to properly make adjustments after ‘cashgate’ and rendered the overall impact of FROIP to be less than what would have otherwise been possible with better M&E and more proactive supervision.

D. RISK TO DEVELOPMENT OUTCOME

Moderate

81. The risk to development outcome is considered Moderate due to the development of legislation that strengthens the role of external and internal audit in Malawi. The Government has shown commitment to the current gains in strengthening the role of audit, but it still needs to solidify these by approving legislation that is currently with Parliament. The Government has been working with the World Bank and other development partners to implement other programs that will allow it to implement or complete the non-IFMIS aspects of FROIP. The GoM is also continuing to move forward with the procurement and implementation of a new IFMIS.

82. The main risks to the achievements under FROIP are from either the failure of hardware purchased or through backlogs beginning to build in the submission of financial reports. Equipment purchased to support the decentralized payroll processing has already begun to fail and because there is no maintenance agreement in place and the disaster recovery plans were not implemented, the possibility that payroll delays will again become an issue is concerning. In addition, the reduction in the backlog of


Page 26 of 62

financial reports was achieved using additional audit firms hired through FROIP and it has not yet been determined if the increased capacity of efficiency of audit staff from FROIP will be enough to keep current with the auditing of financial reports. The GoM is actively pursuing the further implementation of several FROIP achievements including decentralized payroll processing and is collaborating with the World Bank on a new operation ‘Malawi Governance to Enable Service Delivery’ which will provide follow-up support to several FROIP areas, with a focus at the subnational level.

V. LESSONS AND RECOMMENDATIONS

83. Complex procurement activities, such as a new IFMIS, require extensive support and a deep technical expertise to be effective. Procurement of complex information systems has often posed problems in World Bank projects and caused major delays.7 Procurement of the technology platform required for systems implementation—the application software, hardware and networking, and associated integration services require more-than-adequate support throughout the process (from the preparation of bid documents to contract management during implementation). This support can be provided through consultancies but their remains a need to ensure that the World Bank has strong IFMIS technical expertise included in the team. The major consultancy prescribed for the design, procurement, and implementation phases is a diagnostic, design, and project implementation/contract management consultancy.

84. Using composite indicators like PEFA should be avoided for M&E purposes and should introduce intermediate indicators at the beginning of the project. Indicators based on project-generated data and produced on a regular basis may be better suited for public sector reform projects. The Results Framework’s intermediate indicators should not be PEFA based, need to be established at the beginning of the project, and must facilitate the client’s efforts to monitor and report on their progress. A PEFA assessment could be used to assess the overall impact of the reform but not project implementation. Results indicators should be more focused on the specific capacity and system performance issues that will be addressed under the project.

85. Changes in the direction or emphasis of a project need to be clearly communicated to all stakeholders after any significant events that impact the project. After ‘cashgate’ took place early in the implementation of FROIP, no clear direction was provided to the implementing units as to how it would affect their activities. Although it was discussed in meetings that the cost of a new IFMIS would likely reduce the available funding for other activities under FROIP, none of the implementing units were told which activities to drop or alter. This prevented activities from being prioritized in a way to have a sustainable impact. For example, FROIP approved the implementing of new hardware and software but failed to approve the ongoing maintenance or training that is needed to fully utilize it. Even though changes to FROIP fell within the existing guidelines, the emphasis and funding priorities were clearly affected. By working with the implementing units to reduce activities in an organized way could have allowed units to drop whole activities that could not be fully implemented from start to finish.

86. In projects with multiple activities that need proper sequencing and include a capacity-building component, implementing units should be encouraged to hire TA to support implementation. These

7 Hashim, Ali. 2014. A Handbook on Financial Management Information Systems for Government - A Practitioners Guide for Setting Reform Priorities, Systems Design and Implementation (Based on a compilation of experiences in World Bank-financed projects. Washington, DC: World Bank.


Page 27 of 62

implementing units have been targeted for assistance through the project specifically because their performance needs improvement and the hiring of an outside firm can serve dual functions of supporting project implementation to help them stay on schedule and have the added benefit of skills transfer. For example, in FROIP the CIAU hired an international firm called Cowater International to assist in developing and modernizing the audit practices and developing skills of audit staff. With the support of Cowater, the CIAU was able to sequence and implement many of their activities by 2016 and saw major gains in the quality of work being done by staff. Other implementing units under FROIP, without outside TA support, were not as organized or efficient in their implementation.

87. On complex projects or on projects where the factors outside the project make a major impact on the project during implementation, TTLs must have a constant field presence and take full advantage of opportunities to restructure the project. Although frequent TTL turnover is not ideal, there are times where it cannot be avoided. When this occurs, it is imperative to have a continued field presence by the TTLs. For some period of project implementation, the TTL was based outside of Malawi. This made it difficult for the task team to provide swift and intense support when needed. Furthermore, when the restructuring was processed in 2016, the World Bank did not take full advantage of the opportunity to address problems that were already arising in implementation. Instead, the restructuring was used only as a vehicle to extend FROIP implementation time frame and add perfunctory additions such as additional PEFA indicators as intermediate results indicators. The frequent changes in TTLs between 2013 and the restructuring in 2016, combined with the insufficient field presence, contributed to the lack of significant changes at that time.

88. M&E should be a primary focus of any project and its application needs to be continually reviewed. The M&E framework must be the centerpiece of project implementation. The implementing unit should continually review and explain to internal and external audiences how project interventions are expected to lead to results and impacts of importance. As part of the constant dialogue around the results chain, the project Results Framework should be reviewed and updated periodically, and at least during midterm review and each restructuring to ensure that indicators are appropriately defined and measured and that the level of ambition of targets matches the project investment. The PFEM unit intended to hire an outside TA firm to conduct M&E but were unable to find a suitable firm for this activity and decided to do it themselves. Because the M&E is the main instrument to measure ongoing success of project implementation, the World Bank or one of the development partners should have stepped in to ensure the PFEM unit added a qualified M&E person to the unit. Allowing FROIP to continue for five years, including a restructuring and extension without a strong M&E framework in place, was a red flag that could have been corrected by either internal or external stakeholders.

VI. COMMENTS ON ISSUES RAISED BY BENEFICIARY/DEVELOPMENT PARTNERS

89. All comments from development partners have been incorporated into this report.

90. The MoFEPD submitted its Implementation Completion and Results Report in September 2018 and it is attached as annex 5 of this report. Table 4 compares the findings of the Government produced ICR with this ICR. The main differences between the two reports is that the Government’s ICR puts most of emphasis on the accomplishments under FROIP. This ICR does not disagree with those accomplishments but feels that the Government’s ICR did not place equal emphasis on the activities that were either partially or failed to be implemented. The objectives that were not fully achieved and activities not implemented are the main differences between the two ICRs.


Page 28 of 62

Table 4. Differences Between ICR’s

Sub-component 1.1: Accounting and IFMIS: Progress was attained

with regards to improving IFMIS performance through investments in

rehabilitation of fiber back-bone for the Government wide area

network, installation of new servers and other network equipment,

which improved availability and speed of IFMIS.

This is consistent with the ICR teams findings, however, the government

wide area network was not fully implemented, servers and other

equipment were implemented in fewer numbers than planned, and the

government is still relying on an unsupported IFMIS that is in need of

replacement.

Sub-component 1.1: Accounting and IFMIS:

FROIP further supported establishment of interfaces between IFMIS

and other major PFM Systems including HRMIS and Local Authority

IFMIS, hence enhancing controls. More work is however required

considering that the interfaces are not fully operational.

This is consistent with the ICR teams findings. The interface between

major systems has been developed but other factors (i.e. lack of

harmonized CoA) are preventing it from being operational.

Sub-component 1.2: Payroll Management: Under this sub-

component, payroll processing was devolved into 6 local councils

across the country. This led to timely processing of salaries for civil

servants. More to this, ghost workers were eliminated following a

payroll audit and headcount that was undertaken. Equipment was

also installed for efficient performance of HRMIS.

This is consistent with the ICR teams findings. However, the lack of

support and maintenance of hardware has some of the equipment

breaking down, creating a risk that the decentralized payroll processing

cannot be sustained.

Sub-component 1.3: Local Authority IFMIS: Complementing

initiatives by other PFM DPs, Local Authority IFMIS was rolled out to

all councils. In addition, trainings were provided for Management

Information Systems Officers (MISOs) to enable them competently to

backstop the system. This has contributed to Councils Capacity to

Produce Statements in preparation for Audits.

This is consistent with the ICR teams findings, however, several planned

components to strengthen the local IFMIS (VPN connectivity, disaster

recovery hardware) were not implemented.

FROIP supported initiatives to improve capacity in the internal audit

function for the expansion of scope and capacity to carry out

specialized audits including IT audits and Risk based Audits.

Internal audit manual were developed and are currently guiding the

operations of the Internal Audit Function. Relevant trainings were

provided for Internal Auditors. IT audit capacity was also provided for

internal auditors

This is consistent with the ICR teams findings but several initiatives have

been started but not completed. This includes the Enterprise Risk

Management guidelines and the approval of the internal audit policy

and bill.

FROIP supported the automation of external audit function. In this

regard, Audit Management Software (AMS) and CAATs were

procured. This has enabled the NAO to carry out audits of the systems

including IFMIS, and HRMIS.

This is consistent with the ICR teams findings but the roll out of the

IFMIS continuous auditing software was only partially implemented.

Clearance of the audit backlog and maintenance of up to date status

of the audits. FROIP supported this initiative. Further to this, trainings

were provided for Public Accounts Committee (PAC) members for

meaningful scrutiny of the Auditor General’s reports. Further support

was provided to PAC to facilitate deliberation on the produced audit

reports.

This is consistent with the ICR teams findings but the level of attribution

of the reduction of Audits to FROIP is unclear. Additionally it is unclear

if the audit unit has the resources and manpower to remain current on

audit reports.

In addition to normal Project Management Support for the Project

Management component, FROIP supported the Ministry of Finance

with capacity building for effective financial oversight for state-

owned enterprises. Officers were trained in financial analysis. In

addition, tools have been developed to ensure compliance with the

PFM Act in the dealing of the SOEs. It is expected that with the

intervention, issues such as contingent liabilities arising from SOEs

will be managed and that there will be increased remittance of

returns on the Government investment.

This is consistent with the ICR teams findings. However, the project

management component fell short in several key areas including the

hiring of enough staff to monitor and implement the multiple

components of FROIP and in developing an M&E Framework.

Non-implementation of some initially planned activities: this was

mainly due to challenges beyond Project management Control (for

example, procurement of new IFMIS). These challenges include:

litigation, withdraw by the Bank due to limited project duration.

The ICR team found that the procurement of the new IFMIS did force a

change in the number of activities that could be implemented under

FROIP, however, a lack of clear direction and revision to which activities

would proceed or be canceled caused ambiguity among implementing

units on which activities they should focus on.

PDO 1: Accounting and Financial Management

PDO 2: Internal Audit

PDO 3: External Audit

PDO 4: Program Management

Other Areas


Page 29 of 62

ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS

A. RESULTS INDICATORS A.1 PDO Indicators

Indicator Name Unit of Measure Baseline Original Target Formally Revised

Target

Actual Achieved at Completion (PEFA 2018)

Component 1: PEFA PI-23.4 (former PI-18)-Effectiveness of payroll controls

Text C+ A B+ B

30-Jun-2011 March 7, 2013 30-Apr-2018 30-Apr-2018

Comments (achievements against targets):


Target


Component 1: PEFA PI-25 (former PI-20)-Internal controls on non-salary expenditure

Text C+ C B C

30-Jun-2011 March 7, 2013 30-Apr-2018 30-Apr-2018



Page 30 of 62


Target


Component 2: PEFA PI-26 (former PI-21)-Effectiveness of Internal audit

Text D+ C C D+

30-Jun-2011 March 7, 2013 30-Apr-2018 30-Apr-2018



Target


Component 1: PEFA PI-29 (former PI-25)-Accounting, recording and reporting

Text C+ B B D+

30-Jun-2011 March 7, 2013 30-Apr-2018 30-Apr-2018



Target


Component 3: PEFA PI-30 (former 26)-Characteristics of External Audit

Text D+ C C C+

30-Jun-2011 March 7, 2013 30-Apr-2018 30-Apr-2018



Page 31 of 62

A.2 Intermediate Results Indicators


Target


Component 3: Audit coverage and standards (PEFA PI-30.1)

Text D+ N/A B B

30-Apr-2013 30-Apr-2018 30-Apr-2018



Target


Component 1: Payroll audits (PEFA PI 23.4)

Text C N/A B+ B

30-Apr-2013 30-Apr-2018 30-Apr-2018



Target


Component 3: Supreme Audit Institution

International Standards

0 out of 5 Implemented

N/A 4 out of 5 Implemented

2 out of 5 Implemented


Page 32 of 62

Independence (PEFA PI-30.4) 30-Apr-2013 30-Apr-2018 30-Apr-2018



Target


Component 1: Compliance with payment rules and procedures (PEFA PI-25.3)

Text C N/A B D

30-Apr-2013 30-Apr-2018 03-Mar-2016



Target


Component 1: Submission of reports for external audit (PEFA PI-29.2)

Months 48.00 N/A 3.00 4.00

30-Apr-2013 30-Apr-2018 30-Apr-2018


Indicator Name Unit of Measure Baseline Original Target Formally Revised Actual Achieved at Completion (PEFA 2018)


Page 33 of 62

Target

Component 2: Nature of audits and standards applied (PEFA PI-26.2)

Text C N/A B+ C

30-Apr-2013 30-Apr-2018 30-Apr-2018



Page 34 of 62

B. KEY OUTPUTS BY COMPONENT

Objective/Outcome 1: Improved internal controls of the MDA's

Outcome Indicators 1. PI-25 (Formerly PI-20) Improved effectiveness of internal controls for non-salary expenditures 2. PI-23.4 (formerly PI-18) Improved effectiveness of payroll controls

Intermediate Results Indicators 1. PI-25.3 Compliance with payment rules and procedures 2. PI-23.4 Payroll audits

Key Outputs by Component (linked to the achievement of the Objective/Outcome 1)

1. Procurement and Implementation of a new IFMIS system 2. Improve Payroll Management 3. Complete IFMIS roll out and connectivity with districts

Objective/Outcome 2: Improved accounting at the MDA's

Outcome Indicators 1. PI-26 (Formerly PI-21) Improved effectiveness of internal audit

Intermediate Results Indicators 1. PI-26.2 Nature of audits and standards applied 2. PI-30.1 Audit coverage and standards


1. Improve legal framework 2. Improve staff knowledge and techniques of internal auditing

Objective/Outcome 3: Improved reporting of the MDA's

Outcome Indicators 1. PI-29 (Formerly PI-25) Improved quality of annual financial statements

Intermediate Results Indicators 1. PI-29.2 Submission of reports for external audit


1. Improve timeliness of financial statement submissions

Objective/Outcome 4: Improved oversight of the MDA's


Page 35 of 62

Outcome Indicators 1. PI-30 (Formerly PI-26) Scope nature and follow-up of external audit 2. PI-30 (Formerly PI-28) Legislative scrutiny of external audit reports

Intermediate Results Indicators 1. PI-30.1 Audit coverage and standards 2. PI-30.4 Supreme Audit Institution Independence


1. Implement legislation to establish independence of NAO 2. Improve productivity and capability of NAO staff


Page 36 of 62

ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION

A. TASK TEAM MEMBERS

Name Responsibility

PREPARATION

P.K. Subramanian TTL

Trust Chimaliro Financial Management Specialist

Michael Jacobs Team Member

Annie Jere Team Member

Steven Maclean Mhone Procurement Specialist

Khuram Farooq IFMIS Specialist

Nneoma Nwogu Legal

Luis Schwarz Finance Officer

SUPERVISION/Completion

P.K. Subramanian, Hugues Agossou, Daniel Yaw Domelevo, Srinivas Gurazada, Saidu Goje

Task Team Leader(s)

Trust Chimaliro Financial Management Specialist

Srinivas Gurazada, Saidu Goje Program Manager

Steven Maclean Mhone, Anthony Aggrey Msendema, Anjani Kumar

Procurement Specialist(s)

Johanna van Tilburg Team Member

Michael Jacobs Social Safeguards Specialist

Khuram Farooq, Cem Dener IFMIS Specialist(s)

Mercy Chimpokosera-Mseu Environmental Safeguards Specialist

Lozi S. Sapele Team Member

Ikechi B. Okorie Team Member

Michael Eriu Okuny Team Member

Lucy Anyango Musira Team Member

Zione Edith Kansinde Team Member

Appolenia Mbowe Team Member

Annie Jere Team Member

Miriam Sangallo Kalembo Team Member

Kirk Schmidt ICR Author

Dolele Sylla ICR Author


Page 37 of 62

B. STAFF TIME AND COST

Stage of Project Cycle Staff Time and Cost

No. of staff weeks US$ (including travel and consultant costs)

Preparation

Total 0.00 0.00

Supervision/ICR

Total 0.00 22,003.00


Page 38 of 62

ANNEX 3. PROJECT COST BY COMPONENT

Components Amount at Approval

(US$, millions)

Actual at Project Closing (US$,

millions)

Percentage of Approval

Internal Audit 2.8 1.4 51.00

External Audit 4.7 1.4 29.00

PFEMRP Management 1.6 2.2 138.00


9.2 5.9 64.00

Contingencies 0.7 0 0.0

Total 19.0 10.9 57.00


Page 39 of 62

ANNEX 4. EFFICIENCY ANALYSIS

Table 4.1. Efficiency Gains from Improved PFM

Benefit Described in the PAD Progress Under the Project Comment

Improved system controls for accounting and financial management

The current IFMIS infrastructure has been upgraded and IT general and applications controls improved. However, the system reached its end of life and needs replacement.

Gain has partially materialized.

Improved compliance with accounting rules and regulations

Reduced backlogs of financial statements but quality of statements needs improvement


Improved transparency and timeliness of financial reporting

Backlog of financial reporting has been eliminated at the central level.

Gain has materialized.

Improved payroll functions Delays in payroll have been eliminated due to the ability to process payroll at one of six hubs located in districts.


Improved resource utilization across the budget

The interface between national, district, and HRMIS is not fully functional and other modules such as the asset registry were not implemented.

Gain has not materialized.

Improved independence of the NAO The NAO has increased independence that meet international ISSAI standards.


Strengthen human and institutional capacity

The internal and external audit offices have improved capacity, while the Treasury and budget underperformed in capacity and institutional building activities.



Page 40 of 62

ANNEX 5. BORROWER, CO-FINANCIER, AND OTHER PARTNER/STAKEHOLDER COMMENTS

GOVERNMENT OF MALAWI

Financial Reporting and Oversight Improvement Project (FROIP):

Implementation Completion Report

Ministry of Finance Economic Planning and Development

P.O. Box 30049

Capital Hill

Lilongwe

Malawi

September 2018


Page 41 of 62

Contents

LIST OF ABBREVIATIONS AND ACRONYMS .................................................................................. 41

EXECUTIVE SUMMARY .................................................................................................................. 42

1.0 INTRODUCTION ...................................................................................................................... 43

2.0 PUBLIC FINANCE MANAGEMENT REFORMS IMPLEMENTATION OVERVIEW ...................... 44

3.0 BASIC PROJECT INFORMATION ............................................................................................. 44

4.0 RELEVANCE ............................................................................................................................. 45

4.1 Project Development Objective ......................................................................................... 45

4.2 Project Design ................................................................................................................... 45

5.0 PROJECT EFFECTIVENESS ....................................................................................................... 46

5.1 Progress towards Implementation of Project Development Objective ................................. 46

5.2 Project Outcomes (Progress Based on PEFA) ...................................................................... 46

5.3 Progress by Components ................................................................................................... 47

6.0 EFFICIENCY .............................................................................................................................. 49

6.1 Timeliness ......................................................................................................................... 49

6.2 Resource Absorption ......................................................................................................... 49

7.0 MONITORING AND EVALUATIONS ........................................................................................ 50

8.0 FIDUCIARY COMPLIANCE ....................................................................................................... 50

9.0 MAJOR IMPLEMENTATION CHALLENGES .............................................................................. 50

10.0 SUSTAINABILITY ..................................................................................................................... 51

11.0 KEY LESSONS .......................................................................................................................... 51

ACKNOWLEDGEMENTS ................................................................................................................. 52

ANNEXES ........................................................................................................................................ 52

List of Abbreviations and Acronyms

AfDB African Development Bank

DFID Department for International Development

DP Donor Partner

EU European Union

FROIP Financial Reporting and Oversight Improvement Project

GiZ Deutsche Gesellschaft für Internationale Zusammenarbeit GmbH

HRMIS Human Resource Management Information Systems

ICR Implementation Completion Report


Page 42 of 62

IFMIS Integrated Financial Management System

MDA Ministry, Department and Agency

MDTF Multi-Donor Trust Fund

MGDS Malawi Growth and Development Strategy

MISOs Management Information System Officers

NAO National Audit Office

PAC Public Accounts Committee

PDOs Project Development Objectives

PEFA Public Expenditure and Financial Accountability Framework

PFEM Public Finance and Economic Management

PFM Public Financial Management

PFMSD Public Finance Management System Division

RP (PFM) Rolling Plan

Executive Summary

1. The Financial Reporting and Oversight Improvement Project (FROIP) was effective from 28th March 2013 as a first project under the Public Finance management (PFM) Multi-Donor Trust Fund (MDTF that was established to bankroll the implementation of the Public Finance and Economic Management Reform Program (PFEMRP. The Project’s principal objective was to improve the internal controls, accounting, reporting and oversight of government finances at the central and decentralized levels in MDAs and districts in Malawi.

2. The Project was initially designed to be implemented from March 2013- June 2016. However, looking at outstanding activities and implementation progress, Government and the Bank agreed to a Project extension until April 30, 2018, in order for the project to achieve its development objective. A further extension for the project to close on December 31, 2018 was sought but the request was not granted by the Bank. FROIP officially closed on 30th April 2018.

3. Overall, the Project amounted to USD 19 million which was implemented in two phases of USD 8 million and USD 11 million, respectively. the Financing was provided by the European Union (EU), Department for International Development (DFID), Germany (GIZ), Irish Aid, and Norwegian Embassy.

4. The Project implementation period saw one of the most turbulent periods in PFM with the revelation of massive plunder of Public resources in 2013. Reactionary, the project’s focus was redirected to address deficiencies that were revealed by the scandal. More focus was directed towards improving controls for the Integrated Financial Management Information System (IFMIS), and also on enhancing performance of the internal control function.

5. Nevertheless, FROIP attained most of its intended Project Development Objectives (PDOs. With FROIP intervention, payroll processing was devolved to six centers across the country, an initiative that has lead to significant improvement in timeliness and accuracy of salary payments coupled with elimination of ghost workers. Additionally, with FROIP intervention, MDAs are current on the production


Page 43 of 62

of Financial Statements culminating into the clearance of audit backlog and timely submission of Auditor General’s Report to the National Assembly for scrutiny. The financial oversight function has further improved both in terms of scope and coverage. Here, Internal Auditors and External Auditors are able to undertake, in addition to regularity audit; performance audit, and IT Audit among others. In addition, the legal framework for external audit was amended and draft policies for the internal audit developed. More to this, performance of the current IFMIS platform, was sustained and enhanced, despite siting on and outdated Epicor based platform.

6. Promoting use of country systems remains the main key lesson from FROIP. The practice, enhances ownership and sustainability of the initiatives. Through FROIP, capacity has been built in PFMSD such that coordination of PFM reforms is more systematic and sustainable. In addition, investment in change managements is key to delivering sustainable reforms. Lastly,More focus should be on people, since the systems are as good as the people managing those systems.

7. Throughout the implementation period, the PFMSD remained the main coordinating agency for implementation of the Project. The Division is headed by Director, reporting to the Secretary to the Treasury. In the initial years, capacity in terms of numbers was a major challenge affecting smooth project implementation. The Problem was eventually addressed with more staff allocation to PFMSD.

1.0 Introduction

8. The Government of Malawi implemented the Financial Reporting and Oversight Improvement Project (FROIP) with the overall aim of improving the internal controls, accounting, reporting and oversight of government finances at the central and decentralized. The Project was implemented within the Public Finance Management Multi-Donor Trust Fund (MTDF) that was established to bankroll implementation of the Public Finance and Economic Management Reform Programme (PFEMRP.

9. FROIP was conceptualized to respond to the PFEM RP by focusing on improving the Integrated Financial Management System (IFMIS) and oversight functions of internal and external audit for better implementation of the rules and regulations, fuller utilization of IFMIS functionalities and improved service delivery. Implementation of the Project was categorized into four broad components of; accounting and financial management, internal audit, external audit, and programme management.

10. The Project was implemented between 28th March 2013 and 30th April 2018, instead of the initial planned project deadline of June 2016. Another extension was sought from the Bank for a further extension of the project which was not granted. The application for Project extension was made considering that some critical activities were still outstanding due to implementation lags. Implementation lags were mainly due to: revelation of plunder of Public Resources in 2013 which necessitated refocusing of the Project. More to this, the biggest component of the project was obstructed by several factors leading to failure of procurement of new IFMIS platform under FROIP.

11. This Project Implementation Completion Report (ICR) is the Government’s final report that summarizes results and performance after implementation of FROIP. This is done by assessing the extent to which: FROIP development objectives were relevant; implementation effective and efficient; and initiatives sustainable. The report further highlights major challenges and key lessons.


Page 44 of 62

2.0 Public Finance Management Reforms Implementation Overview

12. The Government of Malawi has been reforming its public financial management systems over the last two decades. While this has yielded significant improvements in the legal framework, IT systems and budget procedures, full benefits of these reforms had not yet been felt in terms of aggregate fiscal discipline, strategic allocation of resources and effective service delivery. Implementation of these reform, in the early years were mainly through fragmented Action Plans, the last one of which was developed in 2011 and revised in 2013.

13. Government and development partners (DPs) recognized the need to move to a new phase of the reforms focused in greater implementation of the new rules and regulations, tighter internal controls, greater attention to the benefits of PFM reforms for Ministries, Departments and Agencies (MDA) and sectors and capacity development in the various PFEM institutions. Implementation of this phase of reforms was recommended to be done within a comprehensive PFEM reform strategy, the PFEM Reform Program that covered the period 2011-2014 revised to 2016. Through PFEM RP, Government adopted a more comprehensive approach and provided a strategic direction to provide financing for PFEM reforms through a sector wide approach, within which PFEM MDTF and FROIP were implemented. PFEM MDTF, a common basket funding mechanism, and FROIP were administered by the World Bank and executed by GoM.

14. The Public Finance and Economic Management Reform Program covered ten reform areas (planning and policy analysis, resource mobilization, budgeting, procurement, accounting and financial management including internal audit, cash and debt management, parastatal financing, monitoring and reporting, external auditing, and program management). Of the ten, FROIP covered three main areas: accounting and financial reporting, internal and external audit and program management. This was decided during PFEM Governance meetings based on priority, available resources, and component readiness.

15. Alongside implementation of FROIP, other components were supported by other Development Partners through discreet projects and one-off activity implementation. These DPs include: African Development Bank (AfDB). GIZ, Irish Aid, Norwegian Embassy, USAID, EU, and DFID.

16. The management of the PFM system is mainly concentrated within the Ministry of Finance (MoF). Within the Ministry, the Secretary to the Treasury has a key responsibility for budget planning and execution, while the Accountant General has responsibility for producing timely and appropriate consolidated management and financial reports. Within MDAs it is the Controlling Officers who have overall responsibility for effective PFM. The Public Finance Management Systems Division remained the main coordinating unit for implementation of FROIP and all PFM reforms. The Division was the focal point and secretariat for all key PFM Governance Structures.

3.0 Basic Project Information

1. Project Approval Date 13th February, 2013

2. Date of Effectiveness 28th March 2013

3. Closing Date 30th June 2016

4. Revised Closing Date 30 April 2018

5. Project ID P130878


Page 45 of 62

6. Total Project Cost US$ 19 Million ($ 8 million Phase one and $11 million for phase two)

7. Current Task Team Leader Saidu Dan Goje

8. Previous Task Team Leaders P.K. Subramanian, Srinivas Gurazada, Daniel Domelevo

9. Implementing Agency Ministry of Finance Economic Planning and Development- PFMSD

10 Previous Project Directors Twaib Ali and Victor Lungu

11 Current Director Hetherwick Njati

4.0 Relevance

17. This section focuses on Project relevance that articulates the extent to which PDOs and Project Design were in line with PFM Sector Strategy as well as overall national development strategy, MGDS II.

4.1 Project Development Objective

18. The project’s development objectives were relevant to the development of Malawi which has been reforming its PFM systems over the past two decades. The Project was designed to support Malawi’s growth and development through improving internal controls, accounting, reporting and oversight of government finances at the central and decentralized levels in MDAs and districts. The intervention lead to improved aggregate fiscal discipline and service delivery, in turn. More specifically, salaries for public servants are processed in a timely manner. In addition, financial reporting by MDAs has improved in terms of compliance and timeliness.

19. The Project was in line with the country’s Public Finance Management (PFM) Reform agenda as stipulated in the PFEM Reform Program (2011-2014 extended to 2016), reiterated in the PFM Rolling Plan (2018-2021). More importantly, the Project was in line with the second and third Malawi Growth and Development Strategy (MGDS II and III) under the theme Governance.

4.2 Project Design

20. The project design was relevant to the needs of the country and was aimed at addressing existing challenges. The focus areas of improving internal controls, external audit and overall financial reporting, was a deliberate target at areas that were of high impact with regards to strengthening of PFM systems in Malawi, as at the stage of Project design. PFM situation Analysis and a comprehensive Project appraisal process explored several areas requiring intervention, institutions to be considered as well as the type of support required.

21. Based on the country needs, priorities, recommendations from the analytical works such as 2011 PEFA assessment and project appraisal that was carried out, it was found appropriate that in order to draw maximum benefits, FROIP resources should focus on the improving accounting and financial management, internal audit, External audit, and program management. The Project design was therefore highly relevant to Malawi’s PFM reform agenda.


Page 46 of 62

5.0 Project Effectiveness

22. This section focusses at assessing the extent to which the set PDOs were implemented. The section further comments on progress that was made with regards to implementation of specific project objectives.

5.1 Progress towards Implementation of Project Development Objective

23. The development objective for FROIP was mainly to improve the internal controls, accounting, reporting and oversight of government finances at the central and decentralized levels in MDAs and districts in Malawi. Achievement of the objective were expected to be assessed through impact on relevant Public Expenditure and Financial Accountability (PEFA) performance indicators, using the PEFA 2011 framework. However, throughout project duration no PEFA assessment was carried out.

24. Nevertheless, progress was made with regards to improving financial reporting, in the sense that financial statements are currently being timely produced by MDAs. In-year reporting is also being done. Audit backlog was also cleared and is current for the Central Government. More to this, salary processing for civil servants has improved in terms of timeliness and accuracy. The section below provides progress on each indicator.

5.2 Project Outcomes (Progress Based on PEFA)

25. This section provides an overview of progress that has been attained on each PEFA Indicator, as per the table below, suffice to say that no PEFA assessment was carried out during Project duration.

Table 1. Progress on Project Outcomes Based on PEFA 2011 Indicators

No. Indicator Progress

1. Improved effectiveness of payroll controls and non-salary controls as measured by PEFA PI-18 and PI-20

With FROIP intervention, payment of salaries has tremendously improved for civil servants both in terms of accuracy and timeliness. Payroll Processing was decentralized easing the pressure at Central Government, hence leading to payment within statutory deadlines. Furthermore, since amendments are remotely done, ghost workers have been eliminated and adjustment are accurate

2. Improved compliance with rules as measured by PEFA PI-20;

Within MDAs, Financial Inspectorates Functions were established to pre-audit transactions before payments are effected and ensure compliance with rules and regulations before payments are effected. In addition, Internal audit policy was developed and by the time the Project was closing, the policy was undergoing approval process. Accountants in MDAs were trained in IPSAS. These initiatives are expected to improve compliance to rules and regulations. Two servers were procured and installed for the current IFMIS. Further to that a Heavy Duty UPS was installed to provide backup power. This ensured reliable equipment


Page 47 of 62

No. Indicator Progress

for financial management operations.

3. Improved effectiveness of internal audit as measured by PEFA PI-21

Internal auditors were trained to broaden their scope of audits through trainings in other audit areas like IT, fraud and investigations. In addition, the Internal Audit Function adopted a risk based internal auditing approach which is more effective and value adding approach unlike the system based audit that was in use.

4. Improved information on resources received by service delivery units as measured by PEFA PI-23

MDAs are communicated on information about resources funded to them.

5. Improved quality and timeliness of annual financial statements as measured by PEFA PI-25

MDAs and Accountant General produce statements in a timely manner. This has led to ease of clearing audit backlog by the Auditor General.

6. Improved scope, nature, and follow-up of external audit as measured by PEFA PI-26

Scope of external audits expanded to cover: performance audits and IT audits in addition to regularity and financial audits. Mechanism in place in the Ministry of Finance to ensure follow up of audit recommendations. In Addition, Public Accounts Committee summons controlling officers with outstanding audit issues.

7. Improved scrutiny and response to external audit reports as measured by PEFA PI-28

Capacity building provided for responsible Parliamentarians to scrutinize audit reports. Further support was provided to ensure that outstanding reports were deliberated on.

5.3 Progress by Components

26. FROIP was divided into four components, which had sub-components. This section provides an update on what had been achieved in these components.

Table 2. Progress on Components

No. Component Progress

1 Accounting and Financial Management

1.1 Accounting and IFMIS

Under this sub-component, Progress was attained with regards to improving IFMIS performance through investments in rehabilitation of fibre back-bone for the Government wide area network, installation of new servers and other network equipment, which improved availability and speed of IFMIS. Additionally, the project maintained Service Level Agreement for IFMIS maintenance. More control features were introduced in IFMIS, thereby minimizing occurrence of system fraud. Though the system is outdated, intervention by FROIP enabled to keep it afloat. FROIP further supported establishment of interfaces between IFMIS and other major PFM Systems including HRMIS and Local Authority IFMIS, hence enhancing controls. More work is however required considering that the interfaces are not fully operational. Accountants have been trained in IPSAS and are currently championing timely production of statements in MDAs based on IPSAS standards. This follows Government adoption of Cash Basis IPSAS, through a strategy that was also developed with support from the


Page 48 of 62


Project. Government after long period of not reconciling, had been able to reconcile its accounts and is now on course to be current with reconciliations.

1.2 Payroll Management

Under this sub-component, payroll processing was devolved into 6 local councils across the country. This led to timely processing of salaries for civil servants. More to this, ghost workers were eliminated following a payroll audit and headcount that was undertaken. Equipment was also installed for efficient performance of HRMIS.

1,3 Local Authority IFMIS

Complementing initiatives by other PFM DPs, Local Authority IFMIS was rolled out to all councils. In addition, trainings were provided for Management Information Systems Officers (MISOs) to enable them competently to backstop the system. This has contributed to Councils Capacity to Produce Statements in preparation for Audits.

2.0 Internal Audit FROIP supported initiatives to improve capacity in the internal audit function for the expansion of scope and capacity to carry out specialized audits including IT audits and Risk based Audits. Internal audit manual were developed and are currently guiding the operations of the Internal Audit Function. Relevant trainings were provided for Internal Auditors. IT audit capacity was also provided for internal auditors In addition, FROIP supported improvements in the legal and institutional framework for the internal audit function. In this regard, internal audit policy was developed. By the time the Project closed, the Policy was undergoing through approval processes.

3.0 External Audit FROIP supported the automation of external audit function. In this regard, Audit Management Software (AMS) and CAATs were procured. This has enabled the NAO to carry out audits of the systems including IFMIS, and HRMIS. Complementing the automation, specialized audit trainings were provided thereby improving the capacity of the National Audit Office to carry out specialized audits including: IT Audit, and Performance Audit. In addition, ICT equipment and vehicles were procured to support audit operations. Clearance of the audit backlog and maintenance of up to date status of the audits. FROIP supported this initiative. Further to this, trainings were provided for Public Accounts Committee (PAC) members for meaningful scrutiny of the Auditor General’s reports. Further support was provided to PAC to facilitate deliberation on the produced audit reports. Lastly, the Public Audit Amendment Act was enacted. Its implementation is expected to significantly enhance independence of the Auditor General.

Programme Management

I addition to normal Project Management Support for the Project Management component, FROIP supported the Ministry of Finance with capacity building for effective financial oversight for state-owned enterprises. Officers were trained in financial analysis. In addition, tools have been developed to ensure compliance with the PFM Act in the dealing of the SOEs. It is expected that with the intervention, issues such as contingent liabilities arising from SOEs will be managed and that there will be increased remittance of returns on the Government investment PFMSD led the development of new PFM Rolling Plan (2018-2021). In addition, the


Page 49 of 62


Division facilitated PEFA Assessment which its results were yet to be released. At the time of Project close, PFMSD was coordinating the review of the Public Finance Management Act, 2003.

6.0 Efficiency

27. This section explains the extent to which FROIP has been efficient in terms of timelines as well as resource use.

6.1 Timeliness

28. Considering that the Project was extended initially by 22 months, FROIP was not efficient in this aspect. More to this, at the time of closing, Government had requested a further extension of the project. Delays in implementation of planed activities were mainly due to reasons beyond management control. For instance, the freezing of funds flow in anticipation to Government refund for the ineligible expense coupled with changes in direction on Procurement of new IFMIS platform, arguably, the biggest component of the Project.

6.2 Resource Absorption

29. As at project close, the Project was able to disburse about 57% as depicted in the figure below. Low absorption was mainly due to change in direction with regards to IFMIS procurement which was the main driver of absorption, but its procurement did not materialize during project implementation.

Figure 1: FROIP Financial Performance

30. As captured in figure 1, FROIP was estimated to cost US$19 million, an operation which was financed by the PFEM Reform Programme Multi Donor Trust Fund administered by the World Bank. The


Page 50 of 62

fund was a grant executed by the Government of Malawi. Since inception, World Bank had disbursed US$ 10,900,341,which indicates an absorption rate of 57.37 %.

7.0 Monitoring and Evaluations

31. The results framework contained in the Project Appraisal Document (PAD) had seven high level indicators based on PEFA 2011 framework which formed the main benchmark for following through attainment of PDOs. However, throughout Project lifespan, Government did not carry out a PEFA Assessment which posed significant challenges to objectively analyze project performance in this area. While PEFA is a good tool, the project was well-off developing lower level indicators in a timely manner. A comprehensive M&E framework was however not firmly established and the Project did not recruit an M&E Specialist.

32. Nevertheless, periodic performance assessment of the project and the resulting outcomes were however carried out jointly by the World Bank and the MDTF donors and supported by PFMSD. Agreements made from these reviews guided operations of the Project.

8.0 Fiduciary Compliance

33. FROIP finances were managed by PFMSD within the Ministry of Finance, Economic Planning and Development. PFMSD conducted project accounting initially using excel and later on migrated to SAGE Pastel, after assessing risk associated with using excel. Interim unaudited Financial Statements (IFRs) were produced quarterly during the life cycle of the project.

34. The Bank regularly carried out fiduciary assessments and clinics to provide technical support to PFMSD in the area of financial management. Implementation missions regularly rated PFMSD financial management system as ‘Satisfactory’. Apart from the report that was affected by a mis-procurement of IT equipment’s, Auditors issued unqualified audit opinions on FROIP annual statements.

35. With regards to procurement, apart from one mis-procurement, there were no major concerns from project management. Implementation missions, rates procurement as ‘Satisfactory’. The Project was supported by a highly experienced Procurement Specialist.

9.0 Major Implementation Challenges

36. The following are challenges that affected implementation of FROIP:

(a) Plunder of Public Resources: Barely months into Project implementation, massive plunder of Public resources was revealed, this lead to refocusing of the Project so that the identified loopholes could be filled. The exercise required shelving some initially planned activities.

(b) Procurement of refurbished computers: The bidder who was awarded a contract to supply ICT equipment valued at US$ 519,000, supplied refurbished pieces. This led to freezing disbursements to the project until Government had refunded the funds. Implementation was disrupted during this time, as components lost momentum.


Page 51 of 62

(c) Non-implementation of some initially planned activities: this was mainly due to challenges beyond Project management Control (for example, procurement of new IFMIS). These challenges include: litigation, withdraw by the Bank due to limited project duration.

(d) Changes in direction with regards to activities to be included in project work plans: this was done with a view to accommodate new IFMIS procurement, which in the end did not materialize. This lead to late introduction of some project activities, most of which could not be accommodated.

(e) Capacity in PFMS Division for effective coordination: Throughout the project lifespan, PFMSD was handicapped in terms of numbers. The Ministry has however taken deliberate initiative to address this by allocating personnel to PFMS Division.

(f) Changes in Operational Leadership and Staff Turnover: During its lifespan (5 Years), PFMSD which was implementing FROIP had been led by 3 Heads of the Division. This affected focus and continuity of priorities. In addition, transfers of staff in implementing institutions and PFMSD affected continuity of implementation momentum.

(g) Absence of harmonized Reporting Format: FROIP was implemented alongside other projects funded by other DPs including AfDB, who equally required reports from PFMSD. Multiple requirements meant spreading the capacity thin and compromised team productivity.

(h) Failure to Carry out PEFA Assessments: FROIP results framework was benchmarked on PEFA 2011 framework, failure to carry out an assessment implied non-availability of objective tool to objectively assess progress over time.

10.0 Sustainability

37. At the time of ICR preparation, Government was at an advanced stage in the process of developing a successor PFM Reform Strategy, the PFM Rolling Plan, that shall build on and sustain reforms introduced by FROIP. The PFM RP is a three-year plan developed to guide implementation of PFM reforms from 2018-2021.

38. In addition to strategy, capacity building initiatives under FROIP deliberately targeted Government employees who would use the skills acquired beyond the project lifespan, thereby sustaining the reforms introduced.

39. Lastly, legal reforms that had been carried out aims at institutionalizing the reforms that have been introduced. They include; amendment of the Public Audit Act, Review of PFM Act and development of guidelines, procedures, manuals and policies. All, these will go a long way in sustaining the initiative.

11.0 Key Lessons

(a) Use of Country Systems: Experience from FROIP has shown that using country systems is the most effective way to implement reform project and yield sustainability of those reforms. In addition, using Country systems enhance capacity building further strengthening


Page 52 of 62

the PFM systems. The use of Civil Servants in delivering the results under the Project, for example, shall have long lasting effective due to enhanced ownership of the initiatives.

(b) Harmonized Reporting: Lack of harmonized results reporting framework among DPs overstretched capacity in PFMSD. If DPs could harmonize, reporting would have been easier in the sense that staff would not be overstretched.

(c) Project Design: Project design was clear and applicable to the Malawi PFM architecture. However, experience from implementation, points to the need for feasibility or project preparations before embarking on implementation to boost preparedness and avoid leaving other critical activities not implemented (e.g. the case of new IFMIS procurement)

(d) Consistent Project Management: Not only could this yield sustainable momentum, but also accelerate achievement of results.

(e) Project Flexibility: in the environment FROIP was implemented, flexibility was necessary to accommodate other critical activities that would contribute to attainment of PDO. The experience of refocusing project design to accommodate activities aimed at sealing loopholes identified by revelation of plunder of public resources.

(f) Need for lower level indicators: Apart from ambitious PEFA framework, there is need for lower level indicators that could be easily assessed to inform progress on the Project.

Acknowledgements

The Ministry of Finance acknowledges the tireless support that the World Bank Provided during design and implementation of FROIP through various missions that were carried out. Further acknowledgements goes to MDTF DPs for the financial contributions made towards implementation of FROIP. They include DPs as follows; EU, DFID, Irish Aid, Norwegian Embassy and GIZ.

Lastly but not least the Ministry further commends the Implementing Institutions for the dedication towards implementing activities under FROIP, the sustainability of initiatives is your responsibility. Further acknowledgements goes to all members of relevant PFM Governance structures for effective oversight during implementation

Annexes

Annex 1: Summary of Financial Performance by Component

Disbursement Categories & Expenditure

Budget in USD

Cumulative expenditure From 01/04/2013 to 30/06/2018)

Balance on Budget

Absorption


9,502,000.00 5,877,441.04 3,624,558.96 61.85%

Internal Audit 2,470,000.00 1,432,554.36 1,037,445.64 58.00%

External Audit 4,733,000.00 2,024,998.90 2,708,001.10 42.78%


Page 53 of 62

Disbursement Categories & Expenditure

Budget in USD

Cumulative expenditure From 01/04/2013 to 30/06/2018)

Balance on Budget

Absorption

PFEM RP Mgt 1,626,000.00 1,562,791.70 63,208.30 96.11%

Other 50.00 - 50.00 0.00%

Contingency 669,000.00 2,555.00 666,445.00 0.38%

Total 19,000,050.00 10,900,341.00 8,099,709.00 57.37%


Page 54 of 62

Annex 2: Results Framework and Monitoring

PDO Level Results Indicators*

Co

re

Unit of Measure

(PEFA score)

Baseline (2010/11)

Targets

Frequency

Data Source/ Methodo-logy

Responsibility for Data

Collection

Description (indicator

definition etc.)

Status

Dec. 2013

Dec. 2014

Dec. 2015

Dec. 2016

Indicator One: PEFA PI-18

PI-18 B+ B+ B+ B+ A Annual PEFA Self-Assessment

PFEM Unit Predictability and control in budget execution: Effectiveness of payroll controls

Payroll System Devolved. Salary payment timely and with minimal errors- No PEFA Assessment

Indicator Three: PEFA PI-20

PI-20 C+ C+ C+ C+ C+ Annual PEFA Self-Assessment

PFEM Unit Predictability and control in budget execution: Effectiveness of non-salary controls

Indicator Four: PEFA PI-21

PI-21 D+ D+ D+ C C Annual PEFA Self-Assessment

PFEM Unit Effectiveness of internal audit

Scope, quality and coverage of internal audit improved


Page 55 of 62

Indicator Five: PEFA PI-23

PI-23 D D D D+ D+ Annual PEFA Self-Assessment

PFEM Unit Accounting, recording and reporting: Availability of information on resources received by service delivery units

Reports of finances produced and made public through Parliament

Indicator Six: PEFA PI-25

PI-25 C+ C+ C+ C+ C+ Annual PEFA Self-Assessment

PFEM Unit Accounting, recording and reporting: Quality and timeliness of annual financial statements.

Timely production of financial statements prompting timely audits achieved

Indicator Seven: PEFA PI-26 & 28

PI-26 PI-28

D+ D+

D+ D+

D+ D+

C C

C C

Annual PEFA Assessment

PFEM Unit External audit and legislative scrutiny: Scope, nature and follow-up of external audit & Legislative scrutiny of annual audit reports

External audit scope expanded. Public Accounts Committee Trained in legislative scrutiny. Audit backlog cleared.

Indicator Eight Improved service delivery resulting from PFM reform interventions

Baseline study Q 1

& 2 of project period

To be defined during implementation


Page 56 of 62

Annex 3: Summary of Key Activities Implemented Under FROIP

COMPONENT/Sub/Activity

Responsible Progress to Date Remarks/ Further Plans

1.0 ACCOUNTING AND FINANCIAL MANAGEMENT

1.1 Accounting and IFMIS

Progress to Date

Enhancement of network connectivity for improved IFMIS performance

AGD Government Area Wide Network (GWAN) was rehabilitated, contribution to improved network for enhanced performance of central Government IFMIS. GWAN backbone was rehabilitated for reliable network connectivity within all Government Offices in Capital Hill as well as City Centre. As a result of this investment, most MDAs are currently using GWAN internet. The Rehabilitation of GWAN followed a successful assessment of network connectivity that was undertaken with support from the Project. The study revealed several shortfalls, one of them was an obsolete backbone, hence the recommendation to rehabilitate. Further, network equipment was installed in some MDAs including network termination switches at AGD.

For sustainable usage of the fibre backbone, there has arisen a need for maintenance. As this was not envisaged. Further support required for GWAN maintenance consultant Still on improving network connectivity, an upgrade of Local Area Network (LAN) in MDAs is required.

Procurement, Installation and Maintenance of UPSs

UPSs were installed to facilitate reliable power back up at AGD and DHRMD to avoid loss of data. This has been helpful in the face of power challenges that the economy has been facing recently. The Project has further supported the procurement of maintenance contracts for these UPSs.

With the increased number of servers that have been commissioned, there is need to procure additional (bigger size) UPSsIn addition, there is need to procure a heavy-duty UPS for the Disaster Recovery Site (DRS).

1.1.2 Procurement and Implementation of New IFMIS Platform

Procurement of New IFMIS

AGD

FROIP facilitated the review of business processes and development of functional requirements for the new IFMIS platform. Procurement of the New IFMIS was however withdrawn from the Project following queries that were logged by a Bidder who was not satisfied with the results of the process. The issues was taken to court causing further delays in the

Government went ahead with the procurement process for the new IFMIS platform, using the same results.


Page 57 of 62



process.

1.1.3 Implementation of In-year financial Reporting

Implementation of in-year reporting in line with IFRS and IPSAS

AGD

Officers from the Department of Accountant General were trained as trainers for IPSAS. The trained Officers would support Accountants in MDAs in in-year reporting and financial statements production in line with IPSAS and IFRS. The initiative has lead to timely production of financial statements, further facilitating timely audits.

1.1.4 Procurement for sustaining the operations of the current IFMIS

Maintenance of IFMIS and HRMIS Servers

AGD

Government through the Project procured servers to support IFMIS and HRMIS functionality. The servers were commissioned and has since tremendously lead to the improved performance of central Government IFMIS and HRMIS. Performance has improved both at MDA level as well as consolidation level at the Accountant General’s Department. As a way of ensuring that the improved performance is sustained, Service Level Agreements (SLAs) were made with Server maintenance consultants, through the Project.

Additional servers required for HRMIS.

IFMIS Information Security

AGD

Government contracted the services of an Information Security Expert from the Reserve Bank of Malawi. In addition, analytical tools were installed to support security functions and deter the passing of suspicious transactions in a timely manner. As a way of institutionalizing the initiatives, trainings were provided for Government Officers who would assume information security management responsibilities.

Further capacity building activities in information security. Additionally, equipment in form of ACL servers are required to support information security operations.


Page 58 of 62



Capacity building for IFMIS Users

AGD

The current IFMIS was continually supported by Soft-Tech, the consultant who provided the software. As a way of minimizing reliance on the consultant, Government build in-house capacity by providing relevant trainings to identified personnel who support the system. These trainings include, SQL, Microsoft certifications, VEAM and data back-up and recovery and Project management, among others.

Further specialized trainings required to enable Government competently backstop IFMIS operations

1.2 Improved Payroll Management and HRMIS

Decentralization of Payroll Processing

DHRMD

The project supported the decentralization of payroll processing to six local councils, namely: M’mbelwa, Mulanje, Kasungu, Lilongwe, Zomba and Blantyre. Infrastructure in terms of servers, computers, work stations, and printers were installed in these six councils. HR and IT personnel were trained and management sensitized on the reform. Network connectivity in the councils was procured and currently being maintained through MTL Limited.

More sustainable network connectivity options required as MTL would be expensive on the councils. In addition, more equipment is required for efficient processing of salaries in the councils.

HRMIS Maintenance Contract

DHRMD HRMIS was supplied and maintained by Globe Computer Systems. The project financed the contract.

Further support and capacity building required so that

Capacity Building Support to DHRMD

DHRMD Trainings were provide on information Security Personnel handing the payroll system

ICT Equipment for payroll and HRMIS

Servers and UPS were also procured to support operations of payroll and HRMIS. However, these servers are running out of space and the warranty has expired hence the need to procure additional new servers.

Prepare specifications for additional equipment requirements by March 2018.

1.3.1 Strengthen Financial Management in Local Councils through Training


Page 59 of 62



Procurement of equipment for Local Authority IFMIS

NLGFC

Computers were procured for local Councils for enhance IFMIS performance. Going forward, there is need to improve network connectivity. An assessment has been scheduled; following which network equipment will be procured.

Facilitate network connectivity assessment for local councils in March 2018 Initiate procurement of network equipment in April 2018.

Capacity building for Local Councils Management Information Systems Officers (MISOs) and Accountants

NLGFC 17 MISOs undertook certification trainings in database, network and windows with support from the Project. These MISOs are spread across local councils throughout the country.

Further training session on Windows, Database and Network required for the remaining MISOs.

2.0 INTERNAL AUDIT

Improving Institutional set-up of Internal Audit Service

CIAU

Internal Audit Policy was drafted and it awaits submission to cabinet. In addition, earlier on in the project, Independent Audit Committees were revived. The initiative was however not sustained following Government direction to institutionalize Financial Inspectorate Units in MDAs. Another milestone attained with support from the project is the roll-out of Risk based audit approach in the MDAs. In this regard internal audit manual was revised to reflect the developments. Internal audit strategic plan (2016-21) was developed and reporting framework for internal audit was put in place.

With the interventions, the scope of internal audit significantly expanded from the traditional financial audit to ICT. Internal auditors have been able to audit IFMIS and HRMIS in an exercise jointly undertaken with the National Audit Office (NAO). Furthermore, there is enhanced capacity in investigations.

Improving availability of adequate and skilled Internal Audit Staff

CIAU

Various capacity building activities were undertaken to improve the skill in the internal audit service. 57 Auditors were trained in IT audit techniques. 10 Officers on CAATs. Furthermore, 80 officers were trained on Risk management and internal audit planning. 30 auditors were trained in fraud investigation. In addition, capacity building was also provided in the area of quality assurance with 20 auditors trained. In this regard, a quality assurance

Roll out of continuous audit of major ICT systems is planned. This will foster timely detection of erroneous and potentially fraudulent transactions.


Page 60 of 62



framework was put place. With regards to certifications, one auditor was certified in Information Systems Auditor (CISA), six were Certified in Risk Analysis. 7 completed their first degrees and 5 completed their Masters in Audit Management and Consultancy. In order to improve performance of internal auditors, a balanced score card was developed. These tools were developed with technical assistance from Cowater International, a consulting firm that was recruited under the Project. Lastly, an internal audit website was developed.

Equipment Support

CIAU A motor vehicle, 24 laptops, 9 Desktops and 12 printers were procured to support Internal audit operations. In addition, IDEA was procured to support IT audit function in internal audit.

Further equipment is required in view of the reforms that have taken place with regards to ICT audit.

3.0 EXTERNAL AUDIT (NATIONAL AUDIT OFFICE)

Training for Public Accounts committee Members

NAO

With the clearing of audit backlog, it became eminent to provide support for the responsible parliamentary committee, the Public Accounts Committee to discuss the produced reports to facilitate timely follow-up of the findings.

Automation of the Audit Service

NAO AMS and CAATs were installed at NAO, subscription has been made and the systems are being used. This has significantly improved the capacity of NAO in undertaking IT systems audits. However, NAO lacks sufficient computers to cover all auditors and enable them to use the systems. Suffice to say that the Project had in the previous years procured some computers for NAO.

Procure additional computers for auditors to cover all auditors in the four regions. Upgrade AMS for normal users.

Capacity Building for Specialized Audit Skills

NAO IT Audit techniques, Performance audit and forensic auditing. capacity building activities for external auditors was carried out.


Page 61 of 62



Independence of the National Audit Office

NAO

An amendment Act for the public audit was assented to. The bill comprises among other issues, amendment to improve independence of the Auditor General’s Office. This amendment and other activities were mainly produced with support from KPMG which was engaged with support from the project. Make the amendment act effective.

4.0 PROGRAMME MANAGEMENT

Capacity development for effective skills development

PFMSD

The project management component facilitated implementation of project activities in other components. To ensure effective operations, the Project supported trainings of Officers in Procurement, and financial management. In addition, trainings were undertaken in financial analysis to enable PFMSD effectively carry out financial oversight roles for public enterprises.

Facilitating PFM Reform activities

PFMSD

The Division facilitated the development of medium term PFM reform rolling plan. The plan included M&E framework. PFMSD further facilitated activities relating to PEFA assessment that the Government carried out in 2018. In this regard, 9 key Officers in the PEFA assessment process participated in trainings organized by the PEFA Secretariat.

Printing and publication of the Rolling Plan and finalization of PEFA Assessment

Support for Project operations

PFMSD

The project contracted the services of a procurement specialist to support effective management of procurement function for the Project. Further, monitoring activities are undertaken to inform project management if activities are implemented as planned.


Page 62 of 62

ANNEX 6. SUPPORTING DOCUMENTS

Table 6.1. Comparison of the 2011 PEFA Ratings, PDO Targets, Bank Ratings, and 2018 (draft) PEFA Assessment

PDO Indicators Baseline (2011 PEFA) Target Bank Rated (as of 2016) PEFA Actual (2018 draft)

PI-23.4 (formerly PI-18) Improved effectiveness of payroll controls C+ B+ C+ B

PI-25 (Formerly PI-20) Improved effectiveness of internal controls for non-salary expenditures

C+ B C C

PI-26 (Formerly PI-21) Improved effectiveness of internal audit D+ C C D+

PI-29 (Formerly PI-25) Improved quality of annual financial statements

C+ B C D+

PI-30 (Formerly PI-26) Scope nature and follow-up of external audit D+ C C C+

PI-30 (Formerly PI-28) Legislative scrutiny of external audit reports D+ C C C+

Implementation Completion and Results Report (ICR)...

Documents

Transcript of Implementation Completion and Results Report (ICR)...