Module 5.2

Measuring the performance of PFM systems

INTRODUCTION TO PUBLIC FINANCE MANAGEMENT (PFM)

Module map

• PFM Assessment tools• PEFA Operating Design Principles• Comprehensiveness of PEFAs• PEFA scoring

• NB. there is a dedicated course on PEFA assessments

Module outline

3

Why PFM assessment tools

• Concerns about achievement of development objectives

• Fiduciary and accountability concerns

• Need to link governance with development

• Move towards aid modalities using national PFM procedures

• Need to measure progress

• Inform decision-making on amount, timing and aid modality

• Alignment of processes

• Build capacity and ownership of reform agenda

PFM Assessment

PFM Reform Strategy

PFM Reform action plan

Capacity Development

Toolkit

Do you know a tool to assess a PFM system

PER

CFAA

Performance Audit

PEFA

Compliance Audit

PETSMAPSDeMPA

Guidance on use

FRA - CFAA

MAPS

PETS

PER

PEFA - ROSC

DEMPA

Core features of PFM assessment tools

OECD Methodology for Assessing Procurement Systems (MAPS)

How does the procurement work?• Background

• Limited coverage of procurement issues in PEFA• Need for a detailed procurement tool and

consistent approach

• Introduced in 2006 following the PEFA approach• 12 indicators, 54 sub-indicators• Base Line Indicators (BLIs)• Compliance/Performance Indicators (CPIs)

• Allows to track progress over time

Debt Management Performance Assessment (DeMPA)

How is public debt managed? • Background

• Limited coverage of debt management in PEFA Framework• No insight into the underlying causes of weak performance

• Methodology designed by WB 2006-2009• Guideline revised in 2009

• DeMPA modelled on PEFA• 6 core functions• 15 indicators, 35 dimensions

• Allows to track progress over time

Concluding remarks

•Choice of tool depends on specific needs

•Coordinate assessment as much as possible to avoid assessment fatigue

•Assessment = platform for dialogue versus pass/fail exam

•Country leadership / ownership essential

• PFM Assessment tools• PEFA Operating Design Principles• Comprehensiveness of PEFAs• PEFA scoring

• NB. there is a dedicated course on PEFA assessments

Module outline

13

• Focus on high level performance indicators applied to the central government level

• Widely applicable: relevant to countries at all levels of development

• Assessment must be evidence based• Capable of calibration to capture progress over time • Use data that can be collected cost effectively• Comprehensive: cover all aspects of the PFM cycle• Define the achievement of sound PFM to be based upon the

three PFM objectives

14

The PEFA PMF: Operating Design Principles

• Specify six critical dimensions of a sound PFM system as:• Credibility of the budget• Comprehensiveness and transparency of the budget• Policy based budgeting• Predictability and control in budget execution• Accounting, recording and reporting• External scrutiny and audit

• Incorporate a measure of the impact of donor practices on PFM performance

15

The PEFA PMF: Operating Design Principles

Six core dimensions:• Credibility of the budget• Comprehensiveness and transparency• Budget Cycle

• Policy-based budgeting• Predictability and control in budget execution• Accounting, recording and reporting• External scrutiny and audit

16

The PEFA PMF: Operating Design PrinciplesAn open and orderly PFM system that supports the 3 PFM objectives: Aggregate fiscal discipline

Strategic allocation of resourcesEfficient service delivery

Key elements of operational PFM performance:• 31 performance indicators: 28 for governments and 3 for donors

17

PEFA Model

Scope of the PFM PMF

18

The Central Government Budget

19

PEFA Framework, transparency and corruption

•A comprehensive and proper PFM system makes it harder to divert or misuse funds

•PEFA framework:

• Assesses whether real basics of PFM system are operating well (‘getting the basics right’)

• Provides technical summary of PFM system

• Less focusing on political and cultural aspects relevant for corruption

• Core focus on ‘What’ question, not on ‘Why’ of ‘How’‘what’: identification of weaknesses in PFM system

NOT: ‘why’: insight in causes of budgetary dysfunctions

‘how’: how can these causes be tackled?

No in-depth analysis of underlying causes

No reform recommendations made in PEFA Performance Report

• PFM Assessment tools• PEFA Operating Design Principles• Comprehensiveness of PEFAs• PEFA scoring

Module outline

20

Comprehensiveness of the PEFA PMF

•

21

PEFA PFM Functions IndicatorsStrategic Planning Budget Formulation Budget Preparation Budget Execution

Budget Release Cash Management Debt Management Commitment Control Expenditure Control

Payroll Management Procurement Supply Chain Management No

Internal Audit Financial Reporting and Accounting External Audit Legislative oversight

PEFAFinancial Administrative Network IndicatorsBanking Network NoPostal and Communications Network NoData Network NoElectricity access and reliability No

PEFAPFM Context ReportPolitical Context - accountability and transparency NoLegal and regulatory framework

Institutional arrangements

macroeconomic and economic context

Fiscal framework

Degree of donor harmonisation and reform support

Presence of an organised and engaged civil society NoInformed and active press No

PEFAPFM Capacity IndicatorsRatio of qualified procurement staff to number of procurement units No

Turnover rate of qualified procurement staff No

Ratio of qualified accounting staff to number of spending units No

Turnover rate of qualified accounting staff No

Ratio of qualified internal audit staff to number of spending units NoTurnover rate of qualified internal audit staff No

Ratio of qualified audit staff to number of spending units No

Turnover rate of qualified external audit staff No

Number of technical staff supporting the PAC No

Number of technical staff supporting the Budget Committees No

Ratio of computerised spending units No

Ratio of computerised procurement units No

Ratio of computerised budget offices No

PFM

PEFAPFM Technical Infrastructure IndicatorsThe Budget Instrument Vote book/commitment control ledgers Treasury Single Account NoAccounting Standards Book Keeping Standards (double entry, single entry) NoLinked posts, personnel and payroll database Procurement planning, procurement thresholds, tender evaluation NoWarehouse inventory management NoFixed Asset Register NoAudit Standards PFM Software No

The PEFA Iceberg – how comprehensive?

22

PEFA Assessment

Fixed asset register

Supply chain management

Financial administrative network

Capacity

ProcurementPolitical context

Market

Quality of expenditure management

Engaged civil society

• PFM Assessment tools• PEFA Operating Design Principles• Comprehensiveness of PEFAs• PEFA scoring

Module outline

23

• Calibration and Scoring

• Specific calibration of scores using a four point ordinal scale (A, B, C and D)

• Intermediate scores (B+, C+, D+) possible only for multi-dimensional indicators, where dimensions score differently

• Arrow ▲ can indicate an improvement not reflected in change of indicator score

• Two scoring methods:

• Method M1 ‘weakest link among dimensions’

• Method M2 ‘average of dimensions’

PEFA Scoring

Interdependencies

25

• PFM linkages are analogous to foundation footing linkages – the strength of a foundation is determined by it weakest footing

• That is the reason why the Tower of Pisa leans. The performance of a PFM system is determined by the strength of its weakest PFM activity

• As a general rule PFM scores, like the strengths of foundation footings, cannot be averaged

Measuring PFM system performance: a comparison of tools

26

PER CFAA CPAR ROSC EC Audits FRA OECD/DAC NPA PEFA PMFReview of public expenditure policies and budgetary outcomes X

Review of political incentives X XHigh Level overview of PFM performance X XComprehensive review of PFM functions XIdentification of PFM strengths and weaknesses

X X X X X X X X

Compliance Testing X X XIn-depth analysis of capacity factors X X X X

Recommendations for reform X X X X XAssess risk to public funds X XTrack Progress over time X X

CFAA: Country Financial Accountability Assessment (WB)CPAR: Country Procurement Assessment (WB)ROSC: IMF’s Report on Fiscal Standards and Codes (ROSC)FRA: Fiduciary risk assessment (DfID)NPA: National Program for Action

• Movements in scorings may vary only slowly over time

• The proper rating being based upon the lowest sub-indicator scoring (M1 method) will contributes further to slow changes in indicator scorings

• A number of factors may contribute to a sub-indicator so that even though there may be significant improvements these may not be reflected in a changes score. (see for example PI-17 (i))

Consequences of applying High Level Indicators for managing support

27

Budget Support and PEFA

28PFM Progress over time

Measurements of progress over time using PEFA

Requirements for demonstrated progress to facilitate BS disbursements

Agreed reviewable milestones

Key messages

• The PEFA Framework measures the performance of PFM systems in achieving their objectives

• It generally focuses on the “basics” of PFM systems

• It is used to monitor progress in strengthening systems

• It does not deal with policy issues

• There is a dedicated PEFA course