“The Vocabulary of Performance”

ICGFMDecember 2-4, 2009

CONFIDENTIAL AND PROPRIETARYAny use of this material without specific permission of The World Bank is strictly prohibited

Jim BrumbySector Manager, Public Sector & Governance

2

Table of Contents

• A Transition of Complexity

• Defining Performance

• Inputs, Outputs, Outcomes, Indicators

• Output and Indicator Characteristics

• Is our concern solely with outputs?

• Results Chain

• Basic Concepts: Performance Budgeting

• Objectives of Performance Budgeting

• OECD Practices in Performance Budgeting

A Transition of Complexity

Control & complianceo Rules governedo Limited flexibilityo Non-preventiveo Negative focuso Control of individuals

Performance & Accountabilityo Mission driveno Customer driveno Employee responsiveo Flexibleo Preventiveo Positive team focus

A significant cultural & institutional change

Performance is for the most part determined by where you sit– As a purchaser of a service, your interests are typically different from those of the supplier of the service

o Buyer of car; versus manufacturer of car– As a regulator, your interests are typically different from those of the regulated

o Compliant behavior versus lightness of burden– In some cases, parties may have interests as both purchasers and suppliers (e.g. co-operatives)

Government performance is complex in part because of the variety of interests– Standard problems of government production

With different degrees of clarity, performance budgeting systems attempt to address these various interests

– Institutional fabric is importanto Westminster; Presidential; other Parliamentary etc

Gauging Interests

3

Defining Performance

4 SOURCE: Produced in http://www.wordle.net/.

Inputs, Outputs, Outcomes, Indicators

Inputs: the resources used by government to produce outputs. • Inputs include the labor (the range of skills, expertise and knowledge of employees), capital assets

(including land and buildings, motor vehicles and computer networks), financial assets, and intangible assets (such as intellectual property) which are used in delivering outputs.

Outputs: the goods or services (usually the latter) which government agencies provide for citizens.

• The concept of outputs is not confined to tangible goods and social services delivered directly to citizens. The concept also include more intangible flows of influences on the surroundings from agencies, institutions and other entities delivering on public policies.

• Outputs are potentially largely controllable by government agencies and measurable either quantitatively or qualitatively. Thus outputs can be used for performance management more easily than outcomes.

5

Outcomes: the effects on society of outputs from governmental entitiesOutcomes – whether intended or unintended – are not usually completely controllable by governments. The degree of control depends on the influence of extraneous factors on the goal in question, the effectiveness of implementation and the quality of the policies for reaching the goal.

Indicator: Quantitative or qualitative variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to actions undertaken.

Contextual Factors

Structural, institutional,

and managerial

arrangements

Results Chain (Production Process)

6

Efficiency: indicates the relation between the deployed resources (input) and the delivered products or service (output) (I/O).

Effectiveness: refers to the cause and consequence relation between output and outcome and the extent to which the intervention’s objectives were achieved.

Glossary of Key Terms in Evaluation and Results Based Management. Available from: http://www.aidharmonization.org/download/212150/glossary_of_aid_monitoring_terms.pdf

Source: Dries Verlet and Carl Davos, “Evaluation of performance in the public sector: end or mean?” EGPA Study Group on Evaluation, 2008.

Inputs ActivitiesIntermediate OutcomesOutputs

Final Outcomes

Output and indicator characteristics

Quantity the quality or standard of output expected

Quality the quantity of output to be delivered

Cost the cost of output delivery

Timeliness the timeframe for delivery of the output

Location the physical location where the output will be delivered

Externally focused The output describes a distinct product or service.

Controllable Output specifications appropriately reflect the extent of control over the delivery of each output.

Comprehensive The outputs should incorporate all of the core outputs that were delivered by the entity. This includes outputs where delivery has been subcontracted to another party. It also may include all outputs funded by parties other than ratepayers or taxpayers — for example, outputs funded through user charges.

Measurable Output specifications provide clear descriptions and standards against which actual performance can be judged.

Informative Output descriptions are sufficient to meet the information needs of a relatively new or unfamiliar user of the information.

7

8

Basic Concepts – Performance Budgeting

Performance Budgeting – performance impact of resource allocations and use in order to hold line ministries accountable and efficiently allocate resources.

Weakest form

Moderate form

Strongest form

• Presentational performance budgeting Instrument of transparency

• Performance informed budgeting Instrument of accountability Assists with efficiency Begins more earnest efforts to define, measure, improve

• Direct performance budgeting An unambiguous contractual mechanism

Key Point: One system may have all these, depending on the characteristics of the production process, the nature of the services being supplied, and the preferences and capabilities of the main players

Objectives of Performance Budgeting

9

• Create stronger links between allocated public resources and outputs/outcomes of the public sector

• Impact on decisions in resource allocation by using performance information • Produce and report relevant performance information

• Assist three objectives of public expenditure management 1. Allocative efficiency by challenging the composition of public expenditure with respect

to social needs and priorities 2. Operational efficiency by encouraging cost-effective service delivery and value for

money 3. Aggregate fiscal discipline by affecting the level of public expenditure through

allocative and operational efficiency

The logic is inexorable:

Many countries aspire to performance budgeting;

implementation is a journey rather than a

destination.

Key message: Using performance information in budget decision-making ex-ante (expected targets) and/or ex-post (actual results)

OECD practices in performance budgeting:Spectrum of models

Program budgeting Establish program results along

with funding requirements through the budget process

Disaggregate them to the front-line level

Program classification based on multiple objectives of individual government agencies ( generally corresponds to organizational boundaries)

France, Korea, U.S.

Program budgeting Establish program results along

with funding requirements through the budget process

Disaggregate them to the front-line level

Program classification based on multiple objectives of individual government agencies ( generally corresponds to organizational boundaries)

France, Korea, U.S.

Internal-hierarchical Contractual

Pressure on agencies to boost performance

Improve allocative decisions within agencies

The agency model Specified outcome/output targets linked to

the budgetPSAs (UK), decentralized results framework

(Netherlands, Norway)

Purchaser-provider budgeting Budget agencies are competitive suppliers

of the public sector outputs (quasi-market) Explicit performance contracts with

specified quantity, quality and price of outputs

Australia, New Zealand

Formula-based budgeting Resources are allocated according to a

specified formula Usually used in sectors (e.g. health)

Pressure on agencies to boost performance

Improve allocative decisions within agencies

The agency model Specified outcome/output targets linked to

the budgetPSAs (UK), decentralized results framework

(Netherlands, Norway)

Purchaser-provider budgeting Budget agencies are competitive suppliers

of the public sector outputs (quasi-market) Explicit performance contracts with

specified quantity, quality and price of outputs

Australia, New Zealand

Formula-based budgeting Resources are allocated according to a

specified formula Usually used in sectors (e.g. health)

Performance measures benchmarking program evaluation expenditure reviews CBA

Performance measures benchmarking program evaluation expenditure reviews CBA

Gauge

nonfinancial

performance

Various interests As funder/purchaser As regulator As owner

Need intertemporal consistency

10

End

Thank You.

11