Public Performance Measurement and Public Performance Measurement and Management: From Theory to PracticeManagement: From Theory to Practice

Patricia de Lancer Julnes, Ph.D.Patricia de Lancer Julnes, Ph.D.University of BaltimoreUniversity of Baltimore

Marc Holzer, Ph.D.Marc Holzer, Ph.D.Rutgers University, NewarkRutgers University, Newark

Robert Shick, Ph.D.Robert Shick, Ph.D.Rutgers University, NewarkRutgers University, Newark

International Conference for Administrative Development: International Conference for Administrative Development: Towards Excellence in Public Performance Towards Excellence in Public Performance

Riyadh, Saudi ArabiaRiyadh, Saudi ArabiaNovember, 2009November, 2009

Review of Performance Movement in United States

• Evolution of performance movement• Model of performance process• Performance measurement• Performance-based budgeting• Performance-based management• Approaches to developing a performance system• Challenge to implementing an effective performance

system

Uses of Performance Measurement and Management in the Public Sector• Demonstrate accountability to citizens, elected

officials, and other interested parties by reporting the services efforts and accomplishments associated with a government’s programs and activities

• Improve the resource allocation process by using performance information to inform the development and enactment of a government’s budget

• Assure the achievement of desired results by using performance information to monitor the delivery of services, and make adjustments, if necessary, in the service delivery

Introduction

• Century long effort to define and apply metrics to government services• Driven by management and budget initiatives, PPBS, MBO, TQM, and productivity• Performance measurement broadly defined as ongoing collection of data on program activities and accomplishments• Indicators include: inputs, outputs, and outcomes• Indicators of efficiency, cost-effectiveness, or productivity, ratios of outputs to inputs• Outcomes divided into intermediate outcomes and end-outcomes• Information used for program related decision-making

Evolution of Performance Movement

• New York City, Bureau of Municipal Research, 1906• Brookings Institution predecessor, Institute for Government

Research, whether focus of government should be efficiency, early 1900’s

• Taft Commission 1913, Brownlow Committee, 1st Hoover Commission, Second Hoover Commission

• Nixon National Commission on Productivity, 1970• Carter Quality of Working Life 1977• Clinton National Performance Review 1993• Government Performance and Results Act 1993

Public Sector Performance Process Model

ExamplesMunicipal Function

Input Measures Output/ Workload Measures

Efficiency Measures

Effectiveness measures

Productivity Measures

Explanatory Information

Sanitation The amount of labor-hours of the Sanitation Department, the budget of the Sanitation Department; number of vehicles

Tons of refuse collected; miles of roads cleaned; Number of customers served

Employee-hours per ton of refuse collected; Dollars spent for one mile of snow removal

Percentage of clean streets (e.g., measured by periodical visual inspection; citizen surveys)

Cost per mile of a clean street (i.e., total cost of all road cleaning divided by the total miles of clean streets)

Composition of solid waste; Climatic conditions; Terrain; Crew size of vehicles; Type of vehicles

PM Indicators

Inputs Activities Outputs

Resources dedicated to or consumedby a program.

• funds• staff/staff time• equipment• supplies• facilities

Direct products of program activities, usually measured in terms of volume or work accomplished.

• # permits issued• # brochures mailed• # residents served• # immunizations

PM Indicators

Efficiency Measures:Inputs used per unit of output

Inputs Activities Outputs

PM Indicators

Inputs Activities Outputs

Outcomes

Service Quality

Efficiency

Service Quality:

Degree to which citizens are satisfied with a program, or how accurately or timely a service is provided

- Percent of respondents satisfied with a service - Frequency of repeat repairs- Average days to address a facility work order

PM Indicators

PM Indicators

Outcomes

- Why are we providing this service?- Reduction in fire deaths/injuries- Increased feelings of neighborhood pride- Reduced exposure to safety hazards

A good PM system is…

• Results-Oriented• Reliable• Useful• Quantitative• Realistic• Cost-effective• Easy to Interpret

• Comparable• Credible

Performance Measurement

• Holzer (1977) – demands for improved performance and government accountability have been continually present in the public sector

• Ammons and King (1983) – demands for performance improvement often took a backseat to other pressing government concerns

• Wholey and Hatry (1992) – explained that the value of performance monitoring rests on the information that it provides

Performance-Based Budgeting• Results budgeting or outcomes budgeting• GAO (2001), Melkers and Willoughby (1997), Joyce (1993), and

Grizzle and Pettijohn (2002), Smith (1999), Miller and Robbins (2004), Ngoyi, Miller, and Holzer (2007), and Barnett and Attberry (2007)

• Encouraging• Results budgeting has great potential to improve resource

allocation decisions • Found performance planning a catalyst for setting goals,

objectives, and outcome measures• Performance budgeting has promise to be a truly revolutionary

movement• Performance budgets increasingly required by state legislation

and include: strategic plan, annual performance plan, annual program performance report, and program evaluation

• Budgeting for outcomes hold exceptional promise, and cite steps in budget process including setting measures of annual progress, monitoring, and closing the feedback loop

Performance-Based BudgetingConcerns

• Performance budgeting may be more vulnerable to fraud, falsification, and misrepresentation and shares problems of strategic planning of oversimplification and parochialism, but concedes it has greater philosophical soundness than other budget systems

• Performance budgeting should be approached cautiously, difficult to agree on objectives and most successful projects focused on activities not results

Performance-Based Management• The purposeful use of resources and information to achieve and

demonstrate measurable progress toward agency and program goals• Wholey (1999), Berry and Wechsler (1995), Roberts (2002), Romzek

and Dubnick (2008), Franklin (2000), and Ammons and King (1983)• Encouraging

• Viewed as a way to respond to citizen demands for accountability and productivity

• Report 80% of government manager respondents thought strategic planning important for setting goals, guiding policy, and making budget decisions

• Managers seem to recognize importance of planning and outcomes, but questions what it will take for performance-based management with the required planning and measurement to take hold in public organizations

Performance-Based ManagementConcerns

• New wave of accountability will fail to achieve intended results as current reform neglects agency and personal responsibility and accountability and is deemed as a process of dialogue

• Report of chief local administrators show respondents understand the meaning of performance measurement and management, the top priority concerns they identified were fiscal crisis, capital improvements, and economic development

Performance Used in Federal, State, and Local Government

• Kravchuk and Shack (1996), Berman and Wang (2000), Poister and Streib (1999), and de Lancer Julnes and Holzer (2001)

• Federal level - cautions that formal approaches like GPRA can come at the expense of rational hands-on management and performance measures may misinform if users are not aware of subtle limitations of performance systems

• County level - effectiveness of performance measurement contingent on underlying organizational, political, and technical capacities

• Local level - growing impetus for performance measurement, but rhetoric has outpaced reality, and limited by the extent of commitment to performance measurement has taken hold

• Two stage process – adopting performance measures and implementation

Seven-Step Performance Measurement and Use Methodology

• Step 1. Identification of the Programs to Measure• Step 2. Statement of Purpose• Step 3. Identification of Program Inputs, Outputs,

Efficiency and Productivity Indicators• Step 4. Setting Targets for Accomplishment• Step 5. Monitoring • Step 6. Performance Reporting• Step 7. Analysis and Action

Step 1. Identification of the Programs to Measure

• Programs are groupings of routine activities aimed at providing support for a certain service

• Example, the following four activities--street resurfacing; street patching; seal coating; and curb repairing--are activities that constitute the program that is usually called street maintenance

• Clearly identify and divide activities into distinct programs

Step 2. Statement of Purpose

• One can measure the performance of a program only if he or she knows what the program should accomplish

• Clearly articulate a statement of purpose for a program– a clear mission statement is a good place to start

Step 3. Identification of Program Inputs, Outputs, Efficiency and Productivity Indicators

• Input Indicators – Designed "to report the amount of resources, either financial or other (especially personnel), that have been used for a specific service or program. Input indicators are ordinarily presented in budget submissions and sometimes external management reports.“ GASB

• Output/Workload Indicators - Report units produced or services provided by a program. Workload measures indicate the amount of work performed or the amount of services received.

• Outcome/Effectiveness Indicators - Report the results (including quality) of the service. These measures answer the question How well the organization is meeting its goals.

• Efficiency Indicators- Report whether the desirable ends are achieved with minimal input of resources, or that the resources we employ bring the maximum amount of desirable goal attainment. Efficiency refers to the ratio of the quantity of the service provided to the cost, in dollars or labor, required to produce the service.

• Productivity Indicators - Combine the dimensions of efficiency and effectiveness in a single indicator. The costs (or labor-hours) of faulty meter repairs as well as the costs of effective repairs are included in the numerator of such a calculation, but only good repairs are counted in the denominator--thereby encouraging efficiency and effectiveness of and by meter repair personnel. (Ammons, 1996)

Step 4. Setting Targets for Accomplishment

• Express the goals of a program through date (or period of time) and quantities (including percentages)

• Identify service effectiveness and quality, and explicitly state how we are going to check whether our objectives have been met

Step 5. Monitoring

• Monitoring provides the user with the results needed to decide whether or not the target is accomplished or not

• Each accomplishment target should be monitored on a continuous basis

• Systematic monitoring gives the manager an opportunity to keep tabs on the operation of the program, and take corrective action if requested during the program

Step 6. Performance Reporting

• Performance reports should be an accessible and convenient format including the following: – Program name, organizational jurisdiction, and

statement of purpose– Workloads/outputs, inputs, productivity and/or

efficiency ratios– Targets for accomplishment for crucial indicators,

most notably, effectiveness and efficiency– Brief synopses of additional explanatory information

Prince William County, WashingtonDepartment: Community Improvement

Step 7. Analysis and Action

• A well-developed Performance Measurement System enables users to identify weaknesses and threats, as well as strengths and opportunities

• Gives users the opportunity to diagnose organizational growth capabilities and take relevant actions

First Order Components in a Performance System

• Chief Executive commitment and involvement• Relevant measures of at least outputs and eventually

outcomes• Periodic review and revision of the performance measures• Regular collection of performance data • Comparison of performance data to prior periods• Regular review and analysis of performance results • Chief Executive and other senior management involvement in

the reviews and analysis• Agreement between Chief Executive and department

managers on improvement plans • Follow-up on achievement of improvement plans

Second Order Components in a Performance System

• Targets for performance measures• Comparison of performance data to targets • Comparison of performance data to similar jurisdictions • Support staff involvement in the reviews and analysis of

performance results • Budget reviews and deliberations that consider the

performance targets and results • Budget resources allocated to programs rather than

solely to object classes• Programs for obtaining constituents’ views • Existence of some process for assuring the data’s

reliability• Regular dissemination of performance results

Third Order Components in a Performance System

• Strategic plan for the jurisdiction or departments• Legislative branch involvement in the process• External stakeholder involvement in the process • Cost accounting system and cost of services• Personnel evaluations, promotions, and compensation

adjustments that consider program performance • Involvement of non-government organizations in the process

Challenge

• Insure that the performance measurement and management system is credible, relevant, and useful to the delivery of public sector services through a continued and sustained emphasis by the Chief Executive and Department heads, and to the extent possible, institutionalize the system so it will continue in future administrations