Using Performance Measurements to Evaluate Service Delivery and

ResourcesProposed Suggested Guidelines for Voluntary

Reporting, Service Efforts and Accomplishments Reporting

The opinions expressed in this presentation are those of Mr. Bean. Official positions of the GASB are established only after extensive public due process and deliberation.

• What the project is:– Focus on voluntary reporting– Focus on suggested guidelines– Focus on clarifying GASB’s role

• What the project is not:– Establishing performance measures– Establishing performance benchmarks– Establishing reporting standards– Requiring SEA reporting in any form

SEA Reporting—Scope

Sole Focus Of GASBEfforts

GOVERNMENT PERFORMANCE MANAGEMENT SYSTEM

1

EXTERNAL

REPORTING

Internal

Reporting

Evaluating Performance

Managing Work Processes

Performance- Based

Budgeting

Selecting

Performance Measures

Program or

Activity Planning

Strategic Planning

Government Performance Management

System

3

Suggested Guidelines for Voluntary Reporting, SEA Performance Information composed of three parts:

Four essential components of an effective SEA reportSix qualitative characteristics that are appropriate for

reporting SEA performance informationA discussion of how to effectively communicate SEA

performance information

4

Suggested Guidelines

5

Suggested Guidelines

Suggested guidelines, although voluntary, are intended to assist governments in improving the quality of their reported SEA performance information

Traditional financial statements provide information about fiscal and operational accountability but not the degree to which the government was successful in helping to maintain or improve the well-being of its citizens by providing services

Provide guidance to assist preparers of SEA reports in effectively communicating SEA performance information to users

Purpose and ScopeMajor Goals and ObjectivesKey Measures of SEA PerformanceDiscussion and Analysis of Results and

Challenges

FOUR ESSENTIAL COMPONENTS

6

ESSENTIAL COMPONENTS AND USER NEEDS

7

Essential Components User Needs

Purpose and Scope The knowledge of why the information is being reported and

what portion of a government the information relates to.

Major Goals and Objectives

A basis for determining whether the reported SEA performance measures are relevant measures of what a government intended to accomplish (that is, the measures are associated with the goals and objectives of the program or service).

Key Measures of SEA Performance

The ability to focus on the key measures of SEA performance that are most important to report users.

Discussion and Analysis of Results and Challenges

The ability to understand what has been achieved and what has affected the level of achievement.

8

PURPOSE AND SCOPE

Purpose-

Why a government is publishing the report

Intended audience of the report

What the SEA performance information is intended to communicate

How the reported SEA performance information can assist users in assessing performance and making decisions – Are they operating efficiently and effectively?

How SEA performance information may be used for assessing resource allocation decisions and whether these decisions are consistent with the goals and objectives

9

PURPOSE AND SCOPEScope-

What departments, agencies, programs, or services are

included

Reasons why they were chosen for inclusion

Period covered by the report

Date the report was issued

Ways to contact the government to obtain more informationIdentifies the level of detail within the SEA report and the reasons why a government has decided to report at that level

* Overview─city as a whole* More detailed level─by neighborhood

What level of assurance, if any, was obtained on the SEA performance information

10

MAJOR GOALS AND OBJECTIVES

Major Goals and Objectives-

State the major goals and objectives of the programs and services being reported or what those programs or services are intended to accomplish

State the sources of the major goals and objectives

Explain the link between the major goals and objectives and the measures being reported

11

MAJOR GOALS AND OBJECTIVES

*Also recognizes that depending on users needs, SEA performance information may be presented in other ways, such as by themes, which require a government to integrate results across the organizational structure.

12

KEY MEASURES OF SEA PERFORMANCE

Key Measures of SEA Performance-

Limited number of measures to focus on information that is essential for assessing the level of performance of programs and services in comparison to what it had planned to achieve

Provide users with enough information to develop their own conclusions without overwhelming them

The number and type of key measures may vary depending on the level of reporting

Key measures can be identified based on the following questions:

13

KEY MEASURES OF SEA PERFORMANCE

Key Measures of SEA Performance-

Key measures can be identified based on the following questions:

Does the measure report on results for a major goal or objective?Does the measure address an issue receiving considerable public discussion?Does the measure report information considered important by elected officials, citizens, or experts in the field?Does the measure report information about a program or service that accounts for a large proportion of a government’s use of resources?Does the measure report information about a program or service that affects a large proportion of the population or has potentially significant economic, social, or environmental effects?

14

DISCUSSION AND ANALYSIS OF RESULTS AND CHALLENGES

Discussion and Analysis of Results and Challenges-

Provides an objective explanation of the results being reported

Highlights the major and critical results being reported─both positive and negative

Includes managements understanding of the reasons why the actual results differ from the expected or intended results

Addresses currently known facts and circumstances that could affect results in the future

15

DISCUSSION AND ANALYSIS OF RESULTS AND CHALLENGES

Discussion and Analysis of Results and Challenges-

Discusses the major challenges a government is facing in achieving its goals and objectives

Discusses the consequences (both intended and unintended), and the results (positive or negative) of providing services

Provide further guidance in the application of the essential components—assist users in comprehending and assessing government programs and services

RelevanceUnderstandabilityComparabilityTimelinessConsistencyReliability

SIX QUALITATIVE CHARACTERISTICS

16

QUALITATIVE CHARACTERISTICS AND USER NEEDS

17

Qualitative

Characteristics

User Needs

Relevance The ability to assess the level of accomplishment of a government’s goals and objectives with potentially significant accountability or decision-making implications.

Understandability SEA performance information that is readily comprehendible.

Comparability A clear frame of reference for assessing the SEA performance of a government and its agencies, departments, programs, and services.

Timeliness SEA performance information issued while it is still of value in assessing accountability and making decisions.

Consistency A basis for comparing similar SEA performance information over time.

Reliability SEA performance information that is verifiable, free from bias and a faithful representation of what it purports to represent.

18

RELEVANCE

In order for information presented within an SEA report to be relevant, it needs to:

include information that is capable of making a difference in a user’s assessment of performance (a problem, condition, or event)

have a close logical relationship to the purpose for which it is intended to be used

The relevance of SEA performance information depends on the various user needs for assessing performance and making decisions

For example- Consider the appropriate level of aggregation or disaggregation necessary to represent the performance of different geographical areas

19

RELEVANCE

Determining if information is relevant can be based on the following questions:

Does the SEA performance information present the major goals and objectives of the programs and services being reported?

Does the SEA performance information provide a basis for assessing the level of accomplishment of the major goals and objectives?

Do the major goals and objectives relate to concerns that are important to users?

20

UNDERSTANDABILITY

In order for information presented within an SEA report to be understandable, it needs to:

be expressed simply and clearly

include explanations and interpretations to help users comprehend the information

be communicated in different forms and at different levels of detail so that the information can be understood by those who may not have a detailed knowledge of a government’s programs and services

Governments need to obtain feedback from users of an SEA report in order to enhance the understandability of the reported SEA performance information

21

COMPARABILITY

In order for information presented within an SEA report to be comparable, it needs to:

provide a basis and context for users to assess SEA performance

provide users with the ability to make an assessment of whether the reported SEA performance information is improving, deteriorating, or remaining the same

The types of comparative information reported may depend on issues such as the availability of reliable and relevant information, the purpose of the report, and the needs of the users.

22

COMPARABILITY

Comparable information may include:

time series comparisons─compares results of 2 or more periods

No basis for assessing whether results are at an acceptable level

comparisons with established targets─compares to results anticipated

Targets can be too high or too low

comparisons to industry standards

comparisons to other similar governmentsSignificant differences in measures or circumstances need to be noted and understood by users

23

TIMELINESS

In order for information presented within an SEA report to be timely, it needs to:

be provided to users before it loses its value for assessing accountability and affecting decisions

Timeliness alone does not make information useful, but the passage of time generally diminishes its usefulness.

24

CONSISTENCY

In order for information presented within an SEA report to be consistent, it needs to:

include measures reported in the same way over timecan compare performance and monitor trends over several years

If SEA performance measures are modified or replaced or the manner of presentation is changed, then it is important to communicate to users that a change has taken place and the reasons for the change.

25

RELIABILITY

In order for information presented within an SEA report to be reliable, it needs to:

contain information that is verifiable to provide assurance that the information in an SEA report would be replicated by independent evaluators using the same measurement methods

information needs to be derived from systems producing verifiable data independent verification is ideal system control reviews conducted internally program staff or director evaluations

contain information that is objective and has therefore not been modified by the government to either overstate or understate positive or negative results

26

RELIABILITY

contain information that is comprehensive in coverage or thorough in its presentation of the measures

significant data is not omitted from the information that is necessary to faithfully represent major results nothing is included that would cause the information to be misleading

contain information that is a faithful representation of the results or represent what actually occurred

In order for information presented within an SEA report to be reliable, it needs to:

27

EFFECTIVE COMMUNICATIONEFFECTIVE COMMUNICATION

Provide further guidance on the effective communication of SEA performance information

Intended Audiences Forms of Communication Multiple Levels of Reporting

28

INTENDED AUDIENCES

In order to effectively communicate SEA performance information to users, the intended audiences need to be considered

citizens elected officials appointed officials investors and creditors others

The types of decisions made by potential audiences influence the information that they need in an SEA report.Potential audiences need different types of information and understand and analyze SEA performance information in different ways.

29

INTENDED AUDIENCES

Involving users in the process of identifying: what SEA performance information is valuable and interesting to them how they prefer to receive the SEA performance information

may assist preparers in effectively communicating their results in an SEA report.

30

FORMS OF COMMUNICATION

The effective communication of SEA performance information is, in part, dependent on the forms used to communicate that information

Forms of communication: printed material mailed to households and businesses portable document format (PDF) files on the Internet extensible business reporting language (XBRL) enhanced files on the Internet presentations made in person articles in the print media news segments in the electronic media

31

MULTIPLE LEVELS OF REPORTING

An SEA report that communicates effectively contains different levels of detail so that users can find their appropriate and desired level of detail

Levels of detail can be presented in many ways within one document several separate documents organized by service area

32

MULTIPLE LEVELS OF REPORTING

An SEA report needs to be organized in a hierarchical structure that proceeds through levels (layers) from overview to detail, and needs to have clearly identified links between the levels.

overview introductory summary information about specific programs and services as a whole performance data that is broken down to provide measures of performance about specific strategies or activities within programs and services

Web Sites—www.gasb.org and www.seagov.org

Questions?