Planning for Performance Planning for Performance Measurement: Measurement:

Reviewing the Logic Models Reviewing the Logic Models and Program Areas and Program Areas

Presenters:

Marcia Cohen, DSG, Inc.

Heidi Hsia, OJJDP

What Is Performance What Is Performance Measurement?Measurement?

It is directly related to program goals and objectives.

It measures progress quantitatively.It is not exhaustive.It provides a “temperature” reading—it may

not tell you everything you want to know but provides a quick and reliable gauge of selected results.

Measurement vs. EvaluationMeasurement vs. Evaluation

Impact evaluations are broader and assess the overall or net effects—intended or unintended—of the program as a whole.*

Scope

Time

Process evaluation

Outcome evaluation

Impact evaluation

Performance Measurement

Eval

uatio

n

Outcome evaluations investigate whether the program causes demonstrable effects on specifically defined target outcomes.*

Process evaluations investigate the process of delivering the program, focusing primarily on inputs, activities, and outputs.*

* Evaluation definitions excerpt from: Trochim, William M. The Research Methods Knowledge Base, 2nd Edition. Internet WWW page, at URL: <http://trochim.human.cornell.edu/kb/index.htm> (version current as of Aug. 02, 2000).

Program Monitoring

Outputs Versus OutcomesOutputs Versus Outcomes

Outputs are products of program implementation/activities.

Outcomes are benefits or changes as a result of the program. There are two types of outcomes:

Short-term: The first benefits or changes experienced and the ones most closely related to program outputs.

Long-term: Link a program’s short-term and long-term outcomes. Often they are changes in practice, policy, decision-making, or behavior.

With regard to the Formula Grants and Title V performance measures, to summarize: Outputs are at the micro level and reflect program-level activity.

Outcomes are at the macro level and, when aggregated, will reflect Federal outcomes.

A good performance measurement system should be results oriented and focus on desired outcomes, less on outputs.

Outputs Versus Outcomes (cont’d.)Outputs Versus Outcomes (cont’d.)

Outcome Measure DefinitionsOutcome Measure Definitions

Short-term: Occurs during the program or by the end of the program.

Long-term: Occurs 6 months to 1 year after program completion.

Logic Model: TemplateLogic Model: Template

A graphic representation that clearly lays out

the logical relationships between the problem

to be addressed, program activities, outputs,

and outcomes.

Activities

Outcomes • Short term

• Long term

OutputsProblem

Juvenile Delinquency

Overall Formula Grants and Title V Programs Logic Model

Problems

OUTCOMES

Key

= System-level indicator

= Program-level indicator

Outputs

Juvenile Justice System Improvement Program Areas:19, 23, 31, 33

Core Requirements Program Areas:6, 8, 10, 17, 28

Activities

Increased system

capacity

Improved planning and development

Improved program quality

Improved monitoring of compliance

Implement processes

Implementing Title V programs for keeping at-risk youth from offending or first-time, nonserious youth out of the JJ system.Program Areas:3, 4, 9, 10, 11, 12, 13, 15, 16, 18, 20, 21, 22, 25, 26, 27, 32, 34

Implementing Formula Grants prevention and intervention programsProgram Areas:1, 2, 3, 4, 5, 7, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 22, 24, 25, 26, 27, 29, 30, 32, 33, 34

Improved youth outcomes

Improved policies, procedures, operations, staffing, service delivery

Short Term Long Term

Improved juvenile justice systems

Increased compliance with Core Requirements

Reduced recidivism

Increased accountability

Outcome measures are for illustrative purposes only and are not comprehensive. To see a comprehensive list of outcomes, refer to the individual program area logic models.

To support both State and local prevention and intervention efforts and juvenile justice systems improvements

Objectives

To improve juvenile justice systems by increasing compliance with the Core Requirements and increasing availability and types of prevention and intervention programs

Goals

Title V programs are for keeping at-risk youth from offending or first-time, nonserious youth out of the JJ system.

Title V has 18 Prevention and Early Intervention Program Areas:

3, 4, 9, 10, 11, 12, 13, 15, 16, 18, 20, 21, 22, 25, 26, 27, 32, 34

Title V Program AreasTitle V Program Areas

Three Formula Grants Program Area categories:

Prevention and Intervention Program Areas:1, 2, 3, 4, 5, 7, 9, 10, 11, 12, 13, 14, 15, 16, 18, 20, 21, 22, 24, 25, 26, 27, 29, 30, 32, 33, 34

JJ System Improvement Program Areas:19, 23, 31, 33

Core Requirements Program Areas:6, 8, 10, 17, 28

Formula Grants Program Areas Formula Grants Program Areas