©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation...

©Performance Results, Inc.

Developing and Using Logic Models

for Project Planning and Evaluation

NIH Science Education Projects Annual Conference

May 15, 2012

Why Outcomes Matter

Constructive FailureUnconstructive Failure

http://www.philanthropyroundtable.org/topic/excellence_in_philanthropy/failure_in_philanthropy

Results-based Evaluation



Three buckets of data….

Evaluations

Process

PeopleResults

The Evaluation Process Should Answer:

How has my program made a difference?

How are the lives of the program participants improved as a result of my program?


What are Outcomes?

A target audiences’ changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program.


What is a Program?

Activities and services leading towards intended outcomes

Has a definite beginning and end Designed to change attitudes, behaviors, or

knowledge, or increase skills and abilities related to assumed need


Program Purpose

Program purpose is driven by assumptions about need. It relates to the organization’s mission statement. It defines audience, services, and outcomes.

In summary...We do what, for whom, for what outcome or benefit?


What: We provide assessment, goal development and one-to one tutoring

To whom: Adults living in Reynolds County and surrounding area who express an interest and demonstrate a need for our services

For what outcomes: Adults will improve their reading, writing and communication skills and achieve their personal literacy goals

Program Example: Literacy


What: We will establish a community-wide partnership with health professionals, teachers, parents, children, health researchers and the media to provide raise awareness about obesity among children, promote healthy diets and lifestyles, create family health plans and support parents efforts to help their children reach their diet and fitness goals.

To whom: Parents, children Pre-K and K, and community members in Springfield


Fit from the Start

For what outcomes: 1. Children understand the dangers of bad eating

habits and a sedentary lifestyle2. Children improve their diet3. Children adopt an active lifestyle4. Families support one another in maintaining a

healthy lifestyle5. Families create, use and adopt a regular healthy

meal and exercise plan 6. The community demonstrate a greater

understanding of the cost of poor dietary and exercise habits of its citizens

7. The community make a commitment to sustaining programs to help families become and stay healthy and active

Results-based Evaluation Fit from the Start

Logic Models

The Evaluation Plan:

An outcomes logic model (evaluation plan) is a clear, graphic representation of the links between program activities, the results these activities produce, and how the results will be measured.

Inputs Activities Outputs Outcomes Impact

Resources; people,

materials, supplies, money

Program Process

Products “Counts and Amounts”

Knowledge, Skills, Behaviors,

Attitude changes of targeted

population

Community-level change

EXAMPLE 1

Logic Models

Impact Outcome Indicators Inputs Activities

CommunityResults

Intended individual

change

Observable and measurable behaviors or conditions

Resources; people,

materials, supplies, money

Program Process

EXAMPLE 2

Logic Models

Outcome Indicators Applied to Data Source Data Interval Target

Intended individual

change

Observable and

measurable behaviors or conditions

The population(s)

being measured

Information sources

When information is

collected

The #/% who will achieve

the outcome

The Practical Model

Logic Models

Inputs Services OutputsResources;

people, materials, supplies, money

Program Process

Products “Counts and Amounts”

Start the Plan with Outcomes First

A target audiences’ changed or improved skills, attitudes, knowledge, behaviors, status, or life condition brought about by experiencing a program.

Outcomes

Outcomes Indicators Data Source Applied to Data Interval Target

Outcomes are Logical and Sequential

Immediate Intermediate Long-term IMPACT

Outcomes


Outcomes


Fit from the Start1. Children understand the dangers of bad eating habits

and a sedentary lifestyle2. Children improve their diet3. Children adopt an active lifestyle4. Families support one another in maintaining a healthy

lifestyle5. Families create, use and adopt a regular healthy meal

and exercise plan 6. The community demonstrate a greater understanding

of the cost of poor dietary and exercise habits of its citizens

7. The community makes a commitment to sustaining programs to help families become and stay healthy and active

Indicators Measure the Outcomes

State the measurable conditions or behaviors chosen to show an outcome was achieved.

Show what you hope to see or know Are observable evidence of

accomplishments, changes, or gains.

Indicators


Outcome # 1 Indicator:Children understand the dangers of bad eating habits and a sedentary lifestyle

The #/% of children who can identify at least 2 poor eating habits that harm one’s health and

The #/% of children who can identify at least 3 consequences of an inactive lifestyle

Indicators

Outcome # 2 Indicator:

Children improve their diet

The #/% of children who eat 80% of the six food groups at least three times a week


Children adopt an active lifestyle

The #/% of children who exercise or engage in active play for 30 minutes at least 6 times a week

Fit from the Start

Outcome # 4 Indicator:Families support one another in maintaining a healthy lifestyle

The #/% of families who have more than one family member supporting the child in the Fit from the Start program

Indicators


Families create, use and adopt a regular healthy meal and exercise plan

The #/% of families who take an active role in meal planning and choosing foods which are healthy

The #/% of families who plan and have outings and activities at least 4 times a week month

Fit from the Start

Outcome # 6 Indicator:The community demonstrates an understanding of the cost of poor dietary and exercise habits of its citizens

The #/% of community members who accurately complete the online survey on fitness and diet

Indicators


The community makes a commitment to sustaining programs to help families become and stay healthy and active

The #/% of project partners who complete their obligations to this project and

The #/% of project partner who continue their efforts beyond the grant period

Fit from the Start

Data SourcesData captured at critical points in time

Look at indicators to see what data sources are required.

Follow up – Continued Benefit

End – Results - Outcomes

Middle - Progress

Beginning - Baseline


Data Sources

Used to assess:

Behaviors

Life Condition-Life Status

Attitudes

Skills

Knowledge


Where Will the Information Come From?

Data sources are tools, documents, and locations for information that will show what happened to your target audience. Pre-post test scores Program records Assessments Records from other organizations Observations Portfolio review

Data Sources


What kind of data do you need?

Often data sources are categorized into types:

Primary – Information you have collected yourself Secondary – Existing information collected from

someone else Qualitative – Types of information that is describing

quality rather than quantity – answers questions concerning “how”

Quantitative – Information concerning counts, amounts, percentages; answers questions concerning “how much”

Data Sources


Applied to: Target Audience

Who Gets Measured?

The target audience to whom the indicator applies

Decide if you will measure all participants, completers of the program, or another subgroup

Special characteristics of the target audience can further clarify the group to be measured


Data Intervals

When to Measure?

Data intervals are the points in time when data are collected:

Data can also be collected at the end of an activity or phase and at follow-up

Data are usually collected at program start and end for comparison when increases in skill, behavior, or knowledge are expected


Targets

Targets of Performance

Targets, also known as goals, are the stated expectations for the performance of outcomes.

Stated in terms of a number and/or percent Meets influencers’ expectations May be estimated by the program’s past

performance


Logic Models

Inputs:

The resources you will dedicate to your program.

Staff Facilities Equipment Consultants Materials Curricula

Logic Models

Inputs:

Staff – researchers, health professionals, sports and dance teachers, public relations and web development team

Hands-on activities and curricula Healthy Lifestyle curricula Informal science education PSA’s Print media Website

Fit from the Start

Logic Models

Services:

Services are activities, tasks and events that engage the target audience. Services are designed to have a direct effect on program outcomes.

Workshops Programs Mentoring Classes Training Job Try-outs

Logic Models

Services:

Case assessment Diet and exercise plan and monitoring Age appropriate games and activities Field trips Hands-on experiences Dance and sports orientation Website development Campaign to promote healthy diets and

lifestyles

Fit from the Start

Logic Models

Outputs:

A direct program product, typically measured in numbers:

# Participants # Education sessions

# Plans developed # Online surveys taken

# Plans completed # Activities completed

# Field trips # Non-completers

Characteristics of People Served

Age Race Health Status Income Urban vs. Rural Primary Language Transportation Education Level

Logic Models

Reporting

In summary:

We wanted to do what? We did what? So what?

Three buckets of data….

Evaluations

Process

PeopleResults

Success….

Claudia B. HornPerformance Results, Inc.

20203 Goshen Road Suite 314

Gaithersburg, Maryland 20879

www.performance-results.net240-207-3071

[email protected]

www.resultsonline2.net

mailto:[email protected]

©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation...

Documents

Transcript of ©Performance Results, Inc. Developing and Using Logic Models for Project Planning and Evaluation...