Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.
-
Upload
spencer-dixon -
Category
Documents
-
view
212 -
download
0
Transcript of Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.
![Page 1: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/1.jpg)
Performance Measures
AmeriCorps Project Director TrainingSaratoga, NY
October 7th – 9th, 2013
![Page 2: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/2.jpg)
Theory of Change
Activity
Performance Measure Basics Overview
Activity/Questions
AGENDA
![Page 3: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/3.jpg)
Theory of Change
![Page 4: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/4.jpg)
INTRODUCTIONTheory of Change Perspective
Looks at cause and effect relationships
Identifies specific interventions to achieve the desired result
Shifts thinking from “what are we doing” to focus on “what do we want to achieve”
![Page 5: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/5.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements
STATISTICS documenting the need
STATISTICS documenting the need
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
![Page 6: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/6.jpg)
Prohibited ActivitiesEveryday Life Example
I have strep throat (problem) I will take antibiotics (intervention) I will get better (outcome)
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
![Page 7: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/7.jpg)
Prohibited ActivitiesEveryday Life Example cont.
EVIDENCE Penicillin Amoxicillin×Tetracycline
EVIDENCE Penicillin Amoxicillin×Tetracycline
There are many antibiotics out thereWhich antibiotic do I take for strep?Look at evidence to make the choice
![Page 8: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/8.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements
STATISTICS documenting the need
STATISTICS documenting the need
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
Community Problem/Need is the specific issue your project, with its specific intervention (service activity), is designed to address. What is the extent and severity of this need in the community?
![Page 9: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/9.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Community Need
Data documenting problem/need should answer these questions:
SCOPE: Who and how many are directly affected? How severe is this?
SIGNIFICANCE: What makes this a compelling need? Is it likely to become worse? What will happen if we do nothing?
CAUSE(S): Why does the need exist? How is it perpetuated?
![Page 10: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/10.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Statistics
STATISTICS documenting the need
STATISTICS documenting the need
The best data comes from reputable primary sources like government agencies, institutions, and universities that do their own research that is: Locally relevant Up-to-date
Note: New reports are not primary sources but could be used as back-up to primary evidence.
Data documenting problems/needs should be statistic that document the extent and severity of the problem.
![Page 11: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/11.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Outcome
STATISTICS documenting the need
STATISTICS documenting the need
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
What change are you hoping to make related to the identified need?
![Page 12: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/12.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Outcome
Healthy Futures Program Example
Possible outcomes to measure: Increased knowledge of what is “healthy food” More frequent choice of healthy foods to eat More frequent involvement in physical activity or exercise Improved physical condition
Identifying the Intended Outcome.
![Page 13: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/13.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Intervention
STATISTICS documenting the need
STATISTICS documenting the need
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
An intervention is the specific set of activities in which participants and volunteers will be engaged. What is the best way to achieve the intended outcome?
![Page 14: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/14.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Intervention
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
Design – who does what with whom? Dosage
Frequency – how many sessions a week? Intensity – length of each session. Duration – how many total weeks of sessions?
Describe the design and dosage of your intervention (service activity):
![Page 15: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/15.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Intervention
Design: national service participants implement the Shape Up curriculum with economically disadvantaged urban girls ages 14-16 to increase physical activity (30 minutes/session) and educate them on healthy eating
Frequency: twice a week afterschool Intensity: 60 minutes per session Duration: 12 weeks
Healthy Futures Program Example
![Page 16: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/16.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Evidence
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
Evidence demonstrates that the proposed intervention (design, dosage) is likely to lead to the outcome.
Service activities that are supported by research to demonstrate a high likelihood that these activities will address the need, and result in intended changes.
![Page 17: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/17.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Evidence
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
Evidence provides a reality check for theory of change
New programs: What is the recommended design (specific program activities) and dosage (frequency, intensity, and duration) to achieve an intended outcome? For existing programs: Is there sufficient evidence for the intervention to continue its use? Based on the evidence, are there modifications to the intervention that should be considered to have greater impact? Do you need to choose a new intervention?
EVIDENCE: Information that supports your choice. It is NOT enough to just say “we believe” our intervention is “likely” to be successful.
![Page 18: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/18.jpg)
Overview of Prohibited ActivitiesTheory of Change Elements - Evidence
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
EVIDENCE •Guides choice of intervention
•Supports cause-effect relationship
Your past performance measurement outcome data
Results from an impact evaluation of your program
Research that documents the outcomes of similar programs
Impact evaluations that document outcomes of similar programs
Possible sources of evidence:
![Page 19: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/19.jpg)
Overview of Prohibited ActivitiesTesting Your Theory of Change
PLAUSIBLE: Does the logic of the model seem correct: “if we do these things, will we get the results we expect?”
FEASIBLE: Are resources sufficient to implement the chosen intervention?
MEANINGFUL: Are intended outcomes important? Is the magnitude of expected change worth the effort?
Is your Theory of Change:
![Page 20: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/20.jpg)
Overview of Prohibited ActivitiesTheory of Change Summary
A theory of change identifies cause/effect
The three elements of a theory of change; community problem/need, intervention, intended outcome are supported by data and evidence
Data documenting community need should show scope, significance, and causes
Summary of Key Points:
![Page 21: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/21.jpg)
Case Study
![Page 22: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/22.jpg)
Performance Measure Overview
![Page 23: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/23.jpg)
Overview of Prohibited ActivitiesPerformance Measures
Accountability to funders and stakeholders Tell your story, justify funding
Recognizing progress; measuring change Reliable information collected in a systematic way
Program improvement Spot and correct problems Strengthen the intervention Determine where to allocate limited resources
Why Measure Performance?
![Page 24: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/24.jpg)
Overview of Prohibited ActivitiesPerformance Measures
Using Performance Measurement in Program Management
![Page 25: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/25.jpg)
Overview of Prohibited ActivitiesPerformance Measures
Using Performance Measurement in Program Management continued
![Page 26: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/26.jpg)
Overview of Prohibited ActivitiesPerformance Measures – Outputs and Outcomes
OUTPUTSAmount of services being
providedo People servedo Products createdo Programs developed
OUTCOMESReflects the changes or
benefits that occur
Can reflect changes in individuals, organizations, communities, or environment
Address changes in attitudes/beliefs, knowledge/skill, behavior, or conditions
![Page 27: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/27.jpg)
Overview of Prohibited ActivitiesPerformance Measures – Outcomes
Outcome Example: EDUCATION
![Page 28: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/28.jpg)
Overview of Prohibited ActivitiesPerformance Measures - Alignment
Aligning Theory of Change with Performance Measures
![Page 29: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/29.jpg)
Overview of Prohibited ActivitiesPerformance Measures - Alignment
Alignment: Logical flow and connection between the different elements.
Alignment helps you:
Strengthen your theory of change
Articulate you performance measures
Measure more accurately
Cleary report change
![Page 30: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/30.jpg)
Overview of Prohibited ActivitiesPerformance Measures - Alignment
Example: Aligned Output-Outcome with Performance Measure
![Page 31: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/31.jpg)
Overview of Prohibited ActivitiesPerformance Measures – Impact Evaluation
Differences Between Impact Evaluations and Performance Measures
Key AreasPerformance Measurement
Impact Evaluation
CausalityAssumes causality; does not “prove” theory of change
Seeks to show causality and “prove” theory of change
ImplementationTracks outputs and outcomes on a regular outgoing basis
May occur from time to time, but not on a regular ongoing basis
Time FocusShorter term focus; what can be measured within one year
Longer term focus
DesignBalance rigor with practicality
Uses most rigorous evaluation design
![Page 32: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/32.jpg)
Overview of Prohibited ActivitiesPerformance Measures- Summary
Performance measurement is a systematic process of measuring progress (outputs and outcomes)
Performance measurement does not seek to “prove” a theory of change, but can provide snapshots.
Impact evaluation can determine if results occurred because of the intervention.
Strong performance measures align with the theory of change (need, intervention and outcome)
Key Points
![Page 33: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/33.jpg)
Overview of Prohibited ActivitiesPerformance Measures- Summary
CNCS Priorities and Performance Measures:
http://www.nationalservice.gov/resources/npm/home
Program-Specific Notices of Funding Opportunities and Application Instructions:
http://www.nationalservice.gov
Additional Resources
![Page 34: Performance Measures AmeriCorps Project Director Training Saratoga, NY October 7 th – 9 th, 2013.](https://reader035.fdocuments.in/reader035/viewer/2022070415/56649e4e5503460f94b44c06/html5/thumbnails/34.jpg)
Project Director TrainingSaratoga, NY
October 7th – 9th, 2013
Questions?