Program Evaluation: What is i t?
description
Transcript of Program Evaluation: What is i t?
![Page 1: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/1.jpg)
Brad Rose, PhD.
Program
Development &
Funding
Data Collection
& Outcome
Measurement
Feedback &
Continuous
Improvement
Impact &
Assessment
Reporting
![Page 2: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/2.jpg)
Definition Program Evaluation
• Is an applied research process• Systematically collects, analyzes and interprets data• Addresses activities, characteristics, and outcomes
of programs
![Page 3: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/3.jpg)
Goal of Program Evaluation
To assist stakeholders in making data-informedjudgments about a specific program:
• Impact• Effectiveness• Value
![Page 4: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/4.jpg)
How Does Evaluation Differ from Research?
• Research is an investigation that seeks to find out what is.
• Evaluation is an investigation into how, why, and to what extent objectives or goals are achieved.
• Evaluation is research that compares what is with what should be. It makes a judgment against criteria, expectations, standards.
![Page 5: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/5.jpg)
What is a Program?
• Structured, intentional, intervention to improve the well-being of people, groups, organizations, or communities
• General effort that mobilizes staff and resources toward some defined and funded goals
• Programs vary in size, scope, duration, clarity, and specificity of goals
![Page 6: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/6.jpg)
Basic Purposes/Kinds of Evaluations• Formative Evaluation
Program improvement and strengthening.
• Summative Evaluation (Outcome Evaluation)Ultimate value, merit and impact. • Should the program continue? Be refunded, etc.?
![Page 7: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/7.jpg)
10 Evaluation Questions General questions to consider when conducting a program evaluation:
1. Why is the evaluation being conducted? 2. What is the "it" that is being evaluated? 3. What are the intended outcomes of the program?4. What is the program meant to achieve?5. What are the key questions to be answered? 6. Who is the evaluation for?
![Page 8: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/8.jpg)
10 Evaluation Questions (cont.)
1. Which information will stakeholders find useful?2. How will evaluation findings be used? 3. What will be the “product” of the evaluation?4. What are the potential sources of information data?5. What is the optimal design for the evaluation?6. Which methods will yield the most valid, accurate,
and persuasive evaluation findings and conclusions?
![Page 9: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/9.jpg)
Typical Methods and Tools• Interviews (Individual and Focus Group)• Surveys • Observations• Review of existing data• Collection and statistical analysis of quantitative data
![Page 10: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/10.jpg)
Evaluation Design Quantitative
Non-experimental design: Pre- and post-test, “a single group interrupted time series”
(Observation Treatment Observation)
Experimental design: Compare outcomes among “treatment” and control groupsRandom Observation Treatment Observation
Observation No Treatment Observation
![Page 11: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/11.jpg)
Evaluation Design Qualitative
•Interviews/focus groups with participants (staff and community members)•Observations of program activities•Document analysis•Case studies•And more…
![Page 12: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/12.jpg)
Evaluation Design• Form follows function• Design follows questions
The key to evaluation designThe evaluation design should be determined by the
kind of questions you want to answer.
![Page 13: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/13.jpg)
Review: The Fundamental Questions • What will be changed or different as a result of the
operation of the program?• Attitudes• Knowledge• Behavior • Competencies/Skills
• What will success “look like”?• How will we show that changes occurred? (i.e., which
measures/indicators?)
![Page 14: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/14.jpg)
“If you can‘t say it clearly, you don’t understand it yourself” -- John Searle
“Things should be made as simple as possible, but not any simpler.” -- Albert Einstein
“82.7% of all statistics are made up on the spot.” --Steven Wright
![Page 15: Program Evaluation: What is i t?](https://reader035.fdocuments.in/reader035/viewer/2022062410/56816048550346895dcf6ee4/html5/thumbnails/15.jpg)
Questions?On-line Resources: www.bradroseconsulting.com 617-412-4709