Focusing Evaluation Getting ready. Focusing: Making Ready Stages of Evaluation Evaluability...
-
Upload
oswald-clark -
Category
Documents
-
view
216 -
download
0
Transcript of Focusing Evaluation Getting ready. Focusing: Making Ready Stages of Evaluation Evaluability...
Focusing: Making Ready
• Stages of Evaluation• Evaluability
– Stakeholders– Standards
• Target Indicators– Outcome indicators– Process indicators
• Evaluation Management
Focusing: Making Ready for Process
• Stages of Evaluation: What’s ahead?– Planning process– Focusing: Identifying context, questions, targets– Designing study methods– Selecting study tools– Collecting data– Interpreting data– Using results
Focusing our own process
• Stages of Evaluation: Where are you?
How often do you thinking about what to evaluate after the program is over?
a) Pretty typical
b) Somewhat typical
c) Not typical
d) Never wait until after
e) What evaluation?
Focusing: Making Ready for People
• Stakeholders: Interests and Readiness– Sponsors (STEM careers; mentors from field)– Teachers (STEM achievement; collaboration)– Parents (STEM achievement; volunteerism)– Youth (STEM hands-on; participation)
Not whether it’s evaluation-ready,
but what evaluation it’s ready for.
Focusing: Making Ready for Program
• Standards: Investments and Readiness– Utility (credibility, relevance, collaboration)– Feasibility (management, viability)– Propriety (principled, facilitative)– Accuracy (sound and scientific)
Not whether it’s evaluation-ready,
but what evaluation it’s ready for.
Focusing: Ready Questions
• Evaluation Question:
What do we want to know about a program?
– Outcome questions: What does it do?
– Process questions: How does it work?
Focusing: Making Ready Indicators
• Target Indicators
How do we know a program is worth its salt?
– Outcomes: KASA (Short-term); BA (Long-term)
– Processes: Program Quality, Fidelity
Focusing: Making Ready Quality
• Target Indicators:
Quality First
– Program (Delivery) Processes• In Preparation: Credibility, Consistency• In Process: Engagement, Guided Inquiry
Focusing: Making Ready Outcomes
• Target Indicators: Now & Later
– Program Outcomes for Individuals• In Preparation: Credibility, Consistency• In Process: Content (subject matter concepts); and
Process (inquiry, creativity, problem solving)• In Prospect: Current activities that contribute to
future outcomes
Focusing our own goals
• Evaluation Targets: Where are you?
How often do you check program quality or fidelity before collecting data on outcomes?
a) Pretty typical
b) Somewhat typical
c) Not typical
d) No need to check
e) Fidelity to what?