EfW and its environmental i mpacts – reaching a conclusion without sitting on the fence
E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G...
-
Upload
conrad-todd -
Category
Documents
-
view
214 -
download
2
Transcript of E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G...
![Page 1: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/1.jpg)
EVALUATION PERSPECTIVES:LOGIC MODELS, IMPACTS AND
BUDGETING
2 0 1 1 S Y S T E M S S C I E N C EG R A N T S M A N S H I P W O R K S H O P
U S D A N AT I O N A L I N S T I T U T E O F F O O DA N D A G R I C U LT U R E
AU G U S T 9 , 2 0 1 1Dr. Gary J. SkolitsTiffany L. Smith
Institute for Assessment and Evaluation
865-974-2777
![Page 2: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/2.jpg)
Institute for Assessment and Evaluation, University of Tennessee 2
Evaluation Presentation Topics
1. Logic model basics
2. Documenting project impacts
3. Evaluation do’s and don’ts
4. Evaluation budgeting perspectives
![Page 3: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/3.jpg)
Institute for Assessment and Evaluation, University of Tennessee 3
1. Logic Model Basics
![Page 4: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/4.jpg)
Institute for Assessment and Evaluation, University of Tennessee 4
1. Logic Model Basics
A logic model is a visual depiction of
how a project intervention is
expected to produce a desired
outcome
![Page 5: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/5.jpg)
Institute for Assessment and Evaluation, University of Tennessee 5
General Logic of a Project
Social Need (Problem)
Project Intervention (Action)
Results (Change)
![Page 6: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/6.jpg)
Institute for Assessment and Evaluation, University of Tennessee 6
Logic Model Drivers
Needs Project Interventions Results
Needs Purpose Inputs
Outputs Outcomes
![Page 7: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/7.jpg)
Institute for Assessment and Evaluation, University of Tennessee 7
Logic Model Elements
A. Needs
B. Purpose
C. Inputs
D. Outputs
E. Outcomes
![Page 8: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/8.jpg)
Institute for Assessment and Evaluation, University of Tennessee 8
A. Needs
What problems deserve attention?
oAlways competing needsoSelect critical need to addressoDefine the need
![Page 9: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/9.jpg)
Institute for Assessment and Evaluation, University of Tennessee 9
B. Purpose
What do you seek to accomplish towards addressing this problem?
oSelect aspects of the problem to addressoDefine the specific purpose of your project
![Page 10: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/10.jpg)
Institute for Assessment and Evaluation, University of Tennessee 10
C. Inputs
What resources will you invest?
oMoney
oPeople
oMaterials
oEquipment
oPartners
![Page 11: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/11.jpg)
Institute for Assessment and Evaluation, University of Tennessee 11
D. Outputs
What will you do?
oServices
oTraining
oProducts
Whom will you reach?
oClients (must define this group)
oTypically multiple stakeholders
![Page 12: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/12.jpg)
Institute for Assessment and Evaluation, University of Tennessee 12
E. Outcomes
What results will you achieve?
Short-term (initial impact on participants)
Medium-term (ongoing impact on participants)
Long-term (impact on the problem)
![Page 13: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/13.jpg)
Institute for Assessment and Evaluation, University of Tennessee 13
2. Documenting Project Impacts
![Page 14: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/14.jpg)
Institute for Assessment and Evaluation, University of Tennessee 14
Outcomes – Short-Term
Feedback
Awareness
Commitment
![Page 15: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/15.jpg)
Institute for Assessment and Evaluation, University of Tennessee 15
Outcomes – Medium-Term
Knowledge/Skills
Disposition
Behavior
![Page 16: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/16.jpg)
Institute for Assessment and Evaluation, University of Tennessee 16
Outcomes - Long-Term
Achievement of project purpose
Favorable change in the initial need of concern
![Page 17: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/17.jpg)
Institute for Assessment and Evaluation, University of Tennessee 17
Impact: A Theory of Change
Short -term: If participant reaction is positive, then you theorize that participants are likely to learn desired knowledge, skills, and attitudes.
Medium-term: If participant learning/growth occurs (i.e., knowledge, skills, and dispositions), then you theorize that participants will change their behavior.
Long-term: If behavior changes, then you theorize that there will be a positive change in terms of the initial need.
![Page 18: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/18.jpg)
Institute for Assessment and Evaluation, University of Tennessee 18
Logic Model Template University of Wisconsin-Madison Extension
![Page 19: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/19.jpg)
Institute for Assessment and Evaluation, University of Tennessee 19
EVALUATION: check and verify
What do you want to know?
How will you know it?
PLANNING: start with the end in mind
![Page 20: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/20.jpg)
Institute for Assessment and Evaluation, University of Tennessee 20
3. Evaluation Do’s and Don’ts
![Page 21: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/21.jpg)
Institute for Assessment and Evaluation, University of Tennessee 21
A Few Evaluation Do’s
Talk with colleagues sponsoring similar projects for leads on good evaluators
Engage an evaluator with a good reputation and experience with your type of evaluation need
Engage an evaluator early in project planning
Understand the role of formative and summative project evaluation
Recognize the importance of efficient, reliable data collection
Build evaluation data collection into project operations (when possible)
![Page 22: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/22.jpg)
Institute for Assessment and Evaluation, University of Tennessee 22
A Few Evaluation Dont’s
Be intimidated by evaluation – it is meant to enhance your project
Fail to communicate with your evaluator on a regularly scheduled basis
Forget to address IRB concerns – protect yourself and your clients/stakeholders
Miss the opportunity to use project and evaluation data to add to the literature in your field
![Page 23: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/23.jpg)
Institute for Assessment and Evaluation, University of Tennessee 23
4 . Evaluation Budgeting Perspectives
![Page 24: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/24.jpg)
Institute for Assessment and Evaluation, University of Tennessee 24
4. Budgeting for Evaluation
Some factors to consider
Evaluation costs depend on the required effort of the evaluator
A general rule: 5% to 8% of project budget
Be prepared to negotiate with your evaluator to minimize evaluation costs
![Page 25: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/25.jpg)
Institute for Assessment and Evaluation, University of Tennessee 25
Budgeting for Evaluation
The more data project staff collect, the lower the evaluation costs. (However, some data should be collected by the evaluator.)
Link any proposed evaluation budget to specific evaluation plan and tasks.
Stay on top of the deliverables promised by your evaluator.
![Page 26: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/26.jpg)
Institute for Assessment and Evaluation, University of Tennessee 26
Two Key Resources
University of Wisconsin-Madison (Extension)http://www.uwex.edu/ces/pdande/evaluation/evallogicmodel.html
Kellogg Foundation Logic Model Guide http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx
![Page 27: E VALUATION P ERSPECTIVES : L OGIC M ODELS, I MPACTS AND B UDGETING 2011 S YSTEMS S CIENCE G RANTSMANSHIP W ORKSHOP USDA N ATIONAL I NSTITUTE OF F OOD.](https://reader030.fdocuments.in/reader030/viewer/2022032803/56649e205503460f94b0b647/html5/thumbnails/27.jpg)
Institute for Assessment and Evaluation, University of Tennessee 27
Gary Skolits [email protected]
865-974-2777
Institute for Assessment and EvaluationUniversity of Tennessee, Knoxville
1122 Volunteer Boulevard; A503 Bailey Education ComplexKnoxville, TN 37996
http://web.utk.edu/~iae/