Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation...
Transcript of Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation...
![Page 1: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/1.jpg)
Evaluation:
Beginning with the End in Mind
June Gothberg, Ph.D., Director
Career Connections Research Center
Western Michigan University
https://www.linkedin.com/in/jgothberg/
![Page 2: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/2.jpg)
Imagine
Take a moment and clear your mind
Breathe in deeply, you may even want to close
your eyes
Now visualize two people are walking toward
you about a block away.
![Page 3: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/3.jpg)
At first you can’t
see who they are.
As they get closer
you suddenly
realize it’s you
and one of your
students
![Page 4: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/4.jpg)
Imagine
But it’s not today! It’s five years from now
Now think deeply
▪ What are you talking about with this student?
▪ What have they done over the past five years?
▪ What have they accomplished?
▪ How have you helped them to achieve it?
![Page 5: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/5.jpg)
Imagine
If you actually played along and tried this
exercise, you probably got an idea of what’s the
most important accomplishments you’d like to
see happen in the next five years
![Page 6: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/6.jpg)
Did you envision this? Or this?
![Page 7: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/7.jpg)
Working?
![Page 8: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/8.jpg)
Living?
![Page 9: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/9.jpg)
Still learning?
![Page 10: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/10.jpg)
Hobbies?
![Page 11: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/11.jpg)
Vacations and fun?
![Page 12: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/12.jpg)
Dating and marriage?
![Page 13: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/13.jpg)
Maybe even parenting?
![Page 14: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/14.jpg)
Imagine
These are your longer-term outcomes and
maybe even your impact
This is “beginning with the end in mind”
According to Steven Covey, all things are
created twice — first in the mind, and then in
the real world
![Page 15: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/15.jpg)
How do we get there?
Now we have a vision, an end in mind, what’s
next?
First, we need assess where we are at now, then
▪ we need to create a plan
▪ we need to act on that plan
▪ we need to evaluate how we did
15
![Page 16: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/16.jpg)
Process
This is simply a continuous improvement process
16
![Page 17: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/17.jpg)
Where are you now?
17
![Page 18: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/18.jpg)
What is your plan?
18
![Page 19: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/19.jpg)
What are your actions?
19
![Page 20: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/20.jpg)
What are your actions?
20
![Page 21: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/21.jpg)
Process
Assessment Plan Actions your
What?
21
![Page 22: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/22.jpg)
Evaluation: What are your outcomes?
22
![Page 23: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/23.jpg)
Process
Evaluation can also your
What?
If you only evaluate ‘what’ you did
23
![Page 24: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/24.jpg)
Process
If you want to get to an impact stage, your
Evaluation should your
So What?
So what happened because of ‘what’ you did?
In other words, did you make a difference?24
![Page 25: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/25.jpg)
Process
If you answer the What? And then you answer the
So What? Then you can ask the
Now What? In other words, what are our next steps?
How do we improve, grown, and ultimately
make an impact?25
![Page 26: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/26.jpg)
Some ah-has from your plans and
reports
Most of you are capturing the “what”
A few of you are capturing the “so what”
Hardly any are getting to the “now what”
26
![Page 27: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/27.jpg)
Why?
There seems to be some confusion
27
![Page 28: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/28.jpg)
Why?
There seems to be some confusion between
28
![Page 29: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/29.jpg)
Goal vs. Tasks
Goals are the results you want to achieve
Tasks are the actions you take to support
your goals
29
![Page 30: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/30.jpg)
Outputs vs. Outcomes
Outputs are tangible – what you ‘put out’ as a
result of your tasks
▪ Services you offer
▪ Products you create or provide
▪ Events you implement
Outcomes are the end result, the change(s) you
expect to see
30
![Page 31: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/31.jpg)
Evaluation – When?
During most of these phases
Evaluations can be formative or summative
(ongoing vs. at completion)
“When a cook tastes the soup, it is formative
evaluation; when the dinner guest tastes the soup,
it is summative evaluation.” Harvey, 1998
31
![Page 32: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/32.jpg)
Evaluation – How?
Quantitative vs. Qualitative
▪ Quantitative tends to be based on numbers and
percent
▪ Qualitative tends to be focused on words and
experiences
Informative evaluations that can be used in the
What, So What, Now What process usually
include both 32
![Page 33: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/33.jpg)
Evaluation – How?
33
![Page 34: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/34.jpg)
Evaluation – Design
One area that tends to go unnoticed or
unthought of even in our world is using inclusive
designs to our evaluations
If we want rich data that guides the continuous
improvement process, we need to ensure our
evaluations are accessible, understandable, and
usable to all
34
![Page 35: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/35.jpg)
Evaluation – Design
Remember that Universal Design (UD) is the
design of products and environments to be
usable by all people, to the greatest extent
possible, without the need for adaptation or
specialized design.
35
![Page 36: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/36.jpg)
Evaluation – Design
Universal design principles include
1. Equitable use
2. Flexibility in use
3. Simple and intuitive use
4. Perceptible information
5. Tolerance for error
6. Low physical effort
7. Size and space for approach and use
36
![Page 37: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/37.jpg)
Evaluation – Design
In your materials today I’ve included the
Universal Design for Evaluation Checklist 4th Ed.
37
![Page 38: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/38.jpg)
Evaluation – Design
You may find this checklist useful for planning
your evaluations
▪ Is the space I plan to collect data accessible?
▪ Is the time of day aligned with transportation?
▪ Are the materials readable for the participants?
▪ Piloted with like participants?
▪ Accessible for low vision? Color-blindness?
38
![Page 39: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/39.jpg)
Evaluation – Design
You may find this checklist useful for planning
your evaluations
▪ Is the space I plan to collect data accessible?
▪ Is the time of day aligned with transportation?
▪ Are the materials readable for the participants?
▪ Piloted with like participants?
▪ Accessible for low vision? Color-blindness?
39
![Page 40: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/40.jpg)
Why is Evaluation Important?
What gets measured gets done
If you don’t measure results, you can’t tell
success from failure
If you can’t see success, you can’t reward it
If you can’t reward success, you’re probably
rewarding failure …..
40
![Page 41: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/41.jpg)
Why is Evaluation Important?
If you can’t see success, you can’t learn from it
If you can’t recognize failure, you can’t correct
it
If you can demonstrate results, you can win
public support
From: Osborne & Gaebler, 1992, Chapter 5, “Results Oriented Government”
41
![Page 42: Evaluation: Beginning with the End in Mind Beginning with... · 2020. 12. 9. · Evaluation –When? During most of these phases Evaluations can be formative or summative (ongoing](https://reader036.fdocuments.in/reader036/viewer/2022081410/609edbfc8c2e2b72f45af065/html5/thumbnails/42.jpg)
Discussion and questions