1.1.4 Presentation
Transcript of 1.1.4 Presentation
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 1/21
“Values-engaged Evaluation”
Jennifer GREENE, Univers i ty o f I l l ino is at Urbana-Champaign
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 2/21
VALUES-ENGAGED EVALUATION Webinar Series on Equity-Focused Evaluation
Jennifer C. Greene
University of Illinois atUrbana-Champaign
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 3/21
Welcome
• Importance of this conversation on equity in
evaluation
• Values-engaged evaluation
• Case examples
• Discussion
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 4/21
On valuing in evaluation
• Evaluation intrinsically involves judgments of quality
• Evaluation is thus inherently imbued with values
• But values are rarely named or claimed in
evaluation
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 5/21
On valuing in evaluation (con’t)
Evaluation that is … • Utilization-oriented
• Accountability-focused
• Impact
• Culturally responsive
• Democratic
• Critical
Values advanced … • Use
• Accountability
• Causal knowledge
about outcomes
• Context, respect for difference
• Social justice, equity,voice
• Structural critique
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 6/21
On valuing in evaluation (con’t)
Where do values show up in evaluation planning andpractice?
• Everywhere
• Most importantly:
− Purpose and audience, and intended use(s)
− Key evaluation questions
− Criteria for judging quality
− Communication and reporting (form and content)
− The social relational aspects of the evaluator’s role in context
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 7/21
Values-engaged evaluation• Values-engaged – describes the plurality of stakeholder
values
• Values-committed – prescribes engagement with
particular values inclusion and equity
• Inclusion of all legitimate stakeholder standpoints
• Equity of access, participation, and accomplishment for
all, especially those least well served in that context
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 8/21
Values-engagement in practice
• Evaluation purpose and audience− Enhanced contextualized understanding of promising practices,
especially for those underserved
− Primary audiences of local stakeholders, including intended
beneficiaries
• Intended uses
− Learning
− Program improvement (greater equity)
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 9/21
Practice (continued)
• Key evaluation questions− Appropriate questions on design quality,
implementation quality, and outcome
attainment (each inclusive of multiplestandpoints)
− Question on equity of program access,
experience, and accomplishment
In what ways and to what extent has the programserved to advance the interests and well being of those least well served in this context?
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 10/21
Practice (continued)
• Criteria for judging quality ~ 3 domains
− Quality of program design
− Contextual power of the program design
− Advancement of the interests of underrepresented andunderserved groups
Equity in program access, experiences, andaccomplishments
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 11/21
Practice (continued)
• Communication and reporting
• Evaluator role and relationships
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 12/21
Example 1: Hillside High Math
Program• Context: Regional high school serving multiple rural
areas
• Purpose: “This evaluation aims to investigate the
structure and content of the math pathways at Hillside
High, with a focus on equity of access to and learning
opportunities in mathematics across the whole diversity
of the student body, toward deeper program
understanding and improvement.”
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 13/21
Hillside High (con’t)
• Evaluation questions: “How and with what rationale are
math pathways at Hillside High structured? In what ways
does the structure support equitable opportunities to
learn across the diversity of the school’s student
population?”
• Criteria: “A good public high school math program
affords all students meaningful opportunities and
instruction needed to succeed.”
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 14/21
Hillside High (con’t)
• Communications and reporting
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 15/21
Lower level math classes
On the whole, students in
these classes weredisconnected, disengagedand/or disruptive. Theatmosphere of the classroomwas one of getting through theclass with as little trouble as
possible. Student disruptionswere continuous and constant,and as such, a considerableamount of class time wasspent on the teacher makingrepeated and persistent efforts
to ‘discipline’ acting-outstudents, telling them to “sit in[their] own seats, stop talking,focus and/or pay attention”…
Higher level/honors mathclasses
In general, the majority of the
students in these classes werewell behaved, attentive and
fully engaged, which in turn
brought about interactive and
collaborative-competitive class
atmosphere. Most studentswere highly attentive to the
day’s lesson... For the most
part, the teacher had no
problems eliciting students’
participation in the class, or getting them to stay on task
and complete the
assignments…
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 16/21
Example 2: WWW Access
• Context: A hypothetical technology education
program designed to enhance the individual and
collective “high-tech” capacity of rural residents and
their villages in selected Central American countries.The program is funded by the Gates Foundation.
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 17/21
WWW Access (con’t) • Purpose and audience: Assess service to those most in
needed, include diverse perspectives, toward deep
program understanding and equitable program
enhancements
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 18/21
WWW Access (con’t)
• Quality criteria considerations:
1. Quality of program design – basis in relevant theory and
research
2. Contextual power of program design – “showing up”meaningfully in peoples’ lives
3. Equity of access, experience, and accomplishment
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 19/21
For further information …
• Greene, J.C., Boyce, A., & Ahn, J. (2011). A values-
engaged, educative approach for evaluating educational
programs: A guidebook for practice. University of Illinois.
(available in the American Evaluation Association e-
library, http://www.eval.org)
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 20/21
Discussion ….
7/30/2019 1.1.4 Presentation
http://slidepdf.com/reader/full/114-presentation 21/21
THANK YOU!