Roundtable:

How Do Internal Evaluators Promote

Effective Stakeholder Engagement?

Kelly Bay-Meyer kbay-meyer@collegesuccessfoundation.org

Hilary Loeb hloeb@collegesuccessfoundation.org

College Success Foundation

American Evaluation Association Conference 2012

Friday, October 26, 2012, 1:45-2:30 PM

Introductions

Our Multiple Roles as Internal Evaluators at CSF

Inspiration for Roundtable

Brief Participant Introductions:

1. Name

2. Organization

3. Role

Multiple Roles of Internal Evaluators

Volkov, B.B. (2011). Beyond being an evaluator: The multiplicity of roles of the internal evaluator. In B.B.

Volkov & M.E. Baron (Eds.), Internal evaluation in the 21st century. New Directions for Evaluation, 132, 38.

Michael Quinn Patton’s

Utilization-Focused Evaluation (U-FE)

Assumption that evaluations should be judged

by their utility & actual use.

Potential to increase user buy-in & better tailor

evaluation methods & instruments to the

specific context.

However, U-FE time-consuming for

stakeholders & may reduce independence of

the evaluator through the personalization of the

evaluation process.

Patton, Michael Quinn. (2008). Utilization-Focused Evaluation (4th Ed.). Los Angeles: Sage Publications, Inc.

--(2002). Utilization-Focused Checklist. Retrieved February 20, 2012 from The Evaluation Center at the

University of Western Michigan, Evaluation Checklists website: www.wmich.edu/evalctr/checklists/.

How we try to adopt U-FE principles &

encourage authentic staff participation? 1. Co-creating logic models & theories of change using customizable templates

How we try to adopt U-FE principles &

encourage authentic staff participation?

2. Drafting survey instruments

How we try to adopt U-FE principles &

encourage authentic staff participation?

3. Appreciative Inquiry sessions

How we try to adopt U-FE principles &

encourage authentic staff participation? 4. Reacting to preliminary analyses

Challenges in promoting U-FE staff engagement:

Time constraints

Rapidly changing priorities in a fast-paced

organization

Valuing importance of each step in the evaluation

process

Changing view of evaluation from compliance

“ apple polishing” to useful learning opportunities

to help make real change

Roundtable Discussion

1. How do you encourage authentic staff

engagement in evaluation?

2. What challenges do you face in promoting staff

engagement in evaluation?

3. What are “ best practices in U-FE” that we would

like to share with the AEA community to

confront the challenges that we have identified?

Concluding Remarks & Next Steps

Summary

Participant email addresses to review U-FE best

practices

Send U-FE best practices to AEA Library

U-FE related resources

Thank you!

U-FE Related Resources

Utilization-Focused Evaluation: Patton, Michael Quinn. (2008). Utilization-Focused

Evaluation (4th Ed.). Los Angeles: Sage Publications, Inc. --(2002). Utilization-Focused Checklist. Retrieved February

20, 2012 from The Evaluation Center at the University of Western Michigan, Evaluation Checklists website: www.wmich.edu/evalctr/checklists/.

Appreciative Inquiry: Preskill, Hallie and Tessie Tzavaras Catsambas. (2006).

Reframing Evaluation through Appreciative Inquiry. Thousand Oaks: Sage Publications, Inc.

U-FE Related Resources

Logic Models & Theories of Change:

Pell Institute Evaluation Toolkit : http://toolkit.pellinstitute.org/

Organizational Research Services: http://www.organizationalresearch.com/publications_and_resources_featured.aspx

“ Getting Started: A Self-Administered Guide to Outcome Map Development” (2009)

“ Orientation to Theory of Change” (2008)

“ How to Manage and Analyze Data for Outcome-based Evaluation” (2000)

Kellogg Foundation:

Evaluation Handbook (2004) http://www.wkkf.org/knowledge-center/resources/2010/W-K-Kellogg-Foundation-Evaluation-Handbook.aspx

Logic Model Development Guide (2010) http://www.wkkf.org/knowledge-center/resources/2006/02/WK-Kellogg-Foundation-Logic-Model-Development-Guide.aspx