Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference...
-
Upload
kasey-hamley -
Category
Documents
-
view
213 -
download
1
Transcript of Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference...
![Page 1: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/1.jpg)
Southeastern Association of Educational Opportunity Program Personnel
38th Annual Conference
January 30 – February 3, 2010
Upward Bound Upward Bound Internal & External Program Internal & External Program
EvaluationEvaluation
![Page 2: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/2.jpg)
PresenterPresenter
Elizabeth C. Retamozo, Executive Director
NOSOTROS Education Center NOSOTROS Education Center www.nosotrosedu.org
Phone: 469-450-7137
Email: [email protected]
2
![Page 3: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/3.jpg)
Presentation MaterialsPresentation Materials
• PowerPoint Slide Presentation & Handout
• Located at: www.nosotrosedu.org– Go to Resources Page– Register Your Email Address– Click the Presentations & Links Tab– Click SAEOPP 2010 Folder – UB Internal & External Program Evaluation
3
![Page 4: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/4.jpg)
4
Types of TRIO Evaluation PlansTypes of TRIO Evaluation Plans
Outcome-Based Evaluation
Goals-Based Evaluation
Process-Based Evaluation
![Page 5: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/5.jpg)
5
Common Evaluation Plan TerminologyCommon Evaluation Plan Terminology
Qualitative Values Expressed by impressions, feelings & perceptions
Ex. Surveys, Interviews, Focus Groups, etc.
Quantitative Values Expressed in numerical terms, counted, or
compared on a scale
Ex. Grade Point Averages, Test Scores, etc.
![Page 6: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/6.jpg)
6
Common Evaluation Plan TerminologyCommon Evaluation Plan Terminology
Formative Methods Continuous (daily, weekly, monthly, quarterly,
semester) assessment to track progress
Ex. Monthly Reports
Summative Methods End of year assessment that measures the
results or outcomes of a Project
Ex. Annual Performance Report
![Page 7: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/7.jpg)
7
Sample TRIO Evaluation PlansSample TRIO Evaluation Plans
Outcomes Based Evaluation
• External & Summative
Looks at impacts/benefits/changes to your
students as a result of your project’s efforts after
their participation in your Project (PE
OBJECTIVES).
![Page 8: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/8.jpg)
8
Sample TRIO Evaluation PlansSample TRIO Evaluation Plans
Goals Based Evaluation
• Internal & Formative
Evaluates the extent to which projects are
meeting predetermined internal goals or
objectives.
![Page 9: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/9.jpg)
9
Sample TRIO Evaluation PlansSample TRIO Evaluation Plans
Process Based Evaluation
• External & Summative)
Geared to fully understanding how a Project works
How does it produce that results that it gets?
![Page 10: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/10.jpg)
10
Outcome Based Evaluation Outcome Based Evaluation
Outcomes Based Evaluation (External & Summative)
Measures performance on the four (4) established UB Program objectives by the end of each year (summative).
DE Mandated Objectives = Prior Experience
![Page 11: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/11.jpg)
11
Outcome Based Evaluation Plan Outcome Based Evaluation Plan
1) Identify Major Outcomes (Mandatory Objectives)
2) What Types Of Data Will Be Collected?
3) When Will Various Types Of Data & Reports Be Collected?
4) What Instruments Will Be Developed?
5) How Will The Data Be Analyzed?
6) How Will The Data Collected Determine Accomplishment?
![Page 12: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/12.jpg)
12
Outcome Based Evaluation Outcome Based Evaluation
UB Outcome Based Evaluation Plan
![Page 13: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/13.jpg)
13
Goals Based Evaluation Goals Based Evaluation
Goals Based Evaluation (Internal & Formative)
Evaluates the extent to which Project is meeting predetermined internal goals
Project Developed INTERNAL GOALS based on DE Objectives
Includes quantifiable & qualitative measures to track progress during the Project year (formative)
Monitored and reviewed weekly/monthly
![Page 14: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/14.jpg)
14
Goals Based Evaluation Goals Based Evaluation
1) What Project goals were established?
2) What is the status of the project’s progress towards achieving the goals?
3) Will the goals be achieved according to the timelines specified in the program implementation or operations plan?
4) Do personnel have adequate resources (supplies, facilities, training, etc)?
![Page 15: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/15.jpg)
15
Goals Based Evaluation Goals Based Evaluation
5) How should priorities be changed to focus more on achieving the goals?
6) How should timelines be changed?
7) How should goals be established in the future?
![Page 16: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/16.jpg)
16
Goals Based Evaluation Plan Goals Based Evaluation Plan
1) Develop Internal Goals based on External Objectives
2) Activities & Services Needed to Achieve Each Goal
3) Project Staff Responsible to Carry Out Activities & Services
4) Data that Needs to be Collected
5) Instruments (Forms) that Need to be Developed and Collected
![Page 17: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/17.jpg)
17
Goals Based Evaluation Plan Goals Based Evaluation Plan
6) When is Data Collected?
7) How is Data Collected Analyzed?
8) Was the Goal Met? Yes/No
9) Needed or Recommended Revisions
10) Unanticipated Outcomes
![Page 18: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/18.jpg)
18
Outcome Based Evaluation Outcome Based Evaluation
Sample UB Internal Goals
UB Goals Based Evaluation Plan
![Page 19: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/19.jpg)
19
Process Based EvaluationProcess Based Evaluation
Process Based Evaluation (External & Summative)
Geared to fully understanding how a project works
How does it produce the results that it gets?
Usually External Conducted by Program Evaluator
![Page 20: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/20.jpg)
20
External EvaluationExternal Evaluation
• Recommended in last grant proposal competitions
• Must be conducted if included in grant proposals
• Allowable cost for the project
• May be paid by the grantee agency/institution
• May be provided free of charge – in-kind/donation
![Page 21: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/21.jpg)
21
External EvaluationExternal Evaluation
• May be conducted by person working/employed by agency/institution BUT not for the Project
• Must have experience conducting evaluations for educational opportunity programs & TRIO like programs
• Must state what type of evaluation will be conducted by the evaluator.
![Page 22: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/22.jpg)
22
Process Based Evaluation Plan Process Based Evaluation Plan
1) How does the Project identify, recruit, & select participants?
2) How does the Project decide what specific services & activities to provide to its participants?
3) How does the Project monitor participants’ academic progress?
4) How does the Project deliver services to its participants?
![Page 23: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/23.jpg)
23
Process Based Evaluation Plan Process Based Evaluation Plan
5) How does the Project document services provided to participants?
6) What is the Project’s goals based evaluation plan?
7) What is the Project’s outcome based evaluation plan?
8) How does the Project plan for the next program year?
![Page 24: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/24.jpg)
24
Process Based Evaluation Process Based Evaluation
UB Process Based Evaluation Checklist
External Process Based Evaluation Report
![Page 25: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/25.jpg)
25
Basic Guide to Program Evaluation
Written by Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Copyright 1997-2007.
Adapted from the Field Guide to Nonprofit Program Design, Marketing and Evaluation.
Online Guide: A Basic Guide to Program Evaluation.pdf
Bibliography & ResourcesBibliography & Resources
![Page 26: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/26.jpg)
26
Basic Guide to Goals-Based Evaluation for Nonprofit Organizations with Very Limited Resources
Written by Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Copyright 1997-2007.
Online Guide: Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources.mht
Bibliography & ResourcesBibliography & Resources
![Page 27: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/27.jpg)
27
US Department of Health & Human Services Administration for Children & Families - Office of Planning, Research & Development
The Program Manager's Guide to Evaluation
ACF OPRE The Program Manager’s Guide to Evaluation website.mht
This guide explains program evaluation -what it is, how to understand it, and how to do it. It answers your questions about evaluation and explains how to use evaluation to improve programs and benefit students, families, and staff.
Bibliography & ResourcesBibliography & Resources
![Page 28: Southeastern Association of Educational Opportunity Program Personnel 38 th Annual Conference January 30 – February 3, 2010 Upward Bound Internal & External.](https://reader038.fdocuments.in/reader038/viewer/2022110304/551ab3d855034656628b5187/html5/thumbnails/28.jpg)
Questions & AnswersQuestions & Answers
28