1

Chapter 3 - SAEOPP Priority 1 TrainingChapter 3 - SAEOPP Priority 1 Training

SHOW & TELL:SHOW & TELL:

TRIO PROJECT EVALUATIONTRIO PROJECT EVALUATION

Presenter: Elizabeth C. RetamozoPresenter: Elizabeth C. Retamozo

2

PresenterPresenter

Elizabeth C. Retamozo

Executive Director NOSOTROS Education Center

www.nosotrosedu.org Phone: 469-450-7137

eretamozo@nosotrosedu.org eretamozo@tx.rr.com

3

Session SummarySession Summary

This session will focus on providing TRIO This session will focus on providing TRIO personnel with an overview of:personnel with an overview of:

Program EvaluationProgram Evaluation

Types of Program Evaluations Types of Program Evaluations

Methods & Types of Data Collection Methods & Types of Data Collection

Quantitative & Qualitative Data Quantitative & Qualitative Data

Formative & Summative Methods Formative & Summative Methods

4

EvaluationEvaluation Evaluation -Evaluation - Systematic method for collecting, Systematic method for collecting,

analyzing, and using information to answer basic analyzing, and using information to answer basic questions about your program. questions about your program.

Helps to identify effective & ineffective services, practices, Helps to identify effective & ineffective services, practices, & approaches.& approaches.

Evaluation PlanEvaluation Plan - A written document describing - A written document describing the overall approach or design you anticipate using the overall approach or design you anticipate using to guide your evaluation. to guide your evaluation.

Includes what you plan to do, how you plan to do it, who Includes what you plan to do, how you plan to do it, who will do it, when it will be done, & why the evaluation is will do it, when it will be done, & why the evaluation is being conducted.being conducted.

The evaluation plan serves as a guide for the evaluation.The evaluation plan serves as a guide for the evaluation.

5

EvaluationEvaluation Plan Plan Comprehensive plan to determine the Comprehensive plan to determine the

effectiveness of the program in meeting stated effectiveness of the program in meeting stated objectives.objectives.

Includes the criteria that will determine Includes the criteria that will determine effectiveness & the documentation that objectives effectiveness & the documentation that objectives have been met.have been met.

Describes the methodologyDescribes the methodology

Provides the information about the delivery of Provides the information about the delivery of services & use of resources services & use of resources

Informs & assists program staff to make changes Informs & assists program staff to make changes to improve program effectiveness.to improve program effectiveness.

6

The GRPA ConnectionThe GRPA Connection

Government Performance and Results Act of 1993 Government Performance and Results Act of 1993 (GPRA) (GPRA)

DOE Response to GPRA Requirements: Goals 1-6DOE Response to GPRA Requirements: Goals 1-6

TRIO Programs Performance Indicator: Goal 5TRIO Programs Performance Indicator: Goal 5

TRIO Programs’ Specific Program Measures:TRIO Programs’ Specific Program Measures: Upward Bound, Talent Search, Educational Opportunity Upward Bound, Talent Search, Educational Opportunity

CentersCenters Student Support Services & Ronald E. McNairStudent Support Services & Ronald E. McNair

Prior Experience Criteria & Mandatory ObjectivesPrior Experience Criteria & Mandatory Objectives Project Evaluation PlanProject Evaluation Plan

Activities & Services Specific to Each ProjectActivities & Services Specific to Each Project

7

Types of TRIO Evaluation PlansTypes of TRIO Evaluation Plans

Goals-Based Evaluation Goals-Based Evaluation Goal-based evaluations are evaluating the extent to Goal-based evaluations are evaluating the extent to

which programs are meeting predetermined goals or which programs are meeting predetermined goals or objectives. objectives.

Outcomes-Based Evaluation Outcomes-Based Evaluation Outcomes evaluation looks at impacts/benefits to Outcomes evaluation looks at impacts/benefits to

clients during & after participation in your programs.clients during & after participation in your programs.

Process-Based EvaluationProcess-Based Evaluation Process-based evaluations are geared to fully Process-based evaluations are geared to fully

understanding how a program works -- how does it understanding how a program works -- how does it produce that results that it does. produce that results that it does.

8

TRIO Program Evaluation TRIO Program Evaluation

Shapes the development of the project Shapes the development of the project from the beginning of the grant period. from the beginning of the grant period.

Comprehensive plan to determine the Comprehensive plan to determine the effectiveness of the program in meeting effectiveness of the program in meeting stated objectives.stated objectives.

Includes criteria that will determine Includes criteria that will determine effectiveness & document that objectives effectiveness & document that objectives have been met.have been met.

Provides information about the delivery of Provides information about the delivery of services & their impact & effectivenessservices & their impact & effectiveness

9

TRIO Program EvaluationTRIO Program Evaluation

Provides a mechanism for evaluating & Provides a mechanism for evaluating & implementing programmatic changes to implementing programmatic changes to improve program effectivenessimprove program effectiveness

Provides accountability information about Provides accountability information about the success at the initial site & effective the success at the initial site & effective strategies for replication in other settingsstrategies for replication in other settings

Relates your project to GPRA & the TRIO Relates your project to GPRA & the TRIO Programs Performance IndicatorsPrograms Performance Indicators

10

UB Proposal Evaluation Criteria UB Proposal Evaluation Criteria The Secretary evaluates the quality of the evaluation The Secretary evaluates the quality of the evaluation

plan for the project on the basis of the extent to plan for the project on the basis of the extent to which the applicant’s methods of evaluation - - which the applicant’s methods of evaluation - -

(1) Are appropriate to the project and include both (1) Are appropriate to the project and include both quantitative and qualitative evaluation measures; and quantitative and qualitative evaluation measures; and

(2) Examine in specific and measurable ways the (2) Examine in specific and measurable ways the success of the project in making progress toward success of the project in making progress toward achieving its process and outcome objectives. achieving its process and outcome objectives.

11

TS/EOC Proposal Evaluation CriteriaTS/EOC Proposal Evaluation Criteria The Secretary evaluates the quality of the evaluation The Secretary evaluates the quality of the evaluation

plan for the project on the basis of the extent to which plan for the project on the basis of the extent to which the applicant’s methods of evaluation - - the applicant’s methods of evaluation - -

(1) Are appropriate to the project’s objectives; (1) Are appropriate to the project’s objectives; (2) Provide for the applicant to determine, using (2) Provide for the applicant to determine, using

specific and quantifiable measures, the success of the specific and quantifiable measures, the success of the project in - project in -

(i) Making progress toward achieving its objectives (a (i) Making progress toward achieving its objectives (a formative evaluation); and formative evaluation); and

(ii) Achieving its objectives at the end of the project (ii) Achieving its objectives at the end of the project period (a summative evaluation); and period (a summative evaluation); and

(3) Provide for the disclosure of unanticipated project (3) Provide for the disclosure of unanticipated project outcomes, using quantifable measures if appropriate. outcomes, using quantifable measures if appropriate.

12

SSS Proposal Evaluation Criteria SSS Proposal Evaluation Criteria

The Secretary evaluates the quality of the evaluation The Secretary evaluates the quality of the evaluation plan for the project on the basis of the extent to which plan for the project on the basis of the extent to which - -- -

(1) The applicant’s methods of evaluation - - (1) The applicant’s methods of evaluation - - (i) Are appropriate to the project and include both (i) Are appropriate to the project and include both

quantitative and qualitative evaluation measures; and quantitative and qualitative evaluation measures; and (ii) Examine in specific and measurable ways, using (ii) Examine in specific and measurable ways, using

appropriate baseline data, the success of the project appropriate baseline data, the success of the project in improving academic achievement, retention and in improving academic achievement, retention and graduation of project participants; and graduation of project participants; and

(2) The applicant intends to use the results of an (2) The applicant intends to use the results of an evaluation to make a programmatic changes based evaluation to make a programmatic changes based upon the results of project evaluation. upon the results of project evaluation.

13

McNair Proposal Evaluation CriteriaMcNair Proposal Evaluation Criteria The Secretary evaluates the quality of the evaluation plan for The Secretary evaluates the quality of the evaluation plan for

the project on the basis of the extent to which the the project on the basis of the extent to which the applicant's methods of evaluation– applicant's methods of evaluation–

(1) Are appropriate to the project's objectives; (1) Are appropriate to the project's objectives; (2) Provide for the applicant to determine, in specific and (2) Provide for the applicant to determine, in specific and

measurable ways, the success of the project in—measurable ways, the success of the project in—

(i) Making progress toward achieving its objectives (a (i) Making progress toward achieving its objectives (a formative evaluation); and formative evaluation); and

(ii) Achieving its objectives at the end of the project period (ii) Achieving its objectives at the end of the project period (a summative evaluation); and (a summative evaluation); and

(3) Provide for a description of other project outcomes, (3) Provide for a description of other project outcomes, including the use of quantifiable measures, if appropriate. including the use of quantifiable measures, if appropriate.

14

Common Words & TerminologyCommon Words & Terminology

Appropriate to Project’s ObjectivesAppropriate to Project’s Objectives

Specific & MeasurableSpecific & Measurable

Qualitative & QuantitativeQualitative & Quantitative

Formative & SummativeFormative & Summative

15

Appropriate to Project’s ObjectivesAppropriate to Project’s Objectives

Goals-Based Evaluation - InternalGoals-Based Evaluation - Internal Goal-based evaluations are evaluating the extent Goal-based evaluations are evaluating the extent

to which programs are meeting predetermined to which programs are meeting predetermined goals or objectives. goals or objectives.

Outcomes-Based Evaluation – Outcomes-Based Evaluation – Internal/ExternalInternal/External

Outcomes evaluation looks at impacts/benefits to Outcomes evaluation looks at impacts/benefits to clients during & after participation in your clients during & after participation in your programs.programs.

Process-Based EvaluationProcess-Based Evaluation - - ExternalExternal Process-based evaluations are geared to fully Process-based evaluations are geared to fully

understanding how a program works -- how does it understanding how a program works -- how does it produce that results that it does. produce that results that it does.

16

Specific & MeasurableSpecific & Measurable

Quantify description of program effectiveness for Quantify description of program effectiveness for each objectiveeach objective

Be specific & use quantifiable measuresBe specific & use quantifiable measures

Measures outcomes related to your mandatory Measures outcomes related to your mandatory objectivesobjectives

Measurable: %, time lines, aggregate data, Measurable: %, time lines, aggregate data, descriptive numbersdescriptive numbers

Data examples: Demographics, GPAs, Year to Data examples: Demographics, GPAs, Year to Year Retention, Graduation Rates, College Year Retention, Graduation Rates, College Enrollment & Graduation Rates, & Enrollment in Enrollment & Graduation Rates, & Enrollment in Continuing Education Continuing Education

17

Quantitative DataQuantitative Data Information that can be expressed in Information that can be expressed in

numerical terms, counted or compared on numerical terms, counted or compared on a scale. For example, improvement in a a scale. For example, improvement in a student’ test scores as measured pre and student’ test scores as measured pre and post test scores.post test scores.

Quantifiable – specifically measure (GPA Quantifiable – specifically measure (GPA increase, Retention & Graduation rates, increase, Retention & Graduation rates, academic progress, credit hours earned).academic progress, credit hours earned).

Methods: data collection, test results & Methods: data collection, test results & documentsdocuments

18

Quantitative DataQuantitative Data Grade Point Averages Grade Point Averages Project Retention RatesProject Retention Rates Grade Reports & Test ScoresGrade Reports & Test Scores Secondary/Postsecondary Persistence Secondary/Postsecondary Persistence

StatusStatus Secondary/Postsecondary Graduation Secondary/Postsecondary Graduation

Rates Rates Credit Hour CompletionCredit Hour Completion Good Standing StatusGood Standing Status

19

Qualitative MethodsQualitative Methods May include May include

“meaning”, “meaning”, experience, opinions experience, opinions

Ways to describe Ways to describe program program effectiveness that are effectiveness that are measurable, but may measurable, but may not aggregate in not aggregate in traditional, traditional, quantifiable methodsquantifiable methods

Examples:Examples:

- Focus groups- Focus groups

- Case studies - Case studies

- Open ended interviews- Open ended interviews

- Surveys- Surveys

20

Qualitative DataQualitative Data Information that is difficult to measure, Information that is difficult to measure,

count, or express in numerical terms. For count, or express in numerical terms. For example, a participant's impression about example, a participant's impression about the fairness of a program rule/requirement the fairness of a program rule/requirement is qualitative data. is qualitative data.

Interviews & Focus GroupsInterviews & Focus Groups Used to fully understand someone's Used to fully understand someone's

impressions or experiences, or learn more impressions or experiences, or learn more about their answers to questionnairesabout their answers to questionnaires

21

Qualitative DataQualitative Data ObservationsObservations

Used to gather accurate information about Used to gather accurate information about how a program actually operates, how a program actually operates, particularly about processesparticularly about processes

Questionnaires, Surveys, ChecklistsQuestionnaires, Surveys, Checklists

Used to quickly and/or easily get lots of Used to quickly and/or easily get lots of information from people in a non-information from people in a non-threatening way threatening way

Examples: Sample Evaluation FormsExamples: Sample Evaluation Forms

Pre & Post Test to Measure LearningPre & Post Test to Measure Learning

22

Formative EvaluationFormative Evaluation

Type of process evaluation of programs or Type of process evaluation of programs or services that focuses on collecting data on services that focuses on collecting data on program operations so that needed program operations so that needed changes or modifications can be made to changes or modifications can be made to the program in its early stages. the program in its early stages.

Used to provide feedback to staff about the Used to provide feedback to staff about the program components that are working & program components that are working & those that need to be changed.those that need to be changed.

Example: Changes to Delivery of Services Example: Changes to Delivery of Services

23

Formative EvaluationFormative Evaluation Continuous (ex. monthly) assessment of Continuous (ex. monthly) assessment of

program effectiveness & progress towards program effectiveness & progress towards achieving objectivesachieving objectives

Allows for changes in activities so final Allows for changes in activities so final objectives/outcomes will be met – changes objectives/outcomes will be met – changes in delivery of servicesin delivery of services

Ex: Monthly report of student grades; student/client Ex: Monthly report of student grades; student/client progress reports; monthly student/client contact progress reports; monthly student/client contact reports; client/student evaluation of program; reports; client/student evaluation of program; monthly staff meeting progress reporting; monthly staff meeting progress reporting; student/client attendance & feedback; student pre-student/client attendance & feedback; student pre-post test results post test results

24

Summative EvaluationSummative Evaluation

A type of outcome evaluation that assesses A type of outcome evaluation that assesses the results or outcomes of a program. the results or outcomes of a program.

Concerned with a program's overall Concerned with a program's overall effectivenesseffectiveness

Measurement of objectives or outcomes at Measurement of objectives or outcomes at the end of a pre-defined periodthe end of a pre-defined period

Quantitative data is analyzed for a given Quantitative data is analyzed for a given periodperiod

Qualitative evaluation supplements the Qualitative evaluation supplements the quantitative dataquantitative data

25

Summative Evaluation QuestionsSummative Evaluation Questions

Did you meet program objectives?Did you meet program objectives?

How did you accomplish them?How did you accomplish them?

What is your documentation?What is your documentation?

What changes do you need to make to the What changes do you need to make to the activities or objectives? activities or objectives?

Examples: Student year-end program Examples: Student year-end program evaluations; annual performance reportevaluations; annual performance report

26

Goals-Based Evaluation - InternalGoals-Based Evaluation - Internal

State Mandatory ObjectivesState Mandatory Objectives

State Evaluation CriteriaState Evaluation Criteria

Summative & Formative Evaluation Summative & Formative Evaluation MethodsMethods

Qualitative & Quantitative MeasuresQualitative & Quantitative Measures

Assign Staff ResponsibilityAssign Staff Responsibility

State Time Delineation State Time Delineation

Analyze & Report Findings Analyze & Report Findings

27

Goals-Based Evaluation - Internal Goals-Based Evaluation - Internal

Data Collection ChartData Collection Chart

(1) What types of data will be collected;(1) What types of data will be collected;

(2) When various types of data will be collected; (2) When various types of data will be collected;

(3) What methods will be used; (3) What methods will be used;

(4) What instruments will be developed & when; (4) What instruments will be developed & when;

(5) How the data will be analyzed; (5) How the data will be analyzed;

(6) When reports & outcomes will be available; and(6) When reports & outcomes will be available; and

(7) How the grantee will use the information (7) How the grantee will use the information collected through evaluation to monitor progress of collected through evaluation to monitor progress of the funded project & to provide information about the funded project & to provide information about success at the initial site & effective strategies for success at the initial site & effective strategies for replication in other settings. replication in other settings.

28

Goals-Based Evaluation - Internal Goals-Based Evaluation - Internal

Sample Data Collection ChartSample Data Collection Chart

Talent Search Talent Search Educational Opportunity Educational Opportunity

CentersCenters Upward Bound Upward Bound Student Support Services Student Support Services McNairMcNair

29

Making Progress Toward Achieving its Objectives -Making Progress Toward Achieving its Objectives -

Formative Evaluation:Formative Evaluation:

Project Director will conduct the following Project Director will conduct the following formative evaluation methods of all project formative evaluation methods of all project activities and services that correspond to activities and services that correspond to keep the project running smoothly during keep the project running smoothly during the year.the year.

Project Director will evaluate the results of Project Director will evaluate the results of all daily contacts & services provided all daily contacts & services provided weekly and monthly as it relates to weekly and monthly as it relates to progress to meet project objectives. progress to meet project objectives.

30

Making Progress Toward Achieving its Objectives - Making Progress Toward Achieving its Objectives -

Formative Evaluation:Formative Evaluation:

Progress towards the accomplishment of all Progress towards the accomplishment of all objectives will be tracked through objectives will be tracked through quantitative reports generated by the quantitative reports generated by the project database on a daily, weekly, & project database on a daily, weekly, & monthly basis. monthly basis.

Quantitative reports collect specific Quantitative reports collect specific information documenting what participants information documenting what participants have been served, the types of services have been served, the types of services provided, the frequency of services provided, the frequency of services provided, & by which project staff member. provided, & by which project staff member.

31

Making Progress Toward Achieving its Objectives - Making Progress Toward Achieving its Objectives -

Formative Evaluation:Formative Evaluation:

Project database generates weekly, monthly, Project database generates weekly, monthly, & quarterly performance reports & generates & quarterly performance reports & generates individual participant outcome & contact individual participant outcome & contact reports documenting services provided & the reports documenting services provided & the impact of these services on each project impact of these services on each project participant. participant.

Participants & parents will be asked to Participants & parents will be asked to evaluate specific group activities such as evaluate specific group activities such as workshops, field trips to colleges and career workshops, field trips to colleges and career sites to rate their level of satisfaction with the sites to rate their level of satisfaction with the quality of services they receive – qualitative. quality of services they receive – qualitative.

32

Achieving Objectives at the End of the Project Period - Achieving Objectives at the End of the Project Period -

Summative Evaluation:Summative Evaluation:

Project database will combine all monthly Project database will combine all monthly quantitative reports to produce a report quantitative reports to produce a report documenting & confirming the achievement documenting & confirming the achievement of project objectives. of project objectives.

Participants & parents will complete end-of-Participants & parents will complete end-of-year surveys to measure their level of year surveys to measure their level of satisfaction with the overall quality of satisfaction with the overall quality of project services. project services.

33

Achieving Objectives at the End of the Project Period - Achieving Objectives at the End of the Project Period -

Summative Evaluation:Summative Evaluation:

Target school personnel & project staff will be Target school personnel & project staff will be asked to evaluate specific activities & services asked to evaluate specific activities & services on a numeric rating system to measure their on a numeric rating system to measure their level of satisfaction with the overall quality of level of satisfaction with the overall quality of project services - qualitative. project services - qualitative.

Project Director will address the results of the Project Director will address the results of the summative evaluation to implement any summative evaluation to implement any corrective measures to eliminate potential corrective measures to eliminate potential problems by appropriately modifying & problems by appropriately modifying & improving project activities & services for the improving project activities & services for the next project year. next project year.

34

Sample TRIO Evaluation PlansSample TRIO Evaluation Plans

Upward Bound Upward Bound

Talent Search Talent Search

Educational Opportunity Centers Educational Opportunity Centers

Student Support Services Student Support Services

McNair McNair

35

External EvaluationExternal Evaluation

Recommended in last grant proposal Recommended in last grant proposal competitionscompetitions

Must be conducted if included in grant proposalsMust be conducted if included in grant proposals

Allowable cost for the project Allowable cost for the project

May be paid by the grantee agency/institution May be paid by the grantee agency/institution

May be provided free of charge – May be provided free of charge – in-kind/donationin-kind/donation

36

External EvaluationExternal Evaluation

May be conducted by person May be conducted by person working/employed by agency/institution working/employed by agency/institution BUT not for the project BUT not for the project

Must have experience conducting Must have experience conducting evaluations for educational opportunity evaluations for educational opportunity programs & TRIO like programsprograms & TRIO like programs

Must state what type of evaluation will be Must state what type of evaluation will be conducted by the evaluator.conducted by the evaluator.

37

External EvaluationExternal Evaluation Process-Based EvaluationProcess-Based Evaluation

Process-based evaluations are geared to Process-based evaluations are geared to fully understanding how a program works -- fully understanding how a program works -- how does it produce that results that it does. how does it produce that results that it does.

Useful if programs are long-standing and Useful if programs are long-standing and have changed over the years.have changed over the years.

Useful for accurately portraying to outside Useful for accurately portraying to outside parties how a program truly operates (e.g., parties how a program truly operates (e.g., for replication elsewhere).for replication elsewhere).

38

Process-Based EvaluationProcess-Based Evaluation

Examines the extent to which a program Examines the extent to which a program is operating as intended by assessing is operating as intended by assessing ongoing program operations and whether ongoing program operations and whether the targeted population is being served. the targeted population is being served.

Helps program staff identify needed Helps program staff identify needed interventions and/or change program interventions and/or change program components to improve service delivery. components to improve service delivery.

39

Process-Based EvaluationProcess-Based Evaluation

Involves collecting data that describes Involves collecting data that describes program operations in detail, including:program operations in detail, including:

Types & levels of services providedTypes & levels of services provided

Staffing & location of service deliveryStaffing & location of service delivery

Sociodemographic characteristics of Sociodemographic characteristics of participantsparticipants

Community in which services are providedCommunity in which services are provided

Linkages with collaborating agencies. Linkages with collaborating agencies.

40

External Evaluation PlanExternal Evaluation Plan

Outcomes-Based EvaluationOutcomes-Based Evaluation Outcomes-based evaluation helps Outcomes-based evaluation helps

determine if your organization is really determine if your organization is really doing the right program activities to bring doing the right program activities to bring about the outcomes you believe to be about the outcomes you believe to be needed by your clients/students.needed by your clients/students.

Outcomes are benefits to clients from Outcomes are benefits to clients from participation in the program. Example: participation in the program. Example: Enhanced learning (academic knowledge, Enhanced learning (academic knowledge, perceptions/attitudes or skills, etc). perceptions/attitudes or skills, etc).

41

External Evaluation PlanExternal Evaluation Plan Outcome-Based EvaluationOutcome-Based Evaluation

1.1. Identify the major outcomes that you want to Identify the major outcomes that you want to examine or verify for the program under examine or verify for the program under evaluation. evaluation.

2. Choose the outcomes that you want to examine, 2. Choose the outcomes that you want to examine, prioritize the outcomes.prioritize the outcomes.

3. For each outcome, specify what observable 3. For each outcome, specify what observable measures, or indicators, will suggest that you're measures, or indicators, will suggest that you're achieving that key outcome with your clients. achieving that key outcome with your clients.

4. Specify a "target" goal of clients, i.e., what 4. Specify a "target" goal of clients, i.e., what number or percent of clients you commit to number or percent of clients you commit to achieving specific outcomes achieving specific outcomes

42

External Evaluation PlanExternal Evaluation Plan

Outcome-Based EvaluationOutcome-Based Evaluation

5.5. Identify what information is needed to show Identify what information is needed to show these indicators.these indicators.

6. Decide how can that information be efficiently 6. Decide how can that information be efficiently and realistically gathered & which methods to and realistically gathered & which methods to use (ex. program documentation, observation of use (ex. program documentation, observation of program personnel & clients in the program, program personnel & clients in the program, questionnaires & interviews about clients questionnaires & interviews about clients perceived benefits from the program, case perceived benefits from the program, case studies of program failures & successes, etc.)studies of program failures & successes, etc.)

7. Analyze & report the findings7. Analyze & report the findings

43

•Promotes standards in student affairs, student services, and student development programs since its inception in 1979

• Develops a Book of Professional Standards and Guidelines and Self-Assessment Guides (SAG) that are designed to lead to a host of quality-controlled programs and services.

CAS TRIO/EOP Standards & Guidelines & Self-Assessment Guide (SAG) for TRIO & EOP Programs

Provides a vehicle and opportunity for projects to plan for formative & summative evaluation, to participate in project self-assessment, and plan for project improvements.

The Council for the Advancement of Standards in The Council for the Advancement of Standards in Higher Education (CAS)Higher Education (CAS)

44

Benefits of Program Evaluation Benefits of Program Evaluation

Understand, verify or increase the impact Understand, verify or increase the impact of services on participants or clients of services on participants or clients

Improve delivery mechanisms to be more Improve delivery mechanisms to be more efficient and less costly efficient and less costly

Produce data or verify results that can be Produce data or verify results that can be used for public relations & promoting used for public relations & promoting services in the communityservices in the community

Fully examine & describe effective Fully examine & describe effective programs for duplication elsewhere. programs for duplication elsewhere.

45

Basic Guide to Program Evaluation

Written by Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Copyright 1997-2007.Adapted from the Field Guide to Nonprofit Program Design, Marketing and Evaluation.

Online Guide: http://www.tgci.com/magazine/A%20Basic%20Guide%20to%20Program%20Evaluation.pdf

Bibliography & ResourcesBibliography & Resources

46

Basic Guide to Outcomes-Based Evaluation for Nonprofit Organizations with Very Limited Resources

Written by Carter McNamara, MBA, PhD, Authenticity Consulting, LLC. Copyright 1997-2007.Adapted from the Field Guide to Nonprofit Program Design, Marketing and Evaluation.

Bibliography & ResourcesBibliography & Resources

47

US Department of Health & Human Services Administration for Children & Families - Office of Planning, Research & Development

The Program Manager's Guide to Evaluation

http://www.acf.hhs.gov/programs/opre/other_resrch/pm_guide_eval/reports/pmguide/pmguide_toc.html

This guide explains program evaluation -what it is, how to understand it, and how to do it. It answers your questions about evaluation and explains how to use evaluation to improve programs and benefit students, families, and staff.

Bibliography & ResourcesBibliography & Resources

48

The Council for the Advancement of Standards The Council for the Advancement of Standards in Higher Education (CAS)in Higher Education (CAS)CAS TRIO/EOP Standards & Guidelines & Self-Assessment Guide (SAG) for TRIO & EOP Programs

Provides a vehicle and opportunity for projects to plan for formative & summative evaluation, to participate in project self-assessment, and plan for project improvements.

http://www.cas.edu/http://www.cas.edu/

• Resources • Basics PowerPoint• Handout• Assessment Model• Online Store• Self-Assessment Guide

Bibliography & ResourcesBibliography & Resources

49

Questions & AnswersQuestions & Answers