An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention...
-
Upload
hana-dawkins -
Category
Documents
-
view
217 -
download
0
Transcript of An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention...
![Page 1: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/1.jpg)
An Introduction to Evaluation Research
Duane Shell
Research Associate Professor
Nebraska Prevention Center for Alcohol and Drug Abuse
Department of Educational Psychology
![Page 2: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/2.jpg)
Why Evaluate?
Because somebody
says you have to.
[usually a funder]
![Page 3: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/3.jpg)
What is Evaluation?
Something to be “feared”.
A sneaky way for the “powers that be” to get you.
How “the Man” stifles your creativity.
How “they” can justify taking away your money.
![Page 4: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/4.jpg)
What is Evaluation?
Evaluation is a way for YOU to find out if things you are doing are working.
![Page 5: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/5.jpg)
So Why Evaluate?
You Evaluate for Yourself
Not for others.
Really!!
![Page 6: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/6.jpg)
The difference between
Evaluation
and
Research
![Page 7: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/7.jpg)
Evaluation vs Research
Evaluation and Research DO NOT differ in:
Data collection tools
Methodologies
Analysis methods
![Page 8: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/8.jpg)
The purpose of research is to:
Test hypotheses
Inform theory
Or at least:
Describe or draw conclusions
About some phenomenon
![Page 9: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/9.jpg)
The purpose of evaluation is to:
Inform decision making about project success
by
Determining if objectives are achievedand
Activities are done as planned
![Page 10: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/10.jpg)
So What Is Evaluation?
A systematic way
to collect data
to inform decision making
![Page 11: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/11.jpg)
Inform Decision Making
It is not possible to definitively answer the question:
Was the project successful?
The only thing that can be done is to collect evidence to allow a judgment
![Page 12: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/12.jpg)
Collect Data
The foundation of Evaluation is Data driven decision making.
Data are typically collected from multiple sources
Data are typically both quantitative and qualitative
Multiple data are “triangulated” to make judgments and decisions
![Page 13: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/13.jpg)
Systematic Data Collection
Evaluation is based in a systematic approach to collecting data.
Evaluation uses the same methodologies that are used in research.
Standards of methodological rigor are the same as those used in research.
![Page 14: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/14.jpg)
Systematic Data Collection
Evaluation, though must apply methods and rigor within the context of the real world of the project being evaluated.
Where the exacting standards of research design cannot be done,
Evaluation attempts to collect data in the most systematic and open way possible.
![Page 15: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/15.jpg)
Systematic Data Collection
Recognition of the usual failure to achieve the most exacting standards of research is why evaluation decisions are based on:
multiple triangulated data sources
![Page 16: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/16.jpg)
Evaluation Basics
![Page 17: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/17.jpg)
5 Basic Evaluation Questions
1) What will be assessed?
2) What measures/indicators will be used?
3) Who will be evaluated?
4) What data will be collected?
5) How will data be analyzed?
![Page 18: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/18.jpg)
What Will Be Evaluated?
Formative (aka Process) Evaluation:
Done to help improve the project itself.Gather information on how the project worked.Data is collected about activities: What was done.
Summative (aka Outcome) Evaluation:
Done to determine what results were achieved.Data is collected about outcomes (objectives; goals): What happened.
![Page 19: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/19.jpg)
What Measures Will Be Used?
Formative Evaluation:Completion of planned activitiesAdherence to proposed time linesMeeting budget
Summative Evaluation:
Reaching a criterionChange in knowledge, attitude, skill, behavior
![Page 20: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/20.jpg)
Who will be evaluated?
Formative Evaluation:Those responsible for doing activities/delivering services and those participating in activities.
FacultyAgency personnelStudents
Summative Evaluation:Those who were expected to be impacted by activities.
StudentsClients
![Page 21: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/21.jpg)
What data will be collected?
Formative Evaluation:Program recordsObservationsActivity logsSatisfaction surveys
Summative Evaluation:
ObservationsInterviewsTestsSurveys/questionnaires
![Page 22: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/22.jpg)
How will data be analyzed?
1) Qualitative analysis (more for formative)1) Self-reports2) Documentation3) Description4) Case Study
2) Quantitative analysis (more for summative)1) Group comparison2) Group change3) Individual change4) Comparison to population/reference5) Analysis of relationships
![Page 23: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/23.jpg)
An ExampleThe Cosmic Ray Observatory Project (CROP)
Goal: Establish a statewide collaborative network of expert teachers fully capable of continuing the project locally.
Objectives: Teachers will acquire knowledge about cosmic ray physics and skill in high energy
research methods.Teachers will exhibit increased self-efficacy for conducting CROP research and integrating CROP into their teaching.
Activity: High school physics teachers and students will attend a 3-4 week hands-on summer research experience on cosmic ray physics
at UNL
![Page 24: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/24.jpg)
Formative EvaluationWhat activities were evaluated?
The specific components of the Summer Research ExperienceWhat measures were used?
Completion of activitiesParticipant satisfactionParticipant evaluation of goal attainmentParticipant evaluation of activity effectiveness
Who was evaluated?Participants
What data was collected?InterviewsRating scales
How was data analyzed?Content analysis of interview responsesFrequency and descriptive statistical analysis of rating scales.
![Page 25: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/25.jpg)
Examples of Formative Measures Interview Questions
What was the most effective part of the workshop?
Hands-on work with detectors 6
Information from classroom sessions 4
Teacher Comments (by teacher with coded category(s) indicated):
• For me personal was the activities. The actual connecting and wiring and those things. I don’t sit and take lectures very well. That’s just me. [Hands on work with the detectors]
• Um, I think it was the classroom work. There was a good review for those of us that have had physics and it was a good introduction for those that didn’t. [Information from classroom sessions]
![Page 26: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/26.jpg)
Examples of Formative Measures Rating Scales
1. How effective do you think the workshop was in meeting its goals?
1 2 3 4 5
Not Effective Neither Effective Somewhat Effective Very Effective nor ineffective Effective
4. Indicate how USEFUL you think each of the following workshop components was using the following scale.
1 2 3 4 5 6
Very Unuseful Somewhat Somewhat Useful Very
Unuseful Unuseful Useful Useful
a. Classroom/lecture sessions on particle detectors and experimental techniques.
b. Lab work sessions refurbishing and preparing detectors.
![Page 27: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/27.jpg)
Examples of Formative Measures Rating Scales
How effective do you think the workshop was in meeting its goals?
Very Effective Somewhat Neither Effective Not Effective Effective nor Ineffective Effective
(5) (4) (3) (2) (1) M SD 1 3 1 0 0 4.00 .71
Very Unuseful Somewhat Somewhat Useful Very Unuseful Unuseful Useful Useful (1) (2) (3) (4) (5) (6) M SD Component 0 0 0 0 2 3 5.60 .55 Classroom/lecture
sessions on particle detectors and experimental
techniques. 0 0 0 1 3 1 5.00 .71 Lab work sessions
refurbishing and preparing detectors.
![Page 28: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/28.jpg)
Summative EvaluationWhat Outcomes were evaluated?
Teachers’ increase in knowledge about cosmic ray physics and skill in high energy research methodsTeachers’ change self-efficacy for conducting CROP research and integrating CROP into their teaching
What measures were used?Knowledge gainAchieving criteria level of knowledge/skillIncrease in self-efficacy
Who was evaluated?Teachers
What data was collected?Pre- and post-workshop tests of cosmic ray physics and researchPre- and post-workshop self-efficacy ratings
How was data analyzed?Dependent t-tests of pre-post scoresComparing skill scores to criteria
![Page 29: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/29.jpg)
Summative EvaluationKnowledge Test Questions
1. The energy distribution of primary cosmic rays bombarding the earth has been measured by a number of experiments. In the space below, sketch a graph of the number of observed primary cosmic rays vs. cosmic ray energy, and describe the distribution in a sentence or two.
2. Explain how a scintillation counter works, i.e. write down the sequence of events from the passage of a charged particle through a scintillator to the generation of an electric signal in a photomultiplier tube.
3. Describe some characteristic differences between electromagnetic showers and hadronic showers created when particles impinge on a block of matter or a cosmic ray enters the atmosphere. Hint: think in terms of the type of particle which initiates the shower, the type of secondary particles in the shower, the shape of the shower, depth penetration of the shower particles, etc.
![Page 30: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/30.jpg)
Summative EvaluationData Analysis
Table 9
Participants Pre- and Post-Test Mean Scores on Knowledge Tests
Pre-Test Post-Test
df M SD M SD t ES
Teachers 4 5.00 2.69 19.60 1.71 8.67* 6.64
Note. ES = effect size computed by Cohen's d in averaged pre- and post-test SD units. Teachers, n = 5.
*p < .01.
![Page 31: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/31.jpg)
Summative EvaluationSelf-Efficacy Questions
Please rate how confident you are about each of the following from 0 (completely unconfident) to 100 (completely confident).
1. Your ability to set-up and maintain the CROP research equipment at your school.
2. Your ability to conduct CROP research at your school. 3. Your ability to teach students at your school who
haven't attended the Summer Workshop how to conduct CROP research at your school.
4. Your ability to design your own research projects for your students utilizing the CROP research equipment.
5. Your ability to incorporate lessons and activities in high-energy physics into your classes.
6. Your ability to create "hands-on" projects and activities for students in your classes using the CROP research equipment.
![Page 32: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/32.jpg)
Summative EvaluationData Analysis
Table 11Participants Pre- and Post-Test Mean Self-Efficacy Scores
Pre-Test Post-Test df M SD M SD t ES
Conducting 4 41.00 31.58 77.80 16.10 3.06* 1.54CROP Activities
Integrating CROP 4 45.00 31.37 79.25 17.31 3.32*1.41
Into Classes
Utilizing Distance 4 56.67 17.48 70.33 13.35 4.08* .89
Education
Note. ES = effect size computed by Cohen's d in averaged pre- and post-test SD units. Teachers, n = 5.
*p < .01.
![Page 33: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/33.jpg)
Formative Evaluation Example
To obtain student reactions for the development of the campus specific Web based brief intervention versions, student feedback will be obtained. Beta versions will be evaluated by recruiting a panel of students from the each participating campus. These students will complete the intervention and provide verbal and written feedback on their reactions to the program and their suggestions for improvement. Adjustments to the program will be made based on student feedback.
![Page 34: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/34.jpg)
Summative Evaluation Example
Students will complete the web-based brief alcohol intervention (pre-test). Approximately 6-weeks later, they will again complete the web-based brief alcohol intervention (post-test). Change will be determined by comparing post-test scores to pre-test scores using a Repeated Measures Analysis of Variance (ANOVA). Success will be determined by a statistically significant decrease in drinking and driving (Objective 1) and riding with a driver who has been drinking (Objective 2), with an effect size of at least a 10% pre- to post-test decrease for drunk driving and a 6% decrease for riding with a drinking driver.
![Page 35: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/35.jpg)
Planning Evaluation
The Logic Model
A systematic linkage of project goals, objectives, activities, and outcomes.
![Page 36: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/36.jpg)
Steps in Creating a Logic Model
1) Clarify what the goals of the project/ program are.
2) Clarify what objectives the project should achieve.
3) Specify what program activities will occur.
![Page 37: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/37.jpg)
Goal Clarification
High school physics teachers and students will attend a 3-4 week hands-on summer research experience on cosmic ray physics at UNL.
Is this a goal?
![Page 38: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/38.jpg)
Goal Clarification
Establish a statewide collaborative network of expert teachers fully capable of continuing the project locally.
![Page 39: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/39.jpg)
Developing Objectives
Goal: Establish a statewide collaborative network of expert teachers fully capable of continuing the project locally.
Objectives1. Teachers will acquire knowledge about cosmic
ray physics and skill in high energy research methods.
2. Teachers will exhibit increased self-efficacy for conducting CROP research and integrating CROP into their teaching.
![Page 40: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/40.jpg)
CROP Logic ModelGoal: Establish a statewide collaborative
network of expert teachers fully capable of continuing the project locally.
Objectives: Teachers will acquire knowledge about cosmic ray physics and skill in high energy
research methods.
Teachers will exhibit increased self-efficacy for conducting CROP research and integrating CROP into their teaching.
Activity: High school physics teachers and students will attend a 3-4 week hands-on summer research experience on cosmic ray physics at UNL
![Page 41: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/41.jpg)
Evaluating the Logic Model
• Goal – Objective CorrespondenceAre objectives related to the overall goal?
• Goal – Activity CorrespondenceDo anticipated activities adequately implement the goals?
• Activity – Objective CorrespondenceWill program activities result in achieving objectives?
![Page 42: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/42.jpg)
CROP Logic ModelGoal: Establish a statewide collaborative
network of expert teachers fully capable of continuing the project locally.
Objectives: Teachers will acquire knowledge about cosmic ray physics and skill in high energy
research methods.
Teachers will exhibit increased self-efficacy for conducting CROP research and integrating CROP into their teaching.
Activity: High school physics teachers and students will attend a 3-4 week hands-on summer research experience on cosmic ray physics at UNL
![Page 43: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/43.jpg)
An Example
GOAL 1: Increase the availability of attractive student centered social activities located both on and off the NU campus.
Objective 1.1: Increase by 15% from baseline the number of students aware of campus and community entertainment options available to NU students.
Activity: Develop and maintain an interactive web site describing social and entertainment options
for students.
![Page 44: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/44.jpg)
Another Example
GOAL 7: Reduce high-risk alcohol marketing and promotion practices.
Objective 7.3: Reduce by 25% from baseline, the volume of alcohol advertisements in the Daily Nebraskan, The Reader and Ground Zero that mention high-risk marketing and promotion practices.
Activity: Work with the media to encourage at least 3 newspaper articles or television news stories in the Lincoln market each school year concerning high-risk marketing and promotion practices.
![Page 45: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/45.jpg)
Logic Model Example
Goal Objectives Methodology Completion Date 1. Create active, operational campus task forces at the 11 remaining state-funded institutions of higher education serving undergraduate populations.
1.1 Recruit support from upper administration at each institution to commit personnel to task force coordination and participation.
Administrative luncheon presentations to institution chancellors and presidents on statewide initiatives hosted by University of Nebraska President James Milliken; follow-up identifying key contacts
By November, 2005
1.2 Provide technical assistance and training to assist campuses in campus task force recruitment, organization and development.
Drive-in workshop on coalition development; follow-up teleconferences with organizers, internet resources.
By December, 2005
1.3: Provide regular teleconferencing facilitation to allow interaction between task force coordinators at participating campuses.
Monthly telephone conference of campus task force organizers; agenda that allows sharing of issues, problems, needs, and accomplishments
Ongoing through January, 2007
![Page 46: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/46.jpg)
Logic Model Worksheet
Goals Activities Objectives
(Outcome)
Indicators
Measures
Who Evaluated
Data Sources
Data Analysis
![Page 47: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/47.jpg)
Final Thoughts
Funders including
Federal, State, and Foundation
increasingly want more
Summative (Outcome) Evaluation
![Page 48: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/48.jpg)
Especially Summative Evaluation of
Educational Activities
and
Outreach Activities
![Page 49: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/49.jpg)
To be competitive for funding
You need a strong evaluation
That is more than
Did activities get done
and did attendees like it?
![Page 50: An Introduction to Evaluation Research Duane Shell Research Associate Professor Nebraska Prevention Center for Alcohol and Drug Abuse Department of Educational.](https://reader030.fdocuments.in/reader030/viewer/2022032702/56649cb05503460f94974e0c/html5/thumbnails/50.jpg)
Where to get Help
The SSP Core(Well duh-Why else would we be doing this Brown Bag?)
The SSP can assist with:
Developing an evaluation plan and logic model for funding proposals
Conducting evaluations of ongoing projects