Goal Development & Measurement Celeste Sturdevant Reed, PhD University Outreach and Engagement...

Goal Development & Measurement

Celeste Sturdevant Reed, PhD

University Outreach and Engagement

Destination #3

9:30 – 10:45 am, August 24, 2010

This presentation is based on:

• Your grant proposal to MDE• Your questions

If we don’t know where we’re going, any path will

get us there.

How many of you have seen the proposal you are implementing?

A plan implemented as written

is a plan doomed to failure!

Why do we need to (re)visit our proposal from time to time?

• It is our work “contract” with MDE• Someone who didn’t know what is and is not

allowable by MDE may have written it• Someone who didn’t know students’ needs as

well as the people operating the program may have written it

• Something minor or drastic has changed in our school system

More or other reasons…• We’ve run the program for at least a year now and

know what we need to do to help our students succeed

• We’re unhappy with our vendors’ results and want to replace them

• We’ve had a monitoring visit from our MDE consultant and they have targeted areas for improvement

• We’ve recently met with our local evaluator and the data shows that we need to do more of some things and less of others

NOTE: Some changes may need to be approved by MDE

• Changes in target populations• Changes in our sites (schools or other

locations)• Changes in budget and goals that have been

entered in MEGS

We can’t expect to change things we’re not working on

Grant Proposal – Part D. Project Plan

Project Goals ObjectivesActivities/Tasks/Staff

MeasurementStrategies

Increase academic achievement

48% of regularly participating students will improve by ½ grade in reading /language arts

(these are examples from Cohort F proposals)• pre-identify students needing academic assistance• develop personal learning plan• tutoring• homework help• Reading/literacy time (book clubs, library time)• ELA enrichment (poetry, music clubs)• Math enrichment (math, engineering clubs)

•Reading grades•Math grades•MEAP reading and math scores•Teacher surveys•State required standardized test scores48%... will improve

by ½ grade in math

MEAP reading/ Language arts scores will improve

(+ other federal requirements)

Academic Activities may take several forms:

• Homework help (support for homework completion)

• Tutoring (remedial instruction for 1-3 students)

• Academic enrichment focused on specific subjects

• Embedded learning (academic learning occurring within non-academic enrichment)

Although academic improvement is federally required, last year 5% of students participated in NO academic activities.

One Logic Model Approach

Source: United Way of America, 1996

INPUTS

Resources•money•staff•volunteers•equipment & supplies

Constraints•Laws•regulations•Funders, requirements

Services•shelter•training•education•counseling•mentoring

Products•classes taught•counseling sessions conducted•educational materials distributed•hours of service delivered•participants served

ACTIVITIES OUTPUTS OUTCOMES

Benefits for People• new knowledge• increased skills• changed attitudes or values

• modified behavior

•improved condition•altered status

Pre-inputs (assumptions)

Inputs (activities) Outputs

Initial outcomes

Intermediate outcomes

Ultimate outcomes

The pieces that need to be in place to successfully carry out activities:• Staffing• Resources• Expertise• Partnerships• Leadership

The activities that need to be conducted in order to result in change.• Service programs• Education• Information• Support• Connections

The products that are needed to support your initial outcomes. • # served• # completed• # offered• # contacted• # distributed• # recruited• $ generated

The precursors to the intermediate outcomes (these may be harder to measure than the intermediate outcomes)

Change in:• Knowledge• Skills• Resources• Attitudes• Behaviors (short-term)

The contributors to

the ultimate outcomes.

Change in• Behaviors (long-term)• Practices

The ultimate goal. Usually affected by many other factors. Very difficult to assess your impact at this level.• More vital communities• Healthier, more successful individuals and families• Sustainable practices

Logic Model

Improved academic outcomes

Improved behavioral outcomes

GOALS

Increased academic

engagement /skills

INTERMEDIATE OUTCOMES

IMMEDIATE OUTCOMES

Linkages within school

Family support of

involvement

Engagement with staff

Utilization of programs

Increased valuing of education

Increased internal assets

Improved socio-

emotional functioning

School, program, and community context

Individual and family characteristics

PROGRAM IMPLEMENTATION

Have identified theory of

change/goals

Collaboration with school

and community

Curriculum/ activities tied

to goals

Sufficient staffing/ training

Recruitment/ retention

processes

Improved academic

outcomes

Improved behavioral

outcomes

GOALS

Increased academic

engagement

INTERMEDIATE OUTCOMESIMMEDIATE OUTCOMES


Family support of

involvement



Increased valuing of

education

Increased academic

skills

Improved socio-emotional

functioning





change/goals

Collaboration with school and

community

Curriculum/ activities

tied to goals

Sufficient staffing/

training


processes

Academic Logic Model

Sites will host at least 5

projects directly linked with

the school curriculum.

Teacher surveys will have

a 70% higher satisfaction

rate with the academic

consultation process.

90% of the students will

receive tutoring and

homework assistance at least 2

days/wk.

Academic enrichment

represents at least 40%

of the program.

Incentives for academic

achievement will be

established at each site.

Graduation rates

will increase by

10% by 2011

75% of participating parents

will demonstrate they have

learned positive practices to

support academic

achievement.

Provide academic

enrichment for

students with learning

disabilities.

Student

absenteeism will be

reduced by 5% per

year.

Suspensions &

expulsions will each be

reduced by 10% per

year.

Mastery of new

computer software.

Increase frequency

and enjoyment of

independent reading.

48% of regularly attending

students will improve by ½

grade in reading/LA and

math grades

Map out your academic logic model

• Start with the GOAL and work back• Show all the academic activities that you’re

doing• You may be doing other things with these

groups of people, but we’re just focusing on those things you’re doing specifically to improve students’ grades.

• You also may be doing nothing in these areas.

Students

Students’ Parents

School-Day Teachers

21st CLC Program Staff/Teachers

21st CLC Program Policies

Activities Initial Outcome Intermediate Outcome GOAL

48% of regularly participating

students will improve by ½

grade in reading /language

arts & math

It’s not All Squares BINGO!

• Just because there’s an empty square, you don’t have to fill it.

• Think back to the purpose of this effort.

• Choose links that are most critical to success in this area.

To get improvements in academics

• Balance homework help with other academics

• Embed learning in non-academic activities• Have reasonable proportions of academic

activities to non-academic activities

Activities that improve reading are different than the activities that improve math

• Improved reading grades: – A greater proportion of academic enrichment– A more enjoyable program and supportive staff

• Improved math grades– More total hours of attendance– More hours of homework help/tutoring – Greater proportion of any academic activities and arts

Grant Proposal – Part D. Project Plan

Project Goals ObjectivesActivities/Tasks/Staff

MeasurementStrategies

Increase student learning in non-academic areas

85% of regularly participating students … report that the program helped them in non-academic areas (e.g., leadership, peer relations, community service, sports skills, drug and alcohol resistance skills, etc.)

• gender-specific personal leadership programs• quality non-academic enrichment programs such as culinary arts, conflict resolution, fine arts, etc.• classes such as karate, basketball and dance designed to keep kids active & improve physical fitness

• Student surveys• Parent surveys• Search Institute surveys• Michigan Profile for Healthy Youth

Improved academic

outcomes

Improved behavioral

outcomes

GOALS

Increased academic

engagement



Family support of involvement




education

Increased academic

skills

Improved socio-






change/goals

Collaboration with

school and community


tied to goals


training

Recruitment/ retention processes

Improved Behavior Logic Model

Students report a reduction

in risky behavior such as

smoking and alcohol use.

Increase students’ sense of

empowerment by creating

youth councils at each site.

Personal, social &

emotional enrichment will

be at least 20% of the

program.

Increase opportunities for

family enrichment

activities to foster

improved family relations.

90% of students will

participate in at least one

community involvement

activity.

Expand students’ learning and

developmental experiences by

connecting exceptional after

school program providers to

public school academies.

Students will decrease their

positive attitudes toward

violence by 10%.

Develop social skills of

cooperation, empathy,

assertiveness, self-control &

responsibility.

Student absenteeism

will be reduced by

5% per year.Suspensions & expulsions will

each be reduced by 10% per

year.

95% of the students will

report feeling valued and

respected by staff.

Parents & students will

perceive the schools as

being a community asset

beyond its role in

academics.

Students will be prepared for

their future personally and in the

work place.

Partners effectively

model staff and

community the skills,

attitudes & knowledge

expected of the students.

Increase the cultural

competency of new

immigrant students.

Which sites have higher rates of retention?Sites with:• Staff who say that they give students opportunities for

choice and decision-making• Staff who feel that they are more competent at

connecting with students• Safer environments (observed)• Staff-student interactions that promote more active

participation and leadership (observed)• Stronger relationships with school-day teachers• Less staff turnover

Improved academic

outcomes

Improved behavioral

outcomes

GOALS

Increased academic

engagement



Family support of

involvement




education

Increased academic

skills

Improved socio-






change/goals

Collaboration with

school and community


tied to goals


training


processes

Program Improvement Logic Model

Each site will partner with a

minimum of 8 community-based

organizations for programs &

services.

Before/After school committees will

have identified & initiated a

sustainability plan before the end of

funds that will ensure the continuation

of the program.

A bi-weekly conference will

be offered by the site

coordinator and scheduled at

the parent’s convenience.

By the end of the program

year, a total of (some

number) of students will be

enrolled at the 5 sites.

Provide an array of enjoyable learning

experiences not available during the school

day.

95% of the students will report

feeling valued and respected by

staff.

A ratio of no more than 15 students to 1 adult will be

maintained.

Staff, volunteers and teachers will receive at least 10

hours of in-service training per year.

So how do we get the biggest bang for our buck?

Ensure your program is designed to meet your goals

• Assess objectives and make sure they are designed to meet both federal goals and local needs

• Assess activities and see if they are likely to help you meet your objectives

– Meaningful academics linked to school day– Balance homework help with other academics– Embedded learning– Reasonable proportions of academics to non-

academics

Make sure your staff are well-trained

• Hiring teachers for academics is good practice• Provide continuous professional development

before and during the program year• Staff are better at being warm and supportive

than at providing opportunities for decision-making and meaningful learning—and this is a key point for making a difference

Connect to the school

• Connect to the classroom curricula to give your academic support the biggest bang for the buck

• Make connections with the classroom teachers to get more support and referrals for your program

• Get ideas from teachers about what activities would be helpful

• Better relations may help get better facilities

Goal Development & Measurement Celeste Sturdevant Reed, PhD University Outreach and Engagement...

Documents

Transcript of Goal Development & Measurement Celeste Sturdevant Reed, PhD University Outreach and Engagement...