Evaluation Overview Plus 50 Conference July 29 - 30, 2009 Washington, DC.

Evaluation Overview

Plus 50 Conference July 29 - 30, 2009Washington, DC

Faces to Names and Voices: Your Evaluation Liaisons

Kathleen Hentz• Spokane• Luzerne• Richland

Laura Bekes• Central Florida• Wake Tech• Northern Virginia

Samantha Chen• Cape Cod• Santa Fe• Saint Louis• Clark• Western Dakota

Overview

• Why Evaluate?

• How Do You Evaluate the Plus 50 Initiative?

• How Do You Evaluate the Plus 50 Grantees?

Why Evaluate?

• DOCUMENT implementation

• CAPTURE lessons learned

• TRACK outcomes over time

• SHOW the effectiveness of a model

Overview

• Why Evaluate?



How Do You Evaluate the Plus 50 Initiative?

Putting the evaluation puzzle pieces together:

– Research Questions– Research Tools

•Program Implementation:

•What progress have grantees made in implementing programs?

•Lessons Learned:

•What have grantees learned about what works and doesn’t work in program implementation?

•Participant Outcomes:

•Is program participation having the hoped-for results among the participants? Are they satisfied with the program?


Identifying Research Questions

•Data Collection Worksheets •Participant Surveys•Key Informant Interviews


Identifying Appropriate Research Tools


Implementation Assessment

• What is the progress on workforce training and career development goals for each college?

•Data Collection Worksheets

•Key Informant Interviews


How is implementation progressing?

2 4125

18

1 20

5

10

15

20

25

30

Unique Total Unique Total

Mentor Colleges(n=4)

Demonstration Colleges(n=10)

Ave

rag

e N

um

ber

of C

ou

rses

for S

tud

ents

Ag

e 50

+

Plus 50 Initiative Baseline DataTraining and Re-training Non-credit Courses

for Students Age 50+July 1, 2007 - June 30, 2008

Newly designed Redesigned Existing marketed


Lessons Learned

"The fact that we have monthly enrollment is very helpful for the students. …People are laid off year-round not necessarily aligned with an academic calendar.”

-Plus 50 College

"Short-term is very much in demand, as opposed to something long-term over many semesters. We’re looking at short-term programming that gets people out and on their way.”

-Plus 50 College

"The registration process has a lot of steps. We do it for them – if they had to jump through all those hoops they might give up. The college has also allowed us to skip a few steps.”

-Plus 50 College


Participant Outcomes Evaluation



• What outcomes do Plus 50 participants experience as a result of the their participation in training courses and career services?

•Participant Surveys



6%

5%

5%

6%

6%

5%

34%

35%

31%

55%

54%

59%

0% 25% 50% 75% 100%

Improvement in Technology Skills (n=119)

Improvement in Job Skills (n=131)

Acquisition of New Job Skills (n=130)

Percentage of Respondents

Skills Acquired by Plus 50 Workforce Training Participants

Completely Disagree Mostly Disagree Mostly Agree Completely Agree



20%

18%

20%

15%

25%

29%

13%

14%

33%

29%

38%

31%

22%

25%

30%

40%

0% 25% 50% 75% 100%

Got a raise(n=49)

Got a promotion(n=49)

Advanced in my job

(n=56)

Landed a job(n=107)

Percentage of Respondents

Job Outcomes of Plus 50 Workforce Training Participants

Completely Disagree Mostly Disagree Mostly Agree Completely Agree



“I will always remember you for caring and the important part you played in my success in securing a position with FedEx Express. …I received a call on Thursday to sign my offer letter on Friday at 10AM!!! ….There were times when I was almost running on empty. I wish many more success stories for the 50+ workforce people.”

-Plus 50 Participant



“The classes made me more aware of where I wanted to be going in my career at this time in my life. Also reviewing my skills and abilities, personality traits and strengths, I was able direct my energies toward solid decisions.”




“Perhaps you could offer training in specifically targeted skills. For example, a one-hour [practical workshop] on Saturday morning – practice with e-mailing attachments, practice with storing folders, files, etc…..Practice time with a knowledgeable person would be good.”



Additional Research Questions


Other Things we Want to Know

• How is the Plus 50 Initiative raising public awareness about the issues of Plus 50 learners, and the model?

• What have we learned about what works well in mentorship relationships?

• How has the AACC staff supported grantees?• Are the programs becoming sustainable, and

what strategies work for sustainability?

Overview

• Why Evaluate?



How Do You Evaluate the Plus 50 Grantees?

Looking at Individual Colleges

• Are colleges making progress against the goals set out in the proposal?

• Are colleges appropriately responding to lessons learned by finding ways to stay true to the overarching program objectives while changing implementation tactics to adjust to circumstances?

How Do You Evaluate the Plus 50 Grantees? What is Success for Individual Colleges?

•Information that helps inform AACC on individual college Plus 50 implementation and development:

•Progress Reports

•Site Visits

•Conference Calls

•Data Collection Worksheets

•Participant Surveys

How Do You Evaluate the Plus 50 Grantees? What is Success for Individual Colleges?

• What’s working?

• What would work better in the coming year?

Thoughts on the Year 1 Evaluation

• Participant Surveys

• Data Collection Forms for Year 2

• Key Informant Interviews

Evaluation Next Year

Questions

Thank You!

Evaluation Overview Plus 50 Conference July 29 - 30, 2009 Washington, DC.

Documents

Transcript of Evaluation Overview Plus 50 Conference July 29 - 30, 2009 Washington, DC.