Increasing Student Retention and Graduation: Findings from the ASAP Evaluation
Innovations ConferencePhiladelphia, PA March 6, 2012
ASAP History» 2007: Admit 1,132 students at six CUNY community colleges
with support from the Mayor’s Center for Economic Opportunity (CEO)
» 2010: Exceed 3-year graduation goal and funding made permanent allocation to CUNY by the Mayor’s Office
» 2011: CUNY announces plans to expand ASAP to more students at CUNY community colleges
ASAP History» Fall 2007 enrollment across all six CUNY community colleges:
˃ Borough of Manhattan: 249˃ Bronx: 118˃ Hostos: 82˃ Kingsborough: 247˃ LaGuardia: 208˃ Queensborough: 228
» Fall 2007 cohort was fully skills proficient at time of entry; 28% had developmental need when recruited and addressed over summer
» Since Fall 2009, began to admit students with some developmental need at time of entry
ASAP Goal» At least 50% of ASAP students will complete an
Associate’s degree in no more than three years ready to transfer to a baccalaureate program and/or enter the workforce.
ASAP Theory of ActionSupport students with:
1. Gaining and maintaining academic momentum
2. Developing a connection to the college
3. Accessing timely and relevant resources and services
Students are better engaged and graduate in a timely manner
ASAP Program Design
˃Required full-time study ˃Consolidated class schedule ˃Cohort design by major ˃Winter and summer course taking˃Dedicated full-time staff at each college
ASAP Resources & Services» ASAP Financial Resources:
˃ Tuition waivers for financial aid-eligible students˃ Free Use of Textbooks ˃ Monthly MTA MetroCards
» ASAP Services:˃ Case management advisement˃ Faculty engagement˃ ASAP Seminar˃ Academic support services˃ Career development services˃ Special programs
ASAP Evaluation and Key Outcomes
ASAP Data AnalysisQUANTITATIVE:» Student-level data from the CUNY Office of Institutional
Research and Assessment and colleges» Student Surveys
QUALITATIVE:» Annual Student Focus Groups
ASAP EVALUATION» Constructed Comparison Group (using CUNY data)˃ Entered one year prior to ASAP Cohort
˃ Met same criteria as ASAP students• Full-time Associate students• Begin program with 12 or fewer credits• Not enrolled in College Discovery (program offering similar services as
ASAP)• New York City residents• Enrolled in majors offered to ASAP students• Developmental Education
Cohort 1: Fully skills proficient in reading, writing and mathCohort 2: Allowed up to two remedial courses (in any subject)
ASAP Cohort One Three-Year Outcomes
Summary Profile: Cohort One ASAP and Comparison Group Students
Fall 2007 ASAP Fall 2006
Comparison Group
Total Enrollment N 1,132 1,791Gender
Male % 45.6 46.8Female % 54.4 53.2
Race/EthnicityAmerican Indian/Native Alaskan % 0.2 0.2Asian/Pacific Islander % 10.2 15.6Black % 31.5 26.8Hispanic % 37.2 29.2White % 20.8 28.2
Age Group18 or younger % 58.3 35.119 or 20 % 22.2 36.121 to 23 % 8.5 16.524 or older % 11.0 12.3
Median Age median 19 20Mean Age mean 20 21
Receiving a Pell Grant* % 57.9 67.1Income median $35,008 $29,231**
Admission TypeFirst-Time Freshmen % 75.2 36.9Transfer Students % 9.5 22.3Continuing Students % 15.3 40.8
*Based on dependent students only. **Based on 2007 comparison group's income because income data for 2006 comparison group was not available. Fall 2007 comparison group students met the same criteria as ASAP students but chose not to enroll in ASAP.
Regression Matching Techniques
» Used one-to-one propensity score matching˃ Led to large sample loss
» Used optimal full matching as final procedure˃ One-to-many matching procedure˃ Maintained most of ASAP sample˃ Conducted in partnership with Metis Associates (external evaluators)
ASAP Cohort One Three-Year Graduation Rate: Before and After Optimal Full Matching
Statistical Mean Difference = 28.4 Percentage Points, p < 0.05
0%
10%
20%
30%
40%
50%
60%
Before Matching After Matching
55.0% 54.6%
24.7% 26.9%
ASAP (Fall 2007) Comparison group (Fall 2006)
N=1,132 N=1,791 N=1,104 N=1,242
ASAP Cohort One Three-Year Cumulative Credits and GPA: After Optimal Full Matching
Statistical Mean Difference = 6.6 Percentage Points, p < 0.05
N=1,100 N=1,247
25
30
35
40
45
50
ASAP(Fall 2007)
Comparison group(Fall 2006)
47.3
40.7
˃ GPA Outcome – no statistically significant difference between groups (ASAP GPA: 2.50 , Comparison GPA: 2.46)
Graduation Predictors Study» Research Question
˃ Which variables are good predictors of two-year graduation • studied cumulative GPA, credits earned in 1st semester, and advisement
meetings in 2nd year˃ OUTCOME: Two-Year Graduation
» Sample˃ ASAP students enrolled for four consecutive semesters (N=761)
» Control Variables˃ Gender, Race, Age, Admission type, Parental education, Household income, High
school average, Regents’ scores
» Results˃ All three predictor variables significant
• as predictor variable increases so does probability of graduating in two years
16 meet
1 SD = 7 meetings
RESULT: probability of graduating in 2 years increases by 10 percentage points.
23 meet9 meet
2nd Year Advisement Meetings Outcome:
For every standard deviation increase in advisement meetings during the second year, approximately 7 meetings above the mean of 16.3 meetings, the probability of graduating increases by 10 percentage points.
ASAP Cohort Two Two-Year Outcomes
Summary Profile: Cohort Two ASAP and Comparison Group StudentsCohort 2 ASAP
Students
Fall 2008 Comparison
Group Students*
Total Enrollment N 429 1,510Gender
Male % 40.1 38.1Female % 59.9 61.9
Ethnicity
American Indian/Native Alaskan % 0.2 0.5Asian/Pacific Islander % 9.8 16.8Black % 35.4 27.6Hispanic % 37.5 41.2White % 17.0 13.9
Age Group
18 or younger % 41.0 33.219 % 18.2 17.820 to 22 % 18.4 25.823 to 29 % 13.1 15.730 or older % 9.3 7.5
Mean Age mean 22 21Admission Type
First-time Freshmen % 69.5 50.5Transfer Students % 7.0 17.5Continuing Students % 23.5 32.0
Developmental Students1 % 76.7 75.8Pell Receipt % 84.6 90.5Household Income
Dependent Students2 mean 26,103 26,042Independent Students mean 13,050 11,832
Retention of Cohort Two ASAP Students: Re-enrolled in College of Entry
50%
60%
70%
80%
90%
100%
2nd Semester 3rd Semester 4th Semester
93.9%
84.4%
77.6%
85.8%
66.9%
59.1%
ASAP (Fall 2009) Comparison group (Fall 2008)
One-Year Basic Skills Proficiency Change:ASAP Cohort Two and Comparison Group Students
0%
10%
20%
30%
40%
50%
60%
70%
80%
Dev. Need(s)1st Sem
Dev. Need(s)2nd Sem
Dev. Need(s)3rd Sem
76.0%
43.5%
16.4%
75.1%
59.9%
39.7%
ASAP (Fall 2009) Comparison group (Fall 2008)
Two-Year Graduation Rate of ASAP Cohort Two Students
0%
5%
10%
15%
20%
25%
30%
ASAP(Fall 2009)
Comparison group(Fall 2008)
27.5%
7.2%
ASAP Student Survey and Focus Group Results
ASAP Students’ Educational Aspirations
“I wouldn’t be in school (without ASAP)…I learned the sky is the limit.”
~ Cohort 2 student
0%
20%
40%
60%
80%
100%
Graduate/professional degree
4-year degree 2-year degree
69%
29%
2%
63%
34%
3%
ASAP cohort 1 (Fall 2007)
ASAP cohort 2 (Fall 2009)
Graduating Two Year ASAP Students :Most important ASAP Resource/Service to College Success
“Tuition and books being paid for really helps because there is less to worry about.”
~ Cohort 2 student
0%
20%
40%
60%
80%
100%
FinancialResources
Services - ASAPAdvisor
EarlyRegistration
All Other ASAPServices
70%
20%
2%7%
72%
19%
3% 6%
ASAP cohort 1 (Fall 2007)
ASAP cohort 2 (Fall 2009)
Graduating Cohort 2 Students: ASAP Resources “Very Helpful” to Student Success in College.
“Advisors track students academics; they play a huge role. If you have a problem, you go to your advisor.”
~ Cohort 2 student
0% 20% 40% 60% 80% 100%
Financial resources
Services - ASAPAdvisor
Early registration
Services - ASAP CES
Small class size
ASAP tutoring
Block programming
98%
92%
85%
64%
51%
49%
42%
Multiple responses allowed
Graduating Cohort 2 Students: Advisor Services
“It would be dramatically different without advisors. They know what you need. Without an advisor, the load would be on you to get what you need. It would not feel as personal.”
~ Cohort 2 student
0% 20% 40% 60% 80% 100%
Emphasized the importance of timemanagement
Made sure they were attending classregularly
Provided strategies to help deal withacademic problems
70%
66%
63%
28%
30%
32%
My ASAP Advisor Has:
Strongly Agree Agree
Graduating Cohort 2 Students: How ASAP Has Helped Me
“ASAP has done so many things to change my life…I am overwhelmed by the experience...”
“I’m involved more in college because of ASAP activities. It’s been a great experience.”
~ Cohort 2 students
0% 20% 40% 60% 80% 100%
Come closer to achieving my goals
Inspired by at least one of myprofessors
Learned to ask for help
75%
67%
65%
23%
28%
30%
As a Result of Being in the ASAP Program, I Have:
Strongly Agree Agree
Graduating Two Year ASAP Students :Post-Graduation Plans
“ASAP plays a major role by helping me stay focused”.
~ Cohort 2 student
0%
20%
40%
60%
80%
100%
Attend4-yr college
Work Other Plans
92%
32%
5%
94%
39%
3%
ASAP cohort 1 (Fall 2007)
ASAP cohort 2 (Fall 2009)
Multiple responses allowed
ASAP Focus Groups: Recurring Themes
» ASAP provides connection to the program and between classmates.
“ASAP has helped me to learn how to speak and interact with people. It has opened doors for me to be free.”
» ASAP encourages success, provides direction, and keeps students on track academically.
“ASAP provides a roadmap for what you should focus on.”
» ASAP gives students the skills to navigate the college experience and utilize available resources.
“ASAP tells you where to go, how to contact them, and who you need to talk to.”
» $5 million in additional foundation funds raised to expand ASAP program and evaluation capacity
˃ Transfer scholarship program for ASAP graduates at six CUNY senior colleges
˃ Random assignment study by MDRC at BMCC, Kingsborough, and LaGuardia
» Fall 2011: CUNY announce plans to expand ASAP to more students at CUNY community colleges
ASAP Updates
ASAP Expansion» Expanding ASAP over the next three years at all
community colleges to 4,000 + students by fall 2014» 1,500 will be recruited for fall 2012» Central & college planning teams developing
expansion plans» Citywide outreach & marketing campaign includes
ASAP graduates
Contact Us
www.cuny.edu/asap
Top Related