REPORT College Bridge for All...Summer melt is a phenomenon in which college-intending students with...
Transcript of REPORT College Bridge for All...Summer melt is a phenomenon in which college-intending students with...
Office of Research, Evaluation & Program SupportCUNY Office of the Senior University Dean for Academic Affairs
REPORT
College Bridge for All2018 Descriptive Report
BY GINA AHN AND ELLEN STUDER
November 2018
Office of Research, Evaluation & Program Support | May 2019 2
Acknowledgements We are grateful to our program partners, Laura Myers and Maggie Brown, for their thoughtful collaboration on this report. We would like to acknowledge the contribution of Fahad Ahmed and Christopher Viera in analyzing focus group data and synthesizing findings, and Melissa De Feo (Research and Policy Support Group at NYCDOE) for reviewing an earlier version of the report. Thank you to the College Bridge for All students, college coaches, and hub supervisors for sharing openly about their experiences in the program and providing valuable feedback. College Bridge for All is a partnership between the City University of New York (CUNY) and the New York City Department of Education (NYCDOE).
Office of Research, Evaluation & Program Support | May 2019 3
Executive Summary Summer melt is a phenomenon in which college-intending students with limited resources and support fail to matriculate to college in the fall semester following high school graduation, and it affects between 20 to 40 percent of students every year (Castleman & Page, 2014). The summer melt rate is disproportionately higher for low-income students and students of color, and, in New York City Department of Education (NYCDOE) schools, 74.0 percent of students are economically disadvantaged and 66.5 percent are students of color.
College Bridge for All (CB4A) is an innovative and promising model of intervention for summer melt that utilizes a near-peer mentoring strategy to proactively support college-bound students with a wide range of matriculation tasks. In 2018, CB4A was implemented at 79 NYCDOE schools, where a total of 99 college coaches worked with 6,186 students.
This report is structured to address three central questions: 1. What implementation recommendations from the previous year were successfully addressed? 2. What program processes are under development? 3. What are the preliminary matriculation outcomes of CB4A students, and how do they vary for
different subsets of students?
Drawing on the findings, themes, and recommendations from the 2017 CB4A implementation report while also examining new aspects of implementation that were introduced in 2018, we examined the early successes seen in the CB4A program. In particular, we found that:
In terms of students’ preliminary college matriculation outcomes, we found that, as of October 2018,
While the results in this report are descriptive in nature, we find promising early results for CB4A: The near-peer mentoring strategy appears to have many advantages for the population of students served by the program. This report will provide valuable context to complement the forthcoming 2017 CB4A Impact Report.
1. CB4A successfully integrated several program recommendations from 2017, particularly around coach training and the use of programmatic tools.
2. College coaches initiated contact with students earlier in 2018 versus 2017 and engaged with them frequently both during the school year and over the summer, but the most effective method of communicating with students remains unclear.
3. There continues to be a need for college matriculation advisement during the summer months before college, and CB4A is well-equipped to bridge this gap for New York City’s high school graduates.
77.8% of college-bound CB4A students matriculated to
college in the fall semester
74.8% of the students who matriculated in the
fall enrolled in a CUNY college
86.0% of highly engaged
CB4A students matriculated to
college in the fall semester
Office of Research, Evaluation & Program Support | May 2019 4
Table of Contents
Introduction ................................................................................................................................................................................. 5
Background.................................................................................................................................................................6
Program Description and Structure .................................................................................................................. 7
Methods and Data .....................................................................................................................................................................9
Student Characteristics ......................................................................................................................................................... 11
Program Findings .................................................................................................................................................................... 13
Finding 1. .................................................................................................................................................................... 13
Finding 2. ................................................................................................................................................................... 16
Finding 3.. ................................................................................................................................................................. 20
Program Outcomes ............................................................................................................................................................... 24
Recommendations ................................................................................................................................................................. 28
Conclusion ................................................................................................................................................................................ 29
References ................................................................................................................................................................................ 30
Appendix .................................................................................................................................................................................... 31
Table of Contents
Office of Research, Evaluation & Program Support | May 2019 5
Introduction
Summer melt is a phenomenon in which students intending to go to college fail to actually matriculate, and it affects between 20 to 40 percent of college-intending students every year. The summer melt rate is disproportionately higher for low-income students and students of color, which previous research has found to be as high as 59 percent1. Although college admissions offices have long known about summer melt, research on this issue is only beginning to develop and there is little information about effective interventions to prevent summer melt. Existing interventions that tackle summer melt at a large scale rely on small behavioral “nudges” via automated text messages2.
College Bridge for All (CB4A) is an innovative and promising model of intervention for summer melt that utilizes a near-peer mentoring strategy to proactively support college-bound students with a wide range of matriculation tasks. While nudges and digital campaigns have the advantage of being large-scale and low-cost, students who are at risk of failing to enroll in college often need more involved, hands-on assistance in navigating the college enrollment landscape beyond automated reminders of tasks and deadlines. In CB4A, trained college coaches—who are often alumni of the high school where they are placed—work closely with students during the summer months before college. The activities that college coaches assist students with range from helping them complete matriculation and financial aid paperwork, to visiting college campuses to help students make decisions about where to matriculate.
In New York City Department of Education (NYCDOE) schools, 74.0 percent of students are economically disadvantaged and 66.5 percent are students of color, making tackling summer melt a particularly critical issue in increasing college access. CB4A, the only large-scale summer melt intervention of its kind, employs a comprehensive approach to assist New York City’s college-bound students during the summer after high school graduation. In 2018, CB4A scaled rapidly and engaged 6,186 students at 79 high schools, a 65 percent increase from 2017. Furthermore, CB4A is a partnership between one of the largest urban public university systems in the United States—the City University of New York (CUNY)—and the largest school district in the United States, the NYCDOE. Over three fourths of CUNY’s first-year students in 2016 were NYCDOE graduates, and it has become increasingly important for the two school systems to work together to improve students’ successful transition from high school to college3. This unique partnership presents a valuable opportunity to address a significant blind spot in the college matriculation process, raise awareness about these shortcomings, and refine a promising model of intervention with positive preliminary results.
This report is structured to address three central questions:
1. What implementation recommendations from the previous year were successfully addressed? 2. What program processes are under development? 3. What are the preliminary matriculation outcomes of CB4A students, and how do they vary for
different subsets of students?
1 From Daugherty’s (2012) examination of Summer Link, an intervention in Fort Worth Independent School District https://sdp.cepr.harvard.edu/summer-melt. 2 Georgia State University and the D.C. College Access Program have used this method to remind college-intending students of important deadlines and tasks that need to be completed over the summer to some degree of success. 3 Chellman, C. & Truelsch, S. (2017). The state of CUNY 2017: Where we have been, where we are at, where we are going.
Office of Research, Evaluation & Program Support | May 2019 6
Drawing on the findings, themes, and recommendations from the 2017 CB4A implementation report4 while also examining new aspects of implementation that were introduced in 2018, this report examines the changes made to and early successes seen in the CB4A program. A central goal of this work is to better understand and document the strengths of the CB4A model, as well as any challenges that can yet be learned from.
This report is also intended to provide valuable context to complement the forthcoming 2017-2018 CB4A impact report. In making sense of the impact findings to come, it will be critical to reflect on what we have learned about the program’s implementation to best understand the strengths and potential contributions of the CB4A model in the summer melt intervention landscape.
Background
The months between high school graduation and college matriculation pose steep challenges for many students as they are faced with a complex array of tasks required for successful college enrollment. Low-income and first-generation graduates, in particular, have limited access to professional supports that would help them navigate this work (Castleman & Page, 2014). High school counselors tend to be unavailable in the summer months, and first generation college-goers5 lack the benefit of parents or family members who are familiar with the enrollment process (Castleman & Page, 2014). As a result, many college-intending low-income students fail to matriculate to college in the fall after graduating from high school. To describe this phenomenon, researchers have coined the term “summer melt.” According to Castleman and Page (2014), summer melt affects between 20 to and 40 percent of college-intending students each year, and summer melt rates are higher for students who plan to enroll in community colleges.
A growing body of research has shown that targeted interventions and supports for college-intending high school graduates can help reduce the summer melt phenomenon. Multiple studies have found that providing low-income students with college counseling during the summer effectively increases the rate and quality of college enrollment (Castleman, Arnold, & Wartman, 2012; Castleman, Page, & Schooley, 2014). Additional research has suggested that peer mentor outreach and personalized text messages reminding students of remaining matriculation tasks help to increase college entry among underrepresented populations (Castleman & Page, 2015).
The research on near-peer college advisement during the summer months is limited, but there are a few notable studies that underscore the advantages of the strategy. Castleman, Page, and Snowden (2013) found that, at uAspire and the Mastery charter schools in Lawrence, Massachusetts, peer mentor outreach reduced summer melt and increased college matriculation by four to seven percentage points. A study of near-peer college advising in New Hampshire public schools, where in-person mentorship was offered with cash incentives to boost college matriculation and persistence, found large impacts on college enrollment and persistence (Carrell & Sacerdote, 2013).
4 Ahn, G., Campbell, V., Madsen, C., & Studer, E. (2018). College Bridge for All Implementation Report: Results from the 2017 Program Expansion. 5 40.3 percent of College Bridge for All student survey respondents indicated that if they were to enroll in college, they would be the first person in their family to attend college.
Office of Research, Evaluation & Program Support | May 2019 7
Program Description and Structure
College Bridge for All (CB4A) was launched in 2016 as a part of NYCDOE’s College Access for All Initiative to address summer melt in New York City high schools. CB4A, a partnership between CUNY’s K-16 Initiatives unit and the Office of Postsecondary Readiness (OPSR) at NYCDOE, hires and trains current college students to support graduating seniors at participating high schools to complete all the steps to matriculate into post-secondary institutions. College coaches work closely with high school seniors who are on track to graduate with crucial pre-matriculation tasks, such as completing financial aid forms, registering for classes and placement tests, and navigating obstacles to enrollment that arise during the summer before college. Outlined below are the key work responsibilities of college coaches, called the Lucky 7, and the ten student data points that coaches are responsible for collecting, called the Big 10.
The goals of CB4A are to: • increase the number of NYCDOE graduates who immediately matriculate to college and
participate in academic support programs, with a focus on ensuring strong-fit college choices;
• utilize near-peer mentoring to support student success and matriculation, as well as to develop transferable professional leadership skills in current college students through their work as college coaches; and
• increase each school community’s capacity to provide high-quality college matriculation counseling, while fostering collaborative practices and youth development culture school-wide.
Lucky 7
1. Talk to seniors about what to expect in college
2. Update and verify financial aid 3. Compare and review admissions
offers 4. Help students enroll in CUNY
support programs 5. Help students complete enrollment
paperwork 6. Ensure appropriate first-year class
registration 7. Support last minute college bound
or wavering students
Big 10
1. Postsecondary Plan 2. Deposit Paid or Waived 3. Type of College Student Will Attend 4. Name of College Updated on the
TAP 5. Physical and Immunization Forms
Submitted 6. Accepted Financial Aid Award
Package 7. Placement Test Taken 8. Final Transcript Submitted 9. Enroll in Opportunity Program 10. Registered for Classes
Office of Research, Evaluation & Program Support | May 2019 8
Since its inception, CB4A has used a data-informed approach to working with students. The program trains college coaches to track and monitor student data throughout the summer using the EnrollNYC database. Coaches track student engagement data, including method of outreach (e.g., email, social media, in-person), topic of outreach (e.g., financial aid, applications, matriculation paperwork), and whether or not the student responded to the coach. Having a central database of student data and progress allows coaches to follow up with students who may need assistance. For example, coaches might specifically follow up with students who are attending CUNY community colleges to share information about opportunity and special programs on campus.
A total of 79 NYCDOE schools participated in CB4A in 2018, a 65% increase from 48 schools in 2017 (Figure 1). Unlike previous years, two different models of CB4A were implemented in 2018. In the central model, 71 schools and 93 coaches were managed jointly by a CUNY Program Director and an NYCDOE Program Director. The eight schools in the Student Success Center (SSC) model were managed independently by Program Directors at each SSC. Of the 71 schools in the central model, 23 schools were schools that participated in CB4A in 2017 and 48 schools were new to the program. Two Early College Initiative (ECI) schools6 and seven new transfer schools7 participated in CB4A for the first time in 2018.
6 The CUNY Early College Initiative (ECI) works with 17 high schools in NYC to make higher education more attainable and affordable by providing students with the opportunity to earn college credits during high school at no cost. ECI schools are unlike traditional NYCDOE high schools in that there are a variety of school designs, such as grades 9-14 schools and Career and Technical Education (CTE) schools. Source: http://earlycollege.cuny.edu. 7 Transfer High Schools are different from traditional NYCDOE schools in that they re-engage students who have dropped out or fallen behind in credits by offering a rigorous academic program in addition to career exploration and job readiness services. Source: https://www.schools.nyc.gov/enrollment/other-ways-to-graduate/transfer-high-schools.
Key roles in CB4A:
• College coaches provide college matriculation support five hours per week under the supervision of and in collaboration with the high school’s college counselor.
• College counselors attend planning meetings coordinated by NYCDOE and K-16 Initiatives program leadership. Counselors also hire college coaches, prepare and update a database with coaches’ student caseload, and meet biweekly with coaches to review goals and challenges and ensure they have access to school-based resources.
• Hub supervisors provide individual supervision and group training to coaches.
Office of Research, Evaluation & Program Support | May 2019 9
Figure 1. Breakdown of different types of CB4A schools
Methods and Data This report employs descriptive analyses to examine the implementation of and preliminary matriculation outcomes for the 2018 CB4A cohort. The research team conducted a student survey and a coach survey to explore how the program was experienced by college coaches and by students, both generally and in response to particular implementation modifications unique to the 2018 cohort. To support program development, the research team helped design a coach focus group protocol that was used by hub supervisors, who conducted the focus groups at the conclusion of the program. Program data from EnrollNYC—the database used by college coaches for caseload management and progress tracking—was extracted periodically over the summer and used to assess the frequency and timing of program activities, such as the completion of matriculation and enrollment tasks.
CUNY enrollment data from the CUNYFirst Administrative Data Warehouse (ADW)8 and college enrollment data from the National Student Clearinghouse (NSC) were used to observe preliminary matriculation outcomes for students served by CB4A. In cases of discrepancies regarding a student’s enrollment in CUNY between the ADW and NSC, we deferred to the CUNY enrollment data in ADW. Table 1 below summarizes the data collection for this study.
8 Data were retrieved on October 26, 2018. Although these data are current, enrollment records retrieved from the ADW should be considered preliminary and subject to change.
Office of Research, Evaluation & Program Support | May 2019 10
Table 1: Data Collection for the 2018 CB4A Descriptive Report
Participant Group Method / Data Data Collection Timeline
Students
Retrospective pre-post online survey to capture students’ experiences and perceptions in areas such as: program value add; appropriateness; trust and connectedness to coach; skills and knowledge gained; change in behavior related to matriculation steps; motivation; and educational and career aspirations.
August 2018, n=237 student respondents9
Demographic data from the New York City Department of Education (NYCDOE), including student race, gender, and graduation status.
REPS receives student data from the NYCDOE three times a year. The most recent data were from July 2018.
Program data from EnrollNYC database: coach case notes focusing on the nature of contact between coaches and students such as frequency, type, method, location, and content. Student tracker data, including information on students’ intent to matriculate and progress on matriculation tasks.
REPS extracted EnrollNYC data monthly from June 2018 – August 2018 (3 data pulls)
Matriculation data from the National Student Clearinghouse and CUNYFirst Administrative Data Warehouse
Retrieved October 19, 2018 and October 26, 2018, respectively
College Coaches
Retrospective pre-post online survey to capture training experience; structures and supports received; integration in schools; experiences in service delivery; perceived impact on students; perceptions of program mission; leadership; communication skills; and professionalism.
August 2018, n=95, 96.0% response rate
Focus groups to deepen understanding of program implementation, coach experiences and perceptions.
8 focus groups conducted by hub supervisors in August 2018.
9 Coaches shared a generic link with students they worked with, and as a result, we are not able to calculate the response rate, which takes into account the number of students who received the survey.
Office of Research, Evaluation & Program Support | May 2019 11
Student Characteristics A total of 10,109 high school seniors at 79 CB4A schools were eligible to opt-in to receive college matriculation assistance from a college coach during the summer months before the fall semester. Of these students, 73.5 percent of students graduated from high school before July 2, 2018, 2.1 percent were discharged, 1.1 percent transferred, and 23.2 percent were still enrolled in high school as of July (Table 2).
Of the 7,656 students who graduated from high school, a majority of students (80.8%) intended10 to matriculate to college in the fall semester following high school graduation, while 2.3 percent of students intended to matriculate in the spring semester following high school graduation and 8.2 percent of students planned to explore non-college options (Table 3). Unless otherwise specified, we restrict the analytic sample in this report to the 6,186 college-intending students who graduated from high school before July 2, 2018.
Table 2. High school graduation status for all students at Bridge schools n % Graduated from HS 7,431 73.5 Discharged 216 2.1 Transferred 112 1.1 Still enrolled 2,350 23.2 Total 10,109 100
Source: NYCDOE data files, current as of July 2, 2018 Notes: We assume that students without graduation or discharge dates are still enrolled in a NYCDOE high school. Of the 2,350 students without graduation or discharge dates as of July 2, 2018, 225 students had a fall college enrollment record in the National Student Clearinghouse or CUNYFirst; for these students, we assume that the student has graduated from high school before the fall semester and include them in analyses of college-intending high school graduates.
Table 3. Students’ intent to matriculate to college, restricted to high school graduates June July August n % n % n % Intend to matriculate in the Fall 5,719 74.7 5,871 76.7 6,186 80.8 Intend to matriculate in the Spring 106 1.4 163 2.1 177 2.3 Undecided 360 4.7 267 3.5 268 3.5 Other non-college option 484 6.3 594 7.8 628 8.2 Missing intent data in EnrollNYC 987 12.9 761 9.9 397 5.2 Total 7,656 100 7,656 100 7,656 100
Source: EnrollNYC Notes: College coaches and school counselors were asked to record students’ college plans in Enroll. REPS researchers downloaded these data at the end of each month over the summer
10 Data about students’ intent to matriculate to college was submitted by school counselors before college coaches began working with students, and college coaches updated these data in EnrollNYC over the summer. We use students’ college intent data from August because data become more complete over the summer (Table 3).
Office of Research, Evaluation & Program Support | May 2019 12
Consistent with the participant demographics from last year, CB4A served a diverse group of students that was majority Hispanic (41.7%), black (31.6%), and female (51.4%) (Figure 3). Coaches worked with students from all five boroughs, with the majority of students attending high school in Queens (36.3%) and Brooklyn (27.8%) (Figure 2).
Figure 2. School borough and gender of CB4A students (n=6,186)
Figure 3. Race/ethnicity of CB4A students (n=6,186)
16.0%
31.6%
41.7%
0.2% 1.2%
7.9%
1.4%
Brooklyn28%
Manhattan13%Queens
36%
Staten Island5% Bronx
18%
Female51%
Male48%
Missing1%
Office of Research, Evaluation & Program Support | May 2019 13
Program Findings Finding 1. CB4A successfully integrated several program recommendations from 2017, particularly around coach training and the use of program tools. In the 2017 CB4A implementation assessment, college coaches expressed in the post-program survey and in the focus groups that they would have benefitted from additional training and guidance on topics such as loan counseling, completing tax forms, working with undocumented students, and working with families.
The 2018 cohort of college coaches overall felt very prepared to assist students with most college-going tasks. Notably, coaches’ confidence around loan counseling and completing tax forms improved dramatically from 2017. However, there were certain topics where coaches still expressed a need for additional training, such as assisting undocumented students and completing financial aid forms.
Among coach survey respondents, over 90 percent reported feeling “comfortable” or “very comfortable” with tasks such as updating FAFSA and TAP applications, helping students complete enrollment paperwork, and supporting students in registering for classes (Figure 4).
Figure 4. Coach responses to the question: ‘After receiving the College Bridge for All training, how comfortable did you feel about helping students with the following tasks/ topics?’ C1
25.3%
28.4%
6.4%
12.6%
9.6%
11.6%
2.1%
10.5%
6.4%
2.1%
2.1%
2.1%
3.2%
1.1%
1.1%
36.8%
31.6%
46.8%
34.7%
37.2%
33.7%
43.6%
31.6%
34.0%
37.9%
30.5%
30.5%
25.3%
25.3%
23.4%
25.3%
31.6%
44.7%
47.4%
48.9%
51.6%
52.1%
53.7%
54.3%
56.8%
65.3%
65.3%
69.5%
71.6%
73.4%
Support undocumented students with the college process
Help students explore non-college options
College decision-making (choosing college, choosingmajor)
Support students with reallocation from one college toanother
Support students to apply for college through DirectAdmissions
Accompanying students to college campuses to completeenrollment steps
Comparing and reviewing admissions offers
Supporting students to enroll in ASAP, CUNY Start andother special programs
Completing financial aid verification forms
Completing FAFSA and TAP applications
Support students to register for classes
Helping students complete enrollment paperwork(immunization forms, enrollment deposit)
Support students to understand and register for CUNYplacement test
Updating FAFSA and TAP applications
Talking to seniors about what to expect in college
Very uncomfortable Uncomfortable Neither comfortable nor uncomfortable Comfortable Very comfortable
Office of Research, Evaluation & Program Support | May 2019 14
Although coaches felt adequately trained overall, there were a number of areas they would have liked more training. The task that most coaches reported feeling uncomfortable with was supporting undocumented students with the college process (12.6%) (Figure 4). Accordingly, when asked which topics they would have liked more training on, the majority of college coach respondents (58.9%) selected ‘Supporting undocumented students with the college process’ (Figure 5). A majority of coaches also reported wanting more training on helping students explore non-college options (53.7%). Other areas where a notable proportion of college coaches expressed a need for additional training included financial aid processes, supporting students with reallocation from one college to another (29.5%) and supporting students to enroll in ASAP, CUNY Start and other special programs (23.2%). In explaining the need for additional training on financial aid, one coach reported, “for me, I really did the whole thing, FAFSA and TAP, doing it from the beginning to the end. I think that in the training they didn’t teach you how to.” Another coach added,
I think they should add like more help on financial aid. Like I was really confused with financial aid and I don’t feel like training like really touched base on that. So when I came in April I had to do a lot of financial aid and I was confused and I had to ask for help because training didn’t really help with financial aid like that much.
Figure 5. Coach responses to the question: ‘Of the following topics, which would you have liked additional training on?’ C2
5.3%
5.3%
6.3%
7.4%
8.4%
8.4%
10.5%
11.6%
13.7%
13.7%
14.7%
23.2%
29.5%
53.7%
58.9%
Talking about what to expect in college
Supporting students register for classes
Supporting students to register for CUNY placementtest
College decision-making
Helping students complete enrollment paperwork
Comparing and reviewing admissions offers
Accompanying students to college campuses tocomplete enrollment steps
Supporting students to do direct admissions
Updating FAFSA and TAP applications
Completing FAFSA and TAP applications
Completing financial aid verification forms
Supporting students to enroll in special programs
Supporting students with reallocation from one collegeto another
Helping students explore non-college options
Supporting undocumented students with the collegeprocess
Office of Research, Evaluation & Program Support | May 2019 15
Last year’s implementation report concluded that EnrollNYC utilization varied significantly among coaches. Hub supervisors emphasized in a focus group that EnrollNYC was a useful but underutilized tool, and recommended making EnrollNYC more user-friendly. Specifically, they suggested that more time could be spent during coach training on showing coaches how the platform can be a valuable resource for caseload management. The 2017 report included recommendations that EnrollNYC data entry procedures be streamlined and standardized.
For the 2018 CB4A cohort, a number of changes were made to EnrollNYC to standardize the way coaches entered certain key data points. The changes included: 1) adding a feature on the case note entry page where coaches can indicate whether the student reciprocated the outreach our not; and 2) updating how topics were assigned to case notes, so as to better capture the matters discussed in interactions between coaches and students.
The standardization efforts were also facilitated by the introduction of the Big 10 and the Lucky 7, program tools created to better define and guide coaches’ data entry priorities. As detailed earlier in the report, the Big 10 outlines the ten important student data points that coaches are responsible for collecting, and the Lucky 7 describes coaches’ seven key work responsibilities. The majority of college coach survey respondents felt that the suite of caseload management tools, including EnrollNYC, the Big 10, and the Lucky 7, helped them with their work. Fully 87.4 percent felt that EnrollNYC was a useful tool for organizing and managing student interaction information (Figure 6). 88.4 percent of coaches agreed that the Lucky 7 helped them understand what their responsibilities and priorities were, and that the Big 10 helped them track progress on key matriculation activities.
Figure 6: Coach responses to the question: "To what extent do you agree with the following statements?"
11.6%
8.4%
10.5%
29.5%
30.5%
27.4%
57.9%
57.9%
61.1%
0% 20% 40% 60% 80% 100%
I felt that Enroll was a useful tool for organizingand managing student interaction information.
Big 10 data helped me track progress on keymatriculation activities.
The idea of the Lucky 7 helped me to understandwhat my responsibilities and priorities were.
Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree
Office of Research, Evaluation & Program Support | May 2019 16
Finding 2. College coaches initiated contact with students earlier this year and engaged with them frequently both during the school year and over the summer, but the most effective method of communicating with students remains unclear. In 2017, there was unanimous feedback from students, college coaches, hub supervisors, and school counselors around the timing of the program—many felt a strong need for the program to begin earlier in the spring before major matriculation-related deadlines. One college coach from 2017 explained, “it was so late when we got there… late in the process. A lot was done already when we got to the school.” In-person connections and formal introductions between college coaches and students were identified as a key element to successfully integrating college coaches in 2017.
College coaches initiated contact with students earlier this year11 and connected in-person more frequently during the school year as opposed to over the summer. 90.0 percent of student survey respondents met with their college coach at their high school before graduating and 64.7 percent of students met with a college coach either once a week or once every two weeks during April and May. In contrast, 62.0 percent of student respondents reported meeting with their coaches after graduation (Figure 8).
Another recommendation from 2017 was around establishing benchmarks for frequency of contact with students, as frequency of communication between coaches and students varied widely across the program last year. This year, 94.6 percent of student survey respondents reported receiving communications from their college coaches regarding college paperwork deadlines—including applications and various forms— at various frequencies throughout the summer. 60.2 percent of respondents received these communications either daily or weekly and 34.4 percent received them monthly or biweekly (Figure 7).
Although coaches reported having more in-person interactions with students during the school year compared to the summer, coaches overall had more interactions with students toward the end of the summer (Table 4). Approximately half of all coach-student interactions were recorded in EnrollNYC in August, and over one fourth of all interactions in July. Not surprisingly, college applications were the most prevalent theme for case notes written in April and May, while enrollment paperwork, financial aid, and opportunity and special programs were the most prevalent topics of communication in July and August.
As with previous CB4A cohorts, while college coaches were able to maintain consistent communication with most of their caseloads, unresponsive students remained a persistent issue. When asked about the biggest challenges faced as a college coach, nearly 90 percent of coach respondents (89.5%) marked reaching students who were unresponsive.
11 The 2018 post-program counselor survey that was administered by CB4A program directors shows that an overwhelming majority (63%) of school counselors would prefer to work with college coaches year-round, while 22 percent of counselors would like to work with college coaches between January and August.
Office of Research, Evaluation & Program Support | May 2019 17
Figure 7. Student responses to the question: ‘How frequently did you receive communication from your college coach regarding college paperwork deadlines?’ S4
Figure 8. Student responses to the question: ‘How often did you do the following in the past four months?’ S3
5.4%8.6%
25.8%
34.8%
25.3%
Never Monthly Once every 2 weeks Once a week Daily
76.7%
62.7%
38.0%
39.8%
26.6%
26.9%
10.0%
11.0%
15.0%
25.8%
17.2%
22.5%
18.7%
25.3%
3.2%
7.7%
11.3%
18.1%
24.3%
23.7%
17.2%
9.1%
14.5%
24.9%
24.9%
26.6%
30.6%
47.5%
Have your College Coach speak with your familymember(s)
Speak with your College Coach by phone
Meet with your College Coach at your high school(after graduation)
Use social media to communicate with your CollegeCoach
Use email to communicate with your College Coach
Use text messaging to communicate with yourCollege Coach
Meet with your College Coach at your high school(before graduation)
Never Monthly Once every 2 weeks Once a week
Office of Research, Evaluation & Program Support | May 2019 18
Table 4. Total number of case notes by topic and timing of interaction12
Themes
April May June July August Total n % n % n % n % n % n
College Admissions 531 4.8 1,272 11.5 1,375 12.5 3,051 27.7 4,788 43.5 11,017
College Applications 10 15.4 11 16.9 16 24.6 7 10.8 21 32.3 65
Enrollment Paperwork 281 2.0 1,414 10.2 1,459 10.5 4,274 30.8 6,439 46.4 13,867
Financial Aid 382 2.8 1,760 13.0 1,566 11.5 4,144 30.5 5,728 42.2 13,580
Opportunity/Special Programs 213 2.7 1,056 13.3 739 9.3 1,590 20.0 4,371 54.9 7,969
Other 602 2.9 1,735 8.2 2,346 11.1 5,416 25.7 10,967 52.1 21,066
Total 2,019 3.0 7,248 10.7 7,501 11.1 18,482 27.4 32,314 47.8 67,564 Source: EnrollNYC
As in 2017, coaches utilized several communication strategies to engage with students. However, students and coaches disagree on which types of communication are preferred. While both overwhelmingly agree that in-person meetings and text messaging are the ideal way to connect, they disagree starkly about other platforms, such as email, phone calls, and social media. When asked to rank the preferred methods of communication with their college coach, in-person meetings were the most popular choice among students, with 43.2 percent ranking this communication type first (Figure 9). Accordingly, nearly 90 percent of college coach respondents (89.4%) indicated that in-person meetings are “really effective” (Figure 10). Text messaging was also favored by both groups. 47.8 percent of student survey respondents ranked text messaging as their first or second most preferred method, and 61.1 percent of college coaches agreed that text messaging students was highly effective.
However, as far as email, phone calls, and social media were concerned, students and coaches disagreed on their preferences. While email was another popular communication platform among students, with 47.8 percent ranking it first or second, over a quarter of college coaches (26.9%) indicated that email was not an effective platform for communicating (Figure 10). The student response rate to coaches’ email outreach was among the lowest at 33.0 percent (Table 5). In contrast, phone calls were among the least favored communication type by students—58.8 percent of student respondents ranked phone calls fourth or fifth out of the five options. Meanwhile, over two-thirds of college coach respondents (67.4%) indicated that phone calls were highly effective.
Social media was also ranked very low among student survey respondents, with 54.2 percent ranking it fourth or fifth (Figure 9). Interestingly, a large proportion of coaches reported not using social media at all as part of their outreach strategy to students. 48.4 percent did not use Facebook, and 35.9 percent did not use Instagram. Although students ranked social media as the least preferred method of communication, the student response rate to coach outreach was the highest through social media (Table 5). Regardless of the communication platform, coaches demonstrated flexibility and initiative by utilizing the communication method that they felt their students responded to the most. One returning coach explained,
I didn’t use social media last year... because I was actually not really into the whole social media movement and feeling like that’s the best way... but I actually think that doing it for this cohort, and me taking that initiative, I really do see a big improvement.
12 There were 5,884 unique students with at least one case note. The number of case notes for these students ranged from one to 68 and the average number of case notes was 6.6.
Office of Research, Evaluation & Program Support | May 2019 19
Table 5. Coach outreach and student response
Method of contact
Student did not respond
Student responded
No response data Total
n % n % n % n Email 6,089 33.2 6,062 33.0 6,203 33.8 18,354 In Person 379 8.9 2,752 64.5 1,137 26.6 4,268 Other 543 21.6 1,480 59.0 487 19.4 2,510 Phone Call 754 27.2 1,479 53.4 536 19.4 2,769 Social Media 152 4.7 3,018 92.7 86 2.6 3,256 Text 4,029 53.5 2,243 29.8 1,260 16.7 7,532 Total 11,946 30.9 17,034 44.0 9,709 25.1 38,689
Source: Enroll
Figure 9. Student responses to the prompt: ‘Based on your experience, please rank your preferred method of communication when being contacted by your college coach.’ S5
43.2% 12.1% 16.0% 10.7% 18.0%
20.7% 27.1% 25.6% 21.7% 4.9%
16.3% 33.2% 20.8% 13.4% 16.3%
13.3% 12.3% 20.2% 21.7% 32.5%
9.0% 14.1% 18.1% 30.2% 28.6%
0 1 2 3 4 5 6
In-person meeting with CollegeCoach
Text
Social Media
Phone
Office of Research, Evaluation & Program Support | May 2019 20
Figure 10. Coach responses to the question: ‘What strategies were most effective in engaging students?’ C13
Finding 3. There continues to be a need for college matriculation advisement during the summer months before college, and CB4A is well-equipped to bridge this gap for New York City’s high school graduates. Students widely agreed that their college coaches were helpful in completing an array of matriculation activities, and, as a result of working with their college coaches, they felt more prepared to complete these tasks on their own in the future. Bridge student participants concluded their senior year of high school with varying levels of college knowledge. Nearly 30 percent of student respondents (29.0%), on average, reported having little or no knowledge on topics such as financial aid/FAFSA, scholarships, or course registration prior to meeting with their college coach (Figure 11). This was reinforced by coach reflections in a focus group, where coaches agreed that while students had varying levels of college knowledge, many students were unfamiliar with the college matriculation process. One coach explained,
Like there are [students] at the beginning [who] don’t know what FAFSA is. They don’t know what’s like a placement test. They don’t know all these terms. And then when it gets at the point where they find out what it is, sometimes it’s already too late.
Given this gap, students benefitted greatly from coaches’ help in navigating and completing matriculation activities. Across all matriculation activities that coaches assisted their students with,* an average of 81.7 percent of student respondents found their college coach to be either “extremely helpful” or “somewhat helpful” (Figure 12). Specifically, 80.9 percent of student respondents reported that their college coach was “extremely helpful” with updating and verifying their financial aid. Another 75.0 percent indicated that their coach was “extremely helpful” with completing paperwork to enroll in college.
Furthermore, the support students received from college coaches helped them feel empowered to carry out this work on their own. 82.2 percent of students agreed or strongly agreed that, having worked with a college coach, they felt more comfortable completing tasks including registering for
4.2% 2.1% 4.3%
48.4%35.9%
3.2% 3.2%26.9%
12.9%
17.4%
10.6%
25.3% 33.7%
41.9%
17.2% 29.3%
89.4%
67.4% 61.1%
26.9% 21.5% 17.4%
In-person Phone call Text message Email Facebook Instagram
Did not use Not effective Somewhat effective Really effective
Office of Research, Evaluation & Program Support | May 2019 21
classes, understanding their financial aid package, and knowing where to go on campus if they need help or have questions (Figure 13).
These findings on which matriculation tasks students most benefited from their coaches’ help on are reflected in responses in the coach survey. The matriculation activities coaches reported working on most frequently with students over summer included completing enrollment paperwork, FAFSA and TAP applications, and talking with students about what to expect in college. In particular, 56.8 percent of coaches worked on completing enrollment paperwork weekly with students. 41.5 percent and 42.6 percent of college coaches, respectively, worked weekly on completing, and then updating FAFSA and TAP applications. Additionally, nearly half of coaches (44.2%) also spoke with students about what to expect in college on a weekly basis (Figure 14).
In contrast, activities college coaches worked on least over the summer included supporting undocumented students with the college process, supporting students with reallocation from one college to another, and accompanying students to college campuses to complete enrollment.
Students and coaches alike reported having very positive experiences working with one another, suggesting that coaches were successful in engaging students and serving their needs. 96.8 percent of coach respondents indicated that they had a good relationship with the students in their caseload, and that they were able to relate to the experiences of their students. Of student survey respondents, 94.1 percent agreed that: 1) they felt comfortable approaching their college coach with questions; 2) that they trusted the coach to provide them with good advice; and 3) that if they had a problem or question, the college coach usually knew the answer or would find it out for them (Figure 15). When asked to rate their experience working with a college coach on a scale from one to 10, 47.1 percent of students rated their experience a 10. Less than 10 percent of survey respondents (8.5%) rated their experience a 5 or below (see Appendix).
Figure 11: Student responses to the question: ‘Prior to meeting your college coach, how well-informed were you on the following?’ S2
46.6%
32.8%
36.2%
28.9%
18.1%
30.8%
39.6%
30.2%
30.2%
32.9%
22.6%
27.7%
33.6%
40.9%
48.9%
Non-college options (e.g. career services, technicalschools, military)
Scholarships
Course registration
Opportunity/Special Programs (e.g. ASAP, CLIP,START, EOP, HEOP)
Financial aid/FAFSA
Not at all/ Slightly Moderately Extremely
Office of Research, Evaluation & Program Support | May 2019 22
Figure 12. Student responses to the question: ‘Based on your overall experience, how helpful was your college coach in completing these activities?’ S6
Figure 13. Student responses to the question: ‘Having worked with a college coach, to what extent do you agree with the following statements?’ S
8
29.2%
24.2%
18.6%
17.6%
14.8%
14.4%
8.3%
20.8%
9.3%
8.4%
6.5%
6.5%
6.9%
6.5%
6.0%
3.7%
1.9%
4.6%
4.2%
24.1%
21.9%
23.3%
22.7%
24.5%
21.4%
25.9%
14.4%
26.5%
14.4%
13.0%
37.5%
45.6%
51.6%
53.2%
53.7%
57.7%
59.7%
61.1%
70.2%
75.0%
80.9%
Exploring non-college options (e.g. career services,technical schools, military)
Visiting college campuses to prepare for the first day ofcollege
Registering for your first semester classes
Connecting you with campus resources, academicadvising, and the college as a whole
Helping you navigate placement testing
Helping you enroll in CUNY Special Programs
Comparing and reviewing admissions offers
Making last minute admission or college changes
Helping you understand what to expect in college
Helping you complete paperwork to enroll in college
Updating and verifying your financial aid application
Not Applicable Not at all helpful Somewhat Helpful Extremely Helpful
3.1%
4.6%
2.1%
1.5%
2.6%
30.8%
15.9%
17.0%
12.9%
14.4%
15.9%
9.2%
42.1%
47.2%
48.5%
53.1%
46.9%
45.1%
46.7%
23.6%
31.8%
32.0%
32.0%
35.6%
38.5%
43.6%
I feel comfortable reaching out to other students at mycollege
I know where to go on campus if I need help or have aquestion
I am more confident about attending college
I understand my financial aid package
I will be able to register for classes next semester on myown
Overall, my College Coach made me feel more positivelyabout college
I would recommend to my friends that they meet with aCollege Coach
Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree
Office of Research, Evaluation & Program Support | May 2019 23
Figure 14: Coach responses to the question: ‘Over the summer, how often did you work on the following?’
Figure 15. Student responses to the question: ‘To what extent do you agree with the following statements?’ S7
37.9%
24.2%
24.5%
33.7%
32.6%
23.4%
15.4%
6.3%
7.4%
10.5%4.2%
2.1%
1.1%
1.1%
41.1%
41.1%
60.6%
43.2%
37.9%
33.0%
38.5%
30.5%
26.3%
23.2%
22.1%
25.5%
23.4%
28.4%
11.6%
14.7%
28.4%
8.5%
12.6%
15.8%
25.5%
24.2%
37.9%
34.7%
33.7%
37.9%
30.9%
33.0%
26.3%
31.6%
6.3%
6.3%
6.4%
10.5%
13.7%
18.1%
22.0%
25.3%
31.6%
32.6%
35.8%
41.5%
42.6%
44.2%
56.8%
Support undocumented students with the college process
Support students to do direct admissions
Help students explore non-college options
Support students with reallocation from one college toanother
Accompany students to college campuses to completeenrollment steps
Compare and review admissions offers
College decision-making (choosing college, choosingmajor)
Support students to enroll in ASAP, CUNY Start and otherspecial programs
Support students to understand and register for CUNYplacement test
Complete financial aid verification forms
Support students to register for classes
Complete FAFSA and TAP applications
Update FAFSA and TAP applications
Talk to seniors about what to expect in college
Help students complete enrollment paperwork(immunization forms, enrollment deposit)
Never Monthly (1-2 times over summer) Once every two weeks (4-5 times over summer) Every week
3.9%
5.4%
4.4%
3.9%
4.9%
10.3%
36.3%
46.6%
39.2%
36.3%
40.9%
34.3%
57.4%
45.6%
54.9%
57.8%
53.2%
53.9%
My College Coach was available to me when I hadquestions or problems with the college enrollment process
I was able to relate to my College Coach because he/sherecently experienced college application and enrollment…
I felt comfortable approaching my College Coach to askquestions
I trusted my College Coach to provide me with goodadvice
If I had a problem or question, my College Coach usuallyknew the answer or would find it out for me
Overall, I had a good relationship with my College Coach
Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree
Office of Research, Evaluation & Program Support | May 2019 24
Program Outcomes A central goal of CB4A is to increase the number of NYCDOE graduates who immediately matriculate to college. In alignment to this goal, we examined fall semester college matriculation outcomes for CB4A students who indicated that they planned to enroll in college immediately following high school graduation.
Over three fourths of college-intending CB4A students (77.8%) matriculated to college in the fall semester, while 22.2 percent of college-bound students did not enroll in college (Figure 16). The matriculation rate was two percentage points higher (79.6%) at traditional NYCDOE high schools under the central model (Figure 17). CB4A high schools had considerably higher matriculation rates compared to the citywide NYCDOE college matriculation rate, which was 57 percent for students who entered the DOE in 2012 and graduated in 201613.
Almost thirty percent of students who planned to enroll in the spring semester following graduation enrolled in the fall, 14.8 percent of students who indicated wanting to explore non-college options post-graduation enrolled in college in the fall, and 46.3% of students for whom we did not have college intent information14 matriculated in the fall. A majority (74.8%) of the college-intending students who matriculated to college in the fall enrolled at a CUNY college, and slightly over half (52.4%) of the students enrolled in a 4-year degree program (Figure 18).
As outlined in Figure 1, two different models of CB4A were implemented for the first time in 2018. The college matriculation rate was slightly higher for students in the central model (78.0%) compared to students in the SSC model (75.9%) (Figure 19). The matriculation rate was also slightly higher for students in returning CB4A schools (79.3%) compared to students at schools where CB4A was implemented for the first time (76.5%) (Figure 20). Finally, the matriculation rate at regular NYCDOE high schools was significantly higher (79.2%) than the matriculation rate at ECI schools15 (50.9%) and transfer schools (58.2%) (Figure 21). As previously mentioned, ECI and transfers schools offer different models of secondary education from traditional NYCDOE schools and should not be points of comparison for the majority of CB4A high schools.
Throughout the summer, college coaches documented interactions with students in EnrollNYC using case notes, where they tracked data on topic of discussion, method of communication, and whether or not the student responded to the coach. We used the number of case notes as a proxy for the level of student engagement with a college coach to examine matriculation rates and found that in general, increased levels of student engagement with a college coach was correlated with higher matriculation rates (Table 8). The average matriculation rate for the most highly engaged students—those with 40 or more case notes—was 86.0 percent, whereas the average matriculation rate for students with one to three case notes was 56.8 percent. This suggests that CB4A is most valuable for
13 This percentage includes students who enroll in college, vocational programs, or public-service programs such as the military. Source: https://www.schools.nyc.gov/about-us/news/announcements/contentdetails/2017/11/15/chancellor-fari%C3%B1a-announces-highest-ever-percentage-of-students-enrolling-in-college 14 We assume that students with missing college intent data were not served by CB4A. 15 There are two known possible reasons for the lower matriculation rate at ECI schools. Some ECI students earn Associate degrees while still enrolled in high school, which may preclude them from enrolling in another degree-seeking program after high school graduation. Secondly, the two ECI schools in CB4A are 9-13 and 9-14 grade schools, which means an ECI student may graduate from high school after the 12th grade but may delay matriculation and remain in high school for the 13th and 14th grades to earn college credits.
Office of Research, Evaluation & Program Support | May 2019 25
students who are most actively engaged with college coaches throughout the spring and the summer.
Figure 16. College matriculation rates by students’ intent to enroll in college
Figure 17. College matriculation rate and degree type for students in the central model (excluding ECI and transfer schools)
22.2%
70.1%
53.7%
85.2%
53.7%
31.2%
77.8%
29.9%
46.3%
14.8%
46.3%
68.8%
Intend tomatriculate in
the fall(n=6,186)
Intend tomatriculate in
the spring(n=177)
Undecided(n=268)
Other non-college option
(n=628)
Missing intentdata in Enroll
(n=397)
All graduates ofBridge high
schools(n=7,656)
No Yes
2-year(n=1,976)
47.7%
4-year(n=2,165)
52.3%Matriculated(n=4,141)
79.6%
Did not matriculate(n=1,063)
20.4%
DOE Class of 2016 matriculation rate 57% (n=77,000)
Office of Research, Evaluation & Program Support | May 2019 26
Figure 18. Matriculation at different types of colleges
Figure 19. Matriculation by CB4A model
22.0% 24.1%
78.0% 75.9%
Centrally managed Bridge model(n=5,552)
SSC Bridge model (n=634)
No Yes
74.8%
88.9%
47.6%
60.6%
25.2%
11.1%
52.4%
39.4%
CUNY Non-CUNY(n=4,812)
Public Private(n=4,812)
2-Year 4-Year(n=4,812)
CUNY 2-Year CUNY 4-Year(n=3,597)
Office of Research, Evaluation & Program Support | May 2019 27
Figure 20. Matriculation by new or returning CB4A school status
Figure 21. Matriculation by transfer school status
23.5% 20.7%
76.5% 79.3%
New Bridge school (n=3,299) Returning Bridge school (n=2,887)
No Yes
49.1%41.8%
20.8%
50.9%58.2%
79.2%
ECI school (n=171) Transfer school (n=177) All other DOE schools(n=5,838)
No Yes
Office of Research, Evaluation & Program Support | May 2019 28
Table 6. Matriculation rate by number of case notes
Case notes per student Matriculation rate
n % % 0 4,225 41.8 40.1 1 1,312 13.0 56.2 2-3 1,345 13.3 57.4 4-5 1,309 12.9 67.1 6-7 757 7.5 67.9 8-9 266 2.6 70.7 10-19 620 6.1 47.9 20-29 38 0.4 44.7 30-39 1 0.0 100.0 40-49 127 1.3 83.5 50-59 107 1.1 88.8 60-68 2 0.0 100
Total students with 1+ case notes 5,884 58.2 61.3
Total students 10,109 100.0 52.4 Source: Enroll
Recommendations In triangulating students’ preliminary matriculation data and feedback from students and college coaches, we find two areas of consideration that we believe will strengthen the CB4A model of intervention. 1. Focus on strengthening the in-person communication strategy between college coaches and students.
A distinguishing feature of the CB4A program is that it is more comprehensive than lighter-touch summer melt interventions that are based on nudges, such as personalized text message reminders about forms and deadlines. Students most prone to summer melt often need more involved assistance with the college enrollment process, specifically with tasks such as filling out forms and understanding loan packages. By assigning coaches to work with students over a range of communication platforms and on the specific matriculation needs of each student, the CB4A model presents a distinct advantage. Most notably, both coaches and students alike agreed that in-person meetings are a highly effective way to address these complex needs. Students were most responsive in person (Table 5), and they overwhelmingly reported favoring this communication method over others (Figure 9). A large majority of coaches (89.4%) reported that in-person interactions were very effective (Figure 10). This reflects what other research has found around the advantages of peer-mentoring models. Castleman, Page, and Snowden (2013) reported that similar peer mentor outreach interventions reduced summer melt and increased college matriculation by four to seven percentage points. While research on these models is still emerging, early results speak to the promise of peer mentor interventions like CB4A for reducing summer melt.
Office of Research, Evaluation & Program Support | May 2019 29
2. Optimize college coaches’ time by clearly defining the target participant population of CB4A.
Serving 6,186 students across 79 high schools in 2018, CB4A is a large scale program. As CB4A continues to expand, it will be important to ensure that the program’s resources are being allocated in a way that is optimal to fulfilling its stated goals. Specifically, college coaches’ feedback about persistent challenges in engaging unresponsive students could indicate an opportunity to focus the program’s reach. Both in 2017 and 2018, large shares of coaches (89.8% and 89.5%, respectively) rated “unresponsive students” as the biggest challenge they faced as a coach (Figure A5). While CB4A was designed as a school-level intervention, unresponsive students may represent a group that is not the target beneficiary of the program. By focusing coaches’ time on students who actively take up the coaching services offered, CB4A may have a bigger impact in reducing summer melt among students who need and are eager to receive assistance. Administrators may consider relying on “student intent to enroll” data to identify students who need college coach assistance.
Conclusion The College Bridge for All (CB4A) program addresses an important but less explored shortcoming in the college counseling and matriculation process that disproportionately impacts minority students and students with fewer resources. Although researchers have established that many college-intending students fail to enroll in college due to obstacles faced during the summer months before college, few programs tackle summer melt at the scale of CB4A.
The purpose of this descriptive report was to examine the implementation of CB4A and students’ preliminary college matriculation outcomes. In triangulating a variety of data sources, we found that overall, students and coaches reported having very positive experiences working with one another, suggesting that coaches were successful in engaging students and serving their needs.
While results in this report are descriptive in nature, we find early promising results for CB4A. Over three fourths of college-intending students who worked with a college coach matriculated in the fall semester. The matriculation rate for students who worked most intensively with college coaches was much higher at 86 percent. Future research on CB4A will employ causal inference methods to examine the school-level impact of the program.
Overall, the changes implemented by CB4A as a result of the 2017 implementation report allowed it to achieve significant growth and to improve many of its processes. College coach training was more robust, data entry practices using EnrollNYC became more streamlined, and coaches interacted with students earlier and more frequently both before and throughout the summer. In light of these positive changes, as well as the persistent challenges observed in the present implementation report, recommendations for the next iteration of CB4A include emphasizing in-person communications between coaches and students, and focusing coaches’ time on those students who actively take up the intervention. Given CB4A’s early success in fostering positive experiences among students and coaches and encouraging college enrollment, with continued optimization of its practices, CB4A will offer significant promise as a model of intervention for reducing summer melt.
Office of Research, Evaluation & Program Support | May 2019 30
References Ahn, G., Campbell, V., Madsen, C., & Studer, E. (2018). College Bridge for All implementation report: Results from the 2017 program expansion.
Carrell, S. E., & Sacerdote, B. (2013). Late interventions matter too: The case of college coaching New Hampshire (No. w19031). Cambridge, MA: National Bureau of Economic Research.
Castleman, B. L., Arnold, K., & Wartman, K. L. (2012). Stemming the tide of summer melt: An experimental study of the effects of post-high school summer intervention on low-income students’ college enrollment. Journal of Research on Educational Effectiveness, 5(1), 1-17.
Castleman, B. L., & Page, L. C. (2014). Summer melt: Supporting low-income students through the transition to college. Harvard Education Press. Cambridge, MA.
Castleman, B. L., & Page, L. C. (2015). Summer nudging: Can personalized text messages and peer mentor outreach increase college going among low-income high school graduates? Journal of Economic Behavior & Organization, 115, 144-160.
Castleman, B. L., Page, L. C., & Schooley, K. (2014). The forgotten summer: Does the offer of college counseling after high school mitigate summer melt among college‐intending, low‐income high school graduates?. Journal of Policy Analysis and Management, 33(2), 320-344.
Castleman, B. L., & Page, L. C., & Snowden, A. (2013). SDP summer melt handbook: A guide to investigating and responding to summer melt. Harvard University Center for Education Policy Research.
Office of Research, Evaluation & Program Support | May 2019 31
Appendix
Figure A1. Coach responses to the question: ‘How much do you agree with the following statements?’
Figure A2. Student responses to the question: ‘Overall, how would you rate your experience working with a college coach on a scale of 1 to 10?’ S14
Figure A3. Coach agreement with the prompt: "Being close in age to the students I worked with is an advantage of the program." C14
2.1%
2.1%
8.4%
5.3%
48.4%
46.8%
34.7%
52.6%
48.4%
50.0%
53.7%
41.1%
I was able to relate to the experiences of mystudents.
Overall, I had a good relationship with the studentsin my caseload with whom I had contact.
Being close in age to the students I worked with isan advantage of the program.
Overall, students felt comfortable approaching mewith questions or challenges.
Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree
0.5% 0.5% 1.1% 0.5% 0.5%5.3% 2.6%
6.9%
18.0% 16.9%
47.1%
0 1 2 3 4 5 6 7 8 9 10
1.1% 2.1% 8.4%
34.7%53.7%
Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree
Office of Research, Evaluation & Program Support | May 2019 32
Figure A3. Coach responses to the question: ‘Based on your experience as a college coach, how much do you agree with the following statements?’
Figure A4. Coach responses to the question: ‘To what extent do you agree with the following statements?’ C4
47.4%
40.0%
21.1%
42.1%
57.9%
77.9%
I believe the program is effective in helpingstudents navigate the college matriculation
process.
Overall, the services and activities I providedpositively influenced student behaviors and
attitudes toward college.
Overall, I feel confident that my students know howto access college resources.
Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree
25.3%
24.2%
21.1%
21.3%
17.9%
17.9%
17.0%
16.8%
16.8%
68.4%
72.6%
74.7%
77.7%
80.0%
80.0%
80.9%
81.1%
81.1%
The other coaches in my Hub were integral to mysuccess as a College Coach.
I trusted the other coaches in my Hub to support me inmy work and professional growth.
Hub Day meetings helped me to better fulfill myresponsibilities as a College Coach.
My Hub Supervisor created opportunities for me to takeleadership roles within my hub.
My Hub Supervisor was knowledgeable on collegematriculation processes.
I trusted my Hub Supervisor to support me in my workand professional growth.
I felt comfortable approaching my Hub Supervisor withchallenges related to my work.
My Hub Supervisor provided timely feedback.
My Hub Supervisor was integral to my success as aCollege Coach.
Strongly disagree Disagree Neither disagree nor agree Agree Strongly agree
Office of Research, Evaluation & Program Support | May 2019 33
Figure A5. Coach responses to the question: ‘What were the biggest challenges you experienced as a college coach?’ C17
Figure A6. Coach responses to the question: ‘In CUNY enrollment, which tasks were hardest for students to complete?’ C19
0.0%
4.2%
9.5%
35.8%
37.9%
46.3%
50.5%
89.5%
Lack of training on college matriculation support
Lack of supervisor support during summer (HubSupervisor/Central)
Lack of supervisor support during spring (High SchoolCounselor)
Difficulty meeting with seniors before graduation
Gathering student contact information
Making initial contact with students in April
Not enough hours of work to respond to all mystudents' needs
Reaching students who were unresponsive
14.7%
17.9%
24.2%
28.4%
31.6%
35.8%
45.3%
49.5%
Completing direct admissions to colleges
Helping students register for classes
Helping students connect to an advisor
Directing students to Opportunity/Special Programs(like ASAP, Start, CLIP)
Scheduling CUNY placement tests
Managing holds on account
Turning in matriculation paperwork (deposits,immunization paperwork, etc.)
Confirming financial aid