#pearsoncite - The world's learning company | US...• Past Chair UPCEA Marketing, Enrollment and...
Transcript of #pearsoncite - The world's learning company | US...• Past Chair UPCEA Marketing, Enrollment and...
#pearsoncite
Admission data: What more can it tell us?
How to use data to increase application-to-enrollment conversion rates
Lindsay Baun & Deb Corso-Larson, Pearson Michele Long, University of Denver (Contributor)
The potential in the use of data to inform targeted improvements to retain students throughout the student lifecycle is great. But there can be a lot of confusion about where to begin:
What interventions or teaching and
learning practices work best to retain
students? We have reporting, but
how do we turn that into
action?
What variables have the
largest impact on retention?
How can we assess the
effectiveness of our current
student support practices?
Do we have enough data? Is it the right
data?
How can we effectively
leverage data we already
have?
Institutions are increasingly focused on addressing a variety of retention challenges.
Data
Reporting & Accountability
Melt -- referring to the % of students who are admitted but do not enroll at a given college -- is a vexing problem for many institutions.
The long standing belief at University College of the University of Denver: If you admit them they will come
The landscape of higher education has changed and post-traditional students have options!
University College believed their melt rate had grown to as high as 40%.
University College was struggling to reduce a persistent melt rate.
They knew they had lots of data that could
help them find targeted ways to improve their
melt rate… …but they had limited
internal recourses for data collection, integration, and
analysis.
Pearson Consulting Services and University College partnered to take a data informed approach to addressing their melt challenge.
Pearson Consulting Services North America Higher Ed Services
Lindsay Baun, M.S. Data Analyst
• Biz intelligence & statistical modeling
• Former Data Analyst for Target Corp
Project Consultants:
Deb Corso-Larson, M.A. Academic Consultant
• Research & Evaluation Methods • Data informed approaches to
drive improvements to curriculum & instruction
Project Overview / Goals Learning Analytics Process
Data Model Findings and Insights
Actions Discussion
Michele Long Director of Student Services • 19 years in Continuing Ed • Past Chair UPCEA
Marketing, Enrollment and Student Services Network (MESS)
DU Project Lead: ● Credit Programs:
87% graduate 6% bachelor completion 7% non-degree
● Fall 2014: 71 Lecture courses
and 243 Online courses ● Enrollments: 77% online; 23%
on campus ● Students: 55% in-state; 40%
out-of-state; 5% international
University College of the University of Denver College of Professional and Continuing Education
Desired Project Outcomes What is the best way to reduce melt?
• Understand which admission characteristics matter Establish a benchmark of graduate melt rate and better
understand enrollment predictors
• Pinpoint where to put efforts Strategically utilize limited human resources - admission, student services, academic advisors to impact ‘at-risk’ new admits
• Identify challenges and actions that will positively impact
enrollment
• Be Better! Improve the quality of the experience from inquiry to enrollment
Learning Analytics Process
10
Discovery Analytical Modeling
Strategic Planning
• Articulated the project objectives
• Engaged the right stakeholders
• Defined measurement & success metrics
• Gained permission to share the data
• Conducted a data inventory
• What data do we need?
• Who owns the data? • Integrated and cleaned
multiple data sets • Utilized Decision Tree
Modeling to identify the variables most predictive of melt
• Provided a rigorously grounded analysis of the most important melt drivers
• Implemented an operationalized predictive model to identify at-risk students
• Developed a plan of action to identify at-risk students and start addressing risk factors to reduce melt.
Pearson and University College partnered to engage the right stakeholders on campus, develop a custom analytical/statistical approach, and provide actionable recommendations toward the goal of reducing enrollment melt.
Data Summary
• 55 variables analyzed • Application start, complete, last application requirement submitted, and
admission timing • Financial aid – applied, offered, declined, and received (including dates) • Previous education, distance to DU campus, and demographics • Degree, major, concentration, intended delivery modality, and prior credits
• Data Sources • Banner, Online Application, Financial Aid, Resumes, National Student
Clearinghouse, and Others
*Note: Summer term was excluded from the analysis
Term Student Count
Fall ’13 325
Winter ’14 259
Spring ’14 223
Fall ’14 353
Domestic Graduate Student Admitted in the following terms:
Insights Preview
Factors Discovered to be Responsible for Driving the Majority of Melt at University College: • Financial Aid – have not applied or applied and did not qualify • Distance to DU campus – Current address further than 46
miles from campus • Time between application start & application package
completed – More than a month • Time between admit decision & term start - More than 5
months
Enrollment Melt Benchmark Report:
Fall 2013
Winter 2014
Spring 2014
Fall 2014
Average All Terms
Melt % 15.7% 14.2% 12.7% 20.0% 16.6%
Decision Tree Data Model
• Predicts an applicant’s enrollment
• Sorts applicants from the “root node” of the tree, which
contains all applicants, down to a “leaf node” which
classifies the applicant.
• Uses an algorithm to select the best variables and the
best splits
• Each split is chosen to increase homogeneity within each
node as you travel down the tree.
• Data is split into a test and train dataset to avoid overfitting
• Best model correctly classifies the most applicants in the
test dataset
• Emphasis place on correctly classifying at-risk applicants
Decision Tree Data Model
Applicants in these buckets are predicted to not-enroll (At-Risk)
Applicants in these buckets are predicted to enroll
< 46 miles
> 46 miles
Financial Aid Status at Cut-off Date
Enrolled
Time between Application Start and Application Package Complete
Offered financial aid or applied
and waiting for award
offer
Has not applied for financial aid or applied but did not
qualify
Distance to DU campus
< 34 Days
> 34 Days
Enrolled Not
Enrolled
Not Enrolled
Classification Matrix for Test Dataset Predicted
Actual Not-Enrolled Enrolled Not-Enrolled
81% 36%
Enrolled 19% 64% Overall Misclassification Rate: 33.2%
Building a Replicable Model to Inform Target Actions
Operationalizing the Predictive Model
How do we use the model to identify an at-risk student?
For Fall, Spring, & Winter Terms: 1. Pull the following 4 variables for each
applicant 7 days after their admit decision is made: o Financial Aid Status o Zip Code o Application Start Date o Last Application Requirement
Date
2. Complete the form with the corresponding data to assess risk status
3. Save the student information, data, and risk status to the database table for future tracking
Recommendations
Key Insight Findings
At-Risk Variables: Identification of student’s who are predicted to be at-risk of not enrolling
• Student’s who have not applied or applied and did not qualify for financial aid
• Student’s who live further than 46 miles from DU’s campus (despite the fact that the majority of University College’s enrollments are online - 77%)
• Student’s who take more than a month to complete their application package
• Student’s admitted more than 5 months before term start
Process Change: Limited Financial Aid Advising
• Lack of financial aid is the largest driver of enrollment melt • An analysis of the communication plan revealed not enough is
being done to advise on financial aid options
Process Change: Gap in Communication Plan
• NO communication with applicants after admission letter is sent until registration period begins
• Significant negative impact on applicants who apply early and for applicants who defer to a future term
• Pre-enrollment advisor contact varies by department
New Data Opportunity: Lack of tracking of communications and outreach
• Important pre-enrollment contacts are not currently tracked
Action Plan Going Forward
Key Insight Initiative
At-Risk Variables: Identification of student’s who are predicted to be at-risk of not enrolling
• Create an action plan to implement the predictive model to identify at risk students.
• Define a contact plan with custom interventions around the applicant’s risk variables and execute for each high risk applicant.
Process Change: Limited Financial Aid Advising
• Getting comfortable talking about money and ‘financing your education’
• Staff training with the Financial Aid office • Discussing financial aid intentionally with inquiries and applicants
during initial phone conversation and adding financial aid information to the inquiry communication plan.
Process Change: Gap in Communication Plan
• Make changes to communication plan to fill this gap for students who apply early or defer to a future term
• Fantastic opportunity to build relationship and provide meaningful communications − Develop a Prospective Student Success Communications and
Resource Library
New Data Opportunity: Lack of tracking of communications and outreach
• Scheduled implementation of Technolutions Slate communications tool - fall 2015. This admission CRM will significantly help keep track of pre-enrollment student contacts and pre-advising outreach.
The Ecosystem of a Successful Learning Analytics Initiative
19
Learning analytics can inform improvements across the full circle of components essential to achieving bold, strategic outcomes for student success and academic quality.
Faculty Development
Course & Program Design
Student Support & Success
Instructional Innovation
Outcomes Assessment
• Impact of faculty development
programs
• Targeted curriculum
improvements
• Increased student success and
retention across the student
lifecycle
• Development of personalized
learning and student support
• Assessment of student and
program learning outcomes
Aligning data collection, reporting, and analysis needs with strategic, institutional goals and specific outcomes
Planning
Phase 1
Phase 2
• Create a student success/assessment plan that outlines strategy & desired outcomes
• Identify the people, process & technology needed to achieve outcomes
• Review plan • Collect data • Review/analyze data • Develop improvement
action plan informed by data
• Implement improvement plan
• Collect data • Analyze data • Evaluate improvement plan • Review original student
success/assessment plan
Lindsay Baun Data Analyst Pearson Consulting Services [email protected]
Deb Corso-Larson Academic Consultant Pearson Consulting Services [email protected]
#pearsoncite
Michele Long (Contributor) Director of Student Services University College of the University of Denver [email protected]