USING PREDICTIVE ANALYTICS TO UNDERSTAND HOUSING ENROLLMENTS Heather Kelly, Ed.D., University of Delaware Karen DeMonte, M.Ed., University of Delaware Darlena Jones, Ph.D., EBI MAP-Works

Predictive Analytics: A variety of statistical techniques that analyze current and historical facts to make predictions about future events.

“Study the past, if you would divine the future” Confucius

Why Retention?

• Retention affects… • Enrollment numbers • Institutional image • Institutional rankings • Finances (revenue and funding)

• Retention is often part of our institutional planning. • Retention is essential to student and institution success.

Because retention strongly impacts the institution, it makes sense to

use predictive analytics to better understand retention

Why Predict Retention Risk? • Faculty/staff can’t accurately predict students perceptions

of program or risk of retention • Most faculty are focused on…

• Research • Course load • Advising

• Most staff are focused on… • Putting out fires • Working with students who self-identify issues • Working with students with judicial issues

Not a good strategy?

Why is Predicting Difficult?

• Many factors come into play. • The impact of any one factor can be affected by other

factors. • Some factors are exacerbated by others. • Some factors are mitigated by others.

• Time frame is long. • Our judgment may be affected by:

• Anecdotes • Outlier data points • Issues we care about • Institutional initiatives • Other issues

Our best predictor is not a sum of issues

It’s fairly obvious that “Travis Gatlin” is at risk…

It’s less obvious that “Jessica Anderson”, is

equally at risk.

NATIONAL PREDICTIVE MODELS ACUHO-I/EBI Resident Assessment

Users of Predictive Modeling

Administrators • Understand what

drives success for improvement and accreditation review

CHO • Identifies

predictors for program to focus resources in these areas

Hall Staff • Identifies

predictors for their area/hall (may be different from program) to focus work

Researchers • Understand the

impact of housing on housing and institutional enrollments

Statistical Methods

EBI’s Reporting

Research

• Automatically generated by institution

• Reported in written and online formats

• Uses linear multi-variant regression

• Descriptive Analysis • Correlations • Linear or Logistic Regression • Discriminant Analysis • Classification Trees • Measurement and Path

Models • Neural Networks • Simulation Models

Predicting Residence Hall Retention

Factor Description Predictor Status

Contribution to the Total

Variance

Factor Performance

Overall Resident Satisfaction 1st Predictor 0.151 Moderate

Overall Learning Experience 2nd Predictor 0.032 Moderate

Satisfaction: Room Assignment or Change Process 3rd Predictor 0.031 Moderate

Climate: Sense of Community 4th Predictor 0.024 Good

Climate: Fellow Residents are Respectful 5th Predictor 0.020 Moderate

NOTE: Data from 2011 ACUHO-I/EBI Resident Assessment, 271 institutions, 317,000 responses, R2 = .173

Predicting Satisfaction

Improving these predictors should improve resident satisfaction for future populations

NOTE: Data from 2011 ACUHO-I/EBI Resident Assessment, 271 institutions, 317,000 responses

Predicting Learning Experience

Improving these predictors should improve resident learning for future populations

NOTE: Data from 2011 ACUHO-I/EBI Resident Assessment, 271 institutions, 317,000 responses

Limitations of a National Model

Data definitions

• Using data definitions developed for a national model can make it difficult for some institutions to align with their internal data data

Outlier Institutions

• Some institutions’ housing operations are outside the “average” thus making a national model less applicable

Outlier Populations

• Some student populations are served by specialized institutions (e.g., HBCU) which is outside the average

Solution: Institutional Models

INSTITUTIONAL PREDICTIVE MODEL University of Delaware

New Students

Freshmen

Transfers

Readmitted

Continuing Students

Total number enrolled from prior semester

minus (number of graduates + number of withdrawals)

+

Enrollment Projections: The Variables

Enrollment Projections: The Model

SPRING 2010

FALL 2010

SPRING 2011

FALL 2011

FALL 2009

Historical Data Predicted Data

History

Enrollment Projections: The Output

Persisters Report TABLE 1: ENROLLMENT, DROPOUT RATES AND GRADUATION RATES

FOR FIRST-TIME FRESHMEN ON THE NEWARK CAMPUS (Total)

Enrollment and Dropout Rates Graduation Rates

Entering 1st 2nd 3rd 4th 5th 6th within within withinFall Term Fall Fall Fall Fall Fall Fall 3 yrs 4 yrs 5 yrs Total

1995 N 3154 2673 2439 2355 599 113 21 1721 2219 2344% enrollment 100.0% 84.7% 77.3% 75.3% 19.0% 3.6% 0.7% 54.6% 70.4% 74.3%% dropout 0.0% 15.3% 22.7% 24.7% 26.4% 26.1%

1996 N 3290 2804 2585 2489 606 108 22 1825 2302 2427% enrollment 100.0% 85.2% 78.6% 76.3% 18.4% 3.3% 0.7% 55.5% 70.0% 73.8%% dropout 0.0% 14.8% 21.4% 23.7% 26.1% 26.7%

1997 N 3180 2766 2523 2436 581 117 27 1827 2284 2401% enrollment 100.0% 87.0% 79.3% 77.5% 18.3% 3.7% 0.8% 57.5% 71.8% 75.5%% dropout 0.0% 13.0% 20.7% 22.5% 24.3% 24.5%

1998 N 3545 3080 2830 2762 653 118 22 2079 2621 2727% enrollment 100.0% 86.9% 79.8% 78.5% 18.4% 3.3% 0.6% 58.6% 73.9% 76.9%% dropout 0.0% 13.1% 20.2% 21.5% 22.9% 22.7%

1999 N 3513 3126 2871 2757 526 83 31 2193 2632 2684% enrollment 100.0% 89.0% 81.7% 79.4% 15.0% 2.4% 0.9% 62.4% 74.9% 76.4%% dropout 0.0% 11.0% 18.3% 20.6% 22.6% 22.7%

2000 N 3128 2738 2524 2453 496 85 24 1884 2297 --% enrollment 100.0% 87.5% 80.7% 79.2% 15.9% 2.7% 0.8% 60.2% 73.4%% dropout 0.0% 12.5% 19.3% 20.8% 23.9% 23.8%

2001 N 3358 2976 2746 2674 472 0 31 2138 -- --% enrollment 100.0% 88.6% 81.8% 80.6% 14.1% 0.0% 0.9% 63.7%% dropout 0.0% 11.4% 18.2% 19.4% 22.3% 0.0%

2002 N 3399 3055 2866 2787 0 0 42 -- -- --% enrollment 100.0% 89.9% 84.3% 83.2% 0.0% 0.0% 1.2%% dropout 0.0% 10.1% 15.7% 16.8% 0.0% 0.0%

2003 N 3433 3035 2808 0 0 0 -- -- -- --% enrollment 100.0% 88.4% 81.8% 0.0% 0.0% 0.0%% dropout 0.0% 11.6% 18.2% 0.0% 0.0% 0.0%

2004 N 3442 3064 0 0 0 0 -- -- -- --% enrollment 100.0% 89.0% 0.0% 0.0% 0.0% 0.0%% dropout 0.0% 11.0% 0.0% 0.0% 0.0% 0.0%

2005 2nd Fall Retention Rate: 90.3% 2005 Cohort Graduation Rates: Within 4 yrs: 64.1% Within 5 yrs: 76.1% Within 6 yrs: 78.4% 2010 2nd Fall Retention Rate: 92.5%

Student Retention Benchmarks

Accuracy of Enrollment Projection Model

THE IMPACT OF RETENTION ON HOUSING ENROLLMENTS University of Delaware

The Issue 22

Why the concern?

• Impact of student outcomes and success • Facilities under-utilized • Loss of revenue

23

What is our Goal?

Identify areas for improvement to help reverse the trend of second year

students moving off-campus and re-establish acceptable occupancy rates.

24

Methodology

• Utilize existing information • Student Extracts • National Survey of Student Engagement (NSSE) • Educational Benchmarking, Inc. (EBI) • Undergraduate Student Satisfaction Survey

• Develop new instrument for further exploration • Housing Retention Survey developed by UD administered by Campus

Labs

25

EBI

26

2010 ACUHO-I/EBI Resident Study Overall Results

3.5000

4.0000

4.5000

5.0000

5.5000

6.0000

University Experience Residential Experience Overall Residential Value

Source: 2010 ACUHO-I/EBI Resident Study

Impact on Overall Program Effectiveness

• Which factors predict students’ overall perception of the Full Residential Experience?

27

Regression Variables Performance

Factor R2 β Mean Descr.

Learning Outcomes: Personal Interactions Top Predictor 0.663 0.302 5.29 Good

Satisfaction: Room Assignment or Change Process 2nd Predictor 0.239 5.12 Good

Satistaction: Room/Floor Environment 3rd Predictor 0.153 5.04 Good

Satisfaction: Dining Services 4th Predictor 0.152 4.55 Good

Learning Outcomes: Manage Time, Study, Solve Problems 5th Predictor 0.139 4.85 Good

Climate: Sense of Community 6th Predictor 0.098 5.63 Excellent

Satisfaction: Res Hall Student Staff 7th Predictor -0.039 5.76 Excellent

Source: 2010 ACUHO-I/EBI Resident Study

Impact on Cost to Quality Rating

• Which factors predict students’ rating of the overall value of their residence hall experience?

28

Regression Variables Performance

Factor R2 β Mean Descr.

Satisfaction: Dining Services Top Predictor 0.372 0.303 4.58 Good

Learning Outcomes: Manage Time, Study, Solve Problems 2nd Predictor 0.125 4.58 Good

Satistaction: Room/Floor Environment 3rd Predictor 0.089 5.04 Good

Learning Outcomes: Personal Interactions 4th Predictor 0.120 5.38 Good

Satisfaction: Services Provided 5th Predictor 0.096 5.17 Good

Climate: Sense of Community 6th Predictor 0.068 5.64 Excellent

Satisfaction: Safety and Security 7th Predictor -0.058 5.83 Excellent

Satisfaction: Room Assignment or Change Process 8th Predictor 0.048 5.12 Good

Source: 2010 ACUHO-I/EBI Resident Study

Impact on Intent to Live On Campus

• Which factors predict students’ degree of their intent to live on campus the following year?

29

Regression Variables Performance

Factor R2 β Mean Descr.

Satisfaction: Dining Services Top Predictor 0.116 0.210 4.58 Good

Learning Outcomes: Personal Interactions 2nd Predictor 0.175 5.38 Good

Satisfaction: Facilities 3rd Predictor -0.085 5.14 Good

Satisfaction: Safety and Security 4th Predictor 0.088 5.83 Excellent

Learning Outcomes: Diverse Interactions 5th Predictor 0.065 5.36 Good

Climate: Fellow Residents are Respectful 6th Predictor -0.065 5.27 Good

Source: 2010 ACUHO-I/EBI Resident Study

What does it all mean? 30

• Where should we focus our efforts?

• What areas do we simply monitor?

• What further questions do we ask?

Next Steps

• Establish student needs • Discuss results with our Stakeholders – Dining, Residence Life, Facilities, etc.

• Investigate and Communicate perceived value and student satisfaction factors

• Focus Groups for in-depth discussion • Enhance marketing efforts to parents

31

Changes to Date & Future Recommendations • New marketing messages • Approval to purchase new Housing Administration System

32

Changes to Date & Future Recommendations • Develop Resident Life Living Learning Communities (LLC)

33

Changes to Date & Future Recommendations • Rodney Complex facility improvements

34

Changes to Date & Future Recommendations • East Campus Freshmen Residence Complex and Dining

Hall Projects Planning and Recommendations

35

Source: New dorms to open in 2013, The Review, February 22, 2011.

FINAL THOUGHTS

• Predicting Retention • Group behaviors are easier to predict • Individual student behaviors are impossible to predict • Student decisions strongly impact departmental budgets

• Limitations • National data doesn’t factor in institutional characteristics • Departmental/institutional data may not have large enough

populations

• Accuracy of our predictive models is important, but we must also focus on getting people to use the data.

Don’t forget…this is about informing and motivating behavior

Heather Kelly, Ed.D. , hkelly@udel.edu

Karen DeMonte, M.Ed., kdemonte@udel.edu

Darlena Jones, Ph.D., Darlena@webebi.com

Want to Learn More?

Visit the EBI / MAP-Works booth in the exhibit hall

Attend one of EBI’s free educational webinars (www.webebi.com/community)