Method-ists: Building a Data-Informed Culture

7/31/2019 Method-ists: Building a Data-Informed Culture

1/26

Method-ists: Building a Data-Informed Culture

Los Angeles Harbor College

Kristi Vollmer Blackburn, Ph.D. Dean of Institutional Effectiveness

Rhea Estoya Research Analyst

Elena Reigadas, Ph.D. Faculty Member, SLO Coordinator

RP Group Strengthening Student Success Conference * Session #35 * October 3, 2012


2/26


3/26

Achieving the Dream LAHC Joined AtD Summer 2011 (9 college in District)

Lumina Foundation, AtD is a social justice movement toincrease equity in success by minority students

5 Goals of AtD: Students Successfully complete the courses they take

Students Advance from remedial to credit-bearing courses Students Enroll in and successfully complete gatekeeper

courses

Students Enroll from one semester to the next

Students Earn degrees and/or certificates

Commit to find out why there are gaps in attainment Examine how to eliminate barriers in order to achieve

equity


4/26

Approach of AtD

AtD Data Team (data/evaluation/evidence)

Focus on colleges performance with respect to studentoutcomes, with a special focus on low-income students,students of color and others who face barriers to success

Examine data (qual/quant), present findings in acompelling way

Involve students/faculty to identify strengths/weaknessesof college policies, structures, services

Aid the core team in engaging campus community aboutdata analysis and proposed strategies/goals

AtD Core Team (implementation)


5/26

Who should be on the Data Team?

AtD Suggests: IR/IE Director

Senior Planning Admin

IT representative

Faculty (developmental,

gatekeeper, high enrmt/highfailure courses)

Diversity staff member

Student Services

Students

What we actually did All on the left plus:

SLO/Assessment Coordinator

Director of Financial Aid (althoughcould not participate due to time)

Learning Assistance CenterDirector

Admissions and RecordsSupervisor

CTE/Econ. Workforce Devmt Dean

Other campus constituencies whowanted to participate

Invitations to those who arechange agents


6/26

WHO SHOULD BE ON YOUR DATA TEAM?

ACTIVITY: Identify by position at your college who has

data/evidence that can be brought to the Team

Identify by personality who can be a change

agent on your campus Identify those who have authority to

enact change and how you will

engage them with the Data Team(member?)


7/26

AtD DATA TEAM

SHARED UNDERSTANDING Data discussions need to be held within a context.

Resist the desire to draw conclusions from incomplete data.

Avoid jumping to conclusions.

Data discussions are that: discussions.

Resist desire to draw conclusions about courses/departments/personnelfrom what could be incomplete data.

Be mindful of how we discuss the data and be cautious of how people

hear what we report to theAtD Core Team and CPC.

Data in itself is free from judgmenthow it is interpreted and presented

makes all the difference as to what people do with it.

Keep an open mind and a critical eye. Watch for what makes sense and

what doesnt.

Keep an open heart: the work we are doing is to support Student Success!

Our students will ultimately benefit from this work!


8/26

Component

One

Component

Two

Component

Three

Component

Four

Whats Wrong?(Outcome Measures)

Why?(Underlying Factors)

Intervention(s) Evaluation &Modification

Use Longitudinal,

Disaggregated, Cohortdata

to assess Student Success

Outcomes (e.g., Persistence,Course

Completion rates, Degree comp.

rates) to determine:

1) Which student groups are

less successful than others

(Equity Gaps in Student

Success).

2) Which high enrollment

courses have the lowest

success rates.

Collect, analyze, and use

second set of LOCAL data to

identify the underlying

factors (barriers or

challenges) impedingstudent success:

Focus Groups

Surveys

Literature Reviews

Learning Outcome

Assessment

Use data from Component

Two to revise or design new

interventions to effectively

address the underlying

factors impeding studentsuccess.

Review and consider

changes to existing college

policies that impact the

underlying factors impeding

student success.

Many Colleges:

(a) Skip

(b) Loosely rely on national

literature (Engagement)

(c) Lack a local understanding

based on qualitative data

Collect, analyze, and use

evaluation data to answer:

1) To what extent did the

interventions (or policychanges) effectively address

the underlying factors

impeding student success?

2) To what extent did the

interventions

increase student

success?

Make modifications based

on evaluation results.

Reference: Gonzalez, K. P. (2009). Using data to increase student success: A focus on diagnosis.

Achieving the Dream Inc. www.achievingthedream.org


9/26

DATA TOOLBOX: 2011 Fact Book (hot off the presses!)

Leakage Point Analysis Hand out (fromLumina)

IPEDS Factsheets for 2005, 2006, 2007,

2008, 2009, 2010, 2011

LAHC readiness submission to Lumina for

the grant

Alignment of College/District Strategic Plan

(presented to the Board June, 2011)

Powerpoint of Aligning AtD with

Accreditation activities

Drop Survey results from Spring 2011

Financial Aid data

Learning Assistance Center datawork in

progress

Powerpoint from Dr. RichardsWho are our

Students? Article7 Myths of Student Retention

LAHC Highest Enrollment Courses X

demographic

LAHC Highest failure courses X

demographic

Financial Aid guidelines provided and

discussed

Matriculation Committee Report/

Assessment Data

Summary of Orientation data from E.

Colocho Exit Point Analysis (aka Leakage Points or

momentum points)

Course availability based on placement data

(Report)

Articles: A Period of Adjustment? Race-

adjusted Rates for a State Accountability;

TrickleAcross Theory: Student Flow Into

and Away from the California Community

Colleges

Multiple files on Qualitative data collection

technique (Focus Groups)

ARCC data report from LATTC which has all

colleges in District comparison

In the Beginning


10/26

OUR DATA TEAMS PROCESS

OF INQUIRY AND DIALOGUE:

What are our pain points?

Retention, Completion, Success

What additional data would be useful to know?

What research questions should drive our data

campaign?

What data do we have versus what do we need?

What approach do we want to take?


11/26

GOAL OF AtD DATA SHARED

1. Students Successfullycomplete the courses they take

Factbook; exit point analysis; highest

enrmt/lowest success data; lowest retention

courses data; low success courses data; dropsurvey results Spring2011; in progress:

dismissal/probation student analysis (which

may have some spillover into the other 5

goals)

2. Students Advance from

remedial to credit-bearingcourses

Factbook; exit point analysis; learning center

data (in progress); Course availability report(Matriculation)

3. Students Enroll in and

successfully complete

gatekeeper courses

exit point analysis; Factbook; IPEDS; ARCC

4. Students Enroll from one

semester to the next exit point analysis; Factbook; IPEDS; ARCC

5. Students Earn degrees and/or

certificates

Kick Off presentation data slides; exit point

analysis; IPEDS data; Factbook; ARCC


12/26

44.1%

25.5%

47.0%

30.3%

53.6%

41.5%

53.0%

42.6%

0%

20%

40%

60%

80%

100%

Fa04 Sp05 Fa05

Asian F Asian M AfrAmer F AfrAmer M Hisp F Hisp M White F White M

Lowest Persistence In The First Year: African American Male

African American Female

Hispanic Male

White Male

PERSISTENCE: The Pipelineof First Time College Students by Ethnicity & Gender


13/26

22.5%

27.5%

19.7%

12.1%

34.3%

26.6%

36.5%

27.8%

0%

20%

40%

60%

80%

100%

Fa04 Sp06 Fa06


Lowest Persistence In The Second Year:

African American Male


Hispanic Male

White Male

PERSISTENCEof First Time College Students by Ethnicity & Gender


14/26

13.7%11.8%12.1%

7.6%

18.8%15.9%

21.7%

11.3%

0%

20%

40%

60%

80%

100%

Fa04 Sp07 Fa07


Lowest Persistence In The Third Year:



Hispanic Male

White Male



15/26

8.8%9.8%6.9%

1.5%

7.6%

13.0%11.3%

7.8%

0%

20%

40%

60%

80%

100%

Fa04 Sp08 Fa08


Lowest Persistence In The Fourth Year:



Hispanic Male

White Male

Asian Male



16/26

10.0%

6.3%5.9%6.1%

0.0%

8.8%7.0% 7.0%

0%

20%

40%

60%

80%

100%

Fa04 Sp09 Fa09


Lowest Persistence In The Fifth Year:



Hispanic Male

White Male

Asian Male

White Female



17/26

African American Males

African American Females

Hispanic Males

White Males

PERSISTENCEof First Time College Students -- Summary

Ethnicity and Gender

With Lowest Persistence In 5-Year Trend:


18/26

7

39

65

36

20 19

7

Within 1

year

2 years 3 years 4 years 5 years 6 years More than

6 years

Count

DEGREE & CERTIFICATEAttainment of First Time College Students Over Time


19/26

DEGREE & CERTIFICATEAttainment of First Time College Students - Summary

Within 2 years -- 4% received a degree

or certificate.

Within 4 years 12% received a degreeor certificate.

Overall, 16% (or 193) of 1,209 First TimeCollege Students in Fall 2004 received adegree or certificate.

Awards Received:


20/26

Fall 2004 and Fall 2005 Comparison

FALL 2005:

Age, ethnicity, and gender fairly consistent between the

two semesters.

Percentage of awards received also consistent at 16% (6or more years).

Slight shift on lowest persistence between

disaggregated groups--Fall 2004: African American F African American M Hispanic M

Fall 2005: African American F African American M White M


21/26

English and Math

Within the First Year of College

ENGLISH Out of 1,354, 806 (60%) took

an English course within thefirst year.

-16% (126) Asian-15% (119) White-11% (90) African American-52% (418) Hispanic

Lowest groups tosuccessfully complete:

52% - White M (25 out of 48)45% - Afr Amer M (18 out of 40)52% - Afr Amer F (26 out of 50)62% - Hispanic F (160 out of 259)

MATH Out of 1,354, 749(55 %) took a

Math course within the firstyear.

-15% (112) Asian-15% (109) White-10% (77) African American-54% (408) Hispanic

Lowest groups to successfully

complete Math:33% - White M (15 out of 45)20% - Afr Amer M (7 out of 35)38% - Afr Amer F (16 out of 42)39% - Hispanic M (61 out of 158)


22/26

Fall 2004 and Fall 2005 Comparison

FALL 2005:

Age, ethnicity, and gender fairly consistent between the

two semesters.

Percentage of awards received also consistent at 16% (6or more years).

Slight shift on lowest persistence between

disaggregated groups--Fall 2004: African American F African American M Hispanic M

Fall 2005: African American F African American M White M


23/26

Unexpected Outcomes of Sharing Data Discovering our SIS could not hold major code semester to

semester unless the student declared upon entry Assessment Committee

CTE Deans

Admissions & Records Deans

Recommended change to the D.O. for the SIS New SIS coming on board and resources scarce!

Discussion of Majors fair/Declare Day Students major choice and relationship to math (Focus Groups and

Survey) Students choose their majors based on least amount of math possible

Sequence too long; want acceleration

Students very savvy about other colleges in/out of District Language issues by Instructor awareness, not sure how to address

English 28 is harder than English 101 (Focus Groups)

Overwhelming theme that students know it is their responsibility tolearn

BUT, do they have the skills to meet the responsibility?


24/26

Outcomes leading to changes Student Services Interventions

Created an In-person Orientation

Creation of FYE cohorts with Learning Coaches

Puente

Math/English Interventions Need more Math/English sections

Hiring of Specialists, not just content experts On Course Training

Fast Track/Math acceleration

Equity Intervention Culture of poverty (first year)

Creation of an Urban Center (second year, funding dependent) Evaluation (all of the above)

Data shared

Data discussed


25/26

How will you guide the process on

your campus?

ACTIVITY:

What data will you find useful?

Where is the data located? How will you share the data?

Can you use existing structure on your

campus ie. Assessment Committee


26/26

Wrap Up

Method-ists: Building a Data-Informed Culture

Documents

Transcript of Method-ists: Building a Data-Informed Culture