2009 SHEEO/NCES Network Conference and IPEDS Workshop May 21, 2009 J. Keith Brown

Success is what counts.Success is what counts.

A Better Way to Measure Community College Performance: An Achieving the Dream Cross-State Data Initiative

2009 SHEEO/NCES Network Conference and IPEDS WorkshopMay 21, 2009

J. Keith BrownNorth Carolina Community College System

Success is what counts.

Presentation Summary: Overview of Achieving the Dream and the Cross-

State Data Workgroup

Recommendations for an Alternative Set of Measures for Community College Performance

Introduction of Intermediate Milestones and Final Measures of Student Performance

Implications for Policy and Practice


Achieving the Dream Overview

National initiative to help more community college students succeed (earn degrees, earn certificates, or transfer)

Particularly concerned about student groups that have faced the most significant barriers to success, including low-income students and students of color


82 Institutions in 15 StatesAR, CT, FL, HI, MA, MI, NC, NM, OH, OK, PA, SC, TX, VA, WA


Achieving the Dream Values

Student-centered

Equity and excellence

Culture of evidence, inquiry, accountability, and shared responsibility


Cross-State Data Workgroup

Initial StatesConnecticutFloridaNorth CarolinaOhioTexasVirginia

States JoiningArkansasMassachusettsNew MexicoOklahomaSouth CarolinaWashington


Data Workgroup Goals

Develop a set of indicators to:

More effectively track student performance

Evaluate the effectiveness of interventions

Learn from the strengths of other community college systems


Test Drive: Six States Pilot Better Ways to Measure and Compare Community College Performance


Recommended Changes:Prior EnrollmentCurrent Federal Method ATD Work Group Method

First-time-in-college students only Same as federal method

Intent at time of EnrollmentCurrent Federal Method ATD Work Group Method

Only students seeking a certificate or degree

Same as federal method

Enrollment StatusCurrent Federal Method ATD Work Group Method

Full-time students only Full-time and part-time students


TimeframeCurrent Federal Method ATD Work Group Method

Three years (150% of “normal time” to completion)

Six years

Recommended Changes:

Success Outcomes

Current Federal Method ATD Work Group Method

Earned degree or certificate - Earned degree or certificate- Transferred without an award- Enrolled in year 6 with at least 30 hrs


Tracking transfer students within the 2-year sector


Reporting is based on individual college; doesn’t track outcomes of students who transfer to another college; simply reported as transferred out

Reporting is based on statewide community college system; tracks outcomes within the system (therefore across colleges)



Controlling for factors associated with success


Part-time students excluded from analysis; no disaggregation of results by age at initial enrollment

Disaggregated results by part-time and full-time status and age at initial enrollment



Results of Cross-State Comparison


AGE DISTRIBUTION OF ENROLLMENT

74% 74%

51%

75%83%

76%

23% 23%

43%

22%15%

22%

2% 3% 7% 2% 2% 2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Connecticut Florida North Carolina Ohio Texas Virginia

PERC

ENT

SHAR

E

22< 23-45 46>



GENDER DISTRIBUTION OF ENROLLMENT

45% 46% 44% 47% 46% 44%

55% 54% 56% 53% 54% 56%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


PERC

ENT

SHAR

E

Male Female



Distribution of Awards and Transfer w/ o Award Outcomes (all students)

1%

6%

10%

2%5%

3%

9%

22%

16%

23%

12%

19%

14%

7%

14%

6%

25%

13%

0%

10%

20%

30%

40%

50%

CT FL NC OH TX VA

Award of Less Than AA Degree Associate's Degree Transfer w/ o Award



Students Who Earned an Award or Transferred OR Who Are Still Enrolled with 30+ Credits (all students)

23%36% 40%

30%42%

35%

9%

8%8% 9%

3%5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

CT FL NC OH TX VA

Total Earned Award or Transfer Still Enrolled w/o Award with 30+ Credits


Success is what counts.CT FL NC OH TX VA0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

25%

39% 40%35%

45%38%

10%

8%10%

5%

10%

4%

Younger Students (> or = 22) Who Earned an Award or Transferred and Who Are Still Enrolled with 30+ Credits

Total Earned Award or Transfer Still Enrolled w/o Award with 30+ Credits

Perc

enta

ge



PERCENTAGE OF STUDENTS EARNING AN AWARD WHO SUBSEQUENTLY TRANSFER TO A 4-YEAR INSTITUTION

(FULL-TIME STUDENTS)

17%12%

5% 7% 7%

15%

47%

72%

36%

54% 54%

22%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Connecticut Florida NorthCarolina

Ohio Texas Virginia

PERC

ENTA

GE

% of Certificate Earners % of Associate's Degree Earners



PERCENTAGE OF STUDENTS EARNING AN AWARD WHO SUBSEQUENTLY TRANSFER TO A 4-YEAR INSTITUTION

(PART-TIME STUDENTS)

8% 8% 6% 7% 4% 3%

34%

59%

29%23%

43% 43%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Connecticut Florida NorthCarolina

Ohio Texas Virginia

PERC

ENTA

GE

% of Certificate Earners % of Associate's Degree Earners



Next Steps: Developing Intermediate Benchmarks to Measure Student Progress


Intermediate milestones to track studentsFirst-Year Milestones• Persisted fall to spring• Passed 80% or more of attempted hours• Earned 24 or more hours

Second-Year Milestones• Persisted fall to fall• Completed developmental math by year 2• Earned 48 or more hours

Third-Year Milestones• Passed gatekeeper English or higher by year 3• Passed gatekeeper math or higher by year 3


First-Year Milestones

Returned spring semester 16% increase in final success outcomes for

full-time students; 28% for part-time students

Earned 24 credits (full-time) or 18 credits (part-time) by the end of the first year 25% increase in final success outcomes for


Passed 80% of credits attempted 12% increase in final success outcomes for



Second-Year Milestones

Returned in fall of second year 23% increase in final success outcomes for full-

time students; 53% for part-time students

Returned and earned 42 credits (full-time) or 24 credits (part-time) by the end of the second year 32% increase in final success outcomes for full-

time students; 49% increase for part-time students


Second-Year Milestones

Passed developmental mathematics course by the end of the second year 84% increase in final success outcomes for full-


Passed developmental English course by the end of the third year 17% increase in final success outcomes for full-



Third-Year Milestones

Passed “gatekeeper” mathematics course by the end of the third year 45% increase in final success outcomes for full-


Passed “gatekeeper” English course by the end of the third year 17% increase in final success outcomes for full-



Tracking toward final success measuresFourth- and Sixth-Year Measures• Award of less than associate’s degree w/o transfer

• Award of associate’s degree or higher w/o transfer

• Award of less than associate’s degree and transferred

• Award of associate’s degree or higher and transferred

• Transferred w/o an award

• Still enrolled with 30 or more college hours

• Total success rate


Next Steps…finishing data runs• Run benchmarks at state and institutional

levels• Disaggregate and analyze performance by:o academic readinesso incomeo ethnicityo gender

• Identify and document promising interventions


What does all this mean and what are the policy implications?


Context of the Data Reflects the mission of colleges/state

systems Examples: age distribution, award

distribution Reflects differences in state

priorities/policies Example: college transfer policy

Not all measures pertain to all students

Example: gatekeeper math course


AGE DISTRIBUTION OF ENROLLMENT

74% 74%

51%

75%83%

76%

23% 23%

43%

22%15%

22%

2% 3% 7% 2% 2% 2%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%


PERC

ENT

SHAR

E

22< 23-45 46>


Distribution of Awards and Transfer w/ o Award Outcomes (all students)

1%

6%

10%

2%5%

3%

9%

22%

16%

23%

12%

19%

14%

7%

14%

6%

25%

13%

0%

10%

20%

30%

40%

50%

CT FL NC OH TX VA

Award of Less Than AA Degree Associate's Degree Transfer w/ o Award


Impact of state policy on outcomesDifferences in transfer patterns reflect policy differences

o Encouraging transfer after earning a degree• e.g. FL – 69% transfer after degree & 7% before

o Encouraging transfer without a degree• e.g. TX – 25% transfer without degree

o Absence of strong transfer policies• e.g. OH – 22% transfer after degree & 6% before

o Balanced approach to transfer• e.g. NC – 16% transfer after degree & 14% before


Implications of the Data: College Perspective Identification of at-risk students Student advising Review of policies/practices

Examples: Course taking sequence/timing Drop/add policies


Implications of the Data: State/System Perspective Policy development to improve student

success Review of regulations Performance indicators Development/refinement of student

database Benchmarking


Implications of the Data: National Perspective More appropriate measure of student

success: accountability Financial aid policy Expand the body of knowledge on

successful community colleges/practices


Acknowledgements

The following individuals are gratefully acknowledged for their contributions to this presentation:

Chris Baldwin: Jobs for the FuturePat Windham: FloridaDonna Jovanovich: VirginiaCorby Coperthwaite: Connecticut


Achieving the DreamSuccess is what counts.ww.achievingthedream.org

2009 SHEEO/NCES Network Conference and IPEDS Workshop May 21, 2009 J. Keith Brown

Documents

Transcript of 2009 SHEEO/NCES Network Conference and IPEDS Workshop May 21, 2009 J. Keith Brown