Jane Patton President, ASCCC Michelle Pilati C-ID Faculty Coordinator Vice President, ASCCC.
2010 ASCCC Curriculum Institute Santa Clara Marriott • July 8-10, 2010
description
Transcript of 2010 ASCCC Curriculum Institute Santa Clara Marriott • July 8-10, 2010
2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010
Maps to Success: Using Data
• Richard Mahon, Curriculum Committee• Beth Smith, Grossmont College• Greg Stoup, Cañda College
o Key reference: Data 101
2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010
Maps to Success: Using Data
Accreditation standards require that colleges assess the curriculum at the course, program, and degree levels. What data would allow the curriculum committee to contribute to broader campus conversations about student learning, and how can the curriculum development and approval process help guide faculty toward the thoughtful use of data in evaluating and improving their own effectiveness?
Does your curriculum committee concern itself with data?
2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010
Maps to Success: Using Data
Curriculum, data and institutional processes…
• Program Review• Student Learning Outcome Assessment• Student Equity• Enrollment Management
2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010
Maps to Success: Using Data
System Data: Datamart http://www.cccco.edu/ChancellorsOffice/Divisions/TechResearchInfo/MIS/DataMartandReports/tabid/282/Default.aspx)
ARCC Report:
CalPASS - http://www.cal-pass.org/
•NCES National Center for Education Statistics http://nces.ed.gov/
•CPEC California Postsecondary Commission http://www.cpec.ca.gov/
Local Data?
2010 ASCCC Curriculum InstituteSanta Clara Marriott • July 8-10, 2010
Maps to Success: Using Data
Data for what? • Prerequisites• Labor Market Analysis• Student performance• Student completions (certificates, degrees, transfer)
The Challenges with using college data to
identify strategic interventions
To the outsider, researchers can have an intimidating comfort zoneThe T-Test Procedure Statistics
Lower CL Upper CL Lower CL Upper CLVariable: female N Mean Mean Mean Std Dev Std Dev Std Dev Std Err
Write 0 91 47.975 50.121 52.267 8.9947 10.305 12.066 1.0803Write 1 109 53.447 54.991 56.535 7.1786 8.1337 9.3843 0.7791Write-Write Diff (1-2) -7.442 -4.87 -2.298 8.3622 9.1846 10.188 1.3042
T-Tests
Variable Method Variances DF t Value Pr > |t|
Write Pooled Equal 198 -3.73 0.0002Write Satterthwaite Unequal 170 -3.66 0.0003
Equality of Variances
Variable Method Num DF Den DF F Value Pr > F
Write Folded F 90 108 1.61 0.0187
The wrong format creates dissonance between your message and your audience.
Packaging the message
Consider a simple pathway assessment Basic Skills
Student
Successful Transfer
Learning Community
Hypothesis: Learning Communities improve the transfer success of Basic Skills students
We pull data from our database to test the hypothesis ….
Successful Transfer
Complete SEP
Take Summer Courses
Learning CommunityEnroll Full-
Time
Basic Skills Student
Enroll in MESA
We learn that we need more info to tell the real story
Successful Transfer
Faculty Letter of Recommendation
Library Orientation
Good impression
from Campus
Visit
Talk to Univ. Rep
Nurture Faculty Relationships
Meet with Outreach Professional
Attend Lecture Series
Learning Community
Complete SEP
Take Summer Courses
Participate in Student Govt
Hallway chat with
professor
Basic Skills Student
Enroll in MESA
Enroll Full-Time
Text books in stock
Becomes student tutor
Join Student Club
Placement Test Prep
Faculty suggest outside reading
Faculty recommends MESA
Planning advice from FA Office
But the reality is…
Examples of Program Level Data that can
drive curricular discussion
Cohort Description Headcount
One year Fall-to-Fall Student
Persistence
Two year Fall-to-Fall Student
Persistence FTES
One year Fall-to-Fall
FTES Persistence
Two year Fall-to-Fall
FTES Persistence
Concurrent Enrollment 585 29.5% 15.0% 142.6 43.8% 26.0%
ESL-Only Student 975 32.9% 17.9% 293.5 38.1% 19.9%
Basic Skills Student 1,062 51.0% 35.3% 509.9 48.4% 30.7%
Career-Tech Directed Student 1,328 35.9% 21.3% 304.9 41.8% 21.8%
Transfer Directed Student 2,249 38.2% 23.8% 570.7 40.0% 19.7%
Ed Development Student 606 45.6% 31.9% 117.4 44.6% 26.5%
College Overall 6,805 39.1% 24.4% 2,098 42.4% 23.4%
There is significant variation in persistence across the six segments.
Segment Persistence
College Average 39.1%
College Average 24.4%
One Year Fall-to-Fall
Two Year Fall-to-Fall
Segment Success & Retention
College Average 70.0%
College Average 84.5%
Success Rates
Retention Rates
Can we build on this information?
An estimate of the resource usage patterns of each segment (low, medium, high) might help us develop strategies to more effectively deploy rare resources.
Cohort DescriptionPct of FTES
Fall-to-Fall Persistence
Percent Single Course Takers Six Year Growth Pattern
Resource Usage
Concurrent Enrollment 8.6% 29.5% 68% Accelerating Growing ?
ESL-Only Student 14.3% 32.9% 59% Mild Growth with some volatility ?
Basic Skills 15.6% 51.0% 14% Modest increase with some volatility ?
Career Directed 19.5% 35.9% 58% Several years of steady growth ?
Transfer Student 33.0% 38.2% 40% Slight recovery after extended decline ?
Ed Development 8.9% 45.6% 69% Flattening after steep decline ?
College Overall 100% 39.1% 47% Strong growth following a steep decline ?
ESL Students have low persistence in part because many navigate long course sequences.
ESL / ENGL SequenceEntry Point for Student
Cohort Metric Level 0 Level 1 Level 2 Level 3 Level 4 ESL 400*ENGL 100
Level 0# 1,881 889 443 227 160 42 16% 47.3% 23.6% 12.1% 8.5% 2.2% 0.9%
Level 1# 2,722 1,424 817 602 167 70% 52.3% 29.9% 22.1% 6.1% 2.6%
Level 2# 2,555 1,425 912 312 147% 55.8% 35.7% 12.2% 5.8%
Level 3# 2,173 1,248 489 241% 57.4% 22.5% 11.1%
Level 4# 1,815 806 403% 44.4% 22.2%
ESL Persistence by Student Entry Level(Tracking Period: Fall 2000 – Spring 2008)
* ESL400 category also includes ENGL 836
Persistence patterns generally follow a 50% Rule
Students Taking a Single Course
Students Taking Multiple Courses
In any given term roughly half of our students are Single Course Takers
Why is this important?
• Single Course Takers have a more tenuous connection to the college• Single Course Takers have low persistence• Returning students that become single course takers are 80% less likely to transfer or obtain a degree†
*Persistence Rate = 50.3% *Persistence Rate = 29.8%
*Includes only the Base Segment population of Basic Skills, Career Directed & Transfer Directed Students. †Adjustments were made to account for students whose course load dropped in the final term before transfer or receiving a degree.
Headcounts of Single Course Takers & Percentage of Segment Population
Single Course Takers are not uniformly distributed across the Student Segments
68%
59%
14%
58%
40%
69%
College Average = 47%
INTERMEDIATE ALGEBRA
Sequence Completion Rates by Initial Course Placement
TRANSFER LEVEL MATH
PRE ALGEBRA
ELEMENTARY ALGEBRA
INTERMEDIATE ALGEBRA
Initial Placeme
nt3 years
3.6%
19.1%
47.6%
4 years
5.4%
20.6%
49.2%
5 years
6.1%
22.4%
49.2%
PRE ALGEBRA ELEMENTARY ALGEBRA
2 years
2.3%
15.5%
43.4%
Percent of Students Completing the Algebra Sequence within 2 to 5 Years
Basic Skills Curriculum Sequence
Course Pass % = 55% Course Pass % = 55% Course Pass % = 50% Course Pass % = 75%
Profile of Developmental English & Reading
TRANSFER LEVEL
ENGLISH
Basic Composition
Developmental Reading
Writing Development
Reading Strategies
Course Pass % = 58%
Course Pass % = 58%
Course Pass % = 64%
Course Pass % = 69%
Course Pass % = 67%
ENGL 826
Initial Placeme
nt3 years
23.9%
24.8%
4 years
27.2%
29.9%
5 years
27.7%
30.1%
2 years
20.3%
22.1%
Percent of Students Completing the Basic Skills Sequence within 2 to 5 Years
READ 826
Examples of often overused metrics that, in isolation, can lead to
false conclusions
What are the two measures most widely used by CCCs to assess progress?
• Success Rate• Retention Rate
Consider the multitude of changes over this period:
- faculty/staff turnover- program successes/failures- changing student demographics - budget contractions/expansion- leadership turnover
and yet these performance metrics remained relatively stable.
What to do when you reach the limits of what data can answer
You start off with a large number of
options
The data helped shrink the number of options but
there is still more than one choice
Trust your
intuition and
choose !
( You’ve talked with your colleagues &
reflected on the data )
Handout: An illustration of a collaborative dialogue converging on action
Collaborative Model of Institutional Research
Faculty/Staff
Researcher
Joint Activity
Primary Responsibility
Key Features:
• Dialog-rich• Jointly-driven processes• Priority on the development of the data story
Design Charrette Joint Reflection on Findings
Story Development
Validate Message
Share the Story
1. An attitude of wisdom (knowing what you don’t know)
2. Commitment to framing issues with data
3. Commitment to hearing & telling the truth
4. Adoption of an experimental mindset
5. Oriented toward action
What characterizes a learning organization?
Source: Jeffrey Pfeffer, Professor of Organizational Behavior at the Graduate School of Business, Stanford University.
Nurturing a Culture of Inquiry
DISCUSSION
If you remember only one thing
Make sure you have a room, populated with reflective thinkers, that meet regularly, to discuss data and ask questions
Nurturing a Culture of Inquiry
THANK YOU