Developmental Math Project 2007-2012

Strategic Goal (2008-2011 Plan):5.1 Increase the number of degree/certificate student completers by 1% per year. 5.2

Increase the first time, full time student fall to spring retention rate by 1.5% per year. 5.3 Increase the first time, full

time student fall to fall retention rate by 1.5% per year 5.4 Implement new cross college initiatives designed to

enhance the academic success of students who test into developmental courses: b. Increase by 1% per year the

number of these students who successfully complete the college level course that follows completion of related

developmental courses.

Strategic Goal (2012-2017 Plan): Imperative 1. Northwest College shall be the preeminent two year educational

institution in Wyoming. Goal 1. The highest student completion rates (earned degrees and certificates) of any two

year educational institution in Wyoming.

General Education Goal: Isolate a pertinent question or problem, identify appropriate problem solving strategies, use

mathematical approaches when warranted, draw warranted, logical conclusions, present convincing evidence to

support conclusions.

Program Goals: Choose the appropriate mathematical problem solving technique within the context of a practical

situation and interpret the results. Apply and demonstrate an understanding of numeric and symbolic manipulation

to expressions, relations, and functions.

Budgetary Impacts: One third release time for Developmental Mathematics Coordinator

Computer outfitting of rooms for ALEKS instruction

History – How it happened?

Bob Krenz received a report from Damian and Virginia (2005) when they were doing some work for the Title III

grant that indicated the pass rate in Developmental Math and College Algebra was lower than other community colleges

in the state (possibly 6th out of 7). VPAA Shar indicated we had a lower pass rate in developmental math classes then

desired as reported by the WCCC (ranking had decreased from previous years to an “unacceptable” level) and that math

was a deterrent to completion of degree. Bob Krenz enlisted Lori Austil’s assistance to check the validity of the data

using the year 2003 to 2006 (2006). The data they pulled from our system indicated the success rate in developmental

math in 2003 was 65% and 2006 was 50%. They researched national references and cited national average pass rate in

developmental math was 67%. A proposal to hire someone to do something was developed and presented to ACC

(2007). The someone was called a Developmental Math coordinator, and the something was left up to that person.

Hired Lourdes Pajo to start in fall of 2008. A group was sent to the NAID conference. The information they returned

with indicated that pass rate/student learning was also highly impacted by influences on students outside of the

curriculum, pedagogy, or any methodology. This started the initiative for a College Studies class (sometimes called

Freshman Orientation which continued in a Kick Off Weekend condensed format until the conclusion of the Title III grant

then academic portions discontinued), that died for lack of college support or structure. Emphasis returned to

curriculum, common content, common assessment of student learning, placement, validity of course grades as

indicators, and standardized topics state wide. Inconsistent application and resistance to initiative makes validity of our

data an issue. The Lumina foundation reports phenomenal success rates with artificial intelligence based computer

learning systems in math, specifically MyMathLab and ALEKS (2009). Application for use of technology funds from Title

III to establish enough computer stations for use of ALEKS as Developmental Math learning system. Algebra review

during Intersession 2009 resulted in all students able to progress 1 math level after using ALEKS. Math 0900 class was

set up as half ALEKS and half traditional, this class performed better than pure traditional class.

Inconsistency -Impact 1st Adjustment -Date

Challenges- Adjustments

2nd Adjustment -Date

Current Challenge

Recommendation

Curriculum across the state for Dev Math – comparing different student learning

WYMATC Lourdes takes lead in establishing common topics and credits per class - 2009

Our Math 0930 was more credits/topics-reduced to be in line with statewide

Conversation continues not all have adjusted - 2012

Course Grading – pass rate not valid assessment of student learning

Allocation of percent of overall grade for Dev Math is prescribed for all Dev Math classes - 2008

Cooperation of instructors – Published ready syllabi schedule and instructions for all Dev Math classes and sends to all Dev math instructors each semester

Need to monitor whether instructors are actually following guidelines - 2012

Assessment of learning level and topics greatly varied - 2009

Common finals - 2010

Getting instructors to actually use and turn in results/ change of credits and topics changed finals - 2011

Repeated requests through email and department meetings

Use of Calculators – varied greatly allowing some questions to be just punch instead of concept acquisition-2010

Calculator use policy passed by Math Department - 2011

Adherance to policy and how to handle students with disabilities or differential high school diplomas- ongoing

Students without prerequisites/placement - 2009

Tracking of mis-placed students and cleaning up of rosters by hand and participation in state wide conversation of Compass pre-requisite scores related to pass rates - 2010

Computer did not enforce pre-requisite cut offs –work with registrar to adjust computer enforcement and identify how students are getting in without pre-requisites

Continuing monitoring by hand and paper list

Pacing of class and differential emphasis on

ALEKS pilot program - 2009

Everyone used it differently –

Math department

Verification of compliance by

topics - 2009 some had no lessons presented, some used as a homework system only, some used in conjunction with regular textbook and many graded differently – topics in ALEKS were not pre-set

policy that lessons must be delivered, percentages for overall course grade established, templates for each course -2011

instructors

Summary Table

Action Items –Date completed

Indicators - 2012 Results Presented Conclusion Recommendation

Hire Developmental Math Coordinator -2008 Fall

Longitudinal comparison of pass rates by course NWC

Appendix A Developmental Math Pass Rates

Developmental Math pass rates fluctuated. Terminal Math completion consistently higher than Developmental. Overall years Dev Math pass rate is significantly lower than Terminal Math.

Continue to encourage consistency. Research for type of student successful in each venue and changes in characteristics.

Scale back curriculum to match the rest of the state-2010

Longitudinal comparison of pass rates in Dev Math from 2003-2012 NWC to state

State wide pass rates were not able to be obtained-I have a draft10/1/09 from Bob Krenz alluding to them on the “next page” but no next page. NWC pass rates Appendix A Developmental Math Pass Rates

Standardized course final assessment of student learning

Longitudinal comparison of Final exam data

Appendix B Finals

Math 2009-10 Traditional is lower – rest fairly consistent

Standardized curriculum NWC and state -2010

Longitudinal comparison of success in next dev math class in sequence Longitudinal comparison of pass rate of dev math students who take college level math

Standardize grading – Longitudinal Appendix C Math 0920 and 0930

comparison of course grades

downward trend

Intersession Math Review class on ALEKS - 2009

Grade by year Appendix D Variable

Summer session Math classes ALEKS - 2011

Grade by year Appendix E Variable

Use of ALEKS – 2009 but inconsistently/individual to class

Comparison of ALEKS pass rates to Traditional pass rates Comparison of ALEKS final exam scores to Traditional final exam scores

Appendix F Appendix G

Some significant differences need more data Most significant differences in Math 0900

Personal intervention by division chair with faculty with particularly low pass rates

Not assessed

Enforce pre-requisites – inconsistent 2011

Relationship between COMPASS score and course grade in first math class Longitudinal pass rates of misplaced students versus correctly placed students

Questions and Notes:

Does Intersession success in 2009 and summer school success suggest block schedules/immersion?

Have any student characteristics changed from 2003 to 2007?

GOALS (quoted from Bob Krenz notes)

70% pass rate in Developmental Math classes, 50% pass rate of Developmental Math students in College Level Math.

Actions items attempted to be instituted:

Common finals in all developmental math classes

Coordination and consistency of content, expectations, grading, use of calculator in each developmental math

class

Communication of coordinated content, expectations, grading, use of calculator to every adjunct instructor

State level coordination of content of developmental math courses

Rigorous monitoring and adjustments to students who do not meet the pre-requisites (or placement

requirements)

Use of the ALEKS program of artificial intelligence driven computer software to increase learning and decrease

boredom

Measurement of effectiveness of action items:

Comparison of pass rates before action items to after action items – measuring overarching effect loss of

definition problem- wary that could actually see a changes because of consistent expectations, grading practices, use of

computerized learning, so view this with the completion of sequence data, in whole and by sections taught by adjunct or

full time (do not know if I can get that) and ALEKS non-ALEKS

Comparison of pass rates by proper and improper placement before action items to after action items, ALEKS

and non-ALEKS – measuring effect of placement vigilance – this will indicate effect of proper placement and whether

proper placement has increased – can only do those students with compass scores in the system that have a student id

number assigned

Comparison of completion of the developmental math sequence before action items to after action items,

ALEKS and non-ALEKS – differing level sequences Math 0900-920-930 or any partial sequence.

Comparison of completion of terminal math class (math for graduation 1400,1000,1050) of students who

began in a developmental math before action items to after action items, ALEKS and non-ALEKS – initial indicator of

student graduation challenge of math requirement - any student whose first math class was a 9##, to terminal class

Comparison of grade distributions before action items to after action items, ALEKS and non-ALEKS –

measuring consistency across all classes

Item analysis of Final exam data – measuring topics that are most challenging to inform curricular change or

pacing

Comparison of Final exam data ALEKS to non-ALEKS – measuring effectiveness of ALEKS program on retention

of skills

Comparison of Final exam data Adjunct to Full time – measuring effectiveness of consistency of expectations,

and grading practices

APPENDIX A

Developmental Math Pass Rates

(Data Pull for 2011-2012 not completed)

Pass = A’s B’s C’s Non Pass = D’s, F’s, UF’s, I’s, AU’s, W’s and S’s

Class 2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

Class Pass Rate All Years

MATH-0900 65% 54% 62% 66% 52% 58% 51% 63% 65% 60%

MATH-0920 64% 59% 57% 49% 51% 54% 65% 42% 50% 54%

MATH-0930 69% 66% 65% 62% 63% 63% 64% 64% 61% 64%

MATH-0990 NA NA NA 53% NA NA NA 89% 54% 63%

MATH-1000 78% 70% 81% 68% 69% 68% 70% 63% 62% 69%

MATH-1050 63% 74% 86% 77% 74% 75% 71% 93% 73% 75%

MATH-1100 60% 67% 55% 63% 69% 85% 86% 100% 100% 72%

MATH-1105 82% 84% 78% 89% 58% 94% 100% 92% 81% 84%

MATH-1400 63% 66% 64% 60% 56% 61% 63% 74% 71% 65%

MATH-1405 67% 60% 88% 71% 83% 82% 89% 84% 61% 75%

MATH-1450 53% 59% 63% 68% 86% 73% 81% 59% 73% 68%

MATH-1470 NA NA NA NA NA NA NA NA NA NA

MATH-1490 NA NA NA NA NA NA 100% NA NA 100%

MATH-1510 56% 92% 70% 60% 69% 79% 58% 64% 65% 66%

MATH-2200 76% 62% 61% 80% 73% 91% 76% 75% 63% 73%

MATH-2205 68% 60% 100% 71% 72% 68% 81% 57% 82% 72%

MATH-2210 100% 80% 100% 67% 95% 93% 64% 100% 88% 88%

MATH-2250 100% NA 83% 100% 62% NA 100% 57% NA 74%

MATH-2300 NA NA 100% NA NA 100% NA NA NA 100%

MATH-2310 67% 100% 100% 79% 71% 69% 100% 93% 67% 81%

MATH-2345 NA 100% 60% 100% 75% 80% 80% 100% 100% 85%

MATH-2350 88% 89% 57% 64% 65% 91% 87% 92% 84% 83%

MATH-2355 89% 100% 100% 100% 100% 100% 67% 67% 67% 90%

MATH-2475 NA 100% NA NA NA NA NA NA NA 100%

All Math Classes 67% 64% 65% 62% 61% 65% 66% 64% 64% 64%

Developmental Math(900,920,930)

66% 60% 61% 58% 56% 58% 61% 56% 58% 59%

Terminal College (1000,1400)

66% 67% 67% 62% 58% 62% 64% 73% 70% 66%

0%

10%

20%

30%

40%

50%

60%

70%

80%Y

ear

ly P

ass

Rat

es

Developmental Math Pass (A's,B's,C's) Rates

Non Pass (D's, F's, UF's, I's, W's, Au's, S's)

MATH-0900

MATH-0920

MATH-0930

Developmental Math Refined Pass Rates

(Data Pull for 2011-2012 not completed)

Pass = A’s B’s C’s Non Pass = D’s, F’s based on inability to impact UF (no attendance didn’t finish class), I (they decide

whether to finish), W (usually reasons are beyond instructor control), Au (they meant to audit) and S (satisfactory not

considered a measure of skill attainment)

Refined PASS RATES BY YEAR

Class 2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

Class Pass Rate All Years

MATH-0900 75% 61% 72% 83% 73% 71% 69% 75% 85% 74%

MATH-0920 74% 69% 68% 58% 65% 65% 77% 56% 67% 66%

MATH-0930 81% 75% 74% 77% 77% 75% 78% 76% 78% 77%

MATH-0990 NA NA NA 64% NA NA NA 100% 73% 79%

MATH-1000 89% 79% 85% 84% 79% 81% 75% 75% 71% 80%

MATH-1050 73% 84% 93% 100% 87% 90% 89% 100% 96% 89%

MATH-1100 69% 80% 68% 77% 77% 92% 92% 100% 100% 82%

MATH-1105 90% 84% 88% 100% 64% 100% 100% 100% 94% 91%

MATH-1400 76% 79% 78% 74% 69% 74% 82% 89% 85% 79%

MATH-1405 75% 75% 100% 78% 95% 93% 100% 95% 81% 88%

MATH-1450 65% 87% 71% 79% 96% 95% 91% 70% 83% 81%

MATH-1470 NA NA NA NA NA NA NA NA NA NA

MATH-1490 NA NA NA NA NA NA 100% NA NA 100%

MATH-1510 61% 92% 82% 64% 80% 88% 70% 78% 74% 76%

MATH-2200 76% 82% 74% 94% 79% 93% 87% 87% 72% 83%

MATH-2205 85% 69% 100% 74% 84% 73% 85% 67% 85% 80%

MATH-2210 100% 80% 100% 86% 95% 100% 88% 100% 100% 94%

MATH-2250 100% NA 100% 100% 80% NA 100% 80% NA 89%

MATH-2300 NA NA 100% NA NA 100% NA NA NA 100%

MATH-2310 80% 100% 100% 79% 83% 85% 100% 100% 77% 89%

MATH-2345 NA 100% 100% 100% 100% 100% 100% 100% 100% 100%

MATH-2350 100% 94% 73% 100% 85% 97% 100% 92% 89% 93%

MATH-2355 89% 100% 100% 100% 100% 100% 100% 100% 100% 98%

MATH-2475 NA 100% NA NA NA NA NA NA NA 100%

All Math Classes 77% 74% 75% 75% 75% 76% 80% 78% 80% 77%

Developmental Math(900,920,930)

77% 69% 71% 70% 72% 70% 75% 69% 76% 72%

Terminal College (1000,1400)

78% 79% 79% 76% 70% 75% 81% 88% 84% 79%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

2002-03 2003-04 2004-05 2005-06 2006-07 2007-08 2008-09 2009-10 2010-11

year

ly P

ass

Rat

es

Developmental Math Refined Pass (A's,B's,C's) Rates

Non Pass (D's, F's)

MATH-0900

MATH-0920

MATH-0930

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Ye

arly

Pas

s R

ate

s Refined PASS RATES BY YEAR

DevelopmentalMath(900,920,930)

Terminal College(1000,1400)

0%

10%

20%

30%

40%

50%

60%

70%

80%

Ye

arly

Pas

s R

ate

s

Pass Rates Dev Math and Terminal Math

DevelopmentalMath(900,920,930)

Terminal College(1000,1400)

Appendix B

Finals Data 2009-2012

ALEKS 2009-10

TRAD 2009-10

ALEKS 2010-11

TRAD 2010-11

ALEKS 2011-12

TRAD 2011-12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Math 0900 Finals Scores

ALEKS 2009-10

TRAD 2009-10

ALEKS 2010-11

TRAD 2010-11

ALEKS 2011-12

TRAD 2011-12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Math0920 Finals Data

TRAD 2009-10

ALEKS 2010-11

TRAD 2010-11

ALEKS 2011-12

TRAD 2011-12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Math 0930 Finals Data

Appendix C

Longitudinal Comparison of Course Grades

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

High 3.76 3.46 3.67 3.82 3.25 3.66 3.40 3.62 3.59 3.39

Low 0.47 0.20 0.46 0.58 0.02 0.23 0.08 0.48 0.51 0.22

Mean 2.11 1.83 2.07 2.20 1.63 1.94 1.74 2.05 2.05 1.81

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Math 0900 Mean +- 1StDev

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

High 3.61 3.39 3.46 3.23 3.23 3.41 3.60 2.87 3.12 3.12

Low 0.46 0.30 0.26 0.05 0.08 0.14 0.51 -0.23 0.03 -0.02

Mean 2.03 1.85 1.86 1.64 1.65 1.77 2.05 1.32 1.57 1.55

-0.50

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Math 0920 Mean +- 1 StDev

2002-03

2003-04

2004-05

2005-06

2006-07

2007-08

2008-09

2009-10

2010-11

2011-12

High 3.58 3.60 3.48 3.46 3.55 3.43 3.54 3.63 3.41 3.31

Low 0.66 0.60 0.50 0.38 0.44 0.39 0.47 0.50 0.38 0.36

Mean 2.12 2.10 1.99 1.92 2.00 1.91 2.00 2.06 1.90 1.83

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Math 0930 Mean +-1StDev

Appendix D

Intersession Math 0990 Review class

2005-06 2009-10 2010-11

High 2.93 4.85 3.30

Low 0.01 2.26 0.13

Mean 1.47 3.56 1.71

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50

5.00

Math 0990 Mean +-1StDev

Appendix E

Summer Math Class Grades

0.0

0.5

1.0

1.5

2.0

2.5

3.0

3.5

4.0

2002 2003 2004 2005 2006 2007 2008 2009 2010 2011

Summer Math Classes Average Grade

MATH-0900 MATH-0920 MATH-0930

Appendix F

Aleks Pass Rates to Traditional

Two Prop Z 2011-2012

Nonsignificant

Alpha .05

Pvalue .28

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2010-11 2011-12

Math 0900 Refined Pass Rates

ALEKS

CAMPUS

Two Prop Z 2011-2012

Nonsignificant

Alpha .05

Pvalue .28

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2010-11 2011-12

Math 0900 Refined Pass Rates

ALEKS

CAMPUS

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

2009-10 2010-11 2011-12

Math 0930 Pass Rates

ALEKS

CAMPUS

Chi Square 2010-11 versus 2011-12 (2009-10 only 19 ALEKS)

Significant 2011-12

Alpha .05

Pvalue .02

Appendix G

Aleks versus Traditional Class Performance on Common Final

ANOVA Significant pvalue <.001

Tukey Kramer

Trad 2009-10 Differs from ALEKS 2011-12 pvalue <.0001

Trad 2009-10 Differs from Trad 2011-12 pvalue .0365

ALEKS 2010-11 Differs from ALEKS 2011-12 pvalue .0439

Trad 2010-11 Differs from ALEKS 2011-12 pvalue .0001

ALEKS 2009-10

TRAD 2009-10

ALEKS 2010-11

TRAD 2010-11

ALEKS 2011-12

TRAD 2011-12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Math 0900 Final Exam Grades

TRAD 2009-10

ALEKS 2011-12

0 0.2 0.4 0.6 0.8 1

Math 0900

TRAD 2010-11

ALEKS 2011-12

0 0.2 0.4 0.6 0.8 1

Math 0900

ANOVA Nonsignificant

ANOVA Nonsignificant

ALEKS 2011 n=10

ALEKS 2009-10

TRAD 2009-10

ALEKS 2010-11

TRAD 2010-11

ALEKS 2011-12

TRAD 2011-12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Math 0920 Final Exam Grades

TRAD 2009-10

ALEKS 2010-11

TRAD 2010-11

ALEKS 2011-12

TRAD 2011-12

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

Math 0930 Final Exam Grades