Michele J. Hansen, Ph.D., Director of Assessment, University College

Assessing the Effectiveness of a Learning Community Course Designed

to Improvethe Math Performance of First-Year

Students

Michele J. Hansen, Ph.D., Director of Assessment, University College

Susan G. Meshulam, Lecturer Department of Mathematical Sciences and Adjunct Faculty University College

Brooke N. Watson, Graduate Assistant for Assessment, I/O Psychology Master’s Student

E.C. Moore SymposiumMarch 4, 2010

Indianapolis, Indiana

Research supported by IUPUI University College Faculty Fellowship

IUPUI – University College

Indiana University Purdue University Indianapolis (IUPUI)

Downtown Indianapolis Public comprehensive four year

institution Over 200 academic programs Doctoral/Research Intensive Enrollment - 30,383

• Undergraduate - 22,119• Graduate - 8,264

3

First-Year Students

Large number of under-prepared first-year students struggling to balance work and school commitments.

7% admitted conditionally. 41% of first-year students are first-generation college

students. 86% of first-time, full-time students plan to work while

attending college. 76% of first-time, full-time students commute to

campus.

University Colleges provide a structure to

address the need for a comprehensive

approach to entering students.

5

University College

Academic unit formed in 1998 Houses numerous first-year programs Serves Over 8000 students

6

IUPUI Curriculum

Principles based – no core curriculum

Principles of Undergraduate Learning (PULs) Core Communication and Quantitative Skills Critical Thinking Integration and Application of Knowledge Intellectual Depth, Breadth, and Adaptiveness Understanding Society and Culture Values and Ethics

IUPUI Math-Linked Seminars

First year students

Offered during the Fall Semesters

4 Math-Linked Learning Communities

3 are linked to Introductory Algebra 1 is linked to Intermediate Algebra Meets 1 day each week for 1 hour and 15

minutes 25 students in each learning community Team teaching: Instructor

Student Mentor

Advisor

Librarian

Collaborative Learning

Students get to know peers in math class

Students develop study groups for homework and tests

Study Skills

Students learn how to read a math book

Students are taught how to prepare BEFORE their math class meets

Students discuss math anxiety

Time Management

Students need to do math two hours a day

Students map out a daily schedule using a date planner

Learning Styles

Students take an abbreviated Barsch Learning Styles test in class

Students discuss their strengths and weaknesses in learning new material

Campus Involvement

Students are required to attend campus activities

Students are required to attend the Mathematics Assistance Center (MAC)

Math Professor Panel

Students have the opportunity to ask math instructors questions

Students feel less intimidated around their math instructor

Why Assess Math Learning Community Courses?

Demonstrate worth and value Improve teaching and learning Learn about impacts and goal achievement Contribute to course development and

improvement Obtain student feedback

Need to Improve Math Learning Experiences for First-Year Students

“There are serious problems in K-12 mathematics education, but college faculty also need to look to their own house and think about the first-year experience of their own students.”

“There is a significant gap between student experience of mathematics in high school and the expectations they face on entering college, and there are troubling signs that this gap may be widening.”

David Bressoud, President of the Mathematical Association of America, (2009)

Assessment Methods

Quantitative Questionnaires

administered to students enrolled in Math-Focused Learning Community compared with students enrolled in Math 001 and Math 110 Not Linked.

Academic Performance in Math Classes.

Qualitative Interviews with Students

Richard Light Style of Interviews - Undergraduate students (not in course) hired and trained to conduct the interviews.

Content Analysis of Open-Ended Survey Items

Fall 2007 Math Learning Community Course Students: MATH 001 Introductory Algebra N=58

12.72 - Average Course Load 19.39 - Average Age (range 18 – 36) 426.60 - SAT Verbal 425.11 - SAT Math 2.84 - High School GPA 74% - Conditional Admit 67% - First-Generation 69% - Female

Ethnicity 74% - Caucasian 24% - African American 2% - Asian American

Fall 2007 Math Learning Community CourseStudents: MATH 110 Intermediate Algebra N=21

13.19 - Average Course Load 18.82 - Average Age (range 18 – 20) 437.33 - SAT Verbal 436.67 - SAT Math 3.08 - High School GPA 38% - Conditional Admit 43% - First-Generation 90% - Female

Ethnicity 76% - Caucasian 24% - African American

Questionnaire Method

Administered to students in Math-Focused Learning Community and sample of students enrolled in Math 001 and 110, but NOT enrolled in Math-Focused Learning Community

Administered in class Voluntary participation Response rate = 52% Note – 90% of students in comparison group

also enrolled in a first-year seminar, but not math focused.

Questionnaire Respondents N=139

71% were Females 56% Conditional Admits 19.63 Average Age

Ethnicity 76% Caucasian 18% African American 2% Asian American 2% Latino/a 2% Other SAT Math Score only significant difference between

Respondents (889.22) and Non-Respondents (935.60)

Using Academic Support: The Math Assistance Center

65% of MATH-SEMINAR LINKED Course used Mentoring/Tutoring In MAC 3 or more times.

12% Math courses NOT LINKED used MAC 3 or more times.

Using Academic Support: The Math Assistance Center

93% Math-Focused Learning Community Course would seek out Mentoring/Tutoring In MAC in Future Math Courses

68% Math Courses not linked would seek out the MAC.

Forming Study Groups

54% Math-Focused Learning Community Course formed a study group.

37% Math Courses not linked formed a study group.

Math -Focused LC Areas of Strength ITEM N MATH-

FOCUSED LC MEAN

N NOT MATH-FOCUSED MEAN

I am familiar with the Math Assistance Center

61 4.20 73 3.49

I am familiar with the Principles of Undergraduate Learning (PULs)

61 3.54 75 3.07

I attended activities on campus this semester

61 3.95 77 3.18

I am able to "budget" my time through Time Management activities.

60 3.70 76 3.63

I can maintain a balance between school and work demands

60 4.08 76 3.87

I feel comfortable working in groups. 61 3.97 77 3.88

I feel comfortable speaking in front of the class.

60 3.38 78 3.12

There are a lot of ways around school-related problems I may face.

61 3.80 78 3.69

Only bolded items significantly different based on independent samples t-test (p< .05)

Math-Focused LC Areas In Need of Improvement

ITEM N MATH- FOCUSED LC MEAN

N NOT MATH-FOCUSED MEAN

Overall satisfaction with course 61 3.13 77 4.10

I understand the amount of time I should devote to studying.

60 4.21 77 4.41

I have a good understanding about my future required math courses.

60 3.88 77 4.09

I feel comfortable reading a math book 61 3.70 75 3.84

I feel comfortable studying for the mid-term math exam.

60 3.69 78 3.85

I feel confident I can manage and cope with test anxiety

60 3.68 78 3.81

I feel confident I can do well in future math courses.

61 3.97 77 3.88

Only bolded items significantly different based on independent samples t-test (p< .05)

Helpful Aspects of Math Seminar

MathXL lessonWebmail/Oncourse

Math Professor PanelMidterm Review

Diversity presentationsLearning Styles

Time ManagementRequired MAC Visits

Math Anxiety SessionsTest Anxiety Sessions

Principles of Undergraduate Learning (PULs)

Percent Agree or Strongly Agree

Most Useful Topic of Math-Learning Community Course

Topic Students’ Comments

Time Management “Time management helped a lot, I tend to put things off”

“How to manage my time” “Time management for all my other classes and

for my other activities”

Learning Styles “The important ways to study for the test” “I learned my learning style [which] helped me

with studying” “The different ways that I can study to help me

retain information better”

Resources “helped me get to know all the resources on campus”

“what different areas of the school serve for”

Most Useful Topic of Non Math-Learning Community Course

Topic Students’ Comments

Problem-solving “how to solve difficult problems” “learning about potential mistakes in problem

solving” “the single most useful topic is solving equations

because it is easy and something I will use in the future”

Studying and Strategies

“do problems until you are comfortable with them”

“how to study” “that I have to study” “find a way to understand it, even if its corny”

Resources “the most useful topic I learned is that the MAC is here to help”

“how to use MathXL” “to get help from others, mentoring, tutoring, etc.”

Study Techniques Used

Study Technique Students’ Comments

MathXL “I took all the MathXL review sections” (seminar student)

“I would go through MathXL problems and solve questions that are given…” (non-seminar)

Practice Problems “Writing problems then solving them” (seminar) “I just tried to do as many practice problems as I

could to familiarize myself with the technique or method” (non-seminar)

Book Problems “I did suggested problems as well as chapter tests in the book” (seminar)

“Doing problems out of the book and checking my answers. It was more productive for me” (non-seminar)

2007 Math-Focused Learning Community Students’ Academic Performance and Success

Math 001

* First-year students enrolled in the same section of Math 001, but not enrolled in linked math-focused learning community course.

Math-Focused LC

N SAT Math

H.S. GPA

Fall GPA Math 001 Avg. Grade

Next Math Course Avg. Grade (Spring)

Fall-to-Fall Retention Rate

Yes 58 425.11 2.84 2.35 2.06 1.61 59%

*No 59 438.10 2.89 2.01 1.88 1.63 59%

2007 Math-Focused Learning Community Students’ Academic Performance Math 001

Math-Focused LC

Mean Math 001 Course Grade

Adjusted Math 001 Grade

Yes 2.22 2.23

*No 1.89 1.81

* First-year students enrolled in the same section of Math 001, but not enrolled in math-focused learning community course.

Based on an analysis of covariance (ANCOVA). Math-Focused LC Grade in Math Course marginally significantly higher than non-participants’ Math Course Grade (p =.058). Covariates in the model included Math SAT score and HS GPA.


Math 110


Math-Focused LC

N SAT Math

H.S. GPA




Yes 21 436.67 3.08 2.06 1.64 1.78 62%

*No 12 482.00 3.31 2.41 2.07 1.69 75%


Math 001


Math-Focused LC

N SAT Math

H.S. GPA




Yes 61 428.04 2.87 2.26 2.00 1.43 67%

*No 45 421.94 3.01 2.48 2.00 1.78 62%


Math 110

*First-year students enrolled in the same section of Math 110, but not enrolled in linked math-focused learning community course. **More students attempted a second math course if in the Math-Focused LC (15) compared to those students not in the Math-Focused LC (10).

Math-Focused LC

N SAT Math

H.S. GPA




Yes 20 472.11 2.95 2.59 2.36 1.83** 60%

*No 17 511.43 3.22 2.57 2.33 2.37 59%

Student-to-Student Interviews

Two undergraduate research assistants were trained in the fall of 2007 on how to conduct effective interviews for the purpose of qualitative research.

At the beginning of the spring semester, 2008, the trained undergraduates interviewed 8 of their peers that were enrolled in the math-focused seminar, and simultaneously enrolled in either Math 001 or Math 110.

The purpose of the interviews was gain an in-depth students’ perceptions of effectiveness and affective reactions to the math-focused LC courses.

Student Interviews Overall Findings

Students found any activity dealing with actual practice of math to be the most useful aspect of this class.

Many students reported that if they could modify the course it would be by devoting more of class time to completing math problems and discussing difficulties associated with students’ math courses.

The time management session seemed to be quite effective in influencing students to be more organized with school in general and more diligent about practicing math routinely.

Math anxiety focus of this course seemed to be particularly helpful to students reporting this as a personal issue they are facing.

Students’ Voices

“I think it is a good resource class. I think it will give a lot of help to people with math anxiety or taking any type of math class. I think it is a good course to be taken.”

“It was linked to our math course and we were able to review for tests”

“I gained study habits”

Students’ Voices

“we do math work in class and it’s like you have two classes right in a row where you learn to study the same thing”

“studying the book [one of the things they taught us in class]…take an hour or two out of class and just do practice problems”

“to do practice problems” “basically gave us different skills to study

for a test before it came up”

Students’ Voices

“I learned to take my time, to work through a problem slowly, and that if I didn’t get one at that time to finish the test and come back to it”

“I learned that most problems that people have when it comes to taking a test is that they don’t know how to just relax and take their time and concentrate”

“helped to relax…and think about what I was doing so it helped me learn more than just hurrying up to get done and just to not worry about it”

Students’ Voices

“time should be set aside for doing math problems…so you don’t have to cram for a test and still end up getting a bad grade”

“not to cram myself full of math the night before a test”

“stuff should be on a schedule and should be done at certain times instead of everything just done whenever”

“to plan, not procrastinate”

Students’ Voices

“I will probably have a lot more math in it…doing math problems similar to the type we are doing in class that day”

“do more math practice” “focus more on math practice” “a lot more time in class working with

practice problems and going over tests and stuff”

Closing the Loop – Next Steps Extended class time (35 minutes more) will:

Result in a 2-credit hour class meeting one day a week

Allow for math problem practice time

Allow study groups to form in class (and then hopefully carry on outside of class)

Other changes

Require more MAC visits

Improve the math anxiety lesson

Assign homework – math related

Michele J. Hansen, Ph.D., Director of Assessment, University College

Documents

Transcript of Michele J. Hansen, Ph.D., Director of Assessment, University College