How was the course restructured?
1
The exit from STEM fields is far worse for minority students: After 4 years, only 15% of Latinos, Black and Native American students who had an initial interest in STEM majors graduate with these degrees. (National Research Council, “Expanding Underrepresented Minority Participation” 2011) Restructuring a large, introductory course to help underrepresented minority and first-generation students perform better. Kelly Hogan, Andrea Reubens, and Bob Henshaw (UNC at Chapel Hill) and Erika Bagley (Auburn University.) Key components to course: Face- to- Face Lectu re Masteri ng Biology homewor k Interac tive class activit ies Guided readin g questi ons Poll Everywher e class response Early outreach to students scoring low on first exam Traditio nal 3/wk No No No No No Reformed 3/wk Yes Yes Yes Yes Yes How was the course restructured - One-semester, “Principles of Biology” - 400+ students - Mixed majors; Few (~20%) are Biology majors - Most students are first year students. Biol 101 at UNC: We hypothesized that restructuring the introductory course would help to “level the playing field” for a diverse group of students from various types of high schools across the state. A study from the University of Washington demonstrated that the achievement gap could be reduced inside the classroom setting without additional resources and costs through active learning and more class structure in the form of pre- class reading quizzes, informal group work, weekly exams, and clickers. (Haak, D.C. et al., Science, 2011). We asked if the model could be replicated at UNC and have as similar impact with our at-risk students (under-represented minorities and first generation students). Fall 2009 Fall 2011 0 10 20 30 40 50 60 70 80 90 4-Examp Average TABLE 1: Fall 2009 (traditional) Performance on Exams RESULTS: FIGURE 2: Grades become more evenly distributed in a restructured course. FIGURE 1. A “leveling” of the four-exam average for different races/ethnicities in a restructured (Fall 2011) semester compared to a traditional semester (Fall 2009). 1 Exam 1-3, final exam, and the 4-exam average between 1) African American Students and Caucasian Students, 2) African American Students and Asian Students, 3) Latino/Latina and Caucasian Students, and 4) Latino/Latina and Asian Students are statistically significant at p<.05. In traditional semesters examined, an achievement gap in student performance was seen between underrepresented minorities (African Americans and Latinos) compared to Caucasian/Asian students. A representative semester (Table 1) shows this gap exists throughout the semester for all four exams. In a representative restructured semester (Table 2) the performance gap disappears for Latinos, and is reduced for African Americans. These results have been replicated in three restructured semesters. TABLE 2: Fall 2011 (restructured) Performance on Exams 2 Exam 1-3, final exam, and the 4-exam average between 1) African American Students and Caucasian Students and 2) African American Students and Asian Students, are statistically significant at p<.05. The shift to a more evenly spread of grades is easily noticed when the range of grades is examined (FIGURE 2). Of note, 72% of African Americans had a four-exam average of 50% or below in the traditional format and in the restructured semester, only 32% of African Americans averaged 50% or lower. This is consistent with a reduction in the number of D/Fs in the class as a whole. Figure 3 shows that first generation students (n = 50) performed equally well in the restructured course (Fall 2011) compared to their non-first generation cohorts (n= 328). No significant First g ene rati on 0 40 80 Four-exam average (Mean) Although the exam averages for all student groups were higher in restructured semesters, the goal is not to compare absolute numbers from semester to semester, since the exam questions are new each semester. Figure 1 depicts the “leveling” of the student performance within the restructured semester– as compared to the bigger gap seen within the traditional format. Students scoring below 50% on the first exam were deemed “at-risk” and were encouraged to come to office hours to complete a self-assessment and conversation with the instructor about study skills and resources. The results have varied per semester. In Fall 2010, students who completed the self-assessment (n= 16) after exam 1 had a significantly better performance on all subsequent exams compared to at-risk peers who were invited but did not complete it (data not shown). Yet in Fall 2011, (Figure 4) there was only a significant impact on exam 2 for the students (n = 20) completing the self assessment, when compared to their at-risk peers. These differences did not persist for the third and final exams. Thus, the impact has been replicated for exam 2, but beyond that, the data are unclear. Completion of other interventions along with the self- assessment (i.e. study skills workshop and tutoring) did not significantly add to its impact when examined together (data not shown). Students come to their introductory courses with various levels of preparedness. By increasing the structure to the course as discussed above, students have more chances to practice material inside and outside the classroom at their own pace. Student surveys indicate the number of hours spent studying has increased several hours for all students with the restructured course (data not shown), which might have been expected since study time is positively correlated to academic performance (Brint, S and Cantwell, E.M., Teachers College Record, 2010). Similar to the study by Haak et al., course changes have disproportionately helped certain populations of students traditionally at-risk (Science, 2011). In our study, we added a metacognitive self- assessment intervention, focusing efforts on students who failed the first exam. The self-assessment had an immediate significant impact for students completing it, and thus it is worthwhile to continue to determine if this immediate benefit persists or wanes by the end of the semester. Finding more opportunities to practice content one-on-one (supplemental instruction, tutoring, and office hours) remain important for at-risk students, but their regular attendance remains the challenge. CONCLUSIONS: Fall 2009 Exam Scores (Mean Scores) Race/Ethnicity 1 Exam 1 Exam 2 Exam 3 Final Exam 4-Exam Average African American (non-Hispanic)* 54.44 59.2 52.48 51.19 54.27 Asian* 73.29 79.14 71.43 72.76 73.84 Caucasian (non- Hispanic)* 65.72 72.54 67.27 68.44 68.38 Latino/Latina or Hispanic* 59.83 67.62 62.77 61.87 62.91 Other 61.14 67.33 61.49 66.29 63.77 Fall 2011 Exam Scores (Mean Scores) Race/Ethnicity 2 Exam 1 Exam 2 Exam 3 Final Exam 4-Exam Average African American (non-Hispanic)* 57.83 64.45 80.65 60.96 65.79 Asian 71.51 75.25 85.08 69.59 74.97 Caucasian (non- Hispanic) 73.48 75.87 87.53 71.96 77.11 Latino/Latina or Hispanic 67.49 71.37 83.53 64.52 71.78 Other 72.85 75.71 84.41 68.96 75.36 90 - 100 80 - 89 70 - 79 60 - 69 50 and below 0.0% 20.0% 40.0% 60.0% 80.0% Fall 2009: 4-Exam Average by Race/Ethnicity African American Asian/Pacific Islander Caucasian Latino/Latina 4-Exam Average Ranges Percentage of Students 90 - 100 80 - 89 70 - 79 60 - 69 50 and below 0.0% 10.0% 20.0% 30.0% 40.0% 50.0% 60.0% 70.0% 80.0% Fall 2011: 4-Exam Average by Race/Ethnicity African American Asian/Pacific Islander Caucasian Latino/Latina 4-Exam Average Ranges Percentage of Students Exam 1 Exam 2* Exam 3 Final Exam Average - Exams Only 0 10 20 30 40 50 60 70 80 44.83 64.85 75.73 53.08 59.62 43.23 54.65 73.53 53.4 56.07 Completion vs. Non-Completion of Self Assessment Completed Self Assessment Did Not Complete Self Assessment Exams Mean Note: Mean score from the first to the second exam for students that completed the self assessment increased by 20.02 points vs. students that did not complete the self assessment (increase of 11.41 points). *Statistically significant at p<.05.. n = 66 students invited to complete self-assessment; n= 20 completed it. Of all students originally interested in STEM fields, less than 40% graduate with a degree in STEM fields in four years (National Center for Education Statistics in 2009) The President’s Council of Advisors on Science and Technology (PCAST) suggests our biggest problem may be in our introductory courses. - Meet 3 hr/per week; lab course is separate - Fairly diverse population: at least 25% of the course are underrepresented minorities. - No paid TAs for lecture FIGURE 3: No performance gap for first-generation students. FIGURE 4: The self-assessment significantly impacts at-risk students early in the semester.
description
Restructuring a large, introductory course to help underrepresented minority and first-generation students perform better . Kelly Hogan, Andrea Reubens , and Bob Henshaw ( UNC at Chapel Hill) and Erika Bagley (Auburn University.). - PowerPoint PPT Presentation
Transcript of How was the course restructured?
The exit from STEM fields is far worse for minority students: After 4 years, only 15% of Latinos, Black and Native American students who had an initial interest in STEM majors graduate with these degrees. (National Research Council, “Expanding Underrepresented Minority Participation” 2011)
Restructuring a large, introductory course to help underrepresented minority and first-generation students
perform better. Kelly Hogan, Andrea Reubens, and Bob Henshaw (UNC at Chapel Hill)
and Erika Bagley (Auburn University.) Key components to course:
Face-to-Face
LectureMastering
Biology homework
Interactive class
activitiesGuided reading
questions
Poll Everywhere
class response
Early outreach to students
scoring low on first exam
Traditional 3/wk No No No No No
Reformed 3/wk Yes Yes Yes Yes Yes
How was the course restructured?- One-semester, “Principles of Biology”- 400+ students- Mixed majors; Few (~20%) are Biology majors- Most students are first year students.
Biol 101 at UNC:
We hypothesized that restructuring the introductory course would help to “level the playing field” for a diverse group of students from various types of high schools across the state. A study from the University of Washington demonstrated that the achievement gap could be reduced inside the classroom setting without additional resources and costs through active learning and more class structure in the form of pre-class reading quizzes, informal group work, weekly exams, and clickers. (Haak, D.C. et al., Science, 2011). We asked if the model could be replicated at UNC and have as similar impact with our at-risk students (under-represented minorities and first generation students).
Fall 2009 Fall 20110
10
20
30
40
50
60
70
80
90
4-Ex
amp
Aver
age
TABLE 1: Fall 2009 (traditional) Performance on Exams RESULTS:
FIGURE 2: Grades become more evenly distributed in a restructured course.
FIGURE 1. A “leveling” of the four-exam average for different races/ethnicities in a restructured (Fall 2011) semester compared to a traditional semester (Fall 2009).
1 Exam 1-3, final exam, and the 4-exam average between 1) African American Students and Caucasian Students, 2) African American Students and Asian Students, 3) Latino/Latina and Caucasian Students, and 4) Latino/Latina and Asian Students are statistically significant at p<.05.
In traditional semesters examined, an achievement gap in student performance was seen between underrepresented minorities (African Americans and Latinos) compared to Caucasian/Asian students. A representative semester (Table 1) shows this gap exists throughout the semester for all four exams. In a representative restructured semester (Table 2) the performance gap disappears for Latinos, and is reduced for African Americans. These results have been replicated in three restructured semesters.
TABLE 2: Fall 2011 (restructured) Performance on Exams
2 Exam 1-3, final exam, and the 4-exam average between 1) African American Students and Caucasian Students and 2) African American Students and Asian Students, are statistically significant at p<.05.
The shift to a more evenly spread of grades is easily noticed when the range of grades is examined (FIGURE 2). Of note, 72% of African Americans had a four-exam average of 50% or below in the traditional format and in the restructured semester, only 32% of African Americans averaged 50% or lower. This is consistent with a reduction in the number of D/Fs in the class as a whole.
Figure 3 shows that first generation students (n = 50) performed equally well in the restructured course (Fall 2011) compared to their non-first generation cohorts (n= 328). No significant difference was found in the four-exam averages.
First
generati
on
Non-first ge
neration
020406080
100
Four
-exa
m a
vera
ge
(Mea
n)
Although the exam averages for all student groups were higher in restructured semesters, the goal is not to compare absolute numbers from semester to semester, since the exam questions are new each semester. Figure 1 depicts the “leveling” of the student performance within the restructured semester– as compared to the bigger gap seen within the traditional format.
Students scoring below 50% on the first exam were deemed “at-risk” and were encouraged to come to office hours to complete a self-assessment and conversation with the instructor about study skills and resources. The results have varied per semester. In Fall 2010, students who completed the self-assessment (n= 16) after exam 1 had a significantly better performance on all subsequent exams compared to at-risk peers who were invited but did not complete it (data not shown). Yet in Fall 2011, (Figure 4) there was only a significant impact on exam 2 for the students (n = 20) completing the self assessment, when compared to their at-risk peers. These differences did not persist for the third and final exams. Thus, the impact has been replicated for exam 2, but beyond that, the data are unclear. Completion of other interventions along with the self-assessment (i.e. study skills workshop and tutoring) did not significantly add to its impact when examined together (data not shown).
Students come to their introductory courses with various levels of preparedness. By increasing the structure to the course as discussed above, students have more chances to practice material inside and outside the classroom at their own pace. Student surveys indicate the number of hours spent studying has increased several hours for all students with the restructured course (data not shown), which might have been expected since study time is positively correlated to academic performance (Brint, S and Cantwell, E.M., Teachers College Record, 2010). Similar to the study by Haak et al., course changes have disproportionately helped certain populations of students traditionally at-risk (Science, 2011). In our study, we added a metacognitive self- assessment intervention, focusing efforts on students who failed the first exam. The self-assessment had an immediate significant impact for students completing it, and thus it is worthwhile to continue to determine if this immediate benefit persists or wanes by the end of the semester. Finding more opportunities to practice content one-on-one (supplemental instruction, tutoring, and office hours) remain important for at-risk students, but their regular attendance remains the challenge.
At UNC and many institutions, class sizes are increasing and budget cuts are affecting access to TAs and academic services. Increasing course structure and active learning did not require any additional university resources, although it did require an instructor to make significant change. What will it take for more faculty to change?
CONCLUSIONS:
Fall 2009 Exam Scores (Mean Scores)
Race/Ethnicity1 Exam 1 Exam 2 Exam 3Final Exam
4-Exam Average
African American
(non-Hispanic)* 54.44 59.2 52.48 51.19 54.27
Asian* 73.29 79.14 71.43 72.76 73.84
Caucasian (non-Hispanic)* 65.72 72.54 67.27 68.44 68.38
Latino/Latina or Hispanic* 59.83 67.62 62.77 61.87 62.91Other 61.14 67.33 61.49 66.29 63.77
Fall 2011 Exam Scores (Mean Scores)
Race/Ethnicity2 Exam 1 Exam 2 Exam 3Final Exam
4-Exam Average
African American
(non-Hispanic)* 57.83 64.45 80.65 60.96 65.79Asian 71.51 75.25 85.08 69.59 74.97
Caucasian (non-Hispanic) 73.48 75.87 87.53 71.96 77.11Latino/Latina or Hispanic 67.49 71.37 83.53 64.52 71.78
Other 72.85 75.71 84.41 68.96 75.36
90 - 100 80 - 89 70 - 79 60 - 69 50 and below
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
Fall 2009: 4-Exam Average by Race/Ethnicity
African AmericanAsian/Pacific IslanderCaucasianLatino/Latina
4-Exam Average Ranges
Perc
enta
ge o
f Stu
dent
s
90 - 100 80 - 89 70 - 79 60 - 69 50 and below
0.0%
10.0%
20.0%
30.0%
40.0%
50.0%
60.0%
70.0%
80.0%
Fall 2011: 4-Exam Average by Race/Ethnicity
African AmericanAsian/Pacific IslanderCaucasianLatino/Latina
4-Exam Average Ranges
Perc
enta
ge o
f Stu
dent
s
Exam 1 Exam 2* Exam 3 Final Exam Average - Exams Only
0
10
20
30
40
50
60
70
80
44.83
64.85
75.73
53.08
59.62
43.23
54.65
73.53
53.456.07
Completion vs. Non-Completion of Self Assessment
Completed Self As-sessmentDid Not Complete Self Assessment
Exams
Mea
n
Note: Mean score from the first to the second exam for students that completed the self assessment increased by 20.02 points vs. students that did not complete the self assessment (increase of 11.41 points).*Statistically significant at p<.05.. n = 66 students invited to complete self-assessment; n= 20 completed it.
Of all students originally interested in STEM fields, less than 40% graduate with a degree in STEM fields in four years (National Center for Education Statistics in 2009)
The President’s Council of Advisors on Science and Technology (PCAST) suggests our biggest problem may be in our introductory courses.
- Meet 3 hr/per week; lab course is separate- Fairly diverse population: at least 25% of the course are underrepresented minorities.- No paid TAs for lecture
FIGURE 3: No performance gap for first-generation students.
FIGURE 4: The self-assessment significantly impacts at-risk students early in the semester.
