Examining the Effects of Daily Required Homework on Student Achievement

Mathew Fukuzawa, Dan Koban, Ryan Slocum, Jamar Wright

This paper was completed and submitted in partial fulfillment of the Master Teacher Program, a 2-year faculty professional development program conducted by the Center for Faculty Excellence, United States Military Academy, West Point, NY, 2016.

Abstract The practice of assigning homework is a polarizing topic that has been extensively studied at various levels of education. Despite the body of literature on the topic, there is still debate about its efficacy. To offer stronger objective analysis, we conducted statistical analysis on the effects of incentivizing homework in an introductory mathematics course at the United States Military Academy (USMA). We analyzed the academic performance of 12 classes on homework, course-wide assessments, and various other instructor events. We evenly divided the sections into two groups and used course-wide assessments as our response variable. The treatment group was required to complete daily graded homework assignments, while the control group only completed in-class assessments and quizzes. The results showed no significant difference on the average for course-wide assessments, but they did reveal a significant difference in variance. The treatment group had a smaller variance on course-wide events suggesting that mandatory homework fostered an environment where lower performing students benefitted from a baseline understanding. Furthermore, qualitative feedback from instructors suggests students in the treatment group were generally more engaged in classroom activities. Therefore, we conclude that assigning daily graded homework has beneficial effects, but these effect may not be apparent when looking at average performance.

Introduction One of the academic program goals at USMA is lifelong learning. Cadets need to “engage successfully in deliberate self-directed and collaborative learning experiences” [1]. Cadets also need to demonstrate problem-solving skills, particularly in the STEM fields. Homework is one such method to help achieve these goals. Several of the core math courses at USMA already require daily graded homework—MA104, MA205, and MA206 (Calculus I,II and Probability and Statistics). In general, feedback from students is generally positive regarding homework requirements. However, department records indicate that for at least 15 years, MA103 has not adopted this policy course wide. Although instructors have the freedom to assign points toward homework, it has never been a policy to force homework on the Fourth Class Cadets in MA103. The core math program at USMA has six primary goals [2]:

1. Acquire a body of knowledge. 2. Communicate effectively. 3. Apply technology. 4. Build competent and confident problem solvers. 5. Develop habits of mind.

6. Interdisciplinary perspective. The assigning of homework in MA103, therefore, aims to assess goals 1, 3, 4, and 5 primarily. Homework in its most basic form is a form of repetition or practice to aid in achieving learning mastery. However, the homework assigned within this project was done so with the Thayer Method in mind. The Thayer method intends to place the responsibility for learning on the student. This style of teaching and learning is often described as a form of active learning. Thayer described his method in an 1828 letter to James Monroe:

“A class of 80 students is to be taught Mathematics or Natural Philosophy devoting three hours of each day to the study of the subject at their rooms and three other hours with the Professor. One hour is taken up in the Lecture but this alone is not sufficient. Each student should demonstrate a proposition or explain an investigation at the Black-board and also be interrogated to see that he thoroughly understands the principles. This will require, as experience proves, not less than 15 minutes on an average for each student…Let the class be divided into at least four parts or sections and let each section attend 3 hours daily with an assistant professor to be examined upon the subject of the Lecture or lesson given on the preceding day. The Professor besides lecturing may either have the recitations of one Section himself or what would be the better practice, he might without taking the immediate charge, be present at the recitations, visiting each section in turn and only occasionally putting questions and giving explanations. You know that this is the system of instruction which has been practiced at West Point during the last ten years with what success I leave it for others to say” [3].

Many of the instructors at USMA use various forms of the Thayer Method that strike a balance between active and passive learning. The core math classes in the Math Department generally follow a model where Cadets are given textbook reading assignments and homework problems to complete before arriving at the next scheduled class period. The theory is that this practice allows for a more productive in-class session where instead of a 55-minute instructor led lecture, the Cadets can immediately begin class with their questions and difficulties from the assignment. The remainder of class is devoted to student practice at the chalkboards with the concepts and problems from the lesson. In this manner, instructors can provide immediate feedback to Cadets and also identify weaknesses or gaps in knowledge. Granted, this system is best for a smaller size class, and USMA affords that opportunity where math classes are capped at 18 students. It is not the aim of this study to assess the effectiveness of the Thayer Method or the style of active learning. It is every instructor’s hope, though, that Cadets are driven to independent learning, and many of the skills gained in this area derive from out-of-class preparation with homework. Naturally, the desire to prepare for class comes down to motivation. Even more so at USMA as opposed to civilian universities, Cadets are exposed to various events (not all academic) which demand much of their time. Hence, Cadets must be masters of time management in order to succeed. With all these demands, time for homework must be spent wisely. Hence, motivating Cadets to prepare for class, to do homework or reading assignments, and to achieve success is a complex issue which every instructor struggles with. While Cadets are often individually motivated by time management, they are also individually motivated by effort and ability. As Bempechat [4] points out, under the social cognitive approach to motivational factors in learning, students perceive their own success (or failure) as a result of effort, ability, and external factors. At USMA, while we acknowledge differences in ability, success is highly driven by effort and motivation. Unfortunately, we often motivate by assigning points to an activity. However, as Lowman [5] points out:

“For any course, using grades to motivate compliance with routine homework or reading assignments has the unintended side effect of orienting students more toward the external grades they receive than toward internal intellectual satisfactions. Students’ intrinsic curiosity and pleasure in learning can be diminished considerably by excessive emphasis on grades as an exclusive motivational technique. Instructors who assume that students are interested only in the grade they receive and design their leadership strategies accordingly help produce, ironically, the very grade-oriented behavior they disdain.”

USMA is unique in this aspect in that all Cadets in a year group are ranked according to an order-of-merit list (OML). This list takes into account performance across multiple domains to include academic, physical, and military. The heaviest weight belongs to academics, hence many Cadets are driven to succeed by their OML ranking as this will eventually help to determine branch choice and initial duty station upon commissioning as an Army Officer. If OML ranking is a Cadet’s sole extrinsic motivator, however, we are more apt to see the type of student that Lowman refers to. Boberg [6] examined this concept of motivation to prepare for class in a foreign language setting across two semesters at USMA, with samples of 61 and 64, respectively. Overall, he was interested in the specific rewards and punishments that motivate Cadets to prepare for an introductory Spanish class. In his surveys, Cadets responded that the most important reason for them to prepare was due to a graded event. While this result is only noted for Boberg’s study, the authors can reasonably state that most Cadets feel this way across USMA—Cadets prioritize class preparation by whether or not their work will be rewarded with points. Additionally, one of Boberg’s secondary research questions asked if graded homework affected student performance in language proficiency. Boberg’s definition of homework included two assessments: an out-of-class online homework tool as well as an in-class post-homework comprehension check. The web-based homework was 10% of the student’s final grade, whereas the weight of the comprehension checks varied from control to treatment groups. Given the addition of comprehension checks into the definition of homework, Boberg found improvements in exam performance. However, it is unknown whether the difference was statistically significant, and it is unknown whether the improvement was due to formatting and content changes with other assessments such as in-class quizzes. Other relevant USMA studies include Nowicki, Gifford, and Hunter [7,8,9]. Nowicki used graded homework as a measuring stick against exam performance, but he only showed a positive correlation between the two for one exam and one homework assignment. This study was also performed on a very small sample size, and the homework was assigned in two week blocks. Gifford also employed graded homework against course performance, but his homework assessment was not based on accuracy but rather completion. His treatment groups were given varying degrees of credit for the completion of homework, whereas the control groups were not given any credit for homework. Overall, Gifford was unable to find a statistically significant difference between the mean performances in groups, which he attributed primarily to the Thayer Method. Because his students were receiving one-on-one individual feedback through boardwork in the classroom, Gifford concluded that checking daily homework assignments was not a good use of time. Hunter’s study focused specifically on online homework as a tool to increase motivation in completion. Hunter compared traditional paper-and-pencil homework methods to online homework methods, once again in terms of completion. The two groups had homework assigned at the beginning of an instructional block, and it was due at the end of each block (typically several weeks in duration). Overall, Hunter showed that the online homework group completed more of the homework (33%) compared to the traditional homework group (21.6%). It is unknown as to the weight of a student’s final grade that homework counted to. Remarkably though, student perception of online homework was generally negative, as 60% of Cadets would not recommend online homework for future use.

Deveans and Jackson [10] used WebAssign online homework to help freshman students prepare for the algebra based Fundamental Concepts Exam (FCE) and showed a statistically significant difference in exam performance from one year to the next for the better. 1 Bjerkaas and Wolberg [11] examined the variation of assessment methods between homework and quizzes. Their group that administered only graded homework on a weekly basis did not show a statistically significant correlation with exam grades, and student perception of the right balance of homework and quizzes was mixed. Ahlers [12] replicated the layout of this same study but looked at the frequency of assessment. His homework only group was required to turn in paper-and-pencil assignments “often every lesson”, but the data did not produce a statistically significant correlation with exam grades. Outside of USMA, research on homework has been extensive. Perhaps the most thorough research on homework is owed to Harris Cooper. Cooper’s research is largely longitudinal in nature, covering multiple decades and varying grade levels. In his early work, Cooper found that homework was positively correlated with achievement, more so at the high school level than the junior high and elementary grade levels [13]. In 1998, Cooper et al. [14] discussed attitudes about homework, as well as the amount of homework assigned compared to student achievement. In general, they found that the “more positive student attitudes were associated with more completed homework and higher class grades” [12]. Thus, if the students had a positive attitude about homework (undoubtedly gained from a positive teacher attitude about homework), they generally performed better in class at the higher grade levels. Paschal, Weinstein, and Walberg [15] also conducted a synthesis of homework literature, looking to examine effects of homework on achievement in elementary and secondary students. After accounting for 54 separate variables, they devised a system to quantify the impacts of homework on achievement. Of note, they found large effects for graded homework that teachers wrote comments on. Additionally, assigned homework had larger effects than the absence of homework. Finally, the assignment of daily homework had larger effects than any other frequency period. Further, in 2006, Cooper, Robinson, and Patall conducted a synthesis of homework research from 1987-2003 and concluded that “with only rare exceptions, the relationship between the amount of homework students do and their achievement outcomes was found to be positive and statistically significant. Therefore, we think it would not be imprudent, based on the evidence in hand, to conclude that doing homework causes improved academic achievement” [16]. There are several caveats to Cooper’s research, however. For one, Cooper found that time spent on homework was also a factor. Particularly in the upper grade levels, performance dropped after time spent on homework went much past an hour. For two, their study did not look past secondary education levels. At the university level, various homework studies have been conducted but primarily in the realm of web-based versus traditional based assignments. While this is not our aim to study the difference in delivery methods, the results are worth mentioning. Fynewever compared the use of web-based homework and paper homework. While he did not find a statistically significant difference between the mean posttest scores of the two groups, Fynewever did show a significant correlation between homework grades and student learning outcomes [17]. Bonham, Beichner, and Deardorff conducted a similar study in a university physics class, and although web-based homework sections performed consistently better on exams, the results were not statistically significant enough to conclude a difference in mean scores [18]. Pascarella [19] conducted a somewhat different study of web-based versus traditional homework in a university physics class. In her study, students were divided into web-based and traditional homework groups and their problem-solving strategies were analyzed. At mid semester, these groups switched and assumed the opposite homework type. In each group, students were characterized as Guessers or 1 The FCE is a yearly exam given to all Plebe (freshman) Cadets to assess their skills and competencies in basic algebra. Passing is a requirement to move on to the Calculus sequence.

Traditional Thinkers. Interestingly, Pascarella concluded that the web-based homework tended to promote guessing and low-level problem solving strategies, while traditional homework promoted the use of higher-level problem solving. Perhaps even more alarming is that when groups switched, some Traditional Thinkers switched to being Guessers and vice versa, leading her to believe that web-based homework actually did the students a disservice on the metacognitive level. Richards-Babb et al. [20] reported positive results with the substitution of weekly quizzes with online graded homework. While correlation between homework scores and various course assessments was only mildly positive, both instructors and students gave high praise for the implementation of online homework. On the flip side, there are others who argue against the use of homework. Kohn has been one of the biggest opponents of homework, arguing that teachers should not assign homework unless the benefits are justifiable [21]. He also claims that current research does not prove causation—it does not follow that if a student does more homework, then he/she will have better test grades. Perhaps the more intriguing point that Kohn makes, however, is that “homework studies confuse grades and test scores with learning.” Frankly, it is hard to argue this point, but it is often hard to design objective assessments of learning other than test scores. Especially at the undergraduate level, time is precious and perhaps the best means we have to assess learning is through test scores. Ideas like Kohn’s have also migrated into pop culture as evidenced by the Time magazine article by Wallis in 2006 that argues parents should take a stand against demanding homework policies [22]. Wallis even cites other countries, particularly those whose students perform better on standardized tests versus U.S. students, because these countries assign less homework than classes here in America. Marzano and Pickering [23] counter this by citing a 1994 report by the National Education Commission on Time and Learning. The report states that students in other countries that do better than American students on standardized testing actually spend double the time with the material than students here at home. Marzano and Pickering also take the middle ground in the debate between homework and no homework. They conclude that although the research on homework efficacy is varied, educators should use research, judgment, and experience to formulate their own best practices. Still yet, Trautwein and Kőller explain that existing 20th century homework research only provides weak support for the relationship between homework and achievement [24]. Instead, they propose that future homework research should include an account for three variables: time on task, self-regulation theory and expectancy-value theory, and teacher quality. In summary, the practice of assigning homework is still a topic under contentious debate. We do not aim to add fuel to this fire, but it is our hope that we can learn how to improve instructional techniques in the Department of Mathematical Sciences at USMA as part of normal pedagogical practices to better achieve our academic program goals. We also aim to provide more research of this type at the undergraduate level since very little published research has focused at this level with regard to the assignment of homework. It is our aim to shed some light on this topic to help Cadets and Faculty alike.

Research Question The overall purpose of this study was to evaluate the efficacy of different instructional techniques. The specific technique or strategy being tested was that of daily required homework. Our study was designed to determine whether performance improved on course-wide major graded events when required to complete daily graded homework. We also attempted to examine the following: 1) effects of graded homework on students of different aptitudes; 2) attitudes and beliefs of both the homework requirement and the vehicle by which it is delivered (online). Granted, while web-based homework is our delivery tool, it is not our focus to assess the efficacy of web homework versus paper-and-pencil.

Methods The population for this study was the Plebe (freshman) class of 2019 enrolled in MA103 during the fall academic semester, term 16-01. At USMA, all Cadets are required to enroll in a core math sequence of courses, with the majority of Cadets being placed initially into MA103. For the 16-01 semester, a total of 927 Cadets were enrolled in MA103. Fifty-three sections of the fall iteration of MA103 were randomly assigned to the course instructors. The four authors were assigned three sections each of 16-18 students (total of 197 students involved in the study). The 12 sections in this study were divided into two equal groups of six. The six sections comprising the control group consisted of 100 students who did not have a daily graded homework requirement; instead, 400 points of their grade (20% of total) consisted of other instructor assessments (e.g. quizzes, reflections, portfolios, etc.). There were no restrictions on the type of assessment or the frequency; the only stipulation for the control group was that it did not include a daily graded homework. These six sections were divided amongst the four authors. The six sections comprising the treatment group consisted of 97 students that had graded homework due each day for a score that reflected correctness and accuracy. For these students in the treatment group, 400 points of their grade was based entirely on the required homework. Based on the participants’ standardized test scores, both the control group and the treatment group entered West Point with equal mathematical abilities. The two-sample t-test in Table 1 shows the average math percentile for the control group was an 86.29, compared to that of the treatment group—87.22. Since this is not a statistically significant difference, it is reasonable to assume both groups would perform similarly throughout the course if they underwent the same treatment.

Table 1. Two-sample t-test comparing the average SAT/ACT percentile scores for the treatment and control groups.

Control Max Test Score

(%)

Treatment Max Test Score

(%) Mean 86.290 87.216 Variance 100.652 67.109 Observations 100 97 Hypothesized Mean Difference 0 df 190 t Stat -0.7109303 P(T<=t) one-tail 0.2389996 t Critical one-tail 1.6529129 P(T<=t) two-tail 0.4779992 t Critical two-tail 1.9725282

After each exam (termed a Written Partial Review or WPR), grades from control and treatment groups were compared to determine if correlation was present. At the conclusion of the semester, final exam performance was also compared between treatment and control groups to deduce any effects from the inclusion of graded homework. By daily, we mean that for every new lesson taught, a homework was assigned for that class period. The total number of homework assignments over the semester was 44.

Homework was due at the start of class and included problems from the preceding lesson as well as the following lesson. Additionally, surveys were administered to both treatment and control groups at mid-semester and end of semester to assess student attitudes about the assigned homework. A faculty survey was also administered to gauge instructor opinion on homework (see Appendix A).

Results This study found neither a practical nor statistically significant difference between the treatment group and the control group on course wide graded events. Course wide events included four WPRs and the Term End Exam (TEE). On average, the treatment group scored one percentage point higher than the control group on course wide graded events, which equates to 13 points out of 1,300. The two sample t-test below in Table 2 shows no statistical difference with respect to a student’s average grade on course-wide events; however, there is a noticeably lower variance.

Table 2. Two sample t-test comparing the average grades on course-wide graded events for the control group and the test group.

Control

Avg. on Course Wide Events Treatment

Avg. on Course Wide Events Mean 0.811761538 0.82093973 Variance 0.008623959 0.006558614 Observations 100 97 Hypothesized Mean Difference 0 df 193 t Stat -0.739950224 P(T<=t) one-tail 0.230114665 t Critical one-tail 1.652787068 P(T<=t) two-tail 0.460229329 t Critical two-tail 1.972331676

To determine whether the treatment resulted in a significantly lower variance, an F-test was conducted. The test resulted in a p-value of 0.0985, providing strong evidence that the treatment group had less variance on course-wide graded events than the control group. Since the t-test asserts the average on course-wide graded events for the two groups was equivalent, the F-test implies the treatment group was generally more clustered about the mean. The boxplot in Figure 1 further illustrates what the F-test confirmed by showing noticeably compressed 1st, 2nd and 4th quartiles.

Figure 1. Boxplot comparing student averages on course-wide

graded events based on their group affiliation.

The treatment group had three students’ average less than 65% on course-wide graded events and five students’ average between 65-70%. This was considerably fewer than the control group which had five and eight, respectively. Although the treatment of daily graded homework did not result in mastery of the subject, these results suggest the treatment group had a better collective understanding of the material, and reduced the number of students with D’s or F’s by 40%. While a student’s test scores may ebb and flow with effort, stress, or competing academic requirements over the semester, this study also examined whether requiring daily graded homework would promote long-term understanding of the material. Our metric for this assessment was the TEE. Any student can study hard the night before a test, given the scope of the test is limited and do reasonably well. However, it is much more difficult to cram for an exam that covers 18 weeks-worth of material. The results show no indication that this particular treatment had any effect on long-term understanding or the ability to recall information. The t-test below in Table 3 shows a marginal improvement by the treatment group, but it is not significant.

Table 3. Two sample t-test comparing the average grade on the MA103 Term End Examination (TEE).

Control Tee %

Treatment Tee %

Mean 0.79394 0.799835052 Variance 0.017082219 0.010751723 Observations 100 97 Hypothesized Mean Difference 0 df 188 t Stat -0.351254104 P(T<=t) one-tail 0.362895527 t Critical one-tail 1.652999113 P(T<=t) two-tail 0.725791055 t Critical two-tail 1.972662692

The next research question this study explored was whether daily graded homework benefitted students of different aptitudes. It seems reasonable to assume students with lower aptitudes would benefit from daily

homework because they are forced to stay engaged in the material, whereas members of the control group could go several nights without looking at math. Using the highest ACT or SAT quantitative percentile, this study also examined the effectiveness of the treatment on students in the lower quartile. Students with ACT or SAT scores that ranked below the 80th percentile were subset, creating a sample size of 26 students from the control group and 20 students from the treatment group. Looking strictly at their performance on course-wide graded events, the students from the control group outperformed the treatment students by 1.6%; however, due to the small sample size, this was not statistically significant.

Table 4. Two sample t-test comparing students’ averages on course wide graded events, given the student scored less than the 80th percentile on the quantitative portion of the ACT or SAT.

Q1 Control

Avg. on Course-Wide Events Q1 Treatment

Avg. on Course-Wide Events Mean 0.754645 0.738423 Variance 0.00577 0.006503 Observations 26 20 Hypothesized Mean Difference 0 df 40 t Stat 0.693571 P(T<=t) one-tail 0.24598 t Critical one-tail 1.683851 P(T<=t) two-tail 0.49196 t Critical two-tail 2.021075

This hypothesis test cannot determine the causation, but it is worth speculating why the control group outperformed the treatment group. It is possible students in the treatment group were more focused on the green checkmark (WebAssign’s indication they have answered the question correctly) than learning the material. Or it is possible the control group was accustomed to taking quizzes in a time constrained environment whereas the treatment group did not take quizzes and had unlimited attempts on homework, access to assistance, and multiple days to earn credit for their assignment. In terms of test taking preparation, the former may be the better option. It appears the treatment did not influence the lower quartile’s performance on course-wide graded events, but did it impact their overall grade? Figure 2 shows all 46 students in the lower quartile, with their average on course-wide graded events on the x-axis and their instructor average on the y-axis. Ideally, one might expect a student’s instructor average, which accounts for their performance on either WebAssign or in-class quizzes, to mirror a student’s average on course-wide events. The solid black line shows perfect correlation between a student’s instructor average and average on course-wide events. The two dashed lines represent a difference of ±10% between the two averages.

Figure 2. Scatter plot showing how a student’s average on course-wide

events compared to their instructor average.

Students below the dashed line in the lower-right portion of the graph were students that did better on the course-wide events than on instructor assignments. For example, the circled data point averaged an 87% on course-wide events, but only 70% on instructor quizzes. Students above the dashed line in the upper-left portion of the graph were students that did better on instructor assignments than on course-wide events. Notice the starred data point had a 92% instructor average, yet averaged 60% on course-wide events. Figure 2 shows that the majority of the control group fell within the ±10% window, with multiple students having near perfect correlation. There seems to be an equal number of students whose final grades benefitted from instructor quizzes as those who did not. The treatment group shows a different pattern. There were only three students whose final grade was not inflated by daily graded homework assignments, while the majority of the treatment group’s instructor average was 5% higher than its average on course-wide events. Overall, the treatment had no measurable effect on the lower quartile’s performance other than inflating its instructor average, which was 80.8% to the control group’s 73.9%. The difference in instructor averages balanced out the difference on course-wide graded events, as both the treatment and control groups (in the lower quartile) had final grade averages of 75.8% in the course. The same observation can be made for students in the upper quartile. Table 5 shows a two-sample t test comparing the average grade on course-wide events for both the control group and treatment group, in which the control group averaged 1.4% higher than the treatment group. The hypothesis test resulted in a p-value of 0.547, suggesting there is not sufficient evidence to suggest the treatment group’s average grade is better or worse than the control group’s average.

Table 5. Two sample t-test comparing the average grade on course-wide events for students in the upper quartile. These student scored at or above the 94th percentile in the quantitative portion of the ACT or SAT.

Q4 Control

Avg. on Course-Wide Events Q4 Treatment

Avg. on Course-Wide Events Mean 0.883569 0.871795 Variance 0.00585 0.003423 Observations 25 24 Hypothesized Mean Difference 0 df 45 t Stat 0.606689 P(T<=t) one-tail 0.273553 t Critical one-tail 1.679427 P(T<=t) two-tail 0.547106 t Critical two-tail 2.014103

Figure 4 compares the average grade on course-wide events to instructor averages for students in the upper quartile. The same trend occurs here as in Figure 3, where the majority of the treatment group’s instructor average was higher than its average on course-wide events, resulting in grade inflation. In the lower quartile, there were a handful of students who earned between 90%-100% instructor averages, but the upper quartile students in the treatment group did noticeably better. Interestingly, 19 of 24 students in the treatment group had an instructor average greater than 90%, whereas only 10 of 25 students in the control group earned above 90%.

Figure 3. Scatter plot showing how a student’s average on course-wide events compared to their instructor average.

This study also looked at how the two groups performed on individual course-wide event, in hopes of identifying trends. Table 6 shows the control group performed better on the first exam, but the treatment group did better on the remaining three exams and on the term end exam. In most cases, the difference between the two groups was less than 2%. The most significant difference between the two groups was their instructor average. The treatment group’s average instructor grade was 10% higher than the control

group, which likely explains why the treatment group’s final course average was 2% higher than the control group.

Table 6. Group averages for the five course-wide graded events, instructor average, and average final grade in the course.

Event Control Treatment WPR 1 86.30% 84.75% WPR 2 83.35% 86.35% WPR 3 80.25% 81.25% WPR 4 79.30% 81.25% TEE 79.40% 79.98% Instructor Average 78.66% 88.02% Final Course Average 81.27% 83.74%

Aside from analyzing whether daily graded homework would impact student performance, this study aimed to capture the treatment’s effect on student attitudes towards mathematics. If the treatment improved student’s learning at the expense of their desire to study mathematics, is it worth it? Every stakeholder would have to answer that for themselves, but fortunately, this study found students appreciated having daily homework assignments and felt it furthered their understanding of the material. Throughout the study, daily homework assignments were available to every student in MA103, though those in the control group did not receive credit for attempting the assignments. Halfway through the semester, all students completed a survey regarding the effectiveness of WebAssign and how it impacted their learning. Note, it was not the intent of the survey to specifically question the use of WebAssign as a homework delivery tool. Unfortunately, we predict that many students answered the survey trying to analyze web-based means versus non-web. The survey used a standard Likert-scale, where “1” indicated they strongly agreed with the statement and “5” indicated they strongly disagreed with the statement. Table 6 shows the treatment group largely agreed that WebAssign increased their ability to understand the course and contributed to their learning. Most students in the treatment group felt like the homework assignments could be completed in the USMA time guidelines, which recommend two hours of homework for every hour in class. Overall, students were more neutral when asked whether WebAssign increased their motivation to learn, was a fair and effective technique for learning, and WebAssign provided sufficient feedback for their learning.

Table 7. Mid-course survey on student attitudes and beliefs about WebAssign.

Control Treatment WebAssign increases ability to understand course material 2.50 1.98 WebAssign contributes to my learning 2.46 1.87 WebAssign can be completed within USMA time guidelines 2.76 1.97 Motivation to learn has increased because of this course 2.59 2.43 WebAssign is a fair and effective technique 3.13 2.24 Feedback from WebAssign is sufficient 3.62 2.82

An interesting takeaway from Table 7 is that the control group, which was not required to complete daily graded homework, had a noticeably more negative attitude towards WebAssign. Anecdotal evidence suggested there was a learning curve to using WebAssign—some questions had very specific formatting requirements for the answers, while other questions were sensitive to rounding. Students in the treatment group were forced to work through the learning curve associated with WebAssign, and once they understood how to use the program, there was far less frustration. On the other hand, the control group had the option to stop using WebAssign at any point, so rather than working through the learning curve

they were more likely to get frustrated and quit using it. Additionally, students in the treatment group were more likely to ask questions about homework problems and lesson objectives than the control group. The same survey was administered to both groups at the end of the semester, with the results listed below in Table 8. The survey shows that the views for both the control and treatment groups largely remained the same throughout the semester (plus or minus 0.10), while giving credence to the WebAssign learning curve theory. On the mid-course survey, students agreed WebAssign could be completed within the USMA time guidelines with a score of 1.97, but on the end of course survey, students agreed with a score of 1.85. This was the largest change for any of the questions on the survey, aside from the control group’s response to “Feedback from WebAssign is sufficient” which was skewed by a 6% increase in the number of respondents answering with a value of 6, indicating “not observed.”

Table 8. End of course survey results on student attitudes and beliefs about WebAssign.

Control Treatment WebAssign increases ability to understand course material 2.50 2.03 WebAssign contributes to my learning 2.51 2.00 WebAssign can be completed within USMA time guidelines 2.62 1.85 Motivation to learn has increased because of this course 2.48 2.45 WebAssign is a fair and effective technique 3.19 2.29 Feedback from WebAssign is sufficient 3.43 2.88

Discussion One of the most noticeable outcomes of this study was the higher instructor grades for the treatment group (Table 6). There are several reasons that may explain this observation. First, homework assignments were programed to give students 100 attempts on every question and students were allowed to receive assistance on assignments, so students in the treatment group had plenty of opportunities to receive full credit. Meanwhile, the control group, whose instructor grade was largely determined by its performance on quizzes, only had one attempt to get the correct answer. Another explanation for the treatment groups’ higher average was the opportunity to receive partial credit on late assignments. Unlike a quiz, where a wrong answer is a wrong answer, students were allowed to complete homework assignments late and receive a 10% penalty per 24 hours of lateness. These differences were substantial enough to explain why most of the treatment group’s final grade was inflated by its instructor average. Another interesting outcome of this study was how little the treatment affected student performance on course-wide events. It seems logical to assume students that engaged with the course material at least twice (once the night prior completing their homework and again in class) would perform better than those in the control group only seeing the material in class, but these results showed very little change. One explanation is that students in the control group were better prepared for exams because quizzes prepared them to answer questions in a time constrained environment. WebAssign allowed students 100 attempts per a question, gave hints after an incorrect answer, and allowed students up to 10 days to complete the assignment and receive credit—none of these characteristics bear resemblance to an exam. The timing of control group quizzes may have also influenced the results. A common technique is to give students a quiz a few days before a major graded event, because it forces students to start studying early and allows the instructor to assess their students and provide feedback before the exam. This would certainly serve as a wake-up call for struggling students, but if it bears close resemblance to the exam, it

may give the control an advantage over the treatment group. Because this study did not standardize assignments for the control group, it is unclear how similar quizzes were with the exams. The one statistically significant difference this study revealed was that the treatment reduced the amount of variance on course-wide events. This was good for the 25th percentile because it meant they performed better than those in the control group, but the fact that the treatment group’s 75th percentile was lower was unexpected. It seems reasonable to assume students in the treatment group had seen and attempted every type of problem on an exam, while the control group may have never been tested on a topic. With that in mind, the treatment group was more likely to receive partial credit by recognizing something of limited value about the problem. The control group was more prone to the “you either know it or you don’t” mentality because they had less practice with the material. Upper quartile students in the control group performed better on course-wide events, which may have stemmed from their exposure to similar problems on quizzes. WebAssign was useful for testing numerical, multiple choice, or fill in the blank questions; however, those types of questions only make up a small portion of the exams in MA103. Instead, the exams were geared towards open-ended questions and explaining answers, which the treatment group had little practice with.

Conclusion Will students who are required to do homework every night perform better than students that are not required to do homework? It seems like such an easy question to answer. Surely the daily repetition of problem solving and acquired study habits will result in an increased performance, right? Using two groups of equal mathematical abilities, this study showed that requiring daily graded homework had no visible effect on student exam performance—at any level, for any mathematical aptitude. It seemed reasonable to assume students with weak math backgrounds would benefit by being required to engage the material every day, but this study showed no improvement. The only significant effect the treatment had on student performance was reducing variance. We generally desire less dispersion in scores, and this effect paid off for below average students; however, it also meant above average students did not do as well on course-wide events as those in the treatment group. From a teaching perspective, the control group and treatment group were considerably different. Students in the treatment group generally came to class armed with a basic understanding of the topic. They had questions about the upcoming lesson, or at least a question about the previous night’s homework assignment. On the other hand, the control group generally brought less energy to the classroom, had fewer questions, and it was much more difficult to solicit student participation. Though there was no significant difference in their performance on course-wide events, it was evident most students in the treatment group grew more confident in their abilities and became self-reliant learners as the semester went on. This, in itself, is reason to continue using daily graded homework.

Future Research This study offers several opportunities for continued research. First, it would be valuable to repeat the experiment in future semesters to increase the sample size. Rather than having 200 students involved in

the study, it would be interesting to use all 900 students in the course. In particular, this would shed more light on the impact daily graded homework has on lower-aptitude students. This study had sample sizes of 26 in the control group and 20 in the treatment group, making it difficult to draw significant conclusions. Another aspect of this study that could be refined would be to standardize assignments for the control group. The students in the treatment group all completed the same web-based homework assignments, which eliminated instructor influence from the study. However, each instructor developed and graded their own quizzes and/or homework assignments in the control group, leaving room for an instructor bias. Lastly, rather than focusing on the students’ performance, it would be interesting to study how daily graded homework effected their motivation to learn. This study used a short survey, administered twice throughout the semester, to gain insight in to students’ opinion of WebAssign, but that survey could be lengthened and administered with greater frequency. Questions aimed at measuring confidence, sense of accomplishment, and self-reliance would be ideal topics to focus on, because in terms of education, these attributes are more meaningful than a letter grade.

References [1] Office of the Dean. 2014. Academic Program, Class of 2017. West Point, NY. [2] Department of Mathematical Sciences. Core Mathematics Program. http://www.usma.edu/math/SitePages/Core%20Math.aspx. Accessed 23 Feb 2016. [3] Thayer, Sylvanus. 1828. Letter to President James Monroe. West Point Thayer Papers. 3: 147-148. [4] Bempechat, Janine. 2004. The Motivational Benefits of Homework: A Social-Cognitive Perspective. Theory Into Practice, 43(3): 189-196. [5] Lowman, Joseph. 1995. Mastering the Techniques of Teaching, Second Edition. San Francisco, CA: Jossey-Bass. [6] Boberg, Marc E. 2010. Motivation to Prepare. Master Teacher Program Library. West Point, NY. Available at http://www.usma.edu/cfe/SitePages/MTP_Projects_Search.aspx. [7] Nowicki, Robert A. 2010. The Benefits of Graded Homework in Math, Science, and Engineering Education. Master Teacher Program Library. West Point, NY. Available at http://www.usma.edu/cfe/SitePages/MTP_Projects_Search.aspx. [8] Gifford, James H. 2012. Grading Homework: Is it Really Worth the Instructor’s Time. Master Teacher Program Library. West Point, NY. Available at http://www.usma.edu/cfe/SitePages/MTP_Projects_Search.aspx. [9] Hunter, Scott A. 2015. Does Online Homework Increase Motivation to Complete Homework. Master Teacher Program Library. West Point, NY. Available at http://www.usma.edu/cfe/SitePages/MTP_Projects_Search.aspx. [10] Deveans, Thomas M. and Heather A. Jackson. 2009. An Exploration Assessing WebAssign Using Student and Instructor Feedback. Master Teacher Program Library. West Point, NY. Available at http://www.usma.edu/cfe/SitePages/MTP_Projects_Search.aspx. [11] Bjerkaas, James and Sarah Wolberg. 2012. Homework vs. Quizzes: Which Evaluation Method is Better? Master Teacher Program Library. West Point, NY. Available at http://www.usma.edu/cfe/SitePages/MTP_Projects_Search.aspx. [12] Ahlers, Kris. 2013. Regular Performance Feedback in Relation to WPR Scores. Master Teacher Program Library. West Point, NY. Available at http://www.usma.edu/cfe/SitePages/MTP_Projects_Search.aspx. [13] Cooper, Harris. 1989. Synthesis of Research on Homework. Educational Leadership, 47(3): 85-91. [14] Cooper, Harris et al. 1998. Relationships Among Attitudes About Homework, Amount of Homework Assigned and Completed, and Student Achievement. Journal of Educational Psychology, 90(1): 70-83.

[15] Paschal, Rosanne, T. Weinstein, and H. Walbert. 1984. The Effects of Homework on Learning: A Quantitative Synthesis. Journal of Educational Research, 78(2): 97-104. [16] Cooper, H., J.C. Robinson, and E. Patall. 2006. Does Homework Improve Academic Achievement? A Synthesis of Research, 1987-2003. Review of Educational Research, 76(1): 1-62. [17] Fynewever, Herb. 2008. A Comparison of the Effectiveness of Web-based and Paper-based Homework for General Chemistry. Chemical Educator, 13: 264-269. [18] Bonham, Scott, R. Beichner, and D. Deardorff. 2001. Online Homework: Does it Make a Difference? The Physics Teacher, 39: 293-296. [19] Pascarella, Andrea M. 2004. The Influence of Web-Based Homework on Quantitative Problem-Solving in a University Physics Class. Proceedings of the NARST 2004 Annual Meeting, Vancouver, BC. 2004. [20] Richards-Babb, Michelle et al. 2011. Online Homework, Help or Hindrance? What Students Think and How They Perform. Journal of College Science Teaching, 40(4): 81-93. [21] Kohn, Alfie. 2006. Abusing Research: The Study of Homework and Other Examples. Phi Delta Kappan. 88(1): 9-22. [22] Wallis, Claudia. 2006. The Myth About Homework. Time. 168(10): 59. [23] Marzano, Robert J. and Debra J. Pickering. 2007. The Case For and Against Homework. Educational Leadership. 64(6): 74-79. [24] Trautwein, Ulrich and Olaf Kőller. 2003. The Relationship Between Homework and Achievement—Still Much of a Mystery. Educational Psychology Review. 15(2): 115-145.

Appendix A

While this study aimed to show the effects of instructional techniques on students, we also attempted to gather instructor feedback on the use of homework. MA103 employed 20 instructors to teach the course during the fall term, AYT 16-01, and the 16 instructors not participating in the study were asked to provide opinions about homework at the end of the course. A synopsis of the survey is presented below.

1. What is your stance on requiring daily graded homework?

# Responses (𝒏𝒏 = 𝟏𝟏𝟏𝟏)

% of Response

Strongly Agree 7 44% Agree 7 44% Neutral 0 0% Disagree 0 0% Strongly Disagree 2 12%

2. How did you assign your instructor points?

# Responses (𝒏𝒏 = 𝟏𝟏𝟏𝟏)

% of Response

All to WebAssign 5 31% Some to WebAssign, Some to Other Events

4 25%

All to Other Events 7 44%

3. How often did you discuss/engage/review the questions presented in the WebAssign lessons within your classroom setting?

# Responses (𝒏𝒏 = 𝟏𝟏𝟏𝟏)

% of Response

Daily 3 19% Weekly (1-2 times) 13 81% Bi-weekly 0 0% Monthly 0 0% Never 0 0%

4. On a scale of 1-10, rate all of your sections daily preparedness with the course material. This could include homework completion, class discussion and participation, etc. (1 = no engagement, 5 = moderate engagement, 10 = total engagement).

Section 1 Section 2 Section 3 Instructor 1 7 9 8 Instructor 2 6 8 8 Instructor 3 8 8 8 Instructor 4 7 8 7 Instructor 5 8 7.5 7.5 Instructor 6 5 6 7 Instructor 7 7 7 6 Instructor 8 9 Instructor 9 7 Instructor 10 6 6 Instructor 11 2 7 3 Instructor 12 9 8 9 Instructor 13 5 10 5 Instructor 14 5 7 3 Instructor 15 4.5 9 8 Instructor 16 10 8 AVERAGE LEVEL OF PREPAREDNESS 6.988

5. Given the choice, how would you implement homework?

# Responses (𝒏𝒏 = 𝟏𝟏𝟏𝟏)

% of Response

Online 11 68.8% Paper/Pencil 3 18.7% Other Both online and paper No graded HW; promote

suggested problems

2 12.5%

Questions 1 and 5 reveal a high percentage of instructors believe that daily homework is beneficial. However, questions 2 and 3 perhaps indicate that homework was not reinforced in class often enough to achieve higher levels of participation. Given what we have observed and have thus come to believe about motivation in Cadets, we feel that assigning points to an activity is one of the primary methods for motivating Cadets to come prepared for class. Since roughly only a third of the instructors assigned points to the daily online homework, it is perhaps apparent why engagement and participation levels were not higher and homework was reviewed only once or twice per week. Also, it appears that most instructors prefer online homework over the traditional paper and pencil. While we did not test the difference in homework methods, instructors seem to prefer automated grading and students undoubtedly prefer immediate feedback. Instructors were also asked on this survey to provide suggestions for increasing homework completion rates. There were a range of answers, but several of them reiterated the fact that Cadets generally focus on assignments that have points attached. Several of the comments are provided below.

Have to assign some points to the HW. It has to have points assigned to it. Assign more points to homework. Allow collaboration as long as there is ample documentation.

Make homework applicable to lesson objectives. I know the idea is to have the students want to do the assignments on their own, however, I think

most of them prioritize assignments based on the number of points assigned to them. They must see a decent correlation between homework completion and either grades or learning

(that matters to them) or both. Either doing graded HW or frequent quizzes or frequent tests where test questions are solutions to HW problems.

My recommendation is that the mix of hand written quizzes and WebAssign forces cadets to review the material. Cadets not only worry about points for WA, but they prepare for the quiz.

Daily quizzes; randomly choose a student to brief the class as a daily problem. Maybe do a combo of some WebAssign and some hand written homework.