Cooperative LearningCooperative Learning

Statistical Significance and Effect Statistical Significance and Effect SizeSize

By: Jake Eichten and Shorena By: Jake Eichten and Shorena DolaberidzeDolaberidze

This Chapter This Chapter Examines:Examines:

The problem of declining math The problem of declining math achievement scores in an achievement scores in an

elementary school where Beth elementary school where Beth Miller is a first year principalMiller is a first year principal

The effectiveness of cooperative The effectiveness of cooperative learning strategies in mathematicslearning strategies in mathematics

This chapter is not meant to make you an expert This chapter is not meant to make you an expert in cooperative learning, but rather observe in cooperative learning, but rather observe principal Beth Miller as she undertakes an principal Beth Miller as she undertakes an examination of the problem and develops good examination of the problem and develops good questions to facilitate a review which will help questions to facilitate a review which will help her seek out possible solutions.her seek out possible solutions.

This chapter also demonstrates the importance This chapter also demonstrates the importance of statistical significance and effect size when of statistical significance and effect size when using research as a tool for making important using research as a tool for making important decisions. decisions.

Who, Where, What, and Why?Who, Where, What, and Why?

Beth Miller-First year principal at Pleasant Valley Beth Miller-First year principal at Pleasant Valley Elementary SchoolElementary School

She and staff recognized math deficiencies in grades 3-5She and staff recognized math deficiencies in grades 3-5 Staff members suggested cooperative learning as a Staff members suggested cooperative learning as a

possible solutionpossible solution Miller put together a research team to develop a plan of Miller put together a research team to develop a plan of

actionaction

Two Questions to Guide the Two Questions to Guide the Teams ActionTeams Action

1) What kind of cooperative learning 1) What kind of cooperative learning strategies are available?strategies are available?

2) What cooperative learning strategies 2) What cooperative learning strategies have been found to enhance achievement have been found to enhance achievement in mathematics generally and in problem in mathematics generally and in problem solving skills specifically with 4th and 5th solving skills specifically with 4th and 5th grade students with widely varying levels grade students with widely varying levels of ability? of ability?

Action for Question 1Action for Question 1

The group researched cooperative The group researched cooperative learning methods that are specifically learning methods that are specifically appropriate for mathematics instruction. appropriate for mathematics instruction. They discovered 6 different strategies that They discovered 6 different strategies that were possibilities. were possibilities.

Action for Question 2Action for Question 2

The team researched 4 studies of The team researched 4 studies of cooperative learning techniques that met cooperative learning techniques that met the following criteria:the following criteria:-Focus on mathematics education-Focus on mathematics education-Sample included elementary or middle -Sample included elementary or middle school childrenschool children-Experimental and control conditions -Experimental and control conditions imposed on the subjectsimposed on the subjects

The 4 StudiesThe 4 Studies

The four studies that were chosen for The four studies that were chosen for review met the previously mentioned review met the previously mentioned criteria and were all found to be criteria and were all found to be statistically significant and had an statistically significant and had an acceptable effect size. acceptable effect size.

Statistical SignificanceStatistical Significance

Statistical significance shows the strength Statistical significance shows the strength of the relationship between 2 variablesof the relationship between 2 variables

Statistical significance gives the Statistical significance gives the researchers confidence that the difference researchers confidence that the difference between 2 variables didn’t occur simply between 2 variables didn’t occur simply because of chancebecause of chance

Statistical SignificanceStatistical Significance

Statistical significance is represented as a p Statistical significance is represented as a p value (p=probability)value (p=probability)-An example would be: p < .05, meaning that the -An example would be: p < .05, meaning that the probability of an event occurring by chance is probability of an event occurring by chance is less that 5%less that 5%-The statistical significance of a coin flip would -The statistical significance of a coin flip would be represented as p = .5, meaning that 50% of be represented as p = .5, meaning that 50% of the time the result could be due to chancethe time the result could be due to chance

Statistical Significance Statistical Significance ExampleExample

A computer interactive reading program A computer interactive reading program was compared with a traditional reading was compared with a traditional reading program using 50 5th graders who were program using 50 5th graders who were randomly assigned to treatment and randomly assigned to treatment and control groupscontrol groups

The researchers reported a difference of 4 The researchers reported a difference of 4 points on a reading achievement test in points on a reading achievement test in favor of the computer interactive program. favor of the computer interactive program.

Example ContinuedExample Continued

How do we know that if we do this study How do we know that if we do this study again, the results won’t favor the again, the results won’t favor the traditional reading group?traditional reading group? Statisticians developed techniques to test for Statisticians developed techniques to test for

the possibility that a difference in observed the possibility that a difference in observed scores might have been due to chancescores might have been due to chance

Example ContinuedExample Continued

The researchers found that the statistical The researchers found that the statistical significance in this example would be significance in this example would be

p < .01, meaning the results occurring by p < .01, meaning the results occurring by chance are less than 1%.chance are less than 1%.

Why is Statistical Significance Why is Statistical Significance Important?Important?

When a statistic is significant, it simply When a statistic is significant, it simply means that the statistic is reliable.means that the statistic is reliable.

When using research to assist in making When using research to assist in making important educational decisions, it is important educational decisions, it is important that the data used is reliable.important that the data used is reliable.

Effect SizeEffect Size

Effect size is important for answering the Effect size is important for answering the question “How much of a difference did it question “How much of a difference did it make?” (Cooperative learning strategies)make?” (Cooperative learning strategies)

Effect size allows us to compare the magnitude Effect size allows us to compare the magnitude of experimental treatments from one experiment of experimental treatments from one experiment to anotherto another

Effect size is calculated by the mean difference Effect size is calculated by the mean difference between the treatment and control groups and between the treatment and control groups and dividing it by the average standard deviation of dividing it by the average standard deviation of the two groups (ES=M1-M2/Av.SD)the two groups (ES=M1-M2/Av.SD)

Effect Size ExampleEffect Size Example

Consider a math study with a treatment Consider a math study with a treatment group mean score of 10 and a control group mean score of 10 and a control group mean score of 5, and the average group mean score of 5, and the average standard deviation of the two groups is 10standard deviation of the two groups is 10

ES=M1(10)-M2(5)/Av.SD(10)ES=M1(10)-M2(5)/Av.SD(10)

-The effect size would be .5-The effect size would be .5 The treatment group averaged 1/2 of a SD The treatment group averaged 1/2 of a SD

higher than the control group higher than the control group

Example ContinuedExample Continued

Consider another math study that used a Consider another math study that used a different measure of math achievement with the different measure of math achievement with the mean difference being 10 and the average mean difference being 10 and the average standard deviation being 50standard deviation being 50

The effect size would be .2The effect size would be .2 The effect sizes would be a common The effect sizes would be a common

denominator of the two, which will allow for a denominator of the two, which will allow for a comparison between the two studiescomparison between the two studies

Study 1’s ES (.5) would indicate that the Study 1’s ES (.5) would indicate that the treatment effect was was greater than than that treatment effect was was greater than than that of study 2’s (.2) of study 2’s (.2)

Desired Effect SizeDesired Effect Size

Why is Effect Size Important?Why is Effect Size Important?

It answers the question “How much of a It answers the question “How much of a difference did the treatment make?”difference did the treatment make?”

When researching materials to assist is When researching materials to assist is making important educational decisions, it making important educational decisions, it is important to understand how well a is important to understand how well a treatment worked before deciding to treatment worked before deciding to implement it into your own setting. implement it into your own setting.

Chapter ConclusionsChapter Conclusions

As a result of the research Miller and her As a result of the research Miller and her team conducted, they found:team conducted, they found: Two types of cooperative learning that were Two types of cooperative learning that were

proven successful for teaching math proven successful for teaching math computation and math problem solvingcomputation and math problem solving

Developed an action memo, which they Developed an action memo, which they shared with the full staff explaining their shared with the full staff explaining their findings and their next steps.findings and their next steps.

ReferencesReferences

Haller, E. J., & Kleine, P. F. (2001). UsingHaller, E. J., & Kleine, P. F. (2001). Using

educational research: A schooleducational research: A school

Administrator's guide (1st ed.). New York,Administrator's guide (1st ed.). New York,

NY: Addison Wesley Longman, Inc.NY: Addison Wesley Longman, Inc.