The Student’s T-test and other tests for significance

2

Faced with an Observation

• Question an Observation (I feel more secure using my right hand)

• Why This Question-Background Information (Reaction time can be calculated)

• Hypothesis Ho, H1, H2

• Experimental Design (Test Reaction Time)• Results and Analysis of Data (Statistics)• Conclusions

3

Let’s get started!!State the Hypotheses

• Ho There is no difference between the reaction time of the dominant and non dominant hand (Null Hypothesis)

• H1 = Reaction time in the dominant hand is significantly faster than reaction time in the non dominant hand.

• H2 = Reaction time in the non dominant hand is significantly faster than reaction time in the dominant hand

4

Experimental Design and Analysis of Data

• Collecting reaction time data– Reaction time t = √(2d/g)– How many pieces of data?

Reaction Time: Right Hand vs Left Hand Reaction time t = √(2d/g)

Grp A Grp B38 32 p values

10 1684 57 0.339037829

36 28 21.20062892

50 5535 1273 6148 29

avg 46.75 36.25stdev 23.21 19.02

Grp A Grp B46.75 36.2546.25 36.25 3.02246E-19

46.75 36 5.429855738

47.25 36.2546.75 36.546.25 36.2546.75 3647.25 36.5

avg 46.75 36.25stdev 0.378 0.189

Experiment II

Experiment IHow are these two experiments different even though the groups have the same mean?

Create a Data Table in Excel for yourReaction times: ten repeats on each hand

RIGHT LEFT

TIME(sec)

.012 .014

.018 .016

.018 .015

.016 .018

6

Conclusions?? Significant Difference? What Statistical Method?

• Is there a difference between the right and left hand with regard to reaction time?

• Let’s learn how to do the Student’s T test

Statistical Significance

• If the probability of an event occurring due to chance is less than 5% it is most likely due to the independent variable. This means there is a 95% chance it is due to your independent variable. The “p” value gives you this probability and 5% is defined as alpha and is usually the chosen “cut off” for significance in science. Let us look at some graphs to determine which ones may represent data that is significant and due to the independent variable.

• The t-test assesses whether the means of two groups are statistically different from each other. This analysis is appropriate whenever you want to compare the means of two groups.

What are some representative units for the axis? (ie # students vs Grades (male /female), #of planaria regenerated vs concentration (ant /post), # planaria regenerated vs time (concentration : 0 /70ppb); diameter of zone vs number of disks

• What does it mean to say that the averages for two groups are statistically different?

• Notice that the three situations is that the difference between the means is the same in all three.

• But, the three situations don't look the same -- they tell very different stories.

• The top example shows a case with moderate variability of scores within each group.

• The second situation shows the high variability case.

• the third shows the case with low variability.

• The two groups appear most different or distinct in the bottom or low-variability case.

• Why? Because there is relatively little overlap between the two bell-shaped curves.

Significant???????

• The formula for the t-test is a ratio. The top part of the ratio is just the difference between the two means or averages. The bottom part is a measure of the variability or dispersion of the scores.

• The top part of the formula is easy to compute -- just find the difference between the means. The bottom part is called the standard error of the difference. – To compute it, we take the variance for each group

and divide it by the number of samples in that group. We add these two values and then take their square root

• The formula is:

Formula for the Standard error of the difference between the means

• Remember, the variance is simply the square of the standard deviation.

• The final formula for the t-test is:

Formula for the t-test.

• Cedar-apple rust is a (non-fatal) disease that affects apple trees. Its symptom is rust-colored spots on apple leaves. Red cedar trees are the immediate source of the fungus that infects the apple trees. If you could remove all red cedar trees within a few miles of the orchard, you should eliminate the problem. In the first year of this experiment the number of affected leaves on 8 trees was counted; the following winter all red cedar trees within 100 yards of the orchard were removed and the following year the same trees were examined for affected leaves. The results are recorded on the next panel:

Let’s Try

Data

• Tree number of rusted number of rusted difference: 1-2 leaves: year 1 leaves: year 2

1 38 32 6 2 10 16 -6 3 84 57 27 4 36 28 8 5 50 55 -5 6 35 12 23 7 73 61 12 8 48 29 19

average 46.8 36.2 10.5 standard dev 23 19 12

Determine whether there was a significant change in the number of rusted leaves between years 1 and 2. Did the treatment cure the problem?

Grp A Grp B38 3210 1684 57 0.339037829 ttest

36 2850 5535 1273 6148 29

avg 46.75 36.25stdev 23.21 19.02

Grp A Grp B46.75 36.2546.25 36.25 3.02246E-19 ttest

46.75 3647.25 36.2546.75 36.546.25 36.2546.75 3647.25 36.5

avg 46.75 36.25stdev 0.378 0.189

Experiment II

Experiment IHow are these two experiments different even though the groups have the same mean?

Significant?????• Sometimes, when the statistical In a scientific study, a theory is proposed,

then data is collected and analyzed. The statistical analysis of the data will produce a number that is statistically significant if it falls below 5%, which is called the confidence level. In other words, if the likelihood of an event is statistically significant, the researcher can be 95% confident that the result did not happen by chance.

• Significance of an experiment is very important, such as the safety of a drug meant for humans, the statistical significance must fall below 3%. In this case, a researcher could be 97% sure that a particular drug is safe for human use. This number can be lowered or raised to accommodate the importance and desired certainty of the result being correct.

• Statistical significance is used to reject or accept what is called the null hypothesis. A hypothesis is a statement of the theory that a researcher is trying to prove. The null hypothesis holds that the factors a researcher is looking at have no effect on differences in the data. Statistical significance is usually written, for example, t=.02, p<.05. Here, "t" stands for the statistic test score and "p<.05" means that the probability of an event occurring by chance is less than 5%. These numbers would cause the null hypothesis to be rejected, therefore affirming that the particular theory is true.

Other Statistical Testswww.wikipedia.org

                                                    

  Dark blue is less than one standard deviation from the mean. For the normal distribution, this accounts for 68.27 % of the set; while two standard deviations from the mean (medium and dark blue) account for 95.45 %; and three standard deviations (light, medium, and dark blue) account for 99.73 %.

68-95-99.7 rule

Standard Deviation• The standard deviation of a probability distribution,

random variable, or population or multiset of values is a measure of the spread of its values (wiki).

• The standard deviation is the most common measure of statistical dispersion, measuring how widely spread the values in a data set are. If the data points are close to the mean, then the standard deviation is small. As well, if many data points are far from the mean, then the standard deviation is large. If all the data values are equal, then the standard deviation is zero.

• www.wiki.com

Why???

• The standard deviation can also help you evaluate the worth of all those so-called "studies" that seem to be released to the press everyday. A large standard deviation (greater than 10% of the mean) in a study that claims to show a relationship between eating Twinkies and better SAT scores, for example, might tip you off that the study's claims aren't all that trustworthy.

Coefficient of Variance• The coefficient of variance (CV) measures the

precision of the person/s during a set of individual tests (replicates) performed for one specific water quality parameter. As an example, if a team has just finished collecting data on 5 replicates of dissolved oxygen data, the team can use the coefficient of variance formula to determine how precisely they performed the data. The higher the precision (the lower the %), the higher the likelihood that there was no difference in the way each replicate/ individual test was performed. In other words, data within a data set which is collected consistently should hypothetically have a Coefficient of Variance equal to zero percent!

The Formula• Calculating the Coefficient of Variance 

• s = standard deviation

• X = average

• You can calculate the CV for the 3-5 replicates for a single sampling.

• Distributions with CV < 1 are considered low-variance (that’s good), while those with CV > 1 are considered high-variance (that’s bad).

Percent Error

• The percent error can be determined when the true value is compared to the observed value according to the equation below:

• % error =   | your result - accepted value |    x  100 %                                 accepted value

• Less than 5% error is acceptable

• Additional Slides

A Tale of Two Tails• Directional hypotheses are called one-

tailed– We are only interested in deviations at one tail

of the distribution• Non-directional hypotheses are called two-

tailed–We are usually interested in any

significant deviations from the null hypothesis

How do you decide to use a one-or two-tailed approach?

One Tail or Two? The moderate approach:

• If there’s a strong, prior, theoretical expectation that the effect will be in a particular direction (A>B), then you may use a one-tailed approach. Otherwise, use a two-tailed test.

• Because only an A>B result is interesting, concentrate your attention on whether there is evidence for a difference in that direction.– E.G. does this new educational reform improve

students’ test scores?– Does this drug reduce depression?

One tail or two? The more conservative approach:

• The problem with the moderate approach is that you probably would actually find it interesting if the result went the other way, in many cases.– If the new educational reform leads to worse test scores, we’d want to know!– If the new drug actually increases symptoms of depression, we’d want to know!

One tail or two? The more conservative approach:

• Only use a one-tailed test if you have a strong hypothesis about the directionality of the results (A>B) AND it could also be argued that a result in the “wrong tail” (A<B) is meaningless, and might as well be due to chance.

One tail or two: The most conservative approach

• Always use two-tailed tests!

• Correcting for one-vs. two-tailed tests– If you think a researcher has run the wrong kind

of test, it’s easy to recalculate the p-value yourself:

– P (one-tailed) = ½ P (two-tailed)

Was the result significant?• There is no true sharp dividing line between

probable and improbable results.• There’s little difference between p=0.051 and

p=0.049, except that some journals will not publish results at p=0.051, and some readers will accept results at p=0.049 but not at p=0.051.

• In any case it does not tell us if the result is IMPORTANT!

Decision theory and tradeoffs between types of errors

• Think of a household smoke detector.• Sometimes it goes off and there’s no fire (you

burn some toast, or take a shower).–A false alarm.

–A Type I error.• Easy to avoid this type of error: take out the

batteries!• However, this increases the chance of a Type

II error: there’s a fire, but no alarm.

Decision theory and tradeoffs between types of errors

• Similarly, one could reduce the chances of a Type II error by making the alarm hypersensitive to smoke.

• Then the alarm will very likely go off in a fire.• But you’ll increase your chances of a false

alarm = Type I error. (The alarm is more likely to go off because someone sneezed.)

• There is typically a tradeoff of this sort between Type I and Type II errors.

In statistics, the standard deviation of a data set is the square root of its variance. Standard deviation is a widely used measure of the variability. It may be thought of as the average difference of the data points from the average of distribution or how far they are away from the average. A low standard deviation indicates that the data points tend to be very close to the average, whereas high standard deviation indicates that the data are spread out over a large range of values. The less spread the data points the more precise the data. We want a low standard deviation, which we shall say is less than 10% of the average. If it is greater, then our fish are not similar or our data collection is not precise.

standard deviation

• Add a column in excel to your data table. •Title it T-test•Click on the next open cell next to the temperature readings at 18 degrees•Go up to fx and choose TTEST, hit ok•Array 1 is the readings (of 100%) across the row at 20 C(do not include temp, standard deviation or average)•Array 2 is the readings across the row at 18 C (do not include temp, standard deviation or average)•Type is 2 and tails is 2.•Hit ok•The p value will appear in the box. •Grab the small box in the lower right hand corner of the cell and pull it down to the last row of temp readings at 6C. •The p values will appear which are comparing the readings in that line to the previous (ie 10C respiratory rate to 8C)•When p ≤ 0.05 then there is a statistically significantl difference between the respiratory rate in that row when compared to the previous row readings. •Is there a temp at which it no longer matters how cold the water is (when p is no longer ≤ 0.05)? Is this because there is no longer a temperature effect or because of the variance of the data?????? Really hard question. Is the standard deviation less than 10% of the mean??????•Indicate where the independent variable is having a statistically significant effect of the fish respiratory rate by placing a “*” above the bar on the bar graph of class data. Insert a text box below the graph and type “ p ≤ 0.05” which indicates the independent variable is having a statistically significant effect.

Student T test

Respiratory Rate (% Control) vs TemperatureTemp ºC % control Average

%controlStandard Deviation

T-test p value

20 100 100 100 100 100 100 100 100 0

18 77 86 75 76 58 64 72.7 10.0

1.6172E-0516 63 57 54 58 50 81 54 59.6 10.3

0.0409281214 47 55 45 42 38 18 39 40.6 11.5

0.0067819412 37 38 33 27 27 16 29 29.6 7.5

0.0550431510 29 36 29 19 19 11 21 23.4 8.4

0.17311748 21 28 20 12 12 9 17.0 7.2

0.169591276 29 13 8 4 4 5 7 10.0 8.9

0.15326909