Post on 14-Jan-2016
Intro to StatsSignificance Testing
A statistical method that uses sample data to evaluate a hypothesis about a population
1. State a hypothesis 2. Use the hypothesis to predict the
characteristics the sample should have 3. Obtain a random sample from the
population 4. Compare the obtained sample data with
the prediction made from the hypothesis
Hypothesis Test
Suggests that the difference or relationship is systematic and not due to chance
A significance level gives the risk that the effect is due to chance
At p < .05, there is about a 5% (5/100) chance that an effect just happened in a normal distribution and is not related to any other variable
“Significant”
p < .05 for 2-tailed occurs at z = 1.96
Difference in size between the sample mean and the original population mean
The variability of scores The number of scores in the sample
Qualifying factors
We make certain assumptions about the impact of the methods used to obtain the data sample
Random sampling◦ That the sample represents the population
Independent observations◦ The first measurement has no relationship to the probability
of the second measurement The value of variability is unchanged by the
treatment◦ In most hypothesis-testing situations, we do not have the
“original” population variability A normal distribution
Assumptions Made
Researchers can never be sure that their hypothesis is “true”◦ Sample may not perfectly reflect the population◦ Other influences (confounds) may cause the
results◦ It just might be one of those few chances
These concerns are lessened every time that a finding is replicated
Never Sure
The null is rejected, but there really is no effect (Type I error)
Or fail to reject the null (accept null), but there really is an effect (Type II error)
Can lead to errors
Never know the truth about the null hypothesis
The likelihood of a Type I error is defined by the level of significance◦ p < .05 means there is a 5% chance of rejecting
the null when the null is true (conclude there is a difference when there is none)
Type II error is related to power and sample size
Never Sure 2
Rather than rely on hypothesis testing, it is now typical to include effect sizes◦ Most hypothesis tests simply state that a finding
is UNLIKELY (or not)◦ Significance does not mean it’s important◦ Effect sizes give a way to capture the SIZE of an
effect and make stronger statements about the relationship
◦ This also sidesteps issues of power Power is strongly linked to sample size and therefore
effects may be underestimated in small samples and overestimated in large samples
Effect sizes
Studies may show significant differences but the differences may not be meaningful◦ Small differences in large samples◦ Small differences that come with a large cost
Studies may not show significance but the differences may be meaningful◦ Small differences in small samples◦ Small differences that come with a big benefit and
little cost
Significance vs. Meaning
Inferences made about the population based on a sample
Which test determined by◦ Continuous vs. categorical variables◦ Number of variables◦ Whether variables vary between subjects or
within subjects
*Your book has a useful chart
Inferential Statistics
1. State the null and research hypotheses 2. Set the level of risk (usually .05) 3. Select the appropriate test statistic 4. Compute the test statistic 5. Determine the critical value for rejection of the
null 6. Determine whether the statistic exceeds the
critical value (usually at p < .05) 7&8. If over the critical value, the null hypothesis is
unlikely THEREFORE effect must be due to other variable
If not over the critical value, the null is accepted INTERPRET
Setting the Stage
One-sample z-tests are inferential statistics that allow you to compare a mean from a sample to the average of a population.
One-sample z-tests
1. State hypotheses◦ Null hypothesis: there is no difference between
civic engagement in Texas A&M students and the national average H0: XA&M = μstudents
◦ Research hypothesis: there is a difference between civic engagement at A&M and the national average H1: XA&M ≠ μstudents
Example 1
Z = X – μ SEM X = mean of sample μ = population average SEM = standard error of the mean
SEM = σ/square root of n σ = standard deviation for the population n = size of the sample
Calculating
Civic engagement of A&M students◦ X = 3.42 organizations, n = 200◦ μ = 1.5 organizations, σ = 1.2
Example 1
Civic engagement of A&M students◦ X = 3.42 organizations, n = 200◦ μ = 1.5 organizations, σ = 1.2
◦ SEM = σ/square root of n
Example 1
5. Determine the critical value for rejection of the null
We know that a z of 1.96 or larger is significant at p < .05
Remember this is the smallest value of z that corresponds to a p of .05 or less (falls in the “extreme range”
Bigger zs = more likely to be a significant difference
Example 1
6. Determine whether the statistic exceeds the critical value◦ 24 > 1.96◦ So it does exceed the critical value
◦ THE NULL IS REJECTED IF OUR STATISTIC IS BIGGER THAN THE CRITICAL VALUE – THAT MEANS THE DIFFERENCE IS SIGNIFICANT AT p < .05!!
7. If not over the critical value, fail to reject the null
Example 1
In results◦ Student participants at Texas A&M University
joined more civic organizations per year (M = 3.42) than the national average, z = 24.00, p < .05.
Example 1