Chapter 21: More About Tests “The wise man proportions his belief to the evidence.” -David Hume...
-
Upload
kelly-miller -
Category
Documents
-
view
223 -
download
0
Transcript of Chapter 21: More About Tests “The wise man proportions his belief to the evidence.” -David Hume...
Chapter 21:More About Tests
“The wise man proportions his belief to the evidence.”
-David Hume 1748
The Null Hypothesis The null must be a statement about the value of a
parameter from a model The value for the parameter in the null hypothesis
is found within the context of the problem Use this value to compute the probability that the
observed sample statistic would occur The appropriate null arises from the context of
the problem Think about the WHY of the situation
Another One-Proportion z-Test Null – the therapeutic
touch practitioners are just guessing, so they’ll succeed about half the time.
A one-sided test seems appropriate
Parameter: the proportion of successful identifications
0 50
0 50
: .
: .O
A
H P
H P
Another One-Proportion z-Test Check the
conditions: Independence Randomization 10% condition Success/failure
Independence: the hand choice was randomly selected, so the trials should be independent
Randomization: the experiment was randomized by flipping a coin
10% condition: the experiment observes some of what could be an infinite number of trials
Success/failure:
150 0 5 75 10
150 0 5 75 10
.
.
O
O
np
nq
Another One-Proportion z-Test State the null
model
Name the test
Because the conditions are satisfied, it is appropriate to model the sampling distribution of the proportion with the model
We can perform a one-proportion z-test
,ON p SD p
Another One-Proportion z-Test Find the
standard deviation of the sampling model using the hypothesized proportion,
0 50
5 5
1500 041
.
. .
.
O
O O
p
p qSD p
n
Op
Another One-Proportion z-Test Sketch of
Normal model
Find the z-score Find the P-
value
Observed proportion
is 0.46.p
0.8165 0.7929P P z
Another One-Proportion z-Test Conclusion
Link the P-value to your decision about the null hypothesis
State your conclusion in context
If possible, state a course of action
If the true proportion of successful detections of a human energy field is 50%, then an observed proportion of 46.7% successes or more would occur at random about 80% of the time.
That is not a rare event, so we do not reject the null hypothesis
There is insufficient evidence to conclude that the practitioners are performing better than they would have by guessing.
P-values A P-value is a conditional probability A P-value is the probability of the observed
statistic given that the null hypothesis is true The P-value is not the probability that the null
hypothesis is true A small P-value tells us that our data are rare
given the null hypothesis
P-value observed statistic value(or greater) OP H
Alpha Levels Alpha level
An arbitrarily set threshold for our P-value Also called the significance level Must be selected prior to looking at the data
If our P-value falls below that point, we’ll reject the null hypothesis
The result is called statistically significant When we reject the null hypothesis, we say that the
test is “significant at that level” Common alpha levels: .10, .05, .01
Therapeutic Touch Revisited The P-value was .7929 This is well above any reasonable alpha level Therefore, we cannot reject the null
hypothesis. Conclusion: “we fail to reject the null
hypothesis.” There is insufficient evidence to conclude that the practitioners are performing better than if they were just guessing
Absolutes: Are You Uncomfortable?
Reject/fail to reject decision when we use an alpha level is absolute
If your P-value falls just slightly above the alpha level, you do not reject the null hypothesis. However, if your P-value falls just slightly below, you do reject the null hypothesis
Perhaps it is better to report the P-value as an indicator of the strength of the evidence when making a decision
“Statistically Significant” We mean that the test value has a P-value lower than
our alpha level For large samples, even small deviations from the null
hypothesis can be statistically significant When the sample is not large enough, even very large
differences may not be statistically significant Report the magnitude of the difference between the
statistic and the null hypothesis when reporting the P-value
Critical Values Again Critical values can be used as a shortcut for the hypothesis tests Check your z-score against the critical values Any z-score larger in magnitude than a particular critical value
has to be less likely, so it will have a P-value smaller than the corresponding probability
α 1-sided 2-sided
.05 1.645 1.96
.01 2.28 2.575
.001 3.09 3.29
TT Revisited Again A 90% confidence interval would give
We could not reject because 50% is a plausible value for the practitioners’ true success
Any value outside the confidence interval would make a null hypothesis that we would reject; we’d feel more strongly about values far outside the interval
0 47 1 65 0 04 0 404 0 536. . . . , .
0 50: .OH p
Confidence Intervals & Hypothesis Tests Confidence intervals and hypothesis tests have the
same assumptions and conditions Because confidence intervals are naturally two-sided,
they correspond to two-sided tests A confidence interval with a confidence level of C%
corresponds to a two-sided hypothesis test with an level of 100 – C%
A confidence interval with a confidence of C% corresponds to a on-sided hypothesis test with an level of ½ (100 – C%)
“Click It or Ticket” If there is evidence
that fewer than 80% of drivers are buckling up, campaign will continue
0 80
0 80
: .
: .O
A
H p
H p
Check conditionsIndependence: Drivers are not likely to influence each others’ seatbelt habitsRandomization: we can assume that the drivers are representative of the driving public10%: Police stopped fewer than 10% of driversSuccess/Failure: there were 101 successes and 33 failures; both are greater than 10. The sample is large enough
*Use a one-proportion z-interval
“Click It or Ticket” To test the one-tailed
hypothesis at the 5% level of significance, construct a 90% confidence interval
Determine the standard error of the sample proportion and the margin of error
“Click It or Ticket” – Conclusion We can be 90% confident that between 69%
and 81% of all drivers wear their seatbelts. Because the hypothesized rate of 80% is
within this interval, we cannot reject the null hypothesis.
There is insufficient evidence to conclude that fewer than 80% of all drivers are wearing seatbelts.
Making Errors When we perform a
hypothesis test, we can make mistakes in two ways:
1. The null hypothesis is true, but we reject it.
2. The null hypothesis is false, but we fail to reject it.
The Truth The Truth
HO True HO False
My Decision Reject HO Type I Error Power
My Decision Retain HO OK Type II Error
Type I Errors Type I errors occur when the null
hypothesis is true but we’ve had the bad luck to draw an unusual sample.
To reject HO, the P-value must fall below . When you choose level , you’re setting the
probability of a Type I error.
Type II Errors When HO is false, and we fail to reject it,
we have made a Type II error (). There is no single value for . We can
compute the probability for any parameter value in HA.
Think about effect: how big a difference would matter?
Type I vs. Type II We can reduce for all values in the alternative,
by increasing . If we make it easier to reject the null hypothesis,
we’re more likely to reject it whether it’s true or not
However, we would make more Type I errors The only way to reduce both types of errors is to
collect more data. (Larger sample size)
Power Our ability to detect a false hypothesis is
called the power of a test. When the null hypothesis is actually false,
we want to know the likelihood that our test is strong enough to reject it.
The power of a test is the probability that it correctly rejects a false hypothesis.
Power is the probability that a test fails to reject
a false hypothesis, so the power of a test is
The value of power depends on how far the truth lies from the null hypothesis value.
The distance between the null hypothesis value, pO, and the truth, p, is the effect size
1
What Can Go Wrong??? Don’t change the null hypothesis after you
look at the data. Don’t base your alternative hypothesis on the
data. Don’t make what you want to show into your
null hypothesis Don’t interpret the P-value as the probability
that HO is true
What Can Go Wrong??? Don’t believe too strongly in arbitrary alpha
values Don’t confuse practical and statistical
significance Despite all precautions, errors (Type I or II)
may occur Always check the conditions