Languages
Pages
Legal
PowerPowerPowerPower
The Four Components to a The Four Components to a Statistical ConclusionStatistical Conclusion
The Four Components to a The Four Components to a Statistical ConclusionStatistical Conclusion
The number of units (e.g., The number of units (e.g., people) accessible to studypeople) accessible to study
The salience of the program The salience of the program relative to the noiserelative to the noise
The odds the observed result is The odds the observed result is due to chancedue to chance
The odds you’ll observe a The odds you’ll observe a treatment effect when it treatment effect when it occursoccurs
Sample sizeSample sizeSample sizeSample size
Effect sizeEffect sizeEffect sizeEffect size
Alpha levelAlpha levelAlpha levelAlpha level
PowerPowerPowerPower
The Four Components to a The Four Components to a Statistical ConclusionStatistical Conclusion
The Four Components to a The Four Components to a Statistical ConclusionStatistical Conclusion
Amount of informationAmount of information
Salience of programSalience of program
Willingness to riskWillingness to risk
Ability to see effect that’s thereAbility to see effect that’s there
Sample sizeSample sizeSample sizeSample size
Effect sizeEffect sizeEffect sizeEffect size
Alpha levelAlpha levelAlpha levelAlpha level
PowerPowerPowerPower
The Effect SizeThe Effect SizeThe Effect SizeThe Effect Size
Is a ratio of...Is a ratio of...
The Effect SizeThe Effect SizeThe Effect SizeThe Effect Size
Is a ratio of...Is a ratio of...
SignalSignalSignalSignal
NoiseNoiseNoiseNoise
The Effect SizeThe Effect SizeThe Effect SizeThe Effect Size
Is a ratio of...Is a ratio of...
Difference between groupsDifference between groupsDifference between groupsDifference between groups
Standard error of the differenceStandard error of the differenceStandard error of the differenceStandard error of the difference
SignalSignalSignalSignal
NoiseNoiseNoiseNoise
Given Values for Any Three, You Given Values for Any Three, You Can Compute the Fourth.Can Compute the Fourth.
Given Values for Any Three, You Given Values for Any Three, You Can Compute the Fourth.Can Compute the Fourth.
n = f(effect size, a, power)n = f(effect size, a, power) effect size = f(n, a, power)effect size = f(n, a, power) a = f(n, effect size, power)a = f(n, effect size, power) power = f(n, effect size, a)power = f(n, effect size, a)
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
WhatWhatwe concludewe conclude
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
WhatWhatwe concludewe conclude
Null trueNull true
Alternative falseAlternative falseIn In realityreality... ...
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true
Alternative falseAlternative falseIn In realityreality... ...
Accept null
Reject alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true
Alternative falseAlternative falseIn In realityreality... ...
Accept null
Reject alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
1-1-
THE CONFIDENCE LEVELTHE CONFIDENCE LEVEL
The odds of saying there is The odds of saying there is nono effect or gain when in effect or gain when in
fact there is nonefact there is none
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is nonethere is none
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true
Alternative falseAlternative falseIn In realityreality... ...
Reject null
Accept alternative
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true
Alternative falseAlternative falseIn In realityreality... ...
Reject null
Accept alternative
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
TYPE I ERRORTYPE I ERRORThe odds of saying there The odds of saying there isis
an effect or gain when in an effect or gain when in fact there is nonefact there is none
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is onethere is one
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null falseNull false
Alternative trueAlternative true
In In realityreality... ... • There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null falseNull false
Alternative trueAlternative true
In In realityreality... ...
Accept null
Reject alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null falseNull false
Alternative trueAlternative true
In In realityreality... ...
Accept null
Reject alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
TYPE II ERRORTYPE II ERROR
The odds of saying there is The odds of saying there is no effect or gain when in no effect or gain when in
fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is nonethere is none
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null falseNull false
Alternative trueAlternative true
In In realityreality... ...
Reject null
Accept alternative
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
WhatWhatwe concludewe conclude
Null falseNull false
Alternative trueAlternative true
In In realityreality... ...
Reject null
Accept alternative
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
1-1-
POWERPOWERThe odds of saying there The odds of saying there isis
an effect or gain when in an effect or gain when in fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is onethere is one
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true Null falseNull false
Alternative falseAlternative false Alternative trueAlternative true
In In realityreality... ... In In realityreality... ...
Accept null
Reject alternative
Reject null
Accept alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
1-1-
THE CONFIDENCE LEVELTHE CONFIDENCE LEVEL TYPE II ERRORTYPE II ERROR
The odds of saying there is The odds of saying there is nono effect or gain when in effect or gain when in
fact there is nonefact there is none
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is nonethere is none
The odds of saying there is The odds of saying there is no effect or gain when in no effect or gain when in
fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is nonethere is none
1-1-
TYPE I ERRORTYPE I ERROR POWERPOWERThe odds of saying there The odds of saying there isis
an effect or gain when in an effect or gain when in fact there is nonefact there is none
The odds of saying there The odds of saying there isis an effect or gain when in an effect or gain when in
fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is onethere is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is onethere is one
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true Null falseNull false
Alternative falseAlternative false Alternative trueAlternative true
In In realityreality... ... In In realityreality... ...
Accept null
Reject alternative
Reject null
Accept alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
1-1-
THE CONFIDENCE LEVELTHE CONFIDENCE LEVEL TYPE II TYPE II ERRORERROR
1-1-
TYPE ITYPE I ERRORERROR POWERPOWER
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true Null falseNull false
Alternative falseAlternative false Alternative trueAlternative true
In In realityreality... ... In In realityreality... ...
Accept null
Reject alternative
Reject null
Accept alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
1-1-
THE CONFIDENCE LEVELTHE CONFIDENCE LEVEL TYPE II ERRORTYPE II ERROR
1-1-
TYPE I ERRORTYPE I ERROR POWERPOWER
CORRECTCORRECT
CORRECTCORRECT
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
Whatwe conclude
Null trueNull true Null falseNull false
Alternative falseAlternative false Alternative trueAlternative true
In In realityreality... ... In In realityreality... ...
Accept null
Reject alternative
Reject null
Accept alternative
We say...
• There is no real program effect
• There is no difference, gain
• Our theory is wrong
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
1-1-
THE CONFIDENCE LEVELTHE CONFIDENCE LEVEL TYPE II ERRORTYPE II ERROR
The odds of saying there is The odds of saying there is nono effect or gain when in effect or gain when in
fact there is nonefact there is none
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is nonethere is none
The odds of saying there is The odds of saying there is no effect or gain when in no effect or gain when in
fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is nonethere is none
1-1-
TYPE I ERRORTYPE I ERROR POWERPOWERThe odds of saying there The odds of saying there isis
an effect or gain when in an effect or gain when in fact there is nonefact there is none
The odds of saying there The odds of saying there isis an effect or gain when in an effect or gain when in
fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is onethere is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is onethere is one
If you try to increase power, you increase If you try to increase power, you increase the chance of winding up in the bottom the chance of winding up in the bottom
row and of Type I error.row and of Type I error.
If you try to increase power, you increase If you try to increase power, you increase the chance of winding up in the bottom the chance of winding up in the bottom
row and of Type I error.row and of Type I error.
The Decision MatrixThe Decision MatrixThe Decision MatrixThe Decision MatrixIn realityIn reality
WhatWhatwe concludewe conclude
Null trueNull true Null falseNull false
Alternative falseAlternative false Alternative trueAlternative true
In In realityreality... ... In In realityreality... ...
Accept nullAccept null
Reject alternativeReject alternative
Reject null
Accept alternative
We We saysay......
• There is no real There is no real program effectprogram effect
• There is no difference, There is no difference, gaingain
• Our theory is wrongOur theory is wrong
We say...
• There is a real program effect
• There is a difference, gain
• Our theory is correct
• There is no real program effectThere is no real program effect• There is no difference, gainThere is no difference, gain• Our theory is wrongOur theory is wrong
• There is a real program effectThere is a real program effect• There is a difference, gainThere is a difference, gain• Our theory is correctOur theory is correct
1-1-
THE CONFIDENCE LEVELTHE CONFIDENCE LEVEL TYPE II ERRORTYPE II ERROR
The odds of saying there is The odds of saying there is nono effect or gain when in effect or gain when in
fact there is nonefact there is none
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is nonethere is none
The odds of saying there is The odds of saying there is no effect or gain when in no effect or gain when in
fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is nonethere is none
1-1-
TYPE I ERRORTYPE I ERROR POWERPOWERThe odds of saying there The odds of saying there isis
an effect or gain when in an effect or gain when in fact there is nonefact there is none
The odds of saying there The odds of saying there isis an effect or gain when in an effect or gain when in
fact there is onefact there is one
# of times out of 100 when # of times out of 100 when there is there is nono effect, we’ll say effect, we’ll say
there is onethere is one
# of times out of 100 when # of times out of 100 when there there isis an effect, we’ll say an effect, we’ll say
there is onethere is one
If you try to If you try to decrease Type I decrease Type I
errors, you errors, you increase the increase the
chance of winding chance of winding up in the top row up in the top row
and of Type II and of Type II error.error.
If you try to If you try to decrease Type I decrease Type I
errors, you errors, you increase the increase the
chance of winding chance of winding up in the top row up in the top row
and of Type II and of Type II error.error.
Top Related