Understanding the Variability of Your Data:

Dependent Variable

Understanding the Variability of Your Data:

Dependent Variable

• Two "Sources" of Variability

Understanding the Variability of Your Data:

Dependent Variable

• Two "Sources" of Variability

– Independent (Predictor/Explanatory) Variable(s)

Understanding the Variability of Your Data:

Dependent Variable

• Two "Sources" of Variability

– Independent (Predictor/Explanatory) Variable(s)

– Extraneous Variables

Understanding the Variability of Your Data:

Dependent Variable

• Two Types of Variability

Understanding the Variability of Your Data:

Dependent Variable

• Two Types of Variability

– Unsystematic

Understanding the Variability of Your Data:

Dependent Variable

• Two Types of Variability

– Unsystematic

– Systematic

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Error Variability - unsystematic due to extraneous variables

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Error Variability - unsystematic due to extraneous variables

• Within conditions variability

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Error Variability - unsystematic due to extraneous variables

• Within conditions variability• Individuals in same condition affected differently

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Error Variability - unsystematic due to extraneous variables

• Within conditions variability• Individuals in same condition affected differently• Affects standard deviation, not mean, in long term

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Error Variability - unsystematic due to extraneous variables

Common sources

individual differences

procedural variations

measurement error

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Primary Variability – systematic due to independent variable

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Primary Variability – systematic due to independent variable

• Between conditions variability

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Primary Variability – systematic due to independent variable

• Between conditions variability• Individuals in same condition affected similarly

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Primary Variability – systematic due to independent variable

• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Primary Variability – systematic due to independent variable

• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently• Affects mean, not standard deviation, in long term

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Secondary Variability – systematic due to extraneous variable

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Secondary Variability – systematic due to extraneous variable

• Between conditions variability

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Secondary Variability – systematic due to extraneous variable

• Between conditions variability• Individuals in same condition affected similarly

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Secondary Variability – systematic due to extraneous variable

• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Secondary Variability – systematic due to extraneous variable

• Between conditions variability• Individuals in same condition affected similarly• Individuals in different conditions affected differently• Affects mean, not standard deviation, in long term

Understanding the Variability of Your Data:

Dependent Variable

• Roles played in the Research Situation

– Error Variability• A nuisance – the ‘noise’ in the research situation

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Error Variability• A nuisance – the ‘noise’ in the research situation

– Primary Variability• The focus – the potentially meaningful effect

Understanding the Variability of Your Data:

Dependent Variable

• Three "labels" for the variability

– Error Variability• A nuisance – the ‘noise’ in the research situation

– Primary Variability• The focus – the potentially meaningful effect

– Secondary Variability• The ‘evil’ – confounds the results

Example

• Two sections of the same course

Example

• Two sections of the same course

• Individual’s score as combination of ‘sources’

Statistical decision-making

• The logic behind inferential statistics

• Deciding if there is ‘systematic variability’– primary vs. secondary

• What do the data tell us?

• What decisions should we make?

Statistical decision-making

• A Research Example

– Research Hypothesis

– IF students chant the “Statistician’s Mantra” before taking their Methods exam THEN they will earn higher scores on the exam.

Statistical decision-making

• A Research Example

Your Class (M = 80, SD = 15, n = 25) compared to a known population Mean (M = 70) for a

standardized exam

Statistical decision-making

• A Research Example

Can estimate the Sampling Distribution

See if Population mean ‘fits’

Cause effect relationship not clear

Statistical decision-making

• A Research Example using experimental approach

– Research Hypothesis– IF students chant the “Statistician’s Mantra”

(vs. not chanting) before taking their Methods exam THEN they will earn higher scores on the exam.

Statistical decision-making

• Procedure

– Randomly divide class into two groups

• Chanters – are taught the “Statistician’s Chant” and chant together for 5 minutes before the exam

Statistical decision-making

• Procedure

– Randomly divide class into two groups

• Chanters – are taught the “Statistician’s Chant” and chant together for 5 minutes before the exam

• Non-chanters – sing Kumbaya together for 5 minutes before the exam

Statistical decision-making

• Results

– Compute exam scores for all students and organize by ‘condition’ (levels of IV).

Show ‘changing’ distribution

Statistical decision-making

• Results

– Compute exam scores for all students and organize by ‘condition’ (levels of IV).

– Compare Mean Exam Scores for two Conditions

Statistical decision-making

• Results

– Compute exam scores for all students and organize by ‘condition’ (levels of IV).

– Compare Means Exam Scores for two Conditions

– What will you find?

Statistical decision-making

• Research Hypotheses generally imprecise

– Predictions are not specific

– So “testing” the Research Hypothesis, using the available data, not reasonable

Statistical decision-making

• Null Hypothesis – a precise alternative

– Identifies outcome expected when NO systematic variability is present

Statistical decision-making

• Null Hypothesis – a precise alternative

– Identifies outcome expected when NO systematic variability is present

– But still must decide how close to the predicted outcome you must be to ‘believe’ in the Null Hypothesis

Statistical decision-making

• The Null Hypothesis Sampling Distribution

Statistical decision-making

• The Null Hypothesis Sampling Distribution

– All possible outcomes when the Null Hypothesis is true

• (when there is no ‘systematic’ variability present in the data)

Statistical decision-making

• The Null Hypothesis Sampling Distribution

– All possible outcomes when the Null Hypothesis is true

– Finding all the possible outcomes?

Statistical decision-making

• The Null Hypothesis Sampling Distribution

– All possible outcomes when the Null Hypothesis is true

– Finding all the possible outcomes?

– Seeing where your results fit into the Null Hypothesis Sampling Distribution

Statistical decision-making

• Deciding what to conclude based on the ‘fit’

Statistical decision-making

• Deciding what to conclude based on the ‘fit’

“True” State of the World

• Ho True Ho False• Reject Ho Error Correct Rejection

Decision• Not Reject Ho Correct Error• Nonreject

Statistical decision-making

• Deciding what to conclude based on the ‘fit’

“True” State of the World

• Ho True Ho False• Reject Ho Type 1 (p) Correct Rejection

Decision (power = 1 – Type 2)• Not Reject Ho Correct Type 2• Nonrejection• Deciding what confidence you want to have that you

have not made any errors

Statistical decision-making

• Trade-offs between Types of Errors– I believe I can fly?

Statistical decision-making

• Trade-offs between Types of Errors

• Factors affecting Type 2 Errors (Power)– “Real” systematic variability (size of effect)– Choice of Type 1 probability– Precision of estimates (sample size)

Statistical decision-making

• Trade-offs between Types of Errors

• Factors affecting Type 2 Errors (Power)– “Real” systematic variability (size of effect)

• Assume .5 * SD, a moderate size effect is good

– Choice of Type 1 probability• Use traditional .05

– Precision of estimates (sample size)• Sample of 50 (2 groups of 25)

Statistical decision-making

• Factors affecting Type 2 Errors (Power)

– Type 2 error probability = .59– Power = .41

Statistical decision-making

• Each ‘Decision” has an associated ‘error’

• Can only make Type 1 if “Reject”

• Can only make Type 2 if “Not Reject”

Statistical decision-making

Interpreting “Significant” Statistical Results

• Having decided to “reject” the Null Hypothesis you can:– State probability of Type 1 error– State confidence interval for population value– State percent of variability in DV ‘accounted for’

Statistical decision-making

Interpreting “Significant” Statistical Results

• For Chant vs. No Chant example– State probability of Type 1 error

• .05

– State confidence interval for population value• 95% CI is approximately +2 * SE• Point estimate of 10 + 8 (Real difference between 2

and 18)

– State percent of variability in DV ‘accounted for’• Eta2 = .20, or 20%

Statistical decision-making

Interpreting “Significant” Statistical Results

• Statistical Significance vs. Practical Significance

• How unlikely is the event in these circumstance– versus

• How much of an effect was there

Statistical decision-making

Interpreting “Non-significant” Statistical Results

Having decide you cannot reject the Ho

State the estimated ‘power’ of your design