1)Test the effects of IV on DV
2)Protects against threats to internal validity
Internal Validity – Control through Experimental Design
Chapter 10 – Lecture 10
Causation
Highest Constraint
Comparisons btw grps
Random sampling
Random assignment
Experimental Design
Infer Causality
1)One or more hypothesis
2)Includes at least 2 “levels” of IV
3)Random assignment
4)Procedures for testing hypothesis
5)Controls for major threats to internal validity
Experimental Design(5 characteristics)
Develop the problem statementDefine IV & DVDevelop research hypothesisIdentify a population of interestRandom sampling & Random
assignmentSpecify procedures (methods)Anticipate threats to validityCreate controlsSpecify Statistical tests•Ethical considerations
Experimental Design
Clear Experimental Design…
1.between groups variance (systematic)
Experimental Design2 sources of variance
2. Within groups variance (nonsystematic) (error variance)
drugno drug
Remember…Sampling error
Significant differences…variability btw means is larger than expected on the basis of sampling error alone (due to chance alone)
Variance Need it!Without it…
No go
Between Group Within GroupExperimental Variance
(Due to your treatment)+
Extraneous Variance(confounds etc.)
VARIANCE
Error Variance(not due to treatment – chance)
CON TX Subs
“Partitioning of the variance”
between groups varianceWithin groups variance
Variance: Important for the statistical analysis
F =
Systematic effects + error varianceerror variance
F =
1.00F = No differences btw groups
Variance
Your experiment should be designed to
• Maximize experimental variance
•Control extraneous variance
•Minimize error variance
Maximize “Experimental” Variance
• At least 2 levels of IV (IVs really vary?)
•Manipulation check: make sure the levels (exp. conditions) differ each other
Ex: anxiety levels (low anxiety/hi anxiety) performance on math task
anxiety scale
Control “Extraneous” Variance
1. Ex. & Con grps are similar to begin with
2. Within subjects design (carryover effects??)
3. If need be, limit population of interest (o vs o )
4. Make the extraneous variable an IV (age, sex, socioeconomic) = factorial design
M F
Lo Anxiety
Hi Anxiety
M-low
M-hi
F-low
F-hi
Factorial design(2 IV’s)
YOUR Proposals
1. Ex Post Facto2. Single-group, posttest only 3. Single-group pretest-posttest4. Pretest-Posttest natural control
group
Group A Naturally Occurring Event Measurement
1. Ex Post Facto – “after the fact”
Control through Design – Don’ts
No manipulation
Control through Design – Don’ts
Single group posttest only
Single group Pretest-posttest
Group A TX Posttest
Pretest Group A TX Posttest
Compare
Control through Design – Don’ts
Pretest-Posttest Naturalistic Control Group
Group A Pretest TX Posttest
Group B Pretest no TX Posttest
Compare
NaturalOccurring
• Manipulate IV • Control Group• Randomization
Control through Design – Do’s – Experimental Design
Testing One IV4 Basic Designs
1. Randomized Posttest only, Control Group2. Randomized Pretest-Posttest, Control Group3. Multilevel Completely Randomized Between
Groups4. Solomon’s Four- Group
Randomized Posttest Only – Control Group(most basic experimental design)
R Group A TX Posttest (Ex)
R Group B no TX Posttest (Con)
Compare
Randomized, Pretest-Posttest, Control Group Design
R Group A Pretest TX Posttest (Ex)
R Group B Pretest no TX Posttest (Con)
Compare
Multilevel, Completely Randomized Between Subjects Design (more than 2 levels of IV)
R Group A Pretest TX1 Posttest
R Group B Pretest TX 2 Posttest
R Group C Pretest TX3 Posttest R Group D Pretest TX4 Posttest
Compare
Solomon’s Four Group Design(extension Multilevel Btw Subs)
R Group A Pretest TX Posttest
R Group B Pretest ---- Posttest
R Group C -------- TX Posttest R Group D -------- ---- Posttest
Compare
Powerful Design!
What stats do you use to analyze experimental designs?
Depends the level of measurement
Test difference between groups
Nominal data chi square (frequency/categorical)
Ordered data Mann-Whitney U test
Interval or ratio t-test / ANOVA (F test)
t-Test Compare 2 groups
IndependentSamples (between Subs)
One sample (Within)
Evaluate differences bwt 2 independent groups
Evaluate differences bwt two conditions in a single groups
Assumptions to use t-Test
1. The test variable (DV) is normally distributed in each of the 2 groups
2. The variances of the normally distributed test variable are equal – Homogeniety of Variance
3. Random assignment to groups
Represents the distribution of t that would be obtained if a value of t were calculated for each sample mean for all possible random samples of a given size from some population
t-distribution
Degrees of freedom (df)
When we use samples we approximate means & SD to represent the true population
Sample variability (SS = squared deviations) tendsto underestimate population variability
Restriction is placed = making up for this mathematically by using n-1 in denominator
Degrees of freedom (df): n-1
The number of values (scores) that are free to vary given mathematical restrictions on a sample of observed values used to estimate some unknown population = price we pay for sampling
S2 = variance ss (sum of squares)
df (degrees of freedom)
(x - )2
n-1x
Degrees of freedom (df): n-1
Number of scores free to vary
Data Set you know the mean (use mean to compute
variance)
n=2 with a mean of 6X 8?6
In order to get a mean of 6 with an n of 2…need a sum of 12…second score must be 4… second score is restricted by sample mean (this score is not free to vary)
=x
Group Statistics
10 7.9000 1.1972 .3786
10 2.6000 1.2649 .4000
DRUGdoped
no dope
ENDURANCN Mean Std. Deviation
Std. ErrorMean
Independent Samples Test
.065 .801 9.623 18 .000 5.3000 .5508 4.1429 6.4571
9.623 17.946 .000 5.3000 .5508 4.1427 6.4573
Equal variancesassumed
Equal variancesnot assumed
ENDURANCF Sig.
Levene's Test forEquality of Variances
t df Sig. (2-tailed)Mean
DifferenceStd. ErrorDifference Lower Upper
95% ConfidenceInterval of the
Difference
t-test for Equality of Means
ANOVA
ENDURANC
140.450 1 140.450 92.604 .000
27.300 18 1.517
167.750 19
Between Groups
Within Groups
Total
Sum ofSquares df Mean Square F Sig.
Analysis of Variance (ANOVA)
Two or more groups ….can use on two groups…t2 = F
Variance is calculated more than oncebecause of varying levels (combo of differences)
Several Sources of VarianceSS – between
SS – WithinSS – Total
Sum of Squares: sum of squared deviations from the mean
Partitioning the variance
Assumptions to use ANOVA
1. The test variable (DV) is normally distributed
2. The variances of the normally distributed test variable is equal – Homogeniety of Variance
3. Random assignment to groups
between groups varianceWithin groups variance
F =
Systematic effects + error varianceerror variance
F =
1.00F = No differences btw groups
F = 21.5022 times as much variance betweenthe groups than we would expect by chance
Planned comparisons & Post Hoc tests
A Priori (spss: contrast)
part of your hypothesis…beforedata are collected…prediction is made
A Posteriori
Not quite sure where differences will occur
After Omnibus F…
2 types of errors that you must consider when doing Post Hoc Analysis
Why not just do t-tests!
1. Per-comparison error (PC)2. Family wise error (FW)
Alpha
Inflate Alpha!!!!
FW = c(
c = # of comparisons made= your PC
Ex: IV ( 5 conditions)
1 vs 21 vs 31 vs 41 vs 52 vs 32 vs 42 vs 5
3 vs 43 vs 54 vs 5
FW = c(
10 (0.05) = .50
HSD
Top Related