Comparing 2 Groups

• Most Research is Interested in Comparing 2 (or more) Groups (Populations, Treatments, Conditions)– Longitudinal: Same subjects at different times– Cross-sectional: Different groups of subjects

• Independent samples: No connection between the subjects in the 2 groups

• Dependent samples: Subjects in the 2 groups are “paired” in some manner.

Explanatory Variables/Responses

• Subjects (or measurements) in a study are first classified by which group they are in. The variable defining the group is the explanatory or independent variable.

• The measurement being made on the subject is the response or dependent variable.

• Research questions are typically of the form: Does the independent variable cause (or is associated with) the dependent variable?

I.V. D.V. ?????

Quantitative Responses

• For quantitative outcomes, we wish to compare 2 population means.– Parameter: 2-1

– Estimator: – Standard error:

– Sampling distribution : Approximately normal

12 YY

2

22

1

21

12 nnYY

2

22

1

21

121212

,~nn

NYYYY

Large-Sample CI for 2- 1

• Independent, Large-samples: n120, n220

• Must estimate the standard error, replacing the unknown population variances with sample variances:

• Large-sample (1-)100% CI for 2-1:

2

22

1

21

^

12

n

s

n

sYY

2

22

1

21

2/12

^

2/12 12

n

s

n

szYYzYY YY

Significance Tests for 2- 1

• Typically we wish to test whether the two means differ (null hypothesis being no difference, or effect). For independent samples:

• H0: 2- 1=0 (2= 1)

• Ha: 2- 10 (2 1)

• Test Statistic:

• P-value: 2P(Z |zobs|)

• For 1-sided tests (Ha: 2- 1>0) P=P(Z zobs)

2

22

1

21

12

^

12

12

0

ns

ns

YYYYz

YY

obs

Qualitative Responses

• For quantitative outcomes, we wish to compare 2 population proportions.– Parameter: 2-1

– Estimator: – Standard error:

– Sampling distribution : Approximately normal

1

^

2

^

2

22

1

11 )1()1(1

^

2

^

nn

2

22

1

11121

^

2

^ )1()1(,~

1

^

2

^

nnN

Large-Sample CI for 2-1

• Independent, Large samples (see sample size criteria from Chapter 6 for )

• Estimated standard error of the difference in sample proportions:

• (1-)100% CI for 2-1:2

2

^

2

^

1

1

^

1

^^ )1()1(

1

^

2

^

nn

2

2

^

2

^

1

1

^

1

^

2/1

^

2

^^

2/1

^

2

^ )1()1(1

^

2

^

nnzz

Significance Tests for 2- 1

• Typically we wish to test whether the two proportions differ (null hypothesis being no difference, or effect). For independent samples:

• H0: 2- 1=0 (2= 1)

• Ha: 2- 10 (2 1)

• Test Statistic:

21

2

^

21

^

1^

21

^^

1

^

2

^

^

1

^

2

^

11)1(

0

1

^

2

^nn

nn

nn

zobs

Significance Tests for 2- 1

• P-value: 2P(Z |zobs|)

• For 1-sided tests (Ha: 2- 1>0) P=P(Z zobs)

• When comparing means and proportions, confidence intervals and significance tests with the same levels provide the same conclusions

• Confidence intervals (when available) also provide a range of “believable” values for the difference in the parameter for the 2 groups

Small-Sample (1-100% Confidence Interval for Normal Populations

21

^

,2/12

2121

222

211

^

11

22

)1()1(

nntYY

nndfnn

snsn

Assuming equal population variances, “pool” sample variances to get a better estimate of :

Small-Sample Test for 21Normal Populations

• Case 1: Common Variances (12 = 2

2 = 2)

• Null Hypothesis:• Alternative Hypotheses:

– 1-Sided: – 2-Sided:

• Test Statistic:

0210 : H

021: AH

021: AH

2

)1()1(

11

)(

21

222

211

2^

21

^

021

nn

SnSn

nn

YYtobs

Small-Sample Test for 21Normal Populations

• Observed Significance Level (P-Value)• Special Tables Needed, Printed by Statistical Software

Packages

– 1-sided alternative

• P=P(t tobs) (From the t distribution)

– 2-sided alternative

• P=2P( t |tobs| ) (From the t distribution)

• If P-Value then reject the null hypothesis

Small-Sample Inference for 21Normal Populations

• Case 2: 12 2

2

• Don’t pool variances:

• Use “adjusted” degrees of freedom (Satterthwaites’ Approximation) :

2

22

1

21

^

12

n

s

n

sYY

11

*

2

2

2

22

1

2

1

21

2

2

22

1

21

n

ns

n

ns

ns

ns

Fisher’s Exact Test

• Method of testing for testing whether 2=1 when one or both of the group sample sizes is small

• Measures (conditional on the group sizes and number of cases with and without the characteristic) the chances we would see differences of this magnitude or larger in the sample proportions, if there were no differences in the populations

Example – Echinacea Purpurea for Colds

• Healthy adults randomized to receive EP (n1.=24) or placebo (n2.=22, two were dropped)

• Among EP subjects, 14 of 24 developed cold after exposure to RV-39 (58%)

• Among Placebo subjects, 18 of 22 developed cold after exposure to RV-39 (82%)

• Out of a total of 46 subjects, 32 developed cold• Out of a total of 46 subjects, 24 received EP

Source: Sperber, et al (2004)

Example – Echinacea Purpurea for Colds

• Conditional on 32 people developing colds and 24 receiving EP, the following table gives the outcomes that would have been as strong or stronger evidence that EP reduced risk of developing cold (1-sided test). P-value from SPSS is .079. 2210

2111

2012

1913

1814

Plac/ColdEP/Cold

Example - SPSS Output

Chi-Square Tests

2.990b 1 .084

1.984 1 .159

3.071 1 .080

.114 .079

46

Pearson Chi-Square

Continuity Correctiona

Likelihood Ratio

Fisher's Exact Test

N of Valid Cases

Value dfAsymp. Sig.

(2-sided)Exact Sig.(2-sided)

Exact Sig.(1-sided)

Computed only for a 2x2 tablea.

0 cells (.0%) have expected count less than 5. The minimum expected count is6.70.

b.

TRT * COLD Crosstabulation

Count

10 14 24

4 18 22

14 32 46

EP

Placebo

TRT

Total

No Yes

COLD

Total

Dependent (Paired) Samples

• Same individual receives each “treatment”

• Same individual observed before/after exposure

• Individuals matched on demographic or psychological similarities

• Often referred to as “matched pairs”

Inference Based on Paired Samples (Crossover Designs)

• Setting: Each treatment is applied to each subject or pair (preferably in random order)

• Data: Di is the difference in scores (Trt2-Trt1) for subject (pair) i

• Parameter: D - Population mean difference

• Sample Statistics:

21

2

21

1 DD

n

i id

n

i i ssn

DDs

n

DD

Test Concerning D

• Null Hypothesis: H0:D=0 (almost always 0)

• Alternative Hypotheses: – 1-Sided: HA: D > 0

– 2-Sided: HA: D 0

• Test Statistic:

ns

Dt

D

obs

Test Concerning D

P-value: (Based on t-distribution with =n-1 df)1-sided alternative

P = P(t tobs)

2-sided alternative

P = 2P(t |tobs|)

(1-)100% Confidence Interval for D

n

stD D

,2/

Example - Evaluation of Transdermal Contraceptive Patch In Adolescents

• Subjects: Adolescent Females on O.C. who then received Ortho Evra Patch

• Response: 5-point scores on ease of use for each type of contraception (1=Strongly Agree)

• Data: Di = difference (O.C.-EVRA) for subject i

• Summary Statistics:

1348.177.1 nsD D

Source: Rubinstein, et al (2004)

Example - Evaluation of Transdermal Contraceptive Patch In Adolescents

• 2-sided test for differences in ease of use (=0.05)

• H0:D = 0 HA:D 0

)66.2,88.0(89.077.1)41.0(179.277.1:%95

01.)005(.2)31.4(2

31.441.0

77.1

1348.1

77.1:

CI

tPP

tTS obs

Conclude Mean Scores are higher for O.C., girls find the Patch easier to use (low scores are better)

McNemar’s Test for Paired Samples

• Common subjects being observed under 2 conditions (2 treatments, before/after, 2 diagnostic tests) in a crossover setting

• Two possible outcomes (Presence/Absence of Characteristic) on each measurement

• Four possibilities for each subjects wrt outcome:– Present in both conditions– Absent in both conditions– Present in Condition 1, Absent in Condition 2– Absent in Condition 1, Present in Condition 2

McNemar’s Test for Paired Samples

Condition 1\2 Present Absent

Present n11 n12

Absent n21 n22

McNemar’s Test for Paired Samples• Data: n12 = # of pairs where the characteristic is present in

condition 1 and not 2 and n21 # where present in 2 and not 1

• H0: Probability the outcome is Present is same for the 2 conditions (2 = 1)

• HA: Probabilities differ for the 2 conditions (2 = 1)

|)|(2

:..2112

2112

obs

obs

zZPvalP

nn

nnzST

Example - Reporting of Silicone Breast Implant Leakage in Revision Surgery

• Subjects - 165 women having revision surgery involving silicone gel breast implants

• Conditions (Each being observed on all women)– Self Report of Presence/Absence of Rupture/Leak

– Surgical Record of Presence/Absence of Rupture/Leak

SELF * SURGICAL Crosstabulation

Count

69 28 97

5 63 68

74 91 165

Rupture

No Rupture

SELF

Total

Rupture No Rupture

SURGICAL

Total

Source: Brown and Pennello (2002), “Replacement Surgery and Silicone Gel Breast Implant Rupture”, Journal of Women’s Health & Gender-Based Medicine, Vol. 11, pp 255-264

Example - Reporting of Silicone Breast Implant Leakage in Revision Surgery

• H0: Tendency to report ruptures/leaks is the same for self reports and surgical records

• HA: Tendencies differ

0)0000317(.2)4(2|)|(2

00.4528

528:..

2112

2112

ZPzZPvalP

nn

nnzST

obs

obs