Statistical Issues in Randomized TrialsStatistical Issues in Randomized Trials

• Analysis (very brief):Analysis (very brief):– Standard analysisStandard analysis

– More exotic stuffMore exotic stuff

• Special topics in data analysis in RCT’s (FFD page 300-Special topics in data analysis in RCT’s (FFD page 300-309)309)

– SubgroupsSubgroups

– Adjustment for baseline covariablesAdjustment for baseline covariables

– Multiple endpointsMultiple endpoints

– Slicing and Dicing the endpoint variablesSlicing and Dicing the endpoint variables

• Multiple comparisons in clinical trialsMultiple comparisons in clinical trials

Analysis for clinical trials (review?)Analysis for clinical trials (review?)

• 2 groups simplest2 groups simplest

• Analysis depends on type of outcome variableAnalysis depends on type of outcome variable– ContinuousContinuous

– Binary (y/n)Binary (y/n)

– Binary, time to eventBinary, time to event

Analysis for clinical trials (review?)Analysis for clinical trials (review?)

• 2 groups simplest2 groups simplest

• Analysis depends on type of outcome variableAnalysis depends on type of outcome variable– Continuous (t-test)Continuous (t-test)

– Binary (y/n) (chi-squared)Binary (y/n) (chi-squared)

– Binary, time to event (log rank)Binary, time to event (log rank)

Analysis of trials with continuous outcomesAnalysis of trials with continuous outcomes

• Compare mean in placebo with mean in activeCompare mean in placebo with mean in active– e.g., effect of statins on lipids, b-blocker on MIe.g., effect of statins on lipids, b-blocker on MI

• Usually compare mean change across two groupsUsually compare mean change across two groups– Increased powerIncreased power

– Valid to compare “after” onlyValid to compare “after” only

• For example, weight loss clinical trial last weekFor example, weight loss clinical trial last week– (change in weight is outcome variable)(change in weight is outcome variable)

Multiple Outcomes of Raloxifene Evaluation Multiple Outcomes of Raloxifene Evaluation (MORE Trial)*(MORE Trial)*

• 7,705 postmenopausal women with:7,705 postmenopausal women with:– BMD T below -2.5 or vertebral fracturesBMD T below -2.5 or vertebral fractures

– International 189 centersInternational 189 centers

• Placebo vs. 60 or 120mg raloxifene (a SERM)Placebo vs. 60 or 120mg raloxifene (a SERM)

* Ettinger, Black, et. al. JAMA, 8/99* Ettinger, Black, et. al. JAMA, 8/99

Effect of Raloxifene on BMDEffect of Raloxifene on BMD %

Ch

an

ge

% C

ha

ng

e

MonthsMonths

00

11

22

33

44

00 1212 2424 3636

-1-1

-2-2

Lumbar SpineLumbar Spine

MonthsMonths

-2-2

-1-1

00

11

22

00 1212 2424 3636

33

44 HipHip

PBOPBO

RLXRLX

PBOPBO

RLXRLX

2.5%*2.5%*

2%*2%*

*p<.0001 (t-test)*p<.0001 (t-test)*p<.0001 (t-test)*p<.0001 (t-test)

Little Known Facts about Boring Tests:Little Known Facts about Boring Tests:The t-testThe t-test

• Student’s t-test

• Developed by W.S. Gossett ("Student”) [1876-1937]

• Developed as statistical method to solve problems stemming from his employment in a brewery

• Quiz 1: Which brewery did “Student” work for?

– Ans: Guiness

• Quiz 2: How do you spell t-test?

– a. T-test

– b. t test

– c. t-test

– d. t-test

Little Known Facts about Boring Tests:Little Known Facts about Boring Tests:When is a T-test Valid?When is a T-test Valid?

• If the outcome variable is normally distributed, use a t-If the outcome variable is normally distributed, use a t-test. If the outcome is not normal, use a nonparametric test. If the outcome is not normal, use a nonparametric test such as a Wilcoxin test.test such as a Wilcoxin test.

• True or False?True or False?

When is When is tt-test Valid-test Valid

• tt-test requires that sample means (not individuals) are -test requires that sample means (not individuals) are normally distributed.normally distributed.

• What does CLT stand for?What does CLT stand for?

• (Hint: It’s not a BLT made with chicken.)(Hint: It’s not a BLT made with chicken.)

• Central Limit TheoremCentral Limit Theorem

–The mean from any variable becomes normally distributed The mean from any variable becomes normally distributed as n becomes larger (goes to infinity)as n becomes larger (goes to infinity)

• Practical implication:Practical implication: tt-test-test almost always valid for continuous almost always valid for continuous data as long as n is large enough or variable not too weird.data as long as n is large enough or variable not too weird.

Analysis of trials with continuous outcomesAnalysis of trials with continuous outcomes

• Use Use tt-test usually-test usually

• If radically non-normal, use non-parametric analogueIf radically non-normal, use non-parametric analogue

• 238 P-M women 238 P-M women – 55 to 85 years55 to 85 years– BMD T-score < -2.5, or -2 with risk factorBMD T-score < -2.5, or -2 with risk factor– Minimal previous use of bisphosphonatesMinimal previous use of bisphosphonates

• Randomize (1 year, double blind) to:Randomize (1 year, double blind) to:– PTH alone (119)PTH alone (119)– PTH + Alendronate (59)PTH + Alendronate (59)– Alendronate alone (60)Alendronate alone (60)

• Second year (non-PTH) on-goingSecond year (non-PTH) on-going

• Funded by NIAMSFunded by NIAMS

• NEJM (9/23/03)NEJM (9/23/03)

PTH and Alendronate (PaTH):PTH and Alendronate (PaTH):Study DesignStudy Design

• Treatments (daily)Treatments (daily)– PTH(1-84) injections: 100 PTH(1-84) injections: 100 g (NPS Pharmaceuticals)g (NPS Pharmaceuticals)– Alendronate 10 mg (Merck)Alendronate 10 mg (Merck)– Matching placebos (blinded)Matching placebos (blinded)– Calcium (500 mg) and Vitamin D (400 IU)Calcium (500 mg) and Vitamin D (400 IU)

• EndpointsEndpoints– DXA BMD (spine, hip, radius, whole body)DXA BMD (spine, hip, radius, whole body)– QCT (g/cmQCT (g/cm33, spine and hip), spine and hip)

• Cortical/trabecular density and geometryCortical/trabecular density and geometry

– Markers (BSAP, PINP, serum CTX)Markers (BSAP, PINP, serum CTX)– Safety (serum/urine calcium, AE’s)Safety (serum/urine calcium, AE’s)

PaTH Study Design (cont’d)PaTH Study Design (cont’d)

• Complicated by 3 group designComplicated by 3 group design– Analysis: Analysis:

– Look at changes within groupLook at changes within group

– Compare PTH alone to PTH/ALN & ALN alone to PTH/ALNCompare PTH alone to PTH/ALN & ALN alone to PTH/ALN

• Continuous variables: use t-testContinuous variables: use t-test

PaTH Data AnalysisPaTH Data Analysis

Changes in Trabecular Volumetric BMD Changes in Trabecular Volumetric BMD by QCTby QCT

SpineSpine Total HipTotal Hip00

1010

2020

3030

4040

PTHPTH PTH/ALNPTH/ALN ALNALN

Me

an

Ch

an

ge

(%)

Me

an

Ch

an

ge

(%)

****

**** p<.01 p<.01

Changes in Markers of Bone TurnoverChanges in Markers of Bone Turnover(Use medians and interquartile range)(Use medians and interquartile range)

Me

dia

n C

ha

ng

e (

%)

Me

dia

n C

ha

ng

e (

%)

-100-100

00

100100

200200

300300

00 33 66 99 1212MonthMonth

-100-100

00

100100

200200

300300

400400

00 33 66 99 1212MonthMonth

PTH PTH PTH/ALN PTH/ALN ALN ALN

Resorption (CTX)Resorption (CTX)Formation (P1NP)Formation (P1NP)

Analysis of trials with binary outcomesAnalysis of trials with binary outcomes

• Compare proportion in placebo vs. active groupsCompare proportion in placebo vs. active groups– e.g., occurrence of vertebral fracture on baseline vs. follow-up x-e.g., occurrence of vertebral fracture on baseline vs. follow-up x-

ray (yes/no, don’t know date)ray (yes/no, don’t know date)

• Use a chi-square testUse a chi-square test

3 Years of Raloxifene in MORE: 3 Years of Raloxifene in MORE: Effect on Vertebral FractureEffect on Vertebral Fracture

RR 0.65RR 0.65(0.53, 0.79)(p<.01)(0.53, 0.79)(p<.01)

% w

ith

fra

ctu

re%

wit

h f

ract

ure

PBOPBO RLX 60 RLX 60 RLX120RLX120

Analysis of trials with time-to-event outcomesAnalysis of trials with time-to-event outcomes

• Compare survival curves in active vs. placebo groupsCompare survival curves in active vs. placebo groups

WHI E + P: Coronary Heart DiseaseWHI E + P: Coronary Heart Disease

years 1 2 3 4 5 6 7

WHI E + P: Invasive Breast CancerWHI E + P: Invasive Breast Cancer

years 1 2 3 4 5 6 7

1%

2%

3%

Analysis of trials with time-to-event outcomesAnalysis of trials with time-to-event outcomes

• Compare survival curves in active vs. placebo groupsCompare survival curves in active vs. placebo groups

• Adjust for differential follow-up timeAdjust for differential follow-up time– Due to long recruitment periodDue to long recruitment period

• Conceptual: Conceptual: – Everyone will have the event if followed long enoughEveryone will have the event if followed long enough

– Those without event are censoredThose without event are censored

• Use log rank testUse log rank test– Stratified chi-square at each “failure” timeStratified chi-square at each “failure” time

– Equivalent to proportional hazards model with single binary Equivalent to proportional hazards model with single binary predictorpredictor

p < 0.001p < 0.001

Raloxifene and Risk ofRaloxifene and Risk ofBreast Cancer (MORE trial)Breast Cancer (MORE trial)

YearsYears

0.00

0.25

0.50

0.75

1.00

1.25

0 1 2 3 4

% o

f pa

rtic

ipan

ts%

of

part

icip

ants

PlaceboPlacebo3.8 per 1,0003.8 per 1,000

RaloxifeneRaloxifene1.7 per 1,0001.7 per 1,000

3 Years of Raloxifene Did Not Significantly 3 Years of Raloxifene Did Not Significantly Decrease Risk of Non-spine FracturesDecrease Risk of Non-spine Fractures

% with% withfracturesfractures

0 6 12 18 24 30 36

15

10

5

0

Months

PlaceboRaloxifene (60 + 120)

RH* = 0.91 (0.79, 1.06)RH* = 0.91 (0.79, 1.06)

* relative hazard * relative hazard from PH modelfrom PH model

* relative hazard * relative hazard from PH modelfrom PH model

WHI: Invasive Breast CancerWHI: Invasive Breast Cancer

years 1 2 3 4 5 6 7

1%

2%

3%

Analysis for clinical trials: more exotic stuffAnalysis for clinical trials: more exotic stuff

• Repeated measures analysesRepeated measures analyses– When outcome is repeatedWhen outcome is repeated

– Continuous: several measurements (at different times during Continuous: several measurements (at different times during follow-up)follow-up)

– Dichotomous: more than one occurrence of eventDichotomous: more than one occurrence of event

• Cluster randomization designsCluster randomization designs– Randomize/analyze clustersRandomize/analyze clusters

– Techniques for correlated data (random effects ANOVA, etc.)Techniques for correlated data (random effects ANOVA, etc.)

• Adjusted analysisAdjusted analysis– Use linear regression, logistic or PH to adjust for BL variablesUse linear regression, logistic or PH to adjust for BL variables

– Problematic unless specified Problematic unless specified aprioriapriori

Special topics in Data Analysis in RCT’sSpecial topics in Data Analysis in RCT’s

• SubgroupsSubgroups

• Adjustment for baseline covariablesAdjustment for baseline covariables

• Multiple endpointsMultiple endpoints

• Analysis of adverse eventsAnalysis of adverse events

• Slicing and dicing the endpoint variablesSlicing and dicing the endpoint variables

Special topics in Data Analysis in RCT’sSpecial topics in Data Analysis in RCT’s

• SubgroupsSubgroups

• Adjustment for baseline covariablesAdjustment for baseline covariables

• Multiple endpointsMultiple endpoints

• Analysis of adverse eventsAnalysis of adverse events

• Slicing and dicing the endpoint variablesSlicing and dicing the endpoint variables

• Multiple comparisonsMultiple comparisons

Multiple comparisonsMultiple comparisons

• The general problemThe general problem– Each statistical test has a 5% chance of Type I errorEach statistical test has a 5% chance of Type I error

– We are wrong 1 time out of 20We are wrong 1 time out of 20

– Easy to come up with spurious resultsEasy to come up with spurious results

• Take a worthless drug (placebo 2) compare to placebo 1Take a worthless drug (placebo 2) compare to placebo 1– 1 study: P(type I error)= 5%1 study: P(type I error)= 5%

– 2 studies: P(1 or 2 type I errors)= almost 10%2 studies: P(1 or 2 type I errors)= almost 10%

– 20 studies: P(at least one significant)=64%20 studies: P(at least one significant)=64%

• Publication biasPublication bias

Multiple comparisons: solutions?Multiple comparisons: solutions?

• BonferroniBonferroni– Divide overall p-value by number of testsDivide overall p-value by number of tests

– Unacceptable losses of powerUnacceptable losses of power

• Use common sense/BayesianUse common sense/Bayesian– Does result make sense?Does result make sense?

– Biologic plausibilityBiologic plausibility

– Is result supported by previous data?Is result supported by previous data?

– Was analysis defined Was analysis defined aprioriapriori??

• Special solutions for special situationsSpecial solutions for special situations– Multiple comparison procedures for 3 treatment groupsMultiple comparison procedures for 3 treatment groups

– Interim analysis (later lecture)Interim analysis (later lecture)

Multiple comparisons in RCT’s are pervasiveMultiple comparisons in RCT’s are pervasive

• Monitoring of trials: look at results as they accumulateMonitoring of trials: look at results as they accumulate– Lots of statistical machinery (later lecture, Grady)Lots of statistical machinery (later lecture, Grady)

• Subgroup analysesSubgroup analyses

• Multivariate analysis (adjustment) for BL covariatesMultivariate analysis (adjustment) for BL covariates

• Multiple endpoints in a trialMultiple endpoints in a trial

• Adverse experience analysisAdverse experience analysis

• Slicing and dicing continuous endpointSlicing and dicing continuous endpoint

SubgroupsSubgroups

• After primary analysis, often want to look at subgroupsAfter primary analysis, often want to look at subgroups

• Does effectiveness vary by subgroupDoes effectiveness vary by subgroup

• If drug effective, is it more effective in some If drug effective, is it more effective in some populations?populations?

• If results overall show no effect, does drug work in If results overall show no effect, does drug work in subgroup of participants?subgroup of participants?

• Are adverse effects concentrated in some subgroups?Are adverse effects concentrated in some subgroups?

Levels of subgroups (from FFD)Levels of subgroups (from FFD)

1. Those specified in study protocol have highest validity1. Those specified in study protocol have highest validityEspecially if number is smallEspecially if number is small

2. Those implied by study protocol2. Those implied by study protocoleg. If randomization stratified by age, sex or disease stageeg. If randomization stratified by age, sex or disease stage

3. Subgroups suggested by other trials3. Subgroups suggested by other trials

4. (Weakest) Subgroups suggested by the data themselves 4. (Weakest) Subgroups suggested by the data themselves (“fishing” or “data dredging”)(“fishing” or “data dredging”)

Example: children under 14 born in October (“month of October victimized Example: children under 14 born in October (“month of October victimized by poststudy analyses biased by knowledge of results”)by poststudy analyses biased by knowledge of results”)

5. (Diastrous) Subgroups based post-randomization variables5. (Diastrous) Subgroups based post-randomization variables

Example: Efficacy of Alendronate On Example: Efficacy of Alendronate On Reducing Clinical FracturesReducing Clinical Fractures

• FIT II: Women with BMD T-score < -1.6 FIT II: Women with BMD T-score < -1.6 (osteopenic--only 1/3 osteoporotic)(osteopenic--only 1/3 osteoporotic)

– All without existing vertebral fracturesAll without existing vertebral fractures

• Overall results: Overall results: – 50% reduction in vertebral fractures (p<.01)50% reduction in vertebral fractures (p<.01)

– 14% reduction in non-vertebral fractures (p=.07)14% reduction in non-vertebral fractures (p=.07)

– WimpyWimpy

RR for clinical fracture of alendronateRR for clinical fracture of alendronate(FIT II, Cummings, JAMA 1999) (FIT II, Cummings, JAMA 1999)

BB

BB

BB

00

11

1.51.5

OverallOverall

0.860.86(0.73 - 1.01)(0.73 - 1.01)

Re

lati

ve

Ris

kR

ela

tiv

e R

isk P=0.07P=0.07P=0.07P=0.07

Cummings, Black et. al, JAMA, 1997Cummings, Black et. al, JAMA, 1997

RR for clinical fracture of alendronate RR for clinical fracture of alendronate by baseline BMD groupsby baseline BMD groups

BB

BB

BB

BB

BB

BB

BB

BB

BB

BB

BB

BB

00

11

1.51.5

Baseline Femoral Neck BMD, by T-scoreBaseline Femoral Neck BMD, by T-score

OverallOverall T < -2.5T < -2.5 -2.5 < T < -2.0-2.5 < T < -2.0 T > -2.0T > -2.0

0.860.86(0.73 - 1.01)(0.73 - 1.01)

0.640.64(0.50 - 0.82)(0.50 - 0.82)

1.031.03

(0.77 - 1.39)(0.77 - 1.39)

1.141.14 (0.82 - 1.60)(0.82 - 1.60)

Re

lati

ve

Ris

kR

ela

tiv

e R

isk

Cummings, Black et. al, JAMA, 1997Cummings, Black et. al, JAMA, 1997

What to Do With an What to Do With an Unexpected Subgroup FindingUnexpected Subgroup Finding

• Is this a real finding? Is this a real finding?

• Was it specified in protocol (with Was it specified in protocol (with smallsmall number of other number of other analyses specified)analyses specified)

• Has this been previously observed?Has this been previously observed?– Increase prior probabilityIncrease prior probability

• Ways to verifyWays to verify– Examine for other similar subgrouping variables (BMD at hip, Examine for other similar subgrouping variables (BMD at hip,

spine, radius)spine, radius)

– Examine for other similar endpoints (hip fractures, etc.)Examine for other similar endpoints (hip fractures, etc.)

– Most important: look at other trials, if possible and availableMost important: look at other trials, if possible and available

– Examine biologic plausibilityExamine biologic plausibility

Fosamax International Trial (FOSIT)Fosamax International Trial (FOSIT)

• 1908 women, 34 countries1908 women, 34 countries

• Lumbar spine BMD T-score < -2Lumbar spine BMD T-score < -2

• Alendronate (10 mg) vs. placeboAlendronate (10 mg) vs. placebo

• One year follow-upOne year follow-up

• BMD main endpointBMD main endpoint

• 47% reduction in all clinical fractures (p<.05)47% reduction in all clinical fractures (p<.05)

FOSIT: Relative risk alendronate vs. FOSIT: Relative risk alendronate vs. placebo within BMD subgroupsplacebo within BMD subgroups

BL hip BMD T N RR* 95% CI

Overall 1908 0.53 (0.3,0.9)

> -2 955 1.2 (0.5, 2.9)

-2 to –2..5 279 0.32 (0.07,1.5)

< -2.5 674 0.26 (0.1,0.7)

Black, et. al. World Congress Osteoporosis, 2001Black, et. al. World Congress Osteoporosis, 2001

BMD InteractionBMD Interaction

• Recently also seen in a recent study of the Recently also seen in a recent study of the bisphosphonate ibandronate (T<-3)bisphosphonate ibandronate (T<-3)

Subgroup Analysis During HERSSubgroup Analysis During HERS

• Overall no effect of HRT or perhaps harm in year 1 for Overall no effect of HRT or perhaps harm in year 1 for cardiovascular diseasecardiovascular disease

• Is there subgroup with significant harm?Is there subgroup with significant harm?

• Look at relative hazard (RH) within subgroups defined by Look at relative hazard (RH) within subgroups defined by baseline variablesbaseline variables

– Medication use at baselineMedication use at baseline

– Prior diseasePrior disease

– Health habitsHealth habits

– Compare RH in those with and without risk factorCompare RH in those with and without risk factor• RH in those using beta blockers compared to those not usingRH in those using beta blockers compared to those not using• RH > 1 ==> harmRH > 1 ==> harm• Get p-value for significance of difference of RH in those w and withoutGet p-value for significance of difference of RH in those w and without

HERS: 4 years of HRT increased HERS: 4 years of HRT increased then decreased CHD Eventsthen decreased CHD Events

YearYear E + PE + P PlaceboPlacebo RHRH p-valuep-value

11 5757 3838 1.51.5 .04.04

22 4747 4848 1.01.0 1.01.0

33 3535 4141 0.90.9 .6.6

4 + 54 + 5 3333 4949 0.70.7 .07.07

> 5> 5 ??????

P for trend = 0.009P for trend = 0.009

Subgroups: the final frontier in HERSSubgroups: the final frontier in HERS

•

Relative hazard (E vs. placebo)Relative hazard (E vs. placebo)

Subgroup Within AmongSubgroup Within Among

Subgroup N (%) Subgroup Others p*Subgroup N (%) Subgroup Others p*

history of smoking 1712 (62) 1.01 3.39history of smoking 1712 (62) 1.01 3.39 .01 .01

current smoker 360 (13) 0.55 1.92 .03 current smoker 360 (13) 0.55 1.92 .03

digitalis use 275 (10) 4.98 1.26 .04 digitalis use 275 (10) 4.98 1.26 .04

>= 3 live births 1616 (58) 1.09 2.72 .04 >= 3 live births 1616 (58) 1.09 2.72 .04

lives alone 775 (28) 2.97 1.14 .05 lives alone 775 (28) 2.97 1.14 .05

prior mi by chart review 1409 (51) 2.14 0.93 .05 prior mi by chart review 1409 (51) 2.14 0.93 .05

beta-blocker use 899 (33) 2.89 1.15 .06 beta-blocker use 899 (33) 2.89 1.15 .06

age >= 70 at randomization 1019 (37) 2.65 1.14 .06age >= 70 at randomization 1019 (37) 2.65 1.14 .06

* Statistical significance of interaction* Statistical significance of interaction

Lots of subgroups were analyzed in HERSLots of subgroups were analyzed in HERS

• history of smoking (at rv) 1712 (62) 1.01 3.39 0.30 .01 history of smoking (at rv) 1712 (62) 1.01 3.39 0.30 .01 • current smoker (at rv) 360 (13) 0.55 1.92 0.29 .03 current smoker (at rv) 360 (13) 0.55 1.92 0.29 .03 • digitalis use (at rv) 275 (10) 4.98 1.26 3.96 .04 digitalis use (at rv) 275 (10) 4.98 1.26 3.96 .04 • >= 3 live births 1616 (58) 1.09 2.72 0.40 .04 >= 3 live births 1616 (58) 1.09 2.72 0.40 .04 • lives alone (at rv) 775 (28) 2.97 1.14 2.60 .05 lives alone (at rv) 775 (28) 2.97 1.14 2.60 .05 • prior mi by chart review (cr) 1409 (51) 2.14 0.93 2.30 .05 prior mi by chart review (cr) 1409 (51) 2.14 0.93 2.30 .05 • beta-blocker use (at rv) 899 (33) 2.89 1.15 2.51 .06 beta-blocker use (at rv) 899 (33) 2.89 1.15 2.51 .06 • age >= 70 at randomization 1019 (37) 2.65 1.14 2.32 .06 age >= 70 at randomization 1019 (37) 2.65 1.14 2.32 .06 • prior mi in most distant tertile 447 (16) 2.64 0.93 2.82 .07 prior mi in most distant tertile 447 (16) 2.64 0.93 2.82 .07 • walk 10m or in exercise program (at rv) 1770 (64) 2.35 1.11 2.12 .08 walk 10m or in exercise program (at rv) 1770 (64) 2.35 1.11 2.12 .08 • prior ptca by chart review (cr) 1189 (43) 0.92 1.98 0.46 .08 prior ptca by chart review (cr) 1189 (43) 0.92 1.98 0.46 .08 • prior mi within 2 years 420 (15) 3.20 1.28 2.50 .11 prior mi within 2 years 420 (15) 3.20 1.28 2.50 .11 • tg > median (at rv) 1377 (50) 2.02 1.05 1.93 .12 tg > median (at rv) 1377 (50) 2.02 1.05 1.93 .12 • rales in the lungs (at rv) 80 ( 3) 0.43 1.65 0.26 .13 rales in the lungs (at rv) 80 ( 3) 0.43 1.65 0.26 .13 • digitalis or ace-inhibitor use (at rv) 653 (24) 2.33 1.24 1.88 .16 digitalis or ace-inhibitor use (at rv) 653 (24) 2.33 1.24 1.88 .16 • previous ert for >= 12 months 302 (11) 4.19 1.41 2.98 .18 previous ert for >= 12 months 302 (11) 4.19 1.41 2.98 .18 • serious medical conditions 1028 (37) 1.05 1.81 0.58 .21 serious medical conditions 1028 (37) 1.05 1.81 0.58 .21 • age >= 53 at lmp 578 (21) 3.19 1.38 2.31 .23 age >= 53 at lmp 578 (21) 3.19 1.38 2.31 .23 • hdl > median (at rv) 1315 (48) 1.18 1.95 0.61 .24 hdl > median (at rv) 1315 (48) 1.18 1.95 0.61 .24 • lp(a) > median (at rv) 1378 (50) 1.26 2.08 0.60 .25 lp(a) > median (at rv) 1378 (50) 1.26 2.08 0.60 .25 • use of non-statin llm (at rv) 420 (15) 0.89 1.69 0.52 .25 use of non-statin llm (at rv) 420 (15) 0.89 1.69 0.52 .25 • married (at rv) 1588 (57) 1.26 1.98 0.64 .29 married (at rv) 1588 (57) 1.26 1.98 0.64 .29 • lvef <= 40% 178 ( 6) 2.16 1.01 2.13 .31 lvef <= 40% 178 ( 6) 2.16 1.01 2.13 .31 • prior mi within 4 years 765 (28) 2.07 1.32 1.57 .32 prior mi within 4 years 765 (28) 2.07 1.32 1.57 .32 • previous ert use for >= 1 year 327 (12) 2.86 1.41 2.03 .32 previous ert use for >= 1 year 327 (12) 2.86 1.41 2.03 .32 • prior mi within 1 year 194 ( 7) 2.88 1.43 2.02 .33 prior mi within 1 year 194 ( 7) 2.88 1.43 2.02 .33 • chest pain (at rv) 982 (36) 1.25 1.88 0.67 .33 chest pain (at rv) 982 (36) 1.25 1.88 0.67 .33 • dbp >= 90 mmhg (at rv) 149 ( 5) 0.91 1.62 0.56 .35 dbp >= 90 mmhg (at rv) 149 ( 5) 0.91 1.62 0.56 .35 • prior ptca within 1 year 206 ( 7) 3.94 1.46 2.71 .38 prior ptca within 1 year 206 ( 7) 3.94 1.46 2.71 .38 • prior mi within 3 years 612 (22) 2.05 1.37 1.50 .40 prior mi within 3 years 612 (22) 2.05 1.37 1.50 .40 • prior ptca within 4 years 838 (30) 1.15 1.70 0.68 .40 prior ptca within 4 years 838 (30) 1.15 1.70 0.68 .40 • use of any llm (at rv) 1296 (47) 1.23 1.76 0.70 .40 use of any llm (at rv) 1296 (47) 1.23 1.76 0.70 .40 • diuretic use (at rv) 775 (28) 1.89 1.33 1.42 .41 diuretic use (at rv) 775 (28) 1.89 1.33 1.42 .41 • signs and symptoms of chf (at rv) 118 ( 4) 0.94 1.60 0.58 .42 signs and symptoms of chf (at rv) 118 ( 4) 0.94 1.60 0.58 .42 • ace inhibitor use (at rv) 483 (17) 2.05 1.40 1.46 .44 ace inhibitor use (at rv) 483 (17) 2.05 1.40 1.46 .44 • total cholesterol > median (at rv) 1377 (50) 1.32 1.80 0.74 .47 total cholesterol > median (at rv) 1377 (50) 1.32 1.80 0.74 .47 • l-thyroxine use (at rv) 414 (15) 2.29 1.43 1.60 .47 l-thyroxine use (at rv) 414 (15) 2.29 1.43 1.60 .47 • poor/fair self-rated health (at rv) 665 (24) 1.30 1.72 0.76 .51 poor/fair self-rated health (at rv) 665 (24) 1.30 1.72 0.76 .51 • heart murmur (at rv) 540 (20) 1.89 1.42 1.34 .53 heart murmur (at rv) 540 (20) 1.89 1.42 1.34 .53 • sbp >= 140 mmhg (at rv) 1051 (38) 1.37 1.72 0.80 .59 sbp >= 140 mmhg (at rv) 1051 (38) 1.37 1.72 0.80 .59 • prior ptca within 3 years 695 (25) 1.27 1.61 0.78 .62 prior ptca within 3 years 695 (25) 1.27 1.61 0.78 .62 • s3 heart sounds (at rv) 19 ( 1) 2.74 1.50 1.82 .63 s3 heart sounds (at rv) 19 ( 1) 2.74 1.50 1.82 .63 • htn by physical exam (at rv) 557 (20) 1.32 1.62 0.81 .64 htn by physical exam (at rv) 557 (20) 1.32 1.62 0.81 .64 • >= 2 severely obstructed main vessels 1312 (47) 1.53 1.26 1.22 .69 >= 2 severely obstructed main vessels 1312 (47) 1.53 1.26 1.22 .69 • statin use (at rv) 1004 (36) 1.34 1.59 0.84 .71 statin use (at rv) 1004 (36) 1.34 1.59 0.84 .71 • have you ever been pregnant 2564 (93) 1.55 1.15 1.35 .72 have you ever been pregnant 2564 (93) 1.55 1.15 1.35 .72 • calcium-channel blocker (at rv) 1511 (55) 1.61 1.38 1.17 .73 calcium-channel blocker (at rv) 1511 (55) 1.61 1.38 1.17 .73 • previous hrt for >= least 12 months 132 ( 5) 1.24 1.60 0.78 .77 previous hrt for >= least 12 months 132 ( 5) 1.24 1.60 0.78 .77 • ldl > median (at rv) 1373 (50) 1.44 1.63 0.89 .77 ldl > median (at rv) 1373 (50) 1.44 1.63 0.89 .77 • prior ptca within 2 years 475 (17) 1.35 1.56 0.87 .81 prior ptca within 2 years 475 (17) 1.35 1.56 0.87 .81 • baseline left bundle branch block 212 ( 8) 1.31 1.55 0.85 .82 baseline left bundle branch block 212 ( 8) 1.31 1.55 0.85 .82 • white 2451 (89) 1.48 1.62 0.92 .88 white 2451 (89) 1.48 1.62 0.92 .88 • ever told you had diabetes 634 (23) 1.48 1.53 0.97 .94 ever told you had diabetes 634 (23) 1.48 1.53 0.97 .94 • aspirin use (at rv) 2183 (79) 1.51 1.56 0.97 .95 aspirin use (at rv) 2183 (79) 1.51 1.56 0.97 .95 • any alcohol consumption (at rv) 1081 (39) 1.54 1.57 0.98 .97 any alcohol consumption (at rv) 1081 (39) 1.54 1.57 0.98 .97 • gallstones or gallbladder dis. 633 (23) 1.55 1.52 1.02 .97 gallstones or gallbladder dis. 633 (23) 1.55 1.52 1.02 .97 •

baseline atrial fibrillation/flutter 33 ( 1) - 1.50baseline atrial fibrillation/flutter 33 ( 1) - 1.50 - - - -

Total subgroups examined: 102Total subgroups examined: 102

Total subgroups with p< .05: 6Total subgroups with p< .05: 6

Total subgroups examined: 102Total subgroups examined: 102

Total subgroups with p< .05: 6Total subgroups with p< .05: 6

Subgroups: conclusionsSubgroups: conclusions

• Subgroups are full of statistical problemsSubgroups are full of statistical problems– Multiple comparisons may lead to erroneous conclusionsMultiple comparisons may lead to erroneous conclusions

• Limited power in for subgroup analysesLimited power in for subgroup analyses

• Subgroups based on baseline variables are less badSubgroups based on baseline variables are less bad

• Subgroups based on post-randomization variables are Subgroups based on post-randomization variables are more problematicmore problematic

Adjusted analysis in a randomized trialAdjusted analysis in a randomized trial

• Could view RCT as a prospective trial with binary Could view RCT as a prospective trial with binary predictor (treatment)predictor (treatment)

• Use ANOVA or ANCOVA to adjust if a continuous Use ANOVA or ANCOVA to adjust if a continuous outcomeoutcome

• Could use logistic regression or Cox PH models to Could use logistic regression or Cox PH models to adjust if binary outcomeadjust if binary outcome

• General rule: Variable could be a confounder if it is General rule: Variable could be a confounder if it is related to both outcome and predictor (treatment)related to both outcome and predictor (treatment)

Adjusted analysis in a randomized trialAdjusted analysis in a randomized trial

- What if important prognostic variables (confounders) are - What if important prognostic variables (confounders) are maldistributed by chance alone?maldistributed by chance alone?

eg. Trial of MI: placebos older than treatedeg. Trial of MI: placebos older than treated

Adjust for age?Adjust for age?

- Controversial issue- Controversial issue

If you adjust for enough variables, you will eventually If you adjust for enough variables, you will eventually change the results. High potential for hanky-panky.change the results. High potential for hanky-panky.

Adjusted analysis in a randomized trialAdjusted analysis in a randomized trial

Potential solutions:Potential solutions:

• If a specific variable is highly prognostic, then use If a specific variable is highly prognostic, then use stratified blocking to guarantee balancestratified blocking to guarantee balance

• Perform analysis unadjusted and then adjustedPerform analysis unadjusted and then adjusted

• Pre-specify condition under which adjustment will be Pre-specify condition under which adjustment will be done:done:- eg. If age, BP or ldl are maldistributed (p<.05), then adjust for - eg. If age, BP or ldl are maldistributed (p<.05), then adjust for

that variable only.that variable only.

Multiple endpointsMultiple endpoints

• Often many ways to slice the outcome pieOften many ways to slice the outcome pie– Different subgroups of endpointsDifferent subgroups of endpoints

– Fractures: all, leg, arm, rib, etc. (MORE)Fractures: all, leg, arm, rib, etc. (MORE)

– Multiple comparisons problemsMultiple comparisons problems

• Some solutionsSome solutions– Very explicit predefinition of endpointsVery explicit predefinition of endpoints

– Limit number of endpointsLimit number of endpoints

– FDA: single endpoint onlyFDA: single endpoint only

Multiple Endpoints: Multiple Endpoints: Making a Mountain Out of a MolehillMaking a Mountain Out of a Molehill

• Multiple Outcomes of Raloxifene Evaluation (MORE) trialMultiple Outcomes of Raloxifene Evaluation (MORE) trial

• Main outcome: vertebral fracturesMain outcome: vertebral fractures

• Secondary outcome: non-vertebral fracturesSecondary outcome: non-vertebral fractures– Main osteoporotic subtypes: hip, wristMain osteoporotic subtypes: hip, wrist

• Overall, no effect of raloxifene on NV fracturesOverall, no effect of raloxifene on NV fractures

• Looked at 14 subtypes of fracturesLooked at 14 subtypes of fractures

• One significant: ankle. Wanted to title paper: One significant: ankle. Wanted to title paper: “Raloxifene reduces ankle fractures”“Raloxifene reduces ankle fractures”

WHI: Invasive Breast CancerWHI: Invasive Breast Cancer

years 1 2 3 4 5 6 7

1%

2%

3%

Multiple Comparisons: Multiple Comparisons: Huge Impact on Safety AssessmentHuge Impact on Safety Assessment

• Adverse experiences (“anything bad that happens to a patient”) Adverse experiences (“anything bad that happens to a patient”) are collected in regulatory trials as open text and then are collected in regulatory trials as open text and then categorizedcategorized

• Many categories (1000 or more)Many categories (1000 or more)

• Most have very few eventsMost have very few events

• Some prespecified ones to be taken more seriouslySome prespecified ones to be taken more seriously

• But what about surprises? But what about surprises? – Risedronate and lung cancerRisedronate and lung cancer

– Vioxx and heart diseaseVioxx and heart disease

• How to control for spurious findings?How to control for spurious findings?

• P-values almost meaningless (later lecture)P-values almost meaningless (later lecture)

Slicing and Dicing a Slicing and Dicing a Continuous Outcome VariableContinuous Outcome Variable

• A continuous variable can be analyzed as a comparison A continuous variable can be analyzed as a comparison of two means (generally preferred)of two means (generally preferred)

• Or as dichotomized valueOr as dichotomized value

• Diastolic Blood PressureDiastolic Blood Pressure– Could compare proportions > 90 mm Hg, > 100 mm HgCould compare proportions > 90 mm Hg, > 100 mm Hg

• Could look at variety of dichotomization pointsCould look at variety of dichotomization points

• Nice example on page 309 of FFDNice example on page 309 of FFD

• Specify any potential dichotomizations aprioriSpecify any potential dichotomizations apriori

Statistical issues: SummaryStatistical issues: Summary

• Main analysis generally straightforwardMain analysis generally straightforward– Based on two-group comparison tests or multi-group Based on two-group comparison tests or multi-group

generalizationsgeneralizations

• Multiple comparisons are ubiquitousMultiple comparisons are ubiquitous– MonitoringMonitoring

– Subgroup analysesSubgroup analyses

– Safety analysesSafety analyses

• Where possible, minimize subjectivity and adhoc-nessWhere possible, minimize subjectivity and adhoc-ness