Alternatives and Enhancements to Intention to Treat Analyses of

Randomized Trials

Thomas B. Newman, MD, MPHFebruary 20, 2002

Context

• Topic has interested me since draft lottery study

• Recent renewed interest in context of Appendix 12-B of Designing Clinical Research, 2nd edition

• PROBIT study provides a nice example• Possibility of instrumental variables analysis of

KPMCP phototherapy data

Example 1: Delayed Effects of the Military Draft on Mortality: 1

• Origin of study: Agent Orange concern• Design: “Randomized natural

experiment”using the draft lottery• Data source: computerized death certificate

registries, CA and PA• Predictor variable of interest: military service

Delayed Effects of the Military Draft on Mortality: 2

• Predictor variable measured: date of birth, leading to a draft lottery number below that year’s cutoff

Year of Birth Highest NumberDrafted

1950 195

1951 125

1952 95

BUT: Having an eligible number was a poor measure of military service:

• For consistency with later examples, consider those randomized to be eligible in the Intervention group, and those exempt in the Control group

Lotterycategory

Proportionserving

Interventiongroup (Eligible)

25.6%

Control group(Exempt)

9.3%

Algebraic Correction:

• Assume observed death rates for “intervention” (Ri) and control (Rc) are weighted averages of rates among those serving (A) and not serving (B). (Weighted by proportion serving)

RI = p1A + (1-p1)B RC= p2A + (1-p2)B

p1 = proportion serving in “Intervention” (eligible) groupp2 = proportion serving in “Control” (exempt) group

Algebraic correction 2:

• What we want to know is the relative risk for military service (A/B)

• What we have is the relative risk for draft eligibility (RI/RC)

• Then with algebra it can be shown that :

A/B = 1-RI/RC + 1 p2RI/RC -p1

Results

Cause of Death ObservedRR

ProjectedRR

Suicide 1.13** 1.86

Motor-vehicleaccidents

1.08* 1.53

All causes 1.04* 1.25

** P < 0.01 ; * P < 0.05

Examples of Other Questions Hard to Address with ITT Analyses

• Health benefits of breastfeeding• Health benefits of quitting smoking• Any intervention in which group allocation is

a poor measure of actual predictor variable of interest

Example 2: Promotion of Breastfeeding Intervention Trial

(PROBIT)*• Cluster-randomized trial at 31 sites in Belarus• Subjects 17,046 term singleton infants >2500g initially

breastfed• Intervention: WHO/UNICEF Baby Friendly Hospital

Initiative• Outcomes: BF @ 3,6,9,12 months and allergic,

gastrointestinal and respiratory disease• F/U to 12 months on 16,491 (96.7%)

*Kramer MS et al., JAMA 2001

PROBIT, cont’d• RQ#1: Does a breastfeeding promotion

program increase exclusive breastfeeding?– Predictor = Group assignment– Outcome = Exclusive breast feeding– ITT analysis is fine

• RQ#2: How much does exclusive breastfeeding reduce the risk of atopic eczema in the infant? (What is NNEBF*? )– Predictor = Exclusive breast feeding– Outcome = Atopic eczema– ITT won’t work -- too much misclassification

*Number Needed to Exclusively Breast Feed

Results

• Exclusive BF at 3 months 40% vs 5%*• Eczema 3.3% vs 6.3%; OR = 0.55 (95% CI 0.31-.95

based on GLIMMIX)• Question: if the risk difference and risk ratio in this

study are 3% and 0.55, what can we say about the values for exclusive breast feeding (as opposed to treatment allocation)?

*(Rounded)

Question:

• What is the true effect of breast feeding, undiluted by misclassification bias?

• That is, what would the risk ratio and risk difference for eczema have been if everyone in the intervention group and no one in the control group had exclusively breast fed?

• Might be relevant for helping a working mother make an informed decision about whether to breastfeed exclusively. (NNEBF)

Approach #1• Assume, as in draft lottery study

– There is a rate of eczema for breast fed infants (A) and a different rate for bottle fed infants (B)

– These rates are not dependent on group assignment

• Then if p1 and p2 are proportions breastfed in the intervention and control groups, the observed rates of eczema in the two groups, Ri and Rc are:RI = p1A + (1-p1)B RC= p2A + (1-p2)B

Approach #1, cont’d• To obtain the risk difference, we first

subtract the two equations: RI = p1A + (1-p1)B RC= p2A + (1-p2)B RC -RI =p2A -p1A + B- p2B-B+p1BRC -RI = p1(B-A) +p2(A-B)RC -RI = p1(B-A) -p2(B-A)B-A = (RC- RI)/(p1-p2)

• So difference in risk of eczema for exclusive BF is: (6.3%-3.3%)/(40%-5%)= 8.6%

Approach #1, cont’d• But is assumption that A & B are independent of group

assignment reasonable?• What if babies who are breast fed in response to the

intervention have a different risk of eczema from babies who would have breastfed anyway? (Ai Ac)– E.g. if intervention emphasized that a benefit of breastfeeding is

prevention of allergic disease, those breast feed in response to the intervention might be those most worried about allergic disease

– The risk of eczema in breastfed babies whose mothers breast fed in order to prevent allergic disease might be higher than among those who would breast feed anyway

Problem:

• Breast fed infants are really of two types:– Babies who breast feed anyway– Babies who breast feed only in response to the

intervention– Risk of eczema in these two groups might differ– Only the intervention group includes the second type of

infants• Therefore, our assumption of a single rate of eczema

for breast fed babies in both groups (Ai=Ac) is probably not valid.

Recall question posed was:• What would the risk ratio and risk difference for

eczema be if everyone in the intervention group and no one in the control group had exclusively breast fed?– Answer: can’t tell from this study– Reason: there’s a potential interaction between

effect of intervention on breast feeding and effect of breast feeding on eczema. (Extreme example: the intervention is effective at increasing breastfeeding only in those mothers whose babies are at risk for eczema)

Three types of infants

• Those who would breast feed anyway; Eczema rate=A

• Those who breast feed only with the intervention; Eczema rate = B

• Those that formula feed anyway; Eczema rate = C

Solution

• Explicitly model all types of infants• Change the question we are asking to one that

can be answered by the data

We can estimate the proportions of the three types of infants in each group

InterventionControl

P1

P2BF Anyway

BF from int. only

Bottle feed anyway

P1

P2BF Anyway

BF from int. only

Bottle feed anyway

Breast feedingReason: the groups were randomly assigned, so there is no reason to expect baseline differences in proportions of different types of infants

Predicted eczema rates:

• Ri = p2*Ai + (p1-p2)*Bi + (1-p1)*Ci

• Rc = p2*Ac + (p1-p2)*Bc + (1-p1)*Cc

New assumption

• Ai=Ac; Ci=Cc– Reason: We don’t expect a breastfeeding

intervention to affect the risk of eczema except by altering breastfeeding. In A and C groups (by definition) the intervention has no effect on breast feeding.

• (Violated if intervention covers other topics, like when to introduce solid foods.)

Risk difference

• Predicted Eczema Rates then are: Ri = p2*A + (p1-p2)*Bi + (1-p1)*CRc = p2*A + (p1-p2)*Bc + (1-p1)*C

• Risk difference for BF in response to intervention is then:Rc-Ri = (p1-p2)*(Bc-Bi)Bc-Bi=Rc-Ri/(p1-p2)

• Note that the right side of this equation is exactly the same as before! The point is to be clear on what is being estimated.

What does Bc-Bi estimate?• The expected difference in eczema rates due to

breastfeeding for infants susceptible to the breast feeding intervention

• It is more relevant clinically than the “B-A.”• It is analogous to the expected effect of a

medication on those who take it, which is more relevant to patients than the expected effect of medication on those to whom it is prescribed (which is diluted by those who don’t take it)

Counseling the Undecided Mother

• For a mother who is undecided about exclusive breastfeeding, the difference in eczema rates Bc-Bi is arguably the most relevant quantity!

• Since this is 8.6%, we can say that NNEBF is about 12

What if I want a Risk Ratio, not a Risk Difference?

• Recall:Ri = p2*A + (p1-p2)*Bi + (1-p1)*CRc = p2*A + (p1-p2)*Bc + (1-p1)*C

• Simply taking the quotient of these two and rearranging gives:Bi/Bc = (Ri -X)/(Rc-X) where X = p2*A +(1-p1)*C

• Note that both A and C are estimable from the dataA = risk in control group that GETs the intervention.C = risk in intervention group that does NOT get intervention.

Uh Oh. Does this mean RR in draft lottery paper are wrong?

• Well, the numbers are probably OK, but the text isn’t quite right

• Instead of saying we estimated the RR for military service, we should said we estimated the RR for military service due to the draft lottery. That is, it is quite possible that the effects of serving in the military for those who volunteered are different from those who were drafted, and the study cannot address the former.

Limitations

• Like compliance with medication, breastfeeding is not a dichotomous variable.

• Exclusive breastfeeding may be dichotomous, but its benefits will depend on who is in the comparison group

• This is a model, and models are only as good as their assumptions

• In this case, the assumption that Ci=Cc may be unreasonable, if intervention also increased BF among the 60% not exclusively BF.

Summary and Implication

• Intention to treat analysis gives biased measures of treatment effect when there is substantial misclassification of exposure

• Measuring adherence to the intervention (or success of the intervention on an intermediate endpoint) allows estimates of clinically relevant quantities

• Implication: measuring adherence to intervention (and its predictors) is important in the design of clinical trials