Bala Venkatesh

Professor of Intensive Care

Wesley & Princess Alexandra Hospitals

The George Institute for Global Health

University of Queensland & NSW

Risks of harm in stopping clinical trials early

Scope of this talk

• The DMC or the DSMC – Origins

• Role of the DMC

• Trial stopping rules

• Risk of overestimation of benefit

• Examples of trials stopped early

Controlled Clinical Trials 9:137-148 (1988)

Monitoring of clinical trial progress

• early definitive evidence of benefit

• convincing evidence of harm,

• or sufficient evidence of no potential benefit to render continuation of the trial to be futile.

Recommendations and the discussions arising from the Greenberg report gave birth to the

DMC or DSMC

Role of the DMC

DMC charter

When is a trial stopped early • Treatments found to be convincingly different –

BHAT/CAST/Metoprolol • Trial stopped for futility –ARDSNet Hi vs Lo PEEP/ECMO • Side effects or toxicities are too severe - ILLUMINATE • Information external to the RCT making trial

unnecessary or unethical - CELECOXIB • Data quality is poor • Accrual is slow -CORTICUS • Scientific question is no longer important • Adherence to treatment low - CORTICUS • Resources to perform study are lost or diminished • Study integrity - undermined by fraud or misconduct

Examples of stopping rules - benefit

Stopping rules for futility

Stopping rules for harm

Problems arise

• When there are liberal stopping rules

• When the totality of the evidence is not considered

• Biological plausibility of an effect not taken into consideration

Epidemiology of trials stopped early

1) industry-funded drug trials in the areas of cardiology, hematology-oncology,

(HIV-AIDS).

2) Adequate descriptions of the methods used to inform the decision to stop the

trial were often lacking;

3) 76 of 143 prematurely terminated trials did not note one or more of the

following: a) the planned sample size, b) the interim

analysis after which the trial stopped, or c) whether a statistical

stopping rule informed the decision.

Why is it problematic?

cluster around the true effect; some trials will start and remain close to the true effect as data accumulate.

1) Effects of Multiple Testing and Overestimation of Treatment Effects

1) 91 truncated RCTs with 424 matching non-truncated RCTs 2) Large differences in treatment effect size between truncated and nontruncated RCTs (ratio of relative risks 0.75) occurred with truncated RCTs having fewer than 500 events. 3) The pooled ratio of relative risks in truncated RCTs vs matching nontruncated RCTs was 0.71 (95% confidence interval, 0.65-0.77)

What does the pooled ratio of RR in truncated vs matching non-truncated

RCTs of 0.71 mean?

• This implies that, for instance, if the RR from the non-truncated RCTs was 0.8 (a 20% RR reduction), the RR from the truncated RCTs would be on average approximately 0.57 (a 43% relative risk reduction, more than double the estimate of benefit).

• Non-truncated RCTs with no evidence of benefit—ie, with an RR of1.0— would on average be associated with a 29% relative risk reduction in truncated RCTs addressing the same question.

2) Comprehensive Assessment of

Treatment Impact Limited

• Often, data are lacking on both long term effects and on patient-important outcomes that did not drive the decision to truncate the trial (

• e.g., overall survival

• Quality of life, or

• adverse events).

• Use of composite end points

3) Lack of data on risk benefit ratios

• Fish WH, et al Effect of intramuscular vitamin E on mortality and intracranial hemorrhage in neonates of 1000 grams or less. Pediatrics. 1990;85:578-84.

• Brion LP, et al. Vitamin E supplementation for prevention of morbidity and mortality in preterm infants. Cochrane Database Syst Rev. 2003;4:CD003665.

4) Misguided practice recommendations

• Publication in prominent journals

• Rapid dissemination in media

• Speedy incorporation into practice guidelines and quality assurance initiatives.

Beta blockers in non-cardiac surgery

• A clinical trial of bisoprolol in patients with vascular

• disease having non-cardiac surgery with a planned sample size of 266 stopped early after enrolling 112 patients.

• Two of 59 patients in the bisoprolol group and 18 of 53 in the control group had experienced a composite endpoint event (cardiac death or myocardial infarction).

• The authors reported a 91% reduction in relative risk with a 95% confidence interval of 63% to 98%.

Rapidly incorporated into clinical practice

guidelines

• In 2008 a systematic review and meta-analysis, including over 12 000 patients having non-cardiac surgery

• documented a 35% reduction in the odds of non-fatal myocardial infarction (95% CI 21% to 46%), but a twofold increase in non-fatal strokes (odds ratio 2.1, 27 to 3.68), and a possible increase in all cause mortality (1.20, 0.95 to 1.51)

5) A freezing effect on the conduct of new trials

• Equipoise

• Ethical issues

• Disproportionate weight in SR-MA

Examples of differences between interim analyses and final results

Example 1: ADRENAL - Interim analyses

DMC and close to 1st interim analysis

Sample size 950 – approximately CORTICUS X 2

and Annane X 3

Example 1: ADRENAL - Interim analyses

As the study progressed….

Sample size 1840 – approximately 3.5 X

CORTICUS and Annane X 6

DMC and 2nd interim analysis

Sample size 2900 – approximately CORTICUS X 6

and Annane X 10

Study completion

Example 2: AML trial – 4 vs 5 courses of chemo

Example 2: AML trial – 4 vs 5 courses of chemo

Example 3: Intensive insulin therapy

in critically ill patients

Example 4: Activated protein C

The trial was designed to enroll 2280 patients; two planned interim analyses by

an independent data and safety monitoring board occurred after 760 and 1520

patients had been enrolled. Statistical guidelines to suspend enrollment if

drotrecogin alfa acti- vated was found to be significantly more efficacious than

placebo were determined a priori and used the O’Brien–Fleming spending

function according to the method of Lan and Demets.

At the time of the second interim analysis of data from 1520 patients,

enrollment was suspended because the differences in the mortality rate

between the two groups exceeded the a priori guideline for stopping

Example 5: ECMO trial by Coombes at al stopped for futility

Take home message – Stopping clinical trials early for benefit

• Risk of type 1 error

• Not just look at primary outcome

• Secondary outcomes and safety issues

• Totality of available evidence

• Statistical boundary is just one guideline