EU regulatory concerns about missing data...• EMA (2010). Guideline on Missing Data in...

Prof. Dr. Karl Broich

EU regulatory concerns about missing data: Issues of interpretation and considerations for

addressing the problem

K Broich | EU regulatory issues regarding missing data| 26 September 2016 |

• No conflicts of interest

• Views expressed in this presentation are the author's personal views and not necessarily the views of BfArM or EMA

Disclaimer


• Current regulatory basis

• Sources of missing data

• Missing data and trial validity

• Example: Simulation of a depression trial

• Interpretational issues

• Estimands

• Addressing the problem

• Sensitivity analyses

• Conclusions

Agenda


• Relevant regulatory documents

• ICH E9 Guideline Statistical Principles for Clinical Trials (1998)

• EMA Guideline on Missing Data in Confirmatory Clinical Trials (2010)

• National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials (2010)

• ICH Concept Paper E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials (2014)

Regulatory discussion on missing data in confirmatory Phase III studies (1)


• Issues to discuss:

• How to avoid missing data?

• How to consider/address (non-)adherence?

• measuring efficacy assuming

• perfect adherence or

• real adherence or

• in those patients who tolerate treatment

• How to treat missing data?

• how to treat missing data w.r.t. adherence?

• and how to interpret analysis/missing data imputation

Regulatory discussion on missing data in confirmatory Phase III studies (2)


• Treatment discontinuation

• due to

• adverse events

• lack of efficacy

• others

• often leads to missing data

• follow-up of discontinuing patients avoids missingness

• Study drop-out

• treatment discontinuation and no follow-up

• Intermediate missing data

• usually less relevant

Sources of missing data in confirmatory Phase III studies


• Substantial amount of drop-outs in

• depression

• drop-out may be up to 50%

• average drop-out rate 20 to 30%

• others: schizophrenia, Alzheimer

• Follow-up

• usually poor follow-up of patients that discontinue treatment

• lack of information on non-adherent patients

• but: estimation of “de-facto” efficacy would require

• follow-up of patients that discontinue treatment

• targeting a “treatment-policy estimand”• treatment benefit in all subjects irrespective of treatment

adherence

Missing data in CNS trials


• Analysis in completers only• not compliant with ITT principle• treatment dependent patient selection• biased effect estimates and lack of type-1 error control

• invalid conclusions (e.g. false positive decisions ↑)• Missing data imputation

• based on specific assumptions regarding (unknown) missing data• requires definition of a relevant estimation target (estimand), e.g.

• treatment benefit if all patients adheredor

• treatment benefit in all patients regardless of adherenceor

• treatment benefit attributable to the randomized treatment or

• treatment benefit in those who adhere to treatment

Regulatory concerns about missing data in confirmatory clinical trials (1)


• Missing data imputation

• potential concerns about underlying assumptions and resulting validity

• e.g. LOCF usually invalid in progressive diseases (e.g. dementia)

• potential concerns about target of estimation

• e.g. longitudinal models may target treatment benefit if all patients adhered to treatment• hypothetical target that appears less relevant

• usually several sensitivity analyses required

• to show robustness of the results w.r.t. to underlying assumptions

• to evaluate different estimands

Regulatory concerns about missing data in confirmatory clinical trials (2)


“De-facto“ and “de-jure“ estimands

time

placebo

active treatment

end of trial

de-facto

(difference in all

randomized patients)

treatment dropout“retrieved” data

de-jure

(difference

if all patients

adhered)


Example: Simulation of depression trials BfArM research project on missing data and non-adherence

• Longitudinal data (Hamilton Score)

• Non-adherence: Treatment discontinuation

• Some data were collected after treatment discontinuation

• Different drop-out mechanisms

• treatment dropout (TD)

• study dropout (SD)

• SD time ≥ TD time

• “retrieved data” from TD to SD

• Data generation

• according to a two-piece linear mixed model

Leuchs et al (2014). Statistics in Medicine 33


Example: Simulation of depression trials BfArM research project on missing data and non-adherence

true de-jure effect = 2

(difference if all subjects adhered)

true de-facto (treatment policy)

effect = 0 (difference in all subjects)

Analysis strategies

• 1: Multiple Imputation (Pattern-

Mixture Model)

• 2: Joint Model of drop-out and

outcome

• 3: Mixed Model, all data

• 4: Mixed Model, only data

under treatment

Bias of different analysis strategies for de-jure and de-facto estimands

Leuchs et al (2014). Statistics in Medicine 33Proportion of subjects followed-up

Bia

s f

or

de-f

acto

eff

ect

100% 70% 40% 100% 70% 40%

equal drop-out 30%

unequal drop-out 25% und 35%

Bia

s f

or

de-j

ure

eff

ect


Example: Simulation of depression trials Conclusions

• Longitudinal Mixed Model analysis of on-treatment data

• targets de-jure estimand

• Longitudinal Mixed Model analysis of all data (off- and on-treatment)

• still shows relevant bias w.r.t. de-facto (treatment policy) estimandif follow-up is poor

• “Joint model” of outcome and time to drop-out

• behaves best

• but would require further investigation on robustness


Proposed procedure

Primary estimand

Clinical trial design

Analysis method

Sensitivity analyses

Be clear about the trial‘s objective (i.e. primary estimand) before

deciding trial design and analysis

Select a number of different sensitivity analyses


Customize the design considering the primary estimand

Clinical trial design

Choose a primary analysis applicable for the chosen design and addressing the primary estimand

Analysis method

Leuchs et al (2015). Therapeutic Innovation & Regulatory Science 49.


Regulatory conclusions on missing data and estimands (1)

Which estimand addresses best clinical relevance ?

• “Treatment policy” estimand

• most likely targets clinical relevance for a given population

• Treatment effect in tolerators

• may be relevant for patients

• but require complex causal inference and assumptions for a valid conclusion (without active run-in)

• “De-jure” like estimands (if all patients adhered)

• are hypothetical parameters difficult to justify

• but: may be most sensitive for non-inferiority conclusions

• Many other options to be discussed

• e.g. composite of different estimands related to reasons for drop-out


Regulatory conclusions on missing data and estimands (2)

“Treatment policy” estimand

• fails if no or only few “de-facto” (retrieved) data are available

• requiring unverifiable assumptions

• difference between de-facto and de-jure can hardly be substantiated without data

• strong de-facto conclusions require de-facto data

• patient follow-up after drop-out needed



… to assess the robustness of trial results!

Robustness of the estimation method Robustness of the estimand Robustness with regard to

generalizability of trial results

Internal validity external validity


Conclusions

• Missing data highly relevant issue in depression trials

• interpretational issues related to missing data

• Primary estimand to be agreed upon first

• design and analyse accordingly

• Sensitivity analyses relevant to address

• internal validity (concerning underlying assumptions)

• external validity (concerning clinical relevance addressed by different estimands)

• Treatment policy or attributable estimand relevant for population based conclusions

• treatment policy estimand require follow-up of (most) patients

• lack of follow-up result in the need for unverifiable assumptions


References• ICH Expert Working Group (1999). Statistical principles for clinical trials (ICH E9). Statistics in

Medicine, 18: 1905-1942.

• EMA (2010). Guideline on Missing Data in Confirmatory Clinical Trials.

• National Research Council of the National Academies (2010). The Prevention and Treatment of Missing Data in Clinical Trials. Washington, D.C.: National Academies Press.

• Mallinckrodt CH et al (2012). A structured approach to choosing estimands and estimators in longitudinal clinical trials. Pharmaceut Statist, 11:456–461.

• O’Neill RT and Temple R (2012). The prevention and treatment of missing data in clinical trials: an FDA perspective on the importance of dealing with it. Clin Pharmacol Ther, 91: 550-554.

• Leuchs AK, Zinserling J, Schlosser-Weber G, Berres M, Neuhäuser M, Benda N (2014). Estimation of the treatment effect in the presence of non-compliance and missing data. Statistics in Medicine, 32:193–208.

• ICH concept paper (2014) E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials

• Leuchs AK, Zinserling J, Brandt A, Wirtz D, Benda N (2015). Choosing appropriate estimands in clinical trials. Therapeutic Innovation & Regulatory Science, 49:584-592.

• Leuchs AK, Brandt A, Zinserling J, Benda N (2016). Disentangling estimands and the intention-to-treat principle. Pharmaceut Statist (accepted for publication).

EU regulatory concerns about missing data...• EMA (2010). Guideline on Missing Data in...

Documents

Transcript of EU regulatory concerns about missing data...• EMA (2010). Guideline on Missing Data in...