Prof. Dr. Karl Broich

EU regulatory concerns about missing data: Issues of interpretation and considerations for

addressing the problem

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 2

• No conflicts of interest

• Views expressed in this presentation are the author's personal views and not necessarily the views of BfArM or EMA

Disclaimer

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 3

• Current regulatory basis

• Sources of missing data

• Missing data and trial validity

• Example: Simulation of a depression trial

• Interpretational issues

• Estimands

• Addressing the problem

• Sensitivity analyses

• Conclusions

Agenda

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 4

• Relevant regulatory documents

• ICH E9 Guideline Statistical Principles for Clinical Trials (1998)

• EMA Guideline on Missing Data in Confirmatory Clinical Trials (2010)

• National Research Council. The Prevention and Treatment of Missing Data in Clinical Trials. Panel on Handling Missing Data in Clinical Trials (2010)

• ICH Concept Paper E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials (2014)

Regulatory discussion on missing data in confirmatory Phase III studies (1)

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 5

• Issues to discuss:

• How to avoid missing data?

• How to consider/address (non-)adherence?

• measuring efficacy assuming

• perfect adherence or

• real adherence or

• in those patients who tolerate treatment

• How to treat missing data?

• how to treat missing data w.r.t. adherence?

• and how to interpret analysis/missing data imputation

Regulatory discussion on missing data in confirmatory Phase III studies (2)

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 6

• Treatment discontinuation

• due to

• adverse events

• lack of efficacy

• others

• often leads to missing data

• follow-up of discontinuing patients avoids missingness

• Study drop-out

• treatment discontinuation and no follow-up

• Intermediate missing data

• usually less relevant

Sources of missing data in confirmatory Phase III studies

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 7

• Substantial amount of drop-outs in

• depression

• drop-out may be up to 50%

• average drop-out rate 20 to 30%

• others: schizophrenia, Alzheimer

• Follow-up

• usually poor follow-up of patients that discontinue treatment

• lack of information on non-adherent patients

• but: estimation of “de-facto” efficacy would require

• follow-up of patients that discontinue treatment

• targeting a “treatment-policy estimand”• treatment benefit in all subjects irrespective of treatment

adherence

Missing data in CNS trials

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 8

• Analysis in completers only• not compliant with ITT principle• treatment dependent patient selection• biased effect estimates and lack of type-1 error control

• invalid conclusions (e.g. false positive decisions ↑)• Missing data imputation

• based on specific assumptions regarding (unknown) missing data• requires definition of a relevant estimation target (estimand), e.g.

• treatment benefit if all patients adheredor

• treatment benefit in all patients regardless of adherenceor

• treatment benefit attributable to the randomized treatment or

• treatment benefit in those who adhere to treatment

Regulatory concerns about missing data in confirmatory clinical trials (1)

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 9

• Missing data imputation

• potential concerns about underlying assumptions and resulting validity

• e.g. LOCF usually invalid in progressive diseases (e.g. dementia)

• potential concerns about target of estimation

• e.g. longitudinal models may target treatment benefit if all patients adhered to treatment• hypothetical target that appears less relevant

• usually several sensitivity analyses required

• to show robustness of the results w.r.t. to underlying assumptions

• to evaluate different estimands

Regulatory concerns about missing data in confirmatory clinical trials (2)

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 10

“De-facto“ and “de-jure“ estimands

time

placebo

active treatment

end of trial

de-facto

(difference in all

randomized patients)

treatment dropout“retrieved” data

de-jure

(difference

if all patients

adhered)

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 11

Example: Simulation of depression trials BfArM research project on missing data and non-adherence

• Longitudinal data (Hamilton Score)

• Non-adherence: Treatment discontinuation

• Some data were collected after treatment discontinuation

• Different drop-out mechanisms

• treatment dropout (TD)

• study dropout (SD)

• SD time ≥ TD time

• “retrieved data” from TD to SD

• Data generation

• according to a two-piece linear mixed model

Leuchs et al (2014). Statistics in Medicine 33

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 12

Example: Simulation of depression trials BfArM research project on missing data and non-adherence

true de-jure effect = 2

(difference if all subjects adhered)

true de-facto (treatment policy)

effect = 0 (difference in all subjects)

Analysis strategies

• 1: Multiple Imputation (Pattern-

Mixture Model)

• 2: Joint Model of drop-out and

outcome

• 3: Mixed Model, all data

• 4: Mixed Model, only data

under treatment

Bias of different analysis strategies for de-jure and de-facto estimands

Leuchs et al (2014). Statistics in Medicine 33Proportion of subjects followed-up

Bia

s f

or

de-f

acto

eff

ect

100% 70% 40% 100% 70% 40%

equal drop-out 30%

unequal drop-out 25% und 35%

Bia

s f

or

de-j

ure

eff

ect

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 13

Example: Simulation of depression trials Conclusions

• Longitudinal Mixed Model analysis of on-treatment data

• targets de-jure estimand

• Longitudinal Mixed Model analysis of all data (off- and on-treatment)

• still shows relevant bias w.r.t. de-facto (treatment policy) estimandif follow-up is poor

• “Joint model” of outcome and time to drop-out

• behaves best

• but would require further investigation on robustness

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 14

Proposed procedure

Primary estimand

Clinical trial design

Analysis method

Sensitivity analyses

Be clear about the trial‘s objective (i.e. primary estimand) before

deciding trial design and analysis

Select a number of different sensitivity analyses

Sensitivity analyses

Customize the design considering the primary estimand

Clinical trial design

Choose a primary analysis applicable for the chosen design and addressing the primary estimand

Analysis method

Leuchs et al (2015). Therapeutic Innovation & Regulatory Science 49.

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 15

Regulatory conclusions on missing data and estimands (1)

Which estimand addresses best clinical relevance ?

• “Treatment policy” estimand

• most likely targets clinical relevance for a given population

• Treatment effect in tolerators

• may be relevant for patients

• but require complex causal inference and assumptions for a valid conclusion (without active run-in)

• “De-jure” like estimands (if all patients adhered)

• are hypothetical parameters difficult to justify

• but: may be most sensitive for non-inferiority conclusions

• Many other options to be discussed

• e.g. composite of different estimands related to reasons for drop-out

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 16

Regulatory conclusions on missing data and estimands (2)

“Treatment policy” estimand

• fails if no or only few “de-facto” (retrieved) data are available

• requiring unverifiable assumptions

• difference between de-facto and de-jure can hardly be substantiated without data

• strong de-facto conclusions require de-facto data

• patient follow-up after drop-out needed

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 17

Sensitivity analyses

… to assess the robustness of trial results!

Robustness of the estimation method Robustness of the estimand Robustness with regard to

generalizability of trial results

Internal validity external validity

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 18

Conclusions

• Missing data highly relevant issue in depression trials

• interpretational issues related to missing data

• Primary estimand to be agreed upon first

• design and analyse accordingly

• Sensitivity analyses relevant to address

• internal validity (concerning underlying assumptions)

• external validity (concerning clinical relevance addressed by different estimands)

• Treatment policy or attributable estimand relevant for population based conclusions

• treatment policy estimand require follow-up of (most) patients

• lack of follow-up result in the need for unverifiable assumptions

K Broich | EU regulatory issues regarding missing data| 26 September 2016 | Page 19

References• ICH Expert Working Group (1999). Statistical principles for clinical trials (ICH E9). Statistics in

Medicine, 18: 1905-1942.

• EMA (2010). Guideline on Missing Data in Confirmatory Clinical Trials.

• National Research Council of the National Academies (2010). The Prevention and Treatment of Missing Data in Clinical Trials. Washington, D.C.: National Academies Press.

• Mallinckrodt CH et al (2012). A structured approach to choosing estimands and estimators in longitudinal clinical trials. Pharmaceut Statist, 11:456–461.

• O’Neill RT and Temple R (2012). The prevention and treatment of missing data in clinical trials: an FDA perspective on the importance of dealing with it. Clin Pharmacol Ther, 91: 550-554.

• Leuchs AK, Zinserling J, Schlosser-Weber G, Berres M, Neuhäuser M, Benda N (2014). Estimation of the treatment effect in the presence of non-compliance and missing data. Statistics in Medicine, 32:193–208.

• ICH concept paper (2014) E9(R1): Addendum to Statistical Principles for Clinical Trials on Choosing Appropriate Estimands and Defining Sensitivity Analyses in Clinical Trials

• Leuchs AK, Zinserling J, Brandt A, Wirtz D, Benda N (2015). Choosing appropriate estimands in clinical trials. Therapeutic Innovation & Regulatory Science, 49:584-592.

• Leuchs AK, Brandt A, Zinserling J, Benda N (2016). Disentangling estimands and the intention-to-treat principle. Pharmaceut Statist (accepted for publication).