1

Efficacy ResultsNDA 22092 (MTP-PE)

Laura LuStatistical Reviewer

Office of BiostatisticsFDA/CDER

2

Main Issues for DFS1. The Applicant’s pooled analysis not

appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care.

2. Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset.

3. Conduct of interim analyses complicate the interpretation of DFS results.

3

Main Issues for OS

1. The primary endpoint (DFS) of the study was not met.

2. Patient follow-up for OS was inadequate to perform a meaningful analysis.

4

INT-0133 Efficacy Datasets

• IDM 2003 Dataset – used by COG for analyses published in JCO

• IDM 2006 Dataset

• FDA Dataset

5

FDA Dataset• FDA considered data captured on case report

forms to be “primary source data”.• Where discrepancies identified between CRFs

and IDM 2003 dataset, CRF information used to determine dates of death and relapse.

• FDA did not modify IDM 2006 dataset because CRFs for additional follow-up not submitted. Therefore, FDA could not verify the accuracy of this information.

6

FDA DatasetExcludes• Seven (7) ineligible patients contained in IDM

Dataset 2003Includes • Nine (9) additional events identified in review of

CRFs• Additional follow-up documented on CRFs from one

institutionModified based on FDA review of CRFs • Change in length of disease-free survival for 66 pts• Change in length of overall survival for 68 pts

7

Main Issues for DFS

1. The Applicant’s pooled analysis not appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care.

2. Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset.

3. Conduct of interim analyses complicate the interpretation of DFS results.

8

Study Design

• Randomized to four study arms• One arm (Regimen AΘMTP-PE ) represents the

control arm/standard of care• Three experimental arms, each containing at

least one experimental agent• Evaluation of the efficacy of any experimental

regimen needs to be considered relative to the control regimen.

9

Study Analysis Issues• Study designers considered use of pooled

analysis (A/B MTP-PE vs. AΘ/BΘ MTP-PE) for the effect of MTP-PE.

• CCG/POG discussed the risk of this study design and analysis: “We hope that interactions between MTP-PE and the alternative chemotherapy arms will be similar. In this case it will be possible to analyze the proposed study by a factorial design. If the interactions are different, it will be necessary to consider the study as if it were a four arm analysis.”

10

Pooled AnalysisApplicant's Method

• Compare

Regimen AMTP-PE vs. Regimen AΘMTP-PE

Regimen BMTP-PE vs. Regimen BΘMTP-PE

separately• Pool the results in the two comparisons

(stratified log-rank test).

11

Treatment by Regimen Interaction

• Hazard ratio for AMTP-PE vs. AΘMTP-PE = 0.99

• Hazard ratio for BMTP-PE vs. BΘMTP-PE = 0.62

• The p-value for the treatment of MTP-PE by regimen interaction is 0.067.

12

Kaplan-Meier Curves for DFS by Regimen Based on Primary Data

13

Kaplan-Meier Curves for DFS by Regimen Based on Primary Data

14

Regimen B vs. Regimen A

Hazard ratio of Regimen BΘMTP-PE vs. Regimen AΘMTP-PE is 1.18. Therefore, Regimen BΘMTP-PE performed worse than Regimen AΘMTP-PE.

15

Sensitivity Analysis • Conducted to evaluate impact of inferior

performance of Regimen BΘMTP-PE on pooled outcome.

• In this analysis, results for Regimen AΘMTP-PE (control arm) are substituted for the results in Regimen BΘMTP-PE (inferior experimental chemo arm) in the pooled analysis.

16

Hazard Ratio

P-value

Results by substituting AΘMTP-PE for BΘMTP-PE on FDA Dataset

0.86

0.28

Results of Sensitivity Analysis

17

Analysis Driven by Comparison to Inferior Regimen

This sensitivity analysis shows that the small p-value of the pooled analysis is driven by a comparison to an experimental regimen that performed worse than the control regimen.

18

Invalid Analytic Method

Based on• Evidence of an treatment by regimen interaction

AND• Evidence of pooled analysis driven by a

comparison where experimental regimen (BΘMTP-PE) is inferior to control regimen (AΘMTP-PE)

A pooled analysis is not appropriate.

19

DFS Results by Regimen Relative to Regimen A Based on FDA Dataset

Variable Number of Patients

Number of Events

Hazard Ratio relative to A

P-value

Regimen AΘMTP-PE

171 60 ---

Regimen AMTP-PE

165 57 0.99 0.96

Regimen BΘMTP-PE

166 67 1.18 0.35

Regimen BMTP-PE

169 46 0.73 0.11

20

Main Issues for DFS1. The Applicant’s pooled analysis not

appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care.

2. Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset.

3. Conduct of interim analyses complicate the interpretation of DFS results.

21

DFS Results by Applicant’s Pooled Method Based on FDA Dataset

• Hazard ratio for A/B MTP-PE vs. AΘ/BΘ MTP-PE = 0.78,

p-value = 0.065

22

Main Issues for DFS

1. The Applicant’s pooled analysis not appropriate due to treatment by regimen interaction and a comparison to an experimental arm that performed worse than standard of care.

2. Statistical significance is not reached when applying applicant’s method of pooled analysis to FDA dataset.

3. Conduct of interim analyses complicate the interpretation of DFS results.

23

Type I Error Rate

It is impossible to determine the actual Type I error rate due to the conduct/timing of the interim and final analyses.

24

Interim Analysis Conduct

• According to the final protocol amendment on 6/16/1997, one interim analysis performed with no detailed information for conduct or alpha spending.

• Two additional interim analyses on EFS conducted; timing not based on specific number of events.

25

Final Analysis Conduct

• A final analysis was not conducted according to the protocol (approx 167 DFS events)

• IDM provided results for an analysis performed after 228 events

• If the timing of the final analysis is influenced by the results of the interim analyses, the type I error rate will be impacted.

It thus unclear what alpha should be used for the IDM analysis that included available data as of 4/9/2003.

26

Final Analysis Conduct

• If a pooled analysis is performed based on 167th DFS event using IDM 2003 dataset, the results are not statistically significant (nominal p-value = 0.11).

27

Main issues for OS

1. The primary endpoint of the study (DFS) was not met.

2. Follow-up on OS was inadequate to perform a meaningful analysis.

28

Survival Analysis Issues

• When the primary endpoint was not met, all alpha was spent.

• Any further analysis after the study failed to win on the primary endpoint increases the type I error rate. So literally, the difference in other endpoints should not be considered statistically significant.

29

Survival Analysis Issues

• Analysis plan for OS was not pre-specified. ‘Post-hoc’ analyses make it difficult to interpret the results for OS, since, by continuing to conduct tests for a treatment effect on different endpoints and/or the same endpoint a so-called “statistically significant result” (p < 0.05) can eventually be obtained even when there is no treatment effect.

• When an endpoint is selected based on the study results, the results for that endpoint are biased.

Overall Survival analyses are exploratory

30

Main issues for OS

1. The primary endpoint of the study (DFS) was not met.

2. Follow-up on OS was inadequate to perform a meaningful analysis.

31

Adequacy of Follow-up for OS Analyses

As of the 2003 data cut-off for OS– 22% of patients (148/678) had died per IDM’s

2003 dataset– Among 530 remaining patients who were alive

as of last contact, there were:• 8% with last contact on/before 12/31/1994• 11% with last contact on/before 12/31/1997• 51% with last contact on/before 12/31/2000

32

Adequacy of Follow-up for OS Analyses

• More than 50% of the 530 patients alive at the last contact were lost to follow-up two years prior to data cutoff in 2003

• In a well conducted trial for registration with OS as a primary endpoint, FDA expects substantially less than 5% of patients lost to follow-up at data cutoff.

33

Adequacy of Follow-up for Survival after Disease Occurrence

Among those patients lost to follow-up for survival,

– 26 had active disease (recurrent osteosarcoma or AML) at the last follow-up.

– These patients probably died.