Personalized Predictive Personalized Predictive Medicine and Genomic Medicine and Genomic

Clinical TrialsClinical Trials

Richard Simon, D.Sc.Richard Simon, D.Sc.Chief, Biometric Research BranchChief, Biometric Research Branch

National Cancer InstituteNational Cancer Institutehttp://brb.nci.nih.govhttp://brb.nci.nih.gov

Biometric Research Branch Biometric Research Branch WebsiteWebsite

brb.nci.nih.govbrb.nci.nih.gov

Powerpoint presentationsPowerpoint presentations ReprintsReprints BRB-ArrayTools softwareBRB-ArrayTools software Web based Sample Size Planning Web based Sample Size Planning

Personalized Oncology is Personalized Oncology is Here Today and Rapidly Here Today and Rapidly

AdvancingAdvancing

Key information is in tumor genome, Key information is in tumor genome, not in inherited geneticsnot in inherited genetics

Personalization is based on limited Personalization is based on limited stratification of traditional stratification of traditional diagnostic categories, not on diagnostic categories, not on individual genomesindividual genomes

Personalized Oncology is Personalized Oncology is Here TodayHere Today

Estrogen receptor over-expression in Estrogen receptor over-expression in breast cancerbreast cancer Anti-estrogens, aromatase inhibitorsAnti-estrogens, aromatase inhibitors

HER2 amplification in breast cancerHER2 amplification in breast cancer Trastuzumab, LapatinibTrastuzumab, Lapatinib

OncotypeDx in breast cancerOncotypeDx in breast cancer Low score for ER+ node - = hormonal rxLow score for ER+ node - = hormonal rx

KRAS in colorectal cancerKRAS in colorectal cancer WT KRAS = cetuximab or panitumumabWT KRAS = cetuximab or panitumumab

EGFR mutation or amplification in NSCLCEGFR mutation or amplification in NSCLC EGFR inhibitorEGFR inhibitor

These Diagnostics Have These Diagnostics Have Medical UtilityMedical Utility

They inform therapeutic decision-They inform therapeutic decision-making leading to improved patient making leading to improved patient outcomeoutcome

Tests with medical utility help patients Tests with medical utility help patients and may reduce medical costsand may reduce medical costs

Tests correlated with outcome that are Tests correlated with outcome that are not actionable may increase medical not actionable may increase medical costs without helping patients costs without helping patients

Developing a test and demonstrating Developing a test and demonstrating medical utility for it is a complex medical utility for it is a complex multi-step process that generally multi-step process that generally requires prospective randomized requires prospective randomized clinical trialsclinical trials

Although the randomized clinical trial Although the randomized clinical trial remains of fundamental importance remains of fundamental importance for predictive genomic medicine, for predictive genomic medicine, some of the conventional wisdom of some of the conventional wisdom of how to design and analyze rct’s how to design and analyze rct’s requires re-examinationrequires re-examination E.g. The concept of doing a rct of thousands of patients to E.g. The concept of doing a rct of thousands of patients to

answer a single question about average treatment effect answer a single question about average treatment effect for a heterogeneous target population no longer has an for a heterogeneous target population no longer has an adequate scientific basis in oncologyadequate scientific basis in oncology

Standard Approach is Standard Approach is Based on Assumptions Based on Assumptions

Qualitative treatment by subset Qualitative treatment by subset interactions are unlikelyinteractions are unlikely

i.e. if new treatment T is better than control i.e. if new treatment T is better than control C on average, it is better for all subsets of C on average, it is better for all subsets of patientspatients

““Costs” of over-treatment are less Costs” of over-treatment are less than “costs” of under-treatmentthan “costs” of under-treatment

Cancers of a primary site often Cancers of a primary site often represent a heterogeneous group of represent a heterogeneous group of diverse molecular diseases which diverse molecular diseases which vary fundamentally with regard to vary fundamentally with regard to the oncogenic mutations that cause the oncogenic mutations that cause

them, them, their responsiveness to specific drugstheir responsiveness to specific drugs

How Can We Develop New How Can We Develop New Drugs in a Manner More Drugs in a Manner More Consistent With Modern Consistent With Modern

Tumor Biology and ObtainTumor Biology and ObtainReliable Information About Reliable Information About What Regimens Work for What Regimens Work for What Kinds of Patients?What Kinds of Patients?

Predictive biomarkersPredictive biomarkers Measured before treatment to identify Measured before treatment to identify

who will benefit from a particular who will benefit from a particular treatmenttreatment

Prognostic biomarkersPrognostic biomarkers Measured before treatment to indicate Measured before treatment to indicate

long-term outcome for patients untreated long-term outcome for patients untreated or receiving standard treatmentor receiving standard treatment

Prognostic and Predictive Prognostic and Predictive Biomarkers in OncologyBiomarkers in Oncology

Single gene or protein measurementSingle gene or protein measurement ER protein expressionER protein expression HER2 amplificationHER2 amplification KRAS mutationKRAS mutation

Scalar index or classifier that Scalar index or classifier that summarizes expression levels of summarizes expression levels of multiple genesmultiple genes

Prospective Co-Prospective Co-Development of Drugs and Development of Drugs and

Companion DiagnosticsCompanion Diagnostics1.1. Develop a completely specified Develop a completely specified

genomic classifier of the patients likely genomic classifier of the patients likely to benefit from a new drugto benefit from a new drug

2.2. Establish analytical validity of the Establish analytical validity of the classifierclassifier

3.3. Use the completely specified classifier Use the completely specified classifier to design and analyze a focused to design and analyze a focused clinical trial to evaluate effectiveness clinical trial to evaluate effectiveness of the new treatment and how it of the new treatment and how it relates to the candidate biomarkerrelates to the candidate biomarker

Targeted (Enrichment) Targeted (Enrichment) Design Design

Restrict entry to the phase III trial based Restrict entry to the phase III trial based on the binary predictive classifieron the binary predictive classifier

Using phase II data, develop predictor of response to new drug

Develop Predictor of Response to New Drug

Patient Predicted Responsive

New Drug Control

Patient Predicted Non-Responsive

Off Study

Applicability of Targeted Applicability of Targeted DesignDesign

Primarily for settings where the Primarily for settings where the classifier is based on a single gene classifier is based on a single gene whose protein product is the target whose protein product is the target of the drugof the drug eg trastuzumab eg trastuzumab

Evaluating the Efficiency of Evaluating the Efficiency of Targeted DesignTargeted Design

Simon R and Maitnourim A. Evaluating the efficiency of Simon R and Maitnourim A. Evaluating the efficiency of targeted designs for randomized clinical trials. Clinical targeted designs for randomized clinical trials. Clinical Cancer Research 10:6759-63, 2004; Correction and Cancer Research 10:6759-63, 2004; Correction and supplement 12:3229, 2006supplement 12:3229, 2006

Maitnourim A and Simon R. On the efficiency of Maitnourim A and Simon R. On the efficiency of targeted clinical trials. Statistics in Medicine 24:329-targeted clinical trials. Statistics in Medicine 24:329-339, 2005.339, 2005.

reprints and interactive sample size calculations at reprints and interactive sample size calculations at http://linus.nci.nih.govhttp://linus.nci.nih.gov

Relative efficiency of targeted design Relative efficiency of targeted design depends on depends on proportion of patients test positiveproportion of patients test positive effectiveness of new drug (compared to effectiveness of new drug (compared to

control) for test negative patientscontrol) for test negative patients Specificity of treatmentSpecificity of treatment Sensitivity of testSensitivity of test

When less than half of patients are test When less than half of patients are test positive and the drug has little or no positive and the drug has little or no benefit for test negative patients, the benefit for test negative patients, the targeted design requires dramatically targeted design requires dramatically fewer randomized patientsfewer randomized patients

Stratification DesignStratification Design

Develop Predictor of Response to New Rx

Predicted Non-responsive to New Rx

Predicted ResponsiveTo New Rx

ControlNew RX Control

New RX

Do not use the test to restrict eligibility, but to Do not use the test to restrict eligibility, but to structure a prospective analysis planstructure a prospective analysis plan

Having a prospective analysis plan is essentialHaving a prospective analysis plan is essential “ “Stratifying” (balancing) the randomization is useful to Stratifying” (balancing) the randomization is useful to

ensure that all randomized patients have tissue ensure that all randomized patients have tissue available but is not a substitute for a prospective available but is not a substitute for a prospective analysis plananalysis plan

Size the study for adequate evaluation of T vs C Size the study for adequate evaluation of T vs C separately by marker statusseparately by marker status

The purpose of the study is to evaluate the new The purpose of the study is to evaluate the new treatment overall and for the pre-defined subsets; not treatment overall and for the pre-defined subsets; not to modify or refine the classifier to modify or refine the classifier

The purpose is not to demonstrate that repeating the The purpose is not to demonstrate that repeating the classifier development process on independent data classifier development process on independent data results in the same classifierresults in the same classifier

R Simon. Using genomics in clinical trial R Simon. Using genomics in clinical trial design, Clinical Cancer Research 14:5984-design, Clinical Cancer Research 14:5984-93, 200893, 2008

R Simon. Designs and adaptive analysis R Simon. Designs and adaptive analysis plans for pivotal clinical trials of plans for pivotal clinical trials of therapeutics and companion diagnostics, therapeutics and companion diagnostics, Expert Opinion in Medical Diagnostics Expert Opinion in Medical Diagnostics 2:721-29, 20082:721-29, 2008

Analysis Plan BAnalysis Plan B

(Limited confidence in test)(Limited confidence in test)

Compare the new drug to the control overall Compare the new drug to the control overall for all patients ignoring the classifier.for all patients ignoring the classifier. If pIf poveralloverall ≤ 0.03 claim effectiveness for the ≤ 0.03 claim effectiveness for the

eligible population as a wholeeligible population as a whole Otherwise perform a single subset analysis Otherwise perform a single subset analysis

evaluating the new drug in the classifier + evaluating the new drug in the classifier + patientspatients If pIf psubset subset ≤ 0.02 claim effectiveness for the ≤ 0.02 claim effectiveness for the

classifier + patients.classifier + patients.

Sample size for Analysis Plan BSample size for Analysis Plan B

To have 90% power for detecting uniform To have 90% power for detecting uniform 33% reduction in overall hazard at 3% two-33% reduction in overall hazard at 3% two-sided level requires 297 events (instead of sided level requires 297 events (instead of 263 for similar power at 5% level)263 for similar power at 5% level)

If 25% of patients are positive, then when If 25% of patients are positive, then when there are 297 total events there will be there are 297 total events there will be approximately 75 events in positive patients approximately 75 events in positive patients 75 events provides 75% power for detecting 50% 75 events provides 75% power for detecting 50%

reduction in hazard at 2% two-sided significance reduction in hazard at 2% two-sided significance level level

By delaying evaluation in test positive patients, By delaying evaluation in test positive patients, 80% power is achieved with 84 events and 90% 80% power is achieved with 84 events and 90% power with 109 eventspower with 109 events

Analysis Plan CAnalysis Plan C

Test for difference (interaction) between Test for difference (interaction) between treatment effect in test positive patients treatment effect in test positive patients and treatment effect in test negative and treatment effect in test negative patients at an elevated level (e.g. .10)patients at an elevated level (e.g. .10)

If interaction is significant at that level If interaction is significant at that level then compare treatments separately for then compare treatments separately for test positive patients and test negative test positive patients and test negative patientspatients

Otherwise, compare treatments overallOtherwise, compare treatments overall

Sample Size Planning for Sample Size Planning for Analysis Plan CAnalysis Plan C

88 events in test + patients needed to 88 events in test + patients needed to detect 50% reduction in hazard at 5% two-detect 50% reduction in hazard at 5% two-sided significance level with 90% powersided significance level with 90% power

If 25% of patients are positive, when there If 25% of patients are positive, when there are 88 events in positive patients there are 88 events in positive patients there will be about 264 events in negative will be about 264 events in negative patientspatients 264 events provides 90% power for detecting 264 events provides 90% power for detecting

33% reduction in hazard at 5% two-sided 33% reduction in hazard at 5% two-sided significance levelsignificance level

Does the RCT Need to Be Significant Does the RCT Need to Be Significant Overall for the T vs C Treatment Overall for the T vs C Treatment

Comparison?Comparison?

No No That requirement has been traditionally That requirement has been traditionally

used to protect against data dredging. used to protect against data dredging. It is inappropriate for focused trials of a It is inappropriate for focused trials of a treatment with a companion test.treatment with a companion test.

Web Based Software for Web Based Software for Planning Clinical Trials of Planning Clinical Trials of

Treatments with a Treatments with a Candidate Predictive Candidate Predictive

BiomarkerBiomarker http://brb.nci.nih.gov http://brb.nci.nih.gov

It is difficult to have the right single It is difficult to have the right single completely defined predictive biomarker completely defined predictive biomarker identified and analytically validated by the identified and analytically validated by the time the pivotal trial of a new drug is time the pivotal trial of a new drug is ready to start accrualready to start accrual Changes in the way we do phase II trialsChanges in the way we do phase II trials Adaptive methods for the refinement and Adaptive methods for the refinement and

evaluation of predictive biomarkers in the evaluation of predictive biomarkers in the pivotal trials in a non-exploratory mannerpivotal trials in a non-exploratory manner

Use of archived tissues in focused Use of archived tissues in focused “prospective-retrospective” designs based on “prospective-retrospective” designs based on randomized pivotal trialsrandomized pivotal trials

Multiple Biomarker DesignMultiple Biomarker Design

Have identified K candidate binary Have identified K candidate binary classifiers Bclassifiers B11 , …, B , …, BKK thought to be thought to be predictive of patients likely to predictive of patients likely to benefit from T relative to Cbenefit from T relative to C

Eligibility not restricted by Eligibility not restricted by candidate classifierscandidate classifiers

For notation let BFor notation let B0 0 denote the denote the classifier with all patients positive classifier with all patients positive

Test T vs C restricted to patients positive for Test T vs C restricted to patients positive for BBkk for k=0,1,…,K for k=0,1,…,K Let S(BLet S(Bkk) be log partial likelihood ratio statistic for ) be log partial likelihood ratio statistic for

treatment effect in patients positive for treatment effect in patients positive for BBkk (k=1, (k=1,…,K) …,K)

Let S* = max{S(BLet S* = max{S(Bkk)} , k* = argmax{S(B)} , k* = argmax{S(Bkk)} )} For a global test of significanceFor a global test of significance

Compute null distribution of S* by permuting Compute null distribution of S* by permuting treatment labelstreatment labels

If the data value of S* is significant at 0.05 level, If the data value of S* is significant at 0.05 level, then claim effectiveness of T for patients positive then claim effectiveness of T for patients positive for Bfor Bk*k*

Let S* = max{S(BLet S* = max{S(Bkk)} , k* = argmax{S(B)} , k* = argmax{S(Bkk)} )} in actual datain actual data

The new treatment is superior to control for The new treatment is superior to control for the population defined by k* the population defined by k*

Repeating the analysis for bootstrap Repeating the analysis for bootstrap samples of cases providessamples of cases provides an estimate of the stability of k* (the indication)an estimate of the stability of k* (the indication) an interval estimate of S* (the size of treatment an interval estimate of S* (the size of treatment

effect for the size of treatment effect in the effect for the size of treatment effect in the target population)target population)

Adaptive Signature Adaptive Signature DesignDesign

Boris Freidlin and Boris Freidlin and Richard SimonRichard Simon

Clinical Cancer Research 11:7872-8, Clinical Cancer Research 11:7872-8, 20052005

Adaptive Signature DesignAdaptive Signature DesignEnd of Trial AnalysisEnd of Trial Analysis

Compare E to C for Compare E to C for all patientsall patients at at significance level αsignificance level α00 (eg 0.04) (eg 0.04) If overall HIf overall H00 is rejected, then claim is rejected, then claim

effectiveness of E for eligible patientseffectiveness of E for eligible patients OtherwiseOtherwise

Otherwise:Otherwise: Using only the first half of patients accrued Using only the first half of patients accrued

during the trial, develop a binary classifier that during the trial, develop a binary classifier that predicts the subset of patients most likely to predicts the subset of patients most likely to benefit from the new treatment T compared to benefit from the new treatment T compared to control Ccontrol C

Compare T to C for patients accrued in second Compare T to C for patients accrued in second stage who are predicted responsive to T based stage who are predicted responsive to T based on classifier on classifier

Perform test at significance level 1- αPerform test at significance level 1- α00 (eg 0.01) (eg 0.01)

If HIf H00 is rejected, claim effectiveness of T for subset is rejected, claim effectiveness of T for subset defined by classifierdefined by classifier

Treatment effect restricted to subset.Treatment effect restricted to subset.10% of patients sensitive, 10 sensitivity genes, 10,000 genes, 400 10% of patients sensitive, 10 sensitivity genes, 10,000 genes, 400

patients.patients.

TestTest PowerPower

Overall .05 level testOverall .05 level test 46.746.7

Overall .04 level testOverall .04 level test 43.143.1

Sensitive subset .01 level testSensitive subset .01 level test(performed only when overall .04 level test is negative)(performed only when overall .04 level test is negative)

42.242.2

Overall adaptive signature design Overall adaptive signature design 85.385.3

Cross-Validated Cross-Validated Adaptive Signature Adaptive Signature

DesignDesign

Freidlin B, Jiang W, Simon RFreidlin B, Jiang W, Simon RClinical Cancer Research 16(2) 2010Clinical Cancer Research 16(2) 2010

Prediction Based Analysis Prediction Based Analysis of Clinical Trialsof Clinical Trials

Using cross-validation we can Using cross-validation we can evaluate our methods for analysis of evaluate our methods for analysis of clinical trials, including complex clinical trials, including complex subset analysis algorithms, in terms subset analysis algorithms, in terms of their effect on improving patient of their effect on improving patient outcome via informing therapeutic outcome via informing therapeutic decision makingdecision making

This approach can be used with any This approach can be used with any set of candidate predictor variablesset of candidate predictor variables

Define an algorithm A for developing a Define an algorithm A for developing a classifier of whether patients benefit classifier of whether patients benefit preferentially from a new treatment T preferentially from a new treatment T relative to Crelative to C

For patients with covariate vector x, the For patients with covariate vector x, the algorithm predicts preferred treatmentalgorithm predicts preferred treatment

Applying A to a training dataset Applying A to a training dataset DD provides a classifier model M(A, provides a classifier model M(A, D)D) R(x |M(A, R(x |M(A, D)D) ) = T ) = T R(x | R(x | DD) = C) = C

At the conclusion of the trial randomly partition At the conclusion of the trial randomly partition the patients into K approximately equally sized the patients into K approximately equally sized sets Psets P11 , … , P , … , P1010

Let DLet D-i-i denote the full dataset minus data for denote the full dataset minus data for patients in Ppatients in Pii

Using K-fold complete cross-validation, omit Using K-fold complete cross-validation, omit patients in Ppatients in Pii

Apply the defined algorithm to analyze the data Apply the defined algorithm to analyze the data in Din D-i -i to obtain a classifier Mto obtain a classifier M-i-i

For each patient j in PFor each patient j in Pii record the treatment record the treatment recommendationrecommendation i.e. Ri.e. Rjj=T or R=T or Rjj=C=C

Repeat the above for all K loops of Repeat the above for all K loops of the cross-validationthe cross-validation

All patients have been classified as All patients have been classified as what their optimal treatment is what their optimal treatment is predicted to be predicted to be

Let Let SSTT denote the set of patients for whom denote the set of patients for whom treatment T is predicted optimal i.e. treatment T is predicted optimal i.e. SSTT = {i : = {i : RRjj=T}=T}

Compare outcomes for patients in Compare outcomes for patients in SS who actually who actually received T to those in received T to those in SS who actually received C who actually received C Let zLet zTT= standardized log-rank statistic = standardized log-rank statistic

Let Let SSCC denote the set of patients for whom denote the set of patients for whom treatment C is predicted optimal i.e. treatment C is predicted optimal i.e. SSCC = {i : = {i : RRjj=C}=C}

Compare outcomes for patients in Compare outcomes for patients in SSCC who actually who actually received T to those in received T to those in SS who actually received C who actually received C Let zLet zCC = standardized log-rank statistic = standardized log-rank statistic

Test of Significance for Effectiveness of T vs Test of Significance for Effectiveness of T vs C C

Compute statistical significance of zCompute statistical significance of zTT and and zzCC by randomly permuting treatment by randomly permuting treatment labels and repeating the entire procedurelabels and repeating the entire procedure Do this 1000 or more times to generate the Do this 1000 or more times to generate the

permutation null distribution of treatment permutation null distribution of treatment effect for the patients in each subseteffect for the patients in each subset

The significance test based on comparing The significance test based on comparing T vs C for the adaptively defined subset is T vs C for the adaptively defined subset is the basis for demonstrating that T is more the basis for demonstrating that T is more effective than C for some patients.effective than C for some patients.

Although there is less certainty about Although there is less certainty about which patients actually benefit, which patients actually benefit, classification may be substantially greater classification may be substantially greater than for the standard clinical trial in than for the standard clinical trial in which all patients are classified based on which all patients are classified based on results of testing the single overall null results of testing the single overall null hypothesis hypothesis

70% Response to T in Sensitive Patients70% Response to T in Sensitive Patients25% Response to T Otherwise25% Response to T Otherwise

25% Response to C25% Response to C20% Patients Sensitive20% Patients Sensitive

ASDASD CV-ASDCV-ASD

Overall 0.05 TestOverall 0.05 Test 0.4860.486 0.5030.503

Overall 0.04 TestOverall 0.04 Test 0.4520.452 0.4710.471

Sensitive Subset Sensitive Subset 0.01 Test0.01 Test

0.2070.207 0.5880.588

Overall PowerOverall Power 0.5250.525 0.7310.731

Expected 5-Year DFS Using Expected 5-Year DFS Using New AlgorithmNew Algorithm

Let S(T) = observed 5-year DFS for Let S(T) = observed 5-year DFS for patients in patients in SSTT who received treatment T who received treatment T mmTT such patients such patients

Let S(C) = observed K-year DFS for Let S(C) = observed K-year DFS for patients in patients in SSCC who received treatment C who received treatment C mmCC such patients such patients

Expected K-Year DFS using new algorithm Expected K-Year DFS using new algorithm {m{mT T S(T) + mS(T) + mC C S(C)}/{mS(C)}/{mT T + m+ mCC} } Confidence limits for this estimate can be Confidence limits for this estimate can be

obtained by bootstrapping the complete obtained by bootstrapping the complete cross-validation procedurecross-validation procedure

Expected 5-Year DFS Using Expected 5-Year DFS Using Standard AnalysisStandard Analysis

If the overall null hypothesis is not If the overall null hypothesis is not rejectedrejected Expected 5-Year DFS is the observed 5-Expected 5-Year DFS is the observed 5-

year DFS in the control groupyear DFS in the control group

If the overall null hypothesis is If the overall null hypothesis is rejectedrejected Expected 5-Year DFS is the observed 5-Expected 5-Year DFS is the observed 5-

year DFS in T group year DFS in T group

By applying the analysis algorithm to the By applying the analysis algorithm to the full RCT dataset D, recommendations are full RCT dataset D, recommendations are developed for how future patients should developed for how future patients should be treatedbe treated

R(x|D) for all x vectors.R(x|D) for all x vectors.

The stability of the recommendations can The stability of the recommendations can be evaluated based on the distribution of be evaluated based on the distribution of R(x|D(b)) for non-parametric bootstrap R(x|D(b)) for non-parametric bootstrap samples D(b) from the full dataset D.samples D(b) from the full dataset D.

With Binary OutcomeWith Binary Outcome

ffjj(x) = probability of response for patient (x) = probability of response for patient with covariate vector x who receives rx jwith covariate vector x who receives rx j

Fit separately to data for patients in each Fit separately to data for patients in each treatment group in the training set treatment group in the training set

Logistic regression, stepwise logistic regression, L1 Logistic regression, stepwise logistic regression, L1 penalized logistic regression, CART, random forest, penalized logistic regression, CART, random forest, etcetc

Fit jointly for patients in both treatment groups Fit jointly for patients in both treatment groups combined in the training setcombined in the training set

Logistic model including treatment, selected main Logistic model including treatment, selected main effects, and covariates with large interactions effects, and covariates with large interactions

ˆ ˆEstimate ( ) and ( ) for all covariate vectors

Recommend T for patients in whom

ˆ ˆ( ) ( ) ( )

where ( ) reflects side effects, expense or

inconvenience of T relative to C

Otherwise, recommend

T C

T C

f x f x

f x f x x

x

C

Biotechnology Has Forced Biotechnology Has Forced Biostatistics to Focus on Biostatistics to Focus on

Prediction Prediction This has led to many exciting This has led to many exciting

methodological developments methodological developments p>>n problems in which number of genes p>>n problems in which number of genes

is much greater than the number of casesis much greater than the number of cases Statistics has over-focused on Statistics has over-focused on

inference. Many of the methods and inference. Many of the methods and much of the conventional wisdom of much of the conventional wisdom of statistics are based on inference statistics are based on inference problems and are not applicable to problems and are not applicable to prediction problemsprediction problems

Prediction Based Clinical Prediction Based Clinical TrialsTrials

New methods for determining from New methods for determining from RCTs which patients, if any, benefit RCTs which patients, if any, benefit from new treatments can be from new treatments can be evaluated directly using the actual evaluated directly using the actual RCT data in a manner that separates RCT data in a manner that separates model development from model model development from model evaluation, rather than basing evaluation, rather than basing treatment recommendations on the treatment recommendations on the results of a single hypothesis test. results of a single hypothesis test.

Prediction Based Clinical Prediction Based Clinical TrialsTrials

Using cross-validation we can Using cross-validation we can evaluate new methods for analysis of evaluate new methods for analysis of clinical trials in terms of their clinical trials in terms of their intended use which is informing intended use which is informing therapeutic decision makingtherapeutic decision making

AcknowledgementsAcknowledgements

Boris FreidlinBoris Freidlin Yingdong ZhaoYingdong Zhao Wenyu JiangWenyu Jiang Aboubakar MaitournamAboubakar Maitournam