BLINDED EVALUATIONS OF EFFECT SIZES IN CLINICAL TRIALS ... 2013 Turkoz.pdf · endpoints in addition...
Transcript of BLINDED EVALUATIONS OF EFFECT SIZES IN CLINICAL TRIALS ... 2013 Turkoz.pdf · endpoints in addition...
BASS XX
BLINDED EVALUATIONS OF EFFECT SIZES IN CLINICAL TRIALS COMPARISONS BETWEEN BAYESIAN AND EM ANALYSES
1 Nov 2013
IBRAHIM TURKOZ PhD
Janssen Research amp Development LLC NJ
BASS XX
BASS XX
Introduction
Background
Literature Review
Objective
Methods
EM Algorithm
Bayesian Approaches
Model Selection
Simulations
Set-up
Results
Application
Discussions and Conclusions
Questions
Outline
Nov 2013 2 Outline
BASS XX
Clinical trials can have a major impact on public health
Required information for designing a clinical trial is not fully
available
Ongoing monitoring of accumulating data is necessary
Interim analyses are common
Usually costly
Requires unblinding treatment assignments
Statistical implications
We will investigate the use of the accumulated data for purposes
of modifying aspects of the study design IN BLINDED
SETTING
Background
3 Introduction Nov 2013
BASS XX
Background Motivating Question MDD Trial[1]
4 Introduction
Additional AD Treatment
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
Nov 2013
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 5 Introduction
Additional AD Treatment
RANDOMIZATION for
those who did not respond
6 weeks long but the primary time
point was Week 4
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 6 Introduction
Additional AD Treatment
RANDOMIZATION for
Patients with Caroll
Depression Score gt20
And CGI-S4
6 weeks long but the primary time
point was Week 4
Outcome Parameters
Clinician Rated Patient Reported
bullHRSD-17 (primary efficacy
scale)
bullCGI-S
bullMost Troubling Symptoms
(MTS) or PaRTS-D
bullPatient Global Impression of
Severity PGIS
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial
Nov 2013 7 Introduction
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Introduction
Background
Literature Review
Objective
Methods
EM Algorithm
Bayesian Approaches
Model Selection
Simulations
Set-up
Results
Application
Discussions and Conclusions
Questions
Outline
Nov 2013 2 Outline
BASS XX
Clinical trials can have a major impact on public health
Required information for designing a clinical trial is not fully
available
Ongoing monitoring of accumulating data is necessary
Interim analyses are common
Usually costly
Requires unblinding treatment assignments
Statistical implications
We will investigate the use of the accumulated data for purposes
of modifying aspects of the study design IN BLINDED
SETTING
Background
3 Introduction Nov 2013
BASS XX
Background Motivating Question MDD Trial[1]
4 Introduction
Additional AD Treatment
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
Nov 2013
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 5 Introduction
Additional AD Treatment
RANDOMIZATION for
those who did not respond
6 weeks long but the primary time
point was Week 4
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 6 Introduction
Additional AD Treatment
RANDOMIZATION for
Patients with Caroll
Depression Score gt20
And CGI-S4
6 weeks long but the primary time
point was Week 4
Outcome Parameters
Clinician Rated Patient Reported
bullHRSD-17 (primary efficacy
scale)
bullCGI-S
bullMost Troubling Symptoms
(MTS) or PaRTS-D
bullPatient Global Impression of
Severity PGIS
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial
Nov 2013 7 Introduction
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Clinical trials can have a major impact on public health
Required information for designing a clinical trial is not fully
available
Ongoing monitoring of accumulating data is necessary
Interim analyses are common
Usually costly
Requires unblinding treatment assignments
Statistical implications
We will investigate the use of the accumulated data for purposes
of modifying aspects of the study design IN BLINDED
SETTING
Background
3 Introduction Nov 2013
BASS XX
Background Motivating Question MDD Trial[1]
4 Introduction
Additional AD Treatment
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
Nov 2013
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 5 Introduction
Additional AD Treatment
RANDOMIZATION for
those who did not respond
6 weeks long but the primary time
point was Week 4
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 6 Introduction
Additional AD Treatment
RANDOMIZATION for
Patients with Caroll
Depression Score gt20
And CGI-S4
6 weeks long but the primary time
point was Week 4
Outcome Parameters
Clinician Rated Patient Reported
bullHRSD-17 (primary efficacy
scale)
bullCGI-S
bullMost Troubling Symptoms
(MTS) or PaRTS-D
bullPatient Global Impression of
Severity PGIS
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial
Nov 2013 7 Introduction
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question MDD Trial[1]
4 Introduction
Additional AD Treatment
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
Nov 2013
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 5 Introduction
Additional AD Treatment
RANDOMIZATION for
those who did not respond
6 weeks long but the primary time
point was Week 4
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 6 Introduction
Additional AD Treatment
RANDOMIZATION for
Patients with Caroll
Depression Score gt20
And CGI-S4
6 weeks long but the primary time
point was Week 4
Outcome Parameters
Clinician Rated Patient Reported
bullHRSD-17 (primary efficacy
scale)
bullCGI-S
bullMost Troubling Symptoms
(MTS) or PaRTS-D
bullPatient Global Impression of
Severity PGIS
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial
Nov 2013 7 Introduction
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 5 Introduction
Additional AD Treatment
RANDOMIZATION for
those who did not respond
6 weeks long but the primary time
point was Week 4
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 6 Introduction
Additional AD Treatment
RANDOMIZATION for
Patients with Caroll
Depression Score gt20
And CGI-S4
6 weeks long but the primary time
point was Week 4
Outcome Parameters
Clinician Rated Patient Reported
bullHRSD-17 (primary efficacy
scale)
bullCGI-S
bullMost Troubling Symptoms
(MTS) or PaRTS-D
bullPatient Global Impression of
Severity PGIS
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial
Nov 2013 7 Introduction
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question MDD Trial[1]
Nov 2013 6 Introduction
Additional AD Treatment
RANDOMIZATION for
Patients with Caroll
Depression Score gt20
And CGI-S4
6 weeks long but the primary time
point was Week 4
Outcome Parameters
Clinician Rated Patient Reported
bullHRSD-17 (primary efficacy
scale)
bullCGI-S
bullMost Troubling Symptoms
(MTS) or PaRTS-D
bullPatient Global Impression of
Severity PGIS
Placebo + AD Treatment as usual Placebo + AD Treatment as usual
Active + AD Treatment as usual
1 Mahmoud AR Pandina JG Turkoz I Kosik-Gonzalez C Canuso MC Kujawa JM and Gharabawi GM Risperidone for treatment-refractory major depressive disorder
Annals of Internal Medicine 2007 147 593-602
Prior AD Treatment
BASS XX
Background Motivating Question MDD Trial
Nov 2013 7 Introduction
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question MDD Trial
Nov 2013 7 Introduction
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 8 Introduction
Psychosis
Mania Depression
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 9 Introduction
Psychosis
Mania Depression
PANSS
30-210
YMRS
0-60
HAM-D-21
0-52
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question Schizoaffective Trial[1]
Nov 2013 10 Introduction
BL Wk 6
n=95 PBO
n=216 PALI PR 3 - 12 mgd (start at 6 mg flexible)
Study 3002
N=311
Psychosis
Mania Depression
1 Canuso CM Lindenmayer JP Kosik-Gonzalez C Turkoz I Carothers J Bossie CA Schooler NR A randomized double-blind placebo-controlled study of 2 dose ranges of
paliperidone extended-release in the treatment of subjects with schizoaffective disorder J Clin Psychiatry 2010 May71(5)587-98
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 11 Introduction
Psychosis
Mania
Depression
Changes in PANSS
Total Score at Week 6
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 12 Introduction
Psychosis
Mania
Depression
YMRS
HAM-D-21
PRIMARY ENDPOINT SECONDARY ENDPOINTS
FOR REGISTRATION PURPOSES
Changes in PANSS
Total Score at Week 6
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question Schizoaffective Trial
Nov 2013 13 Introduction
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 14 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background Motivating Question ADHD Trial 1
Nov 2013 15 Introduction
1 Wigal SB Wigal T Schuck S Brams M Williamson D Armstrong RB and Starr HL Academic Behavioral and Cognitive Effects of OROSreg Methylphenidate on Older Children
with Attention-DeficitHyperactivity Disorder J Child Adolesc Psychopharmacol 2011 April 21(2) 121ndash131
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background
Nov 2013 16 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background
Nov 2013 17 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background
Nov 2013 18 Introduction
It is important to evaluate secondary endpoints to differentiate your
drug from other drugs in market place
Another symptom
Functionality
Medication Satisfaction
Certain Adverse Reactions
Cognition
Etc
Failure to consider important outcomes can limit the
conclusions of clinical trials
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Background
Nov 2013 19 Introduction
bull Examination of appropriate statistical methods to test secondary
endpoints requires careful consideration
bull Fixed sequence gate-keeping procedures [1234] are widely used
because of their ease of application and interpretation in many
circumstances
bull These gate-keeping procedures often require prospective ordering
of null hypotheses for secondary endpoints
bull Can we use information that is available to order the null
hypotheses based on interim effect sizes without breaking the
blind
1 Demitrienko A Tamhane A Bretz F Multiple testing problems in pharmaceutical statistics 2010 ChapmanampHallCRC Biostatistics Series
2 Westfall PH Krishen A Optimally weighted fixed sequence and gatekeeper multiple testing procedure J of Statistical Planining and Inference 99 (2001) 25-40
3 Weins B Dmitrienko A (2005) The fallback procedure for evaluating a single family of hypotheses Journal of Biopharmaceutical Statistics 15 929ndash942
4 Bretz F Maurer W Brannath W Posch M A graphical approach to sequentially rejective multiple test procedures Stat in Med 2009 28586ndash604
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Literature Review
Nov 2013 20 Introduction
bull EM algorithm to estimate the within group variance for sample size re-
estimation without unblinding at interim stages has been suggested [123]
bull Enrollment order of subjects and the randomization block sizes to estimate the
within group variance have also been used [4]
bull Risk of unblinding the treatment effect from blinded inferences based on
knowledge of the randomization block size for continuous and binary
outcomes [5]
bull The recent draft FDA guidance on adaptive designs [6] discusses possible
study design modifications such as selection andor order of secondary
endpoints in addition to sample size re-estimation
bull Blinded treatment effects for survival endpoints[7] were also examined
1Gould AL Shih WJ Sample size re-estimation without unblinding for normally distributed outcomes with unknown variance Communications in Statistics Theory and Methods
1992 21 2833ndash2853
2 Shih WJ Sample size reestimation in clinical trials In Biopharmaceutical sequential statistical applications Peace K (ed) Marcel Dekker New York 1992 285ndash301
3 Shih WJ Sample size reestimation for triple blind clinical trials Drug Information Journal 1993 27 761ndash764
4 Xing B Ganju J A method to estimate the variance of an endpoint from an on-going blinded trial Statistics in Medicine 2005 24 1807-1814
5 Miller F Friede T Kieser T Blinded assessment of treatment effects utilizing information about randomization block length Statistics in Medicine 2009 28 1690-1706
6 Guidance for industry Adaptive design clinical trials for drugs and biologics
httpwwwfdagovdownloadsDrugsGuidanceComplianceRegulatoryInformationGuidancesucm201790pdf
7 Xie J Quan H Zhang J Blinded assessment of treatment effects for survival endpoint in an ongoing trial Pharmaceutical Statistics 2012
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Objective
Nov 2013 21 Introduction
bull Instead of formal interim analyses on unblinded data use the
information from blinded data
bull Assess the feasibility of estimating magnitude of treatment effect
on various secondary endpoints in ongoing trial
bull Primary objective is to compare posterior ordering of the signal-
to-noise ratios (effect sizes) using available data
points
d
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Model
Nov 2013 Methods 22
( 1 01 12 ) is the response variable for the endpoint from
20 ~ (0 )
th thjijkY i n j k K k j
ijk k k ik ik ijk ky z N i i d
20 0
21 1
0
treatment group on subject
~ ( )
~ ( )
the mean placebo response
magnitude of the treatment effect
latent binary treatment assignment
We assume that lower scores a
i k k k
i k k k
k
k
ik
i
Y N
Y N
z
y
21 0~ ( )
re better and the average effect of the treatment is
k k k k
k
Y N
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Model
Nov 2013 Methods 23
For a two component univariate normal mixture model it is appropriate to specify the
complete likelihood function for a given response in the form
( |p y z
2 20 0 1 1 0
1 0 =p(z 1)
where ( ) ( ) is randomization weight
The signal-to-noise ratio for endpoint ( 1 )
) ( | ) (1 ) ( | )
Our objective is
k k kk d k K
p y p y
1 0
θ θ
θ θ θ
1 2
to compare the EM and Bayesian algorithm induced
posterior ordering of the signal-to-noise ratios of endpoints ( | )kP d d d y
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
EM Algorithm
Nov 2013 Methods 24
=p(z 1)
MMLEs are the most common way to estimate parameters of interest however MMLEs
often do not have closed form solutions EM seeks the MMLE by iterativel
1 0( | ) ( | ) (1 ) ( | )p y z p y p y 1 0θ θ θ
( )
y applying
Expectation and Maximizing steps
ˆE-Step log ( | ) |
M-Step the conditional expectation is maximized with respect to This yields the new estimates
tE p y z yθ θ
θ
( 1) ( 1)ˆ for and a distribution for t tz θ
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
EM Algorithm
Nov 2013 Methods 25
( ) ( ) 2( )( 1) ( ) 0
( ) ( )( ) 2( ) 2( )0 0
Conditional expectation of latent treatment assignment at iteration ( 1) is
ˆˆ ˆ( | )ˆ( 1| )
ˆˆ ˆ ˆ ˆ( | ) (1 ) ( | )
In the M step the mean pa
t t tt t
i t tt t t
t
N yP z y
N y N y
θ
20
( 1) ( 1) ( 1)( ) ( )0
( 1) ( 1)1 10
( 1) ( 1)( ) ( )
1
ˆˆ ˆrameters and variance parameter at iteration are ( 1)
ˆ ˆ ˆ( 0 | ) ( 1| )( )
ˆˆ
ˆ ˆ( 0 | ) ( 1| )
n nt t tt t
i ii it ti i
nt tt t
i i
i i
t
P z y y P z y y
P z y P z y
θ θ
θ θ
1)
1
( 1) ( 1) ( 1) ( 1)( ) 2 ( ) 21 0
2(t 1
ˆ ˆˆ ˆ( 1| )( ) ( 0 | )( )
ˆ
n
nt t t tt t
i ii i
i
P z y y P z y y
n
θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 26
The likelihood function of the mixture distribution and the prior distributions of and z
are combined to obtain the joint posterior distribution which is assumed to contain all the information
abou
θ
t the unknown parameters Using the Bayes theorem the posterior distribution satisfies
where
( | ) is the likelihood
( | ) is the prior probability of latent tre
( | ) ( | ) ( | ) ( )
p y z
p z
p z y p y z p z p
θ
θ
θ θ θ θ
atment assignment
( ) is the prior distribution of p θ θ
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 27
The conditional distribution of the treatment allocation is proportional
this reduces to latent treatment allocation probability in EM
Prior Distribution
( | ) ( | ) ( | ) ( )
( | ) ( | )
p z y p y z p z p
p z y p y z
θ θ θ θ
θ θ
20 0 0
21 0 1 1
2
s
in order to reduce sensitvity
For simplicity purposes we have
~ ( )
~ ( )
~ ( )
( )
j N
N y s
N y s
Gamma
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 28
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 29
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for treatment
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 30
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Likelihood for placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 31
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for treatment arm
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 32
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
i i
i i
z z
y
y y
s s
Prior for Placebo arm
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 33
Regardless of the structure of the prior distribution the posterior distribution does not have
a simple closed form In order to make inferences about the unknown parameters MCMC
methodologies are em
2 2 20 0 0
2
2 2
ployed to generate samples from the posterior distribution The joint
posterior probability distribution of these parameters satisfies
( | ) ( | ) ( ) ( )
1 1exp (
2
n
p z y p y z p z p
y
2 20 02
1 0
12 220 1 0 0
2 2 21 0
1) exp ( )
2
( ) ( ) 1exp exp exp( )
2 2
In computations =3 and small scale hyperparameter ( 001) are assumed
i i
i i
z z
y
y y
s s
Prior for Sigma
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach
Nov 2013 Methods 34
0 12 2 2
0 110
2 2 2 2 2 20 1 0 1
0
1 0 12
The posterior conditional distributions of parameters of interest in equation are given by
1~
1 1 1 1
( )
~
i i
i
i i
i
y z y y
s sN
n n
s s s s
y zy
sN
21
1 12 2 2 2
1 1
20
2 12
3
1
1 1
( )
~
i i
i i
i i
i
n
z z
s s
y z
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 35
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 36
We assume a trial with stratified randomization The sample is divided into subsamples on the
basis of stratifaction factor (covariate additional medication usage and groups)
Let denote ti
yes no
g
22
0 2
he known strata membership of the subject 1 if observation belongs to the
strata and 0 if it does not The joint posterior distribution then takes the form
1( | ) e
i
n
yes no
i g yes
p z y
202
1 01
2 20 12 0 0
02 2 21 00 01
12
2
1xp ( )
2
( ) ( )1exp ( ) exp exp
2 2 2
1exp( )
i
i i
i i
i i
i g
z g
g gi
z g
y
y yy
s s
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 37
202
2 20 02 2
02
Conditional posterior probability treatment assignment at each iteration is given by
1exp ( )
2 Yes strata
1 1exp ( ) exp ( )
2 2
( 1)
1exp (
2
i yes
i yes i
i
i
y
y y
P z
y
2
2 20 02 2
)
No strata1 1
exp ( ) exp ( )2 2
no
i no iy y
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate
Nov 2013 Methods 38
1 102 2 2
0 110
2 2 2 2 2 20 1 0 1
0
The posterior distribution for parameters of interest are computed to be
( ) ( ) ( )
1~
1 2 1 2
( )
~
i i yes no yes yes no no
i
i
yes
y z y yy
s sN
n n
s s s s
y
N
0
1 1 0 1 1 2 0 1
2 2 2 21 1
1 1 1 1 1 2 1 2
2 2 2 2 2 2 2 21 1 1 1
20 0
2
( )
1 1 ~
1 1 1 1
( ) (
~
i i i
i g yes i g no
noi i i i
i g i g i g i g
i i yes i i
z y zy y
s sN
z z z z
s s s s
y z y z
2
1 1 1 2
23
)
no
i g i g
n
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 39
0
[123]0
0 0
0
Denote the historical data by and current data by
[01] to control the influence of the historical data on the current study
1 approaches full barrowing from
0 approaches no ba
D D
D
0
0
0
rrowing from
The basic idea is to use the power parameter to control the influence of the historical data on the current study
Let assume
( | ) is the past data likelihood
D
L D
0 0 0 0 0
0 0
( | ) is the current data likelihood
( | ) ( | ) ( ) ( ) is the posterior given past likelihood
Then the full likelihood is proportional to ( | ) ( | )
L D
p D L D p p
p D L D
1 Ibrahim J Chen MH Power prior distributions for regression models Statistical Science 2000 1546-60
2 Duan Y Ye K Smith EP Evaluating water quality using power priors to incorporate historical information Environmetrics 2006 17 95-106
3 Hobbs BP Carlin BP Mandrekar S and Sargent DJ Hierarchical commensurate and power prior models for adaptive incorporation of historical information in
clinical trials Biometrics 2011 671047-1056
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 40
0
0 0
22
0 0 2 2
The joint posterior distribution can be extended to include the prior for
Think of as the historical effective sample size
The result takes the form
1 1( | ) exp (
2
n
y n
n
p z y y
0 0
0 0
0
20
1
21 0 1 1 2
02 21 10
21 0 1 1
21 1
0 0 0
0
)
( )1exp ( ) exp
2 2
( )exp
2
( )exp
i
i
i g
z g
p yes yes yes p yesi
z
p no no no p no
p
n y ny
s s
n y n
s s
n y
s
0
2 10 2
2 20
1 10 0
1exp( )
2
(1 )
p
a b
n
s
Prior for Power Parameter
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 41
0
0
1 2 20 0 0 1 1 12
0 00 2 20 1
0 1
The posterior distribution of parameters of interest is computed to be
( ) ( )(1 ) exp
2
( ) exp
2
aop yes yes p yesb
no no
y n y n
s s
y
0 0 0
0 0 0 0
21
21
0 0 1 1 1 1
2 2 20 11
00 1 0 1
2 2 2 2 2 20 1 0 1
( ) ( )( )
1~
p no
p yes yes p yes no no p noi i yes no
i
p p p p
n
s
y n y n y ny z
s sN
n n n nn n
s s s s
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Bayesian Approach with Covariate and a Power Prior
Nov 2013 Methods 42
0
0 0
0
0 1 1 1 1
2 21
1 1 1 1 1 1
2 2 2 21 1
The posterior distribution of parameters of interest is computed to be
( )( )
1~
(
~
i i
yes p yesi g
yesi i
p yes p yesi g i g
no
y zy n
sN
z zn n
s s
y
N
0
0 0
0
0 1 1 1 2
2 21
1 1 1 2 1 2
2 2 2 21 1
2 20 0
1 1 1 22
23
)( )
1
( ) ( )
~
i i
no p noi g
i i
p no p noi g i g
i i yes i i no
i g i g
n
zy n
s
z zn n
s s
y z y z
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Model Selection
Nov 2013 Methods 43
By contrast with frequentist analogues Bayesian model comparison distinguishes which likelihood and prior
combinations better fit the data These measures of performance can be used to choose a singl
[1]
e ldquobestrdquo model
or improve estimation via model averaging
The deviance information criterion (DIC) provides a natural measure of performance in this setting
Models with smaller DIC should be preferre
0 0
0 0
d to models with larger DIC
=2 ( ) since ( )
ˆˆ =2 ( ) ( )
ˆˆ 2 ( ) ( )
d
d
DIC p D
D D p D D
D D
Expectation over Expectation over
θ θ
1 DJ Spiegelhalter NG Best BP Carlin A Linde Bayesian measures of model complexity and fit (with discussion) Journal of the Royal Statistics
Society Series B (Statistical Methodology)2002 64 583ndash639
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Model Selection
Nov 2013 Methods 44
0
20
0 021
2
0
( )( ) this is a posterior expectation over ( )average after all the iterations
ˆ
ˆ averaged after all the iterations
updates at each iter
0 0ˆˆ2 ( ) ( )
i i
i
y zD E
DIC D D
20
0 21
ation
updates at each iteration
treatment assignment at each iteration
ˆˆ( )ˆˆ( ) this is a posterior expectation over it is using averaged parametersˆ
ˆ
i
i
i iz i
i
z latent
y zD E z
2
0
averaged after all the iterations
ˆ averaged after all the iterations
ˆ averaged after all the iterations
treatment assignment at each iterationiz latent
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
MCMC Gibbs Sampling
Nov 2013 Methods 45
An important feature of the Gibbs sampler is that each simulated posterior parameter value is always accepted
The main drawback of the Gibbs sampler in this setting is the lack of mixing
We employed a Metropolis-Hastings algorithm with an acceptance-rejection sampling
This involved subsampling 10 of the treatment and placebo group observations and proposing a label switch
1) Draw random candidate samples fromfor switching treatment
2) Generate from the Uniform (01) distribution
(1 ( ( 1))3) If make label change otherwise return to s
( 1)
i i
i i
u
p zu
p z
tep 1
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Set-Up
Nov 2013 Simulations 46
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt ~0 170
4 10 gt 05 gt ~0 72
5 10 gt 05 gt ~0 468
1d 3d2d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Set-Up
Nov 2013 Simulations 47
Sample Size and Power Computations using a Two-Sample t-test with Two-Sided Type I Error=005
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Set-Up
Nov 2013 Simulations 48
1 2 3
2 treatment arms with a 11 treatment allocation ratio
3 uncorrelated endpoints and
No missing data
For each case 20 data sets are generated ( 1 20)
Different prior for each of
k y y y
l l
the endpoints
20 0 0
21 0 1 1
2
2 20 0 1
Generate Input Data Sets
( ) ~ ( )
( ) ( ) ( ) ~ ( )
~ ( ) 123 1 20
Without loss of generality we assumed that
10 10
k k
k k
k k
k k k
k l N Y s
k l k l k l N Y s
IG b k l
Y s s
Scenario N
1 05 = 05 = 05 170
2 08 gt 05 gt 03 170
3 10 gt 05 gt 0 170
4 10 gt 05 gt 0 72
5 10 gt 05 gt 0 468
2d 3d1d
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Set-Up
Nov 2013 Simulations 49
2 20 1 Each true input clinical trial data set is based on ~ (1010 ) and with normal distribution with variance 10
and corresponding mean value to generate effect sizes for a given scenario
Initi
N
20
0
al values of ( ) for simulations are based on the true input data set parameters and the initial
power parameter estimate is given as 025 and 05
Posterior inferences are based on 1000 i
th
terations
The first 200 iterations are considered to be the burn in period for both EM and Bayesian algorithms
Every 4 point after the burn in period is stored for the Bayesian simulations to re
duce correlations
to emulate thinning process
Eight hundred simulation results in EM and 200 results in Bayesian methods are summarized
Posterior parameter estimates are summuarized
Empirical pr
obability of each triplet combination of effect size ordering is examined
In each case for a given data set the last 20 simulation results are kept for effect size orderings
For each of the study data sets a total of 20 20 20 8000 possible triplet combinations of orderings are available
For 20 different study data sets a total of 160000 possible orderings are available
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results
Nov 2013 Simulations 50
Posterior Probability of Effect Size Ordering by Scenario
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results
Nov 2013 Simulations 51
Posterior Distribution of Effect Sizes by Scenario Averaged Over 20 Data Sets
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results
Nov 2013 Simulations 52
Posterior Distribution of Effect Sizes by Scenario
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 53
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 54
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 55
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 56
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 1
Nov 2013 Simulations 57
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 58
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 2
Nov 2013 Simulations 59
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 60
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 3
Nov 2013 Simulations 61
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 62
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 4
Nov 2013 Simulations 63
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 64
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Simulation Results Scenario 5
Nov 2013 Simulations 65
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
MDD Trial
Nov 2013 Application 66
Primary Time Point
Week 4 or Week 6
Primary Endpoint
HRSD-17 or PaRTS-D
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
MDD Trial Results
Nov 2013 Application 67
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
MDD Trial Results
Nov 2013 Application 68
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
MDD Study Simulations
Nov 2013 Application 69
4 6
Decision Rule 1
Calculate the posterior probability that the HRSD 17 effect size at Week 4 is larger than that of Week 6
( ( ) ( ) | )
Decision Rule 2
Calculate 95 Credible Intervals for W
Wk WkP d H d H
Y
4 4 4
6 6 6
4
2 204 4 0 0
14 4
eek 4 and Week 6 effect sizes
( ) ( ) ( )
( ) ( ) ( )
Priors for HRSD 17 placebo and treatment means at Week 4 ( ) were assumed to be
( ) ~ ( 15 7 )
( ) ~
Wk Wk Wk
Wk Wk Wk
Wk
Wk
Wk
d H d H d H
d H d H d H
H
H N Y s
H
2 21 1( 12 7 )N Y s
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
MDD Study Simulations
Nov 2013 Application 70
Summary of Effect Sizes for HRSD-17 and PaRTS-D
At Week 4 EM Results were Not Available
Posterior Ordering of Effect Sizes
for HRSD-17
Week 4 Week 6
EM 00 1000
Bayesian 477 523
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
MDD Study Simulations
Nov 2013 Application 71
DIC Scores on HRSD-17 and PaRTS-D
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 72
We investigated whether or not the researchers can obtain reliable estimates of effect sizes
without knowing treatment assignments
Instead of relying on the clinicianrsquos subjective evaluations the
suggested methodologies
provide numerical assistance
We showed how to order secondary null hypotheses while the study is ongoing
without unblinding the treatments
without losing the validity
of the testing procedure
and with maintaining the integrity of the trial
In our simulations we used prior distributions whose hyper parameters reflect the true values
of the model parameters I
n MDD analyses we used prior distributions whose hyper parameters
reflect the original study design elements
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Discussion and Conclusion
Nov 2013 Discussion 73
The EM algorithm frequently failed to generate posterior parameter estimates
Both the Bayesian and the EM algorithms overestimated treatment differences and
standardized effect sizes
The Bayesian
algorithms performed better than existing EM algorithm counterparts in ordering
the standardized effect sizes
With the Bayesian algorithm the posterior probability for identifying the ground-truth ordering
increased both as a function of the effect size differences and as a function of the sample size
For large sample sizes the proportion of times the true ordering was selected was high (ab ove 35)
and the variability of standardized effect sizes was low
With the EM algorithm the probability for identifying the ground-truth ordering was low
(sometimes near zero) and the variability of
standardized effect sizes was high
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS
BASS XX
Discussion
Nov 2013 Discussion 74
THANK YOU
QUESTIONS