Prior sensitivity and model choice · DIC1 = 271 DIC2 = 98 DIC1 - DIC2 = 172 Conclude: Overwhelming...
Transcript of Prior sensitivity and model choice · DIC1 = 271 DIC2 = 98 DIC1 - DIC2 = 172 Conclude: Overwhelming...
Sensitivity DIC
Prior sensitivity and model choice
Alex Cook
Week 9
Alex Cook, ST5219, Bayesian Hierarchical Modelling 1/30
Sensitivity DIC
HIV trial
x = 51
n = 8197
x ∼ Bin(n, p)
p ∼ Be(1, 1)
E(p) = 0.5
V(p) ≈ 0.292
p
dens
ity
0% 1% 2%
Alex Cook, ST5219, Bayesian Hierarchical Modelling 2/30
Sensitivity DIC
HIV trial
x = 51
n = 8197
x ∼ Bin(n, p)
p ∼ Be(1, 1)
E(p) = 0.5
V(p) ≈ 0.292
p
dens
ity
0% 1% 2%
Alex Cook, ST5219, Bayesian Hierarchical Modelling 3/30
Sensitivity DIC
HIV trial
x = 51
n = 8197
x ∼ Bin(n, p)
p ∼ Be(1, 97)
E(p) = 0.01
V(p) = 0.012
p
dens
ity
0% 1% 2%
Alex Cook, ST5219, Bayesian Hierarchical Modelling 4/30
Sensitivity DIC
HIV trial
x = 51
n = 8197
x ∼ Bin(n, p)
p ∼ Be(1, 97)
E(p) = 0.01
V(p) = 0.012
p
dens
ity
0% 1% 2%
Alex Cook, ST5219, Bayesian Hierarchical Modelling 5/30
Sensitivity DIC
HIV trial
x = 51
n = 8197
x ∼ Bin(n, p)
p ∼ Be(90, 809)
E(p) = 0.1
V(p) = 0.012
p
dens
ity
0% 1% 2%
Alex Cook, ST5219, Bayesian Hierarchical Modelling 6/30
Sensitivity DIC
HIV trial
x = 51
n = 8197
x ∼ Bin(n, p)
p ∼ Be(90, 809)
E(p) = 0.1
V(p) = 0.012
p
dens
ity
0% 1% 2%
Alex Cook, ST5219, Bayesian Hierarchical Modelling 7/30
Sensitivity DIC
HIV trial
x = 51
n = 8197
x ∼ Bin(n, p)
p ∼ Be(90, 809)
E(p) = 0.1
V(p) = 0.012
p
dens
ity
0% 10% 20%
Alex Cook, ST5219, Bayesian Hierarchical Modelling 8/30
Sensitivity DIC
Take home
Lesson learnt
Informative priors can have big effects on posteriors
Alex Cook, ST5219, Bayesian Hierarchical Modelling 9/30
Sensitivity DIC
Simulated Poisson data
set.seed(54321)
n=100
lambda=10
x=rpois(n,lambda)
dlambda=0.01
lambda vector
=seq(1,20,dlambda)
x
Fre
quen
cy0 10 20
010
20
Alex Cook, ST5219, Bayesian Hierarchical Modelling 10/30
Sensitivity DIC
Simulated Poisson data
> prior1=dunif(lambda vector,0,100)
> logprior1=log(prior1)
> a=logprior1
> for(i in 1:n)a=a+
dpois(x[i],lambda vector,log=TRUE)
> a=exp(a-max(a));posterior1=a(sum(a)*dlambda)
> plot(lambda vector,posterior1,type=’l’,
xlab=expression(lambda),ylab=’density’)
> lines(lambda vector,prior1,lty=2)
Alex Cook, ST5219, Bayesian Hierarchical Modelling 11/30
Sensitivity DIC
Simulated Poisson data
prior1=dunif(lambda vector,0,100)
0 5 10 15 20
λ
dens
ity
0.0
0.5
1.0
Alex Cook, ST5219, Bayesian Hierarchical Modelling 12/30
Sensitivity DIC
Simulated Poisson data
prior2=dexp(lambda vector,0.01)
0 5 10 15 20
λ
dens
ity
0.0
0.5
1.0
Alex Cook, ST5219, Bayesian Hierarchical Modelling 13/30
Sensitivity DIC
Simulated Poisson data
prior3=dnorm(lambda vector,0,100)
0 5 10 15 20
λ
dens
ity
0.0
0.5
1.0
Alex Cook, ST5219, Bayesian Hierarchical Modelling 14/30
Sensitivity DIC
Take home
Lesson learnt
Informative priors can have big effects on posteriors
Non-informative priors: form doesn’t matter much
Alex Cook, ST5219, Bayesian Hierarchical Modelling 15/30
Sensitivity DIC
Classical model selection
yi ∼ N(µi , σ2)
M1 : µi = a + bxi
M2 : µi = a + bxi + cx2i
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
2 3 4 5 6
100
110
120
130
140
x
y
You were probably taught. . .
NHST of H0 : c = 0, if p < α choose M2
Alex Cook, ST5219, Bayesian Hierarchical Modelling 16/30
Sensitivity DIC
Classical model selection
yi ∼ N(µi , σ2)
M1 : µi = a + bxi
M2 : µi = a + bxi + cx2i
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
2 3 4 5 6
100
110
120
130
140
x
y
You were probably taught. . .
NHST of H0 : c = 0, if p < α choose M2
Alex Cook, ST5219, Bayesian Hierarchical Modelling 16/30
Sensitivity DIC
Classical model selection
yi ∼ N(µi , σ2)
M1 : µi = a + bxi
M2 : µi = a + c exp(xi)
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
2 3 4 5 6
100
110
120
130
140
x
y
What to do now?
NHST of H0 :?
Alex Cook, ST5219, Bayesian Hierarchical Modelling 17/30
Sensitivity DIC
Classical model selection
yi ∼ N(µi , σ2)
M1 : µi = a + bxi
M2 : µi = a + c exp(xi)
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●●
●
●
●● ●
●●
● ●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●●
●
●
●
●
●
●
● ●●
●●
●
●
●●
●
●
● ●
●
●
●
●
●
●
2 3 4 5 6
100
110
120
130
140
x
y
What to do now?
NHST of H0 :?
Alex Cook, ST5219, Bayesian Hierarchical Modelling 17/30
Sensitivity DIC
Classical model selection
Alex Cook, ST5219, Bayesian Hierarchical Modelling 18/30
Sensitivity DIC
Bayesian model selection
DIC:
Deviance: D(θ) = −2 log f (data|θ)
Posterior mean deviance: D̄ =∫
D(θ)f (θ|data) dθ
Effective # parameters: pD = D̄ − D(θ̄)
DIC: DIC = D̄ + pD
Spiegelhalter et al (2002, J Roy Stat Soc ser B64:583–639) recommend same scale as AIC
Alex Cook, ST5219, Bayesian Hierarchical Modelling 19/30
Sensitivity DIC
Bayesian model selection
DIC:
Deviance: D(θ) = −2 log f (data|θ)
Posterior mean deviance: D̄ =∫
D(θ)f (θ|data) dθ
Effective # parameters: pD = D̄ − D(θ̄)
DIC: DIC = D̄ + pD
Spiegelhalter et al (2002, J Roy Stat Soc ser B64:583–639) recommend same scale as AIC
Alex Cook, ST5219, Bayesian Hierarchical Modelling 19/30
Sensitivity DIC
Radish data
days post sowing
% in
fect
ed
0 7 14 21
0
20
40
60
80
100
Alex Cook, ST5219, Bayesian Hierarchical Modelling 20/30
Sensitivity DIC
Radish data
Model 1: no difference model
Xi ∼ Bin(ni , p)
p ∼ Be(1, 1)
Alex Cook, ST5219, Bayesian Hierarchical Modelling 21/30
Sensitivity DIC
Radish data
Model 2: hierarchical model
Xi ∼ Bin(ni , pi)
pi ∼ Be(a, b)
a ∼ exp(0.0001)
b ∼ exp(0.0001)
Alex Cook, ST5219, Bayesian Hierarchical Modelling 22/30
Sensitivity DIC
Calculating the DIC
logposterior1=function(current,data){
current$loglikelihood =sum(dbinom(data$I,data$N,current$p,log=TRUE))
current$D = -2*current$loglikelihoodcurrent$logprior = dbeta(current$p,1,1,log=TRUE)current$logposterior = current$logprior +
current$loglikelihoodcurrent
}
Alex Cook, ST5219, Bayesian Hierarchical Modelling 23/30
Sensitivity DIC
Calculating the DIC
phat=mean(dump$p)current$p=phatcurrent=logposterior1(current,data)Dbar1=mean(dump$D)pD1=Dbar1-current$D
DIC1=Dbar1+pD1
Alex Cook, ST5219, Bayesian Hierarchical Modelling 24/30
Sensitivity DIC
Calculating the DIC
logposterior2=function(current,data){
current$loglikelihood =sum(dbinom(data$I,data$N,current$p,log=TRUE))
current$D = -2*current$loglikelihoodcurrent$logprior =
sum(dbeta(current$p,current$a,current$b,log=TRUE))+dexp(current$a,0.001,log=TRUE)dexp(current$b,0.001,log=TRUE)
current$logposterior = current$logprior +current$loglikelihood
current
}
Alex Cook, ST5219, Bayesian Hierarchical Modelling 25/30
Sensitivity DIC
Calculating the DIC
phat=c();for(i in 1:13)phat[i]=mean(dump$p[,i])current$p=phatcurrent=logposterior2(current,data)Dbar2=mean(dump$D)pD2=Dbar2-current$D
DIC2=Dbar2+pD2
Alex Cook, ST5219, Bayesian Hierarchical Modelling 26/30
Sensitivity DIC
Fitted models
0.70 0.75 0.80 0.85 0.90 0.95 1.00
020
4060
8010
0
probability of disease
dens
ity
Alex Cook, ST5219, Bayesian Hierarchical Modelling 27/30
Sensitivity DIC
Fitted models
100 150 200 250
0.0
0.2
0.4
0.6
0.8
deviance
dens
ity
Alex Cook, ST5219, Bayesian Hierarchical Modelling 28/30
Sensitivity DIC
Comparing the DIC
DIC1 = 271
DIC2 = 98
DIC1 - DIC2 = 172
Conclude:
Overwhelming evidence against homogeneous diseaserates
Alex Cook, ST5219, Bayesian Hierarchical Modelling 29/30
Sensitivity DIC
Better Bayesian model selection
Alex Cook, ST5219, Bayesian Hierarchical Modelling 30/30