Confidence Region Hypothesis Testing
Transcript of Confidence Region Hypothesis Testing
-
8/13/2019 Confidence Region Hypothesis Testing
1/8
Confidence region Hypothesis testing
Posterior mean
Confidence level data Y parameter
Find a region C(Y) of the parameter based on posterior belongs to this region with prob 1-
Natural confidence region Highest posterior density (HPD)
C(Y) = : P(|Y) > K- chose K st. P(C(Y)|Y) = 1-
Optimal HPD gives the region with smallest volume for given
Bayesian depends on prior best one can do if prior is correct inference is relative easy conceptually if
one can compute posterior HPD is well-defined.
Frequentist: confidence region should cover true parameter with probability 1- (on larger) regardless
of prior. Harder problem
Normal known variance and unknown mean
Y ~ N(,2) ~N(0,
2)
Posterior: ~N(Y, 2)
Y= 2Y/(
2+
2)
-2=
-2
-2 when inf
hat
2 2
Y
-
8/13/2019 Confidence Region Hypothesis Testing
2/8
C= *YK, Y+ K+
Kis /2 quantile of Cis classical confidence interval
Full normal with unkown mean and unknown variance in 1-D
Y= {Yi} Yi ~N(,2) =,2) both and 2 are unknown
Likelihood: exponential family form
Sufficient stat: Ybar =( 1/n)SumYi
SY= Sum (YiYbar)2
P(Y|) = ProductP(Yi|)
= Product 1/*sqrt(2)+exp-(Yi)2/2
2}
1/nexp{-[sum(YiYbar)2+ n(Ybar)2+/22}
1/nexp{-[Sy +nYbar
2-2nYbar n
2+/2
2
[SY, nYbar2, 2nYbar, n]T[-1/22, -1/22, -1/22, -1/22]
Conjugate prior four parameter , , ,
P() (2)- exp{- *(-)2 +/22}
Posterior P(,2|Y) (2
2)^- exp* (-)
2 +/2
2}1/
nexp{-[SY+ n(Ybar)
2+/2
2}
(22)^- -n/2exp* (-)2 -SY- n(Ybar)2+/22}
Y= ( nYbar)/+n
=*,+ =* ,2]
Jeffreys prior
P(,2) ~1/
3
A more common prior (non-informative) is
P(,2) ~1/
2
Posterior
P(,2|Y) (
2)^- 1-n/2 exp{[ SY
2- n(Ybar)
2+/2
2}
-
8/13/2019 Confidence Region Hypothesis Testing
3/8
Interested in confident interval for look at the marginal posterior
P(|Y) = integrationP(,2|Y)d 2
[ SY2- n(Ybar)
2]^(-n/2)
n-1distribution
Bayesian confidence interval (credible interval)
C= [Ybart/2,n-1SY/sqrt(n(n-1)), Ybar + t/2,n-1SY/sqrt(n(n-1))]
Bayesian t-confidence region whether =0 or not
Hypothesis testing
Deciding about assumption of given observation
Null hypothesia Ho: 0
Vs alternative hypothesis H1: 1
Example: 0 = 0-
1 = R\{0}
Binary outcome
1: accet Null hypothesis
0 reject Null hypothesis
Possible outcomes
Decision\ Truth H0 is true H0 is false
Accept H0 Right decision Type II error
Reject H0 Type I error Right decision
Generally want probability of type I error small, say
-
8/13/2019 Confidence Region Hypothesis Testing
4/8
(,d) = 0 if d=I(0)
a0 if 0& d=0 (type I error)
a1 if 0& d=1 (type II error)
d{0,1} a0>=0, a1>=0
Bayes optimal estimator
1 if P(0|Y) >a1/(a0+a1)
0 otherwise
Normal-normal
: known variance unknown mean
~ N(0, 2) Y~N(,2)
Posterior ~N(Y, 2)
Y= 2Y/( 2+ 2)
0 = :
-
8/13/2019 Confidence Region Hypothesis Testing
5/8
Alternative: try to alleviate the provlem
Bayes factor (assuming prior of M0 = prior of M1)
B10= P(Y|M1)/P(Y|M0) = P(Y|1)/ P(Y|0)= integral 1P(Y|)P()/P(1)d-/integral0
P(Y|)P()/P(0)d)
B10=[ P(1|Y)/P(1)]/ *P(0|Y) /P(0)]
Likelihood ratio test
LR10= sup 1P(Y|)/sup 0P(Y|)
N withproper prior
LR10~ B10
Bayes factor can handle point null hypothesis
Cannot handle improper prior because it cannot be renormalized.
One should avoid improper prior for hypothesis testing and model selection
How to balance the two models have to be careful
Jeffreys scale
Bayes factor
Rule of thumb interpretation for hypothesis testing (analogous to p-value)
Log10B10 (0, 0.5) evidence against H0 is poor
(0.5, 1) substantial
(1, 2) decisive
>=2 significant level about 0.01
Analogous to p-value
-
8/13/2019 Confidence Region Hypothesis Testing
6/8
Improper prior: P() 1
Normalization not well defined
P(Y|M1) /P(Y|M0) = integral exp{-(Y-)2/2d--,/exp-Y
2/2}=sqrt(2pi)exp{y
2/2}
Proper prior
~ N(0, 2) (later let )
P(Y|M1) /P(Y|M0) =[1/sqrt(1+ 2)+exp2y2/2(1+ 2)- 0
Bartletts paradox
Bayesian Computation
Monte Carlo Method
Given prior P()
Likelihood P(Y|)
Want to fin posterior P(|Y) P()P(Y|)
Computation associated with posterior integration of functions of parameter with respect to posterior
(mean, variance)
Sample form the posterior (monte Carlo)
Mode finding or approximate posterior by simple distribution
Interior integration
E~P(.|y)h() = h()P(|y)d
Generally let f() = P(|y) be density
Want to calculate
J = integral h()f()d
h(): function such as mean, variance
Monte Carlo integration draw sample *1, , n+ from f()
Approximate J by
J hmbar = 1/m h(i)
-
8/13/2019 Confidence Region Hypothesis Testing
7/8
If i are independent, by law of large numbers m hmbarJ. in probability
Variance
E(hmbarJ)2= 1/mE(h() J)
2= 1/m( h()-J)
2f()d
Posterior simulation methods
Independent sample techniques
Inverse transformation sampling
important sampling
Rejection sampling
Adv: convergence easy to understand
Disadvantage: hard to deal with complex high dimensional problem
Dependent sampling
MCMC(Markov-chain Monte Carlo)
Techniques
Gibbs Sampler
Metropolis/Metropolis-Hastings
Easy to deal with complex problems
Convergence harder to determine
Inverse transformation sampling
Assume has pdf f()
Cdf F(a) that invertible
Want to sample 1. . . m~f()
Algorithm j= 1,,m
Generate random number uj~U(0,1)
Let j be the number such that F(j) = uj (i.e. j= F-1(uj))
Claim j~ f()
Proof cdf of jis P(j
-
8/13/2019 Confidence Region Hypothesis Testing
8/8
Example: exponential
Pdf: f() = e-(>=0)
Cdf : F() = 1- e-(>=0)
Inverse: F-1(u) = -ln(1-u)/
Algorithm
uj ~ U(0, 1)
j=( -1/)ln(1-u)
Example: Weibull Distribution
Pdf: f() = (/)-1exp{-(/)}
Cdf: F() = 1 - exp{-(/)}
F-1(u) = -ln(1-u)]1/
Sampling algorithm
Generate ujU(0,1)
j-ln(1-u)]1/