Construction, concentration, and calibration of
21
Construction, concentration, and calibration of Gibbs posteriors 12 Ryan Martin North Carolina State University www4.stat.ncsu.edu/ ~ rmartin 6th African International Conference on Statistics Arsi University, Ethiopia May 27th, 2019 1 Some joint work with my former student, Nick Syring 2 Partially supported by NSF DMS–1811802; previously by the U.S. Army 1 / 21
Transcript of Construction, concentration, and calibration of
Construction, concentration, and calibration ofGibbs posteriors12
Ryan MartinNorth Carolina State University
www4.stat.ncsu.edu/~rmartin
6th African International Conference on StatisticsArsi University, Ethiopia
May 27th, 2019
1Some joint work with my former student, Nick Syring2Partially supported by NSF DMS–1811802; previously by the U.S. Army
1 / 21
Introduction
Model misspecification is usually viewed as a bad thing.
But there are cases where working with a misspecified modelhas certain advantages, e.g., quantile regression.
A “correctly specified model” can be too complex
lots of (potentially unnecessary) parametersmarginalization
A strategically misspecified model can:
reduce computational burdensimplify prior specificationeliminate the need for marginalization
But there’s a risk of misspecification bias.
How to balance benefits and risks?
2 / 21
Intro, cont.
I’m going to focus on Gibbs posterior distributions
similar to Bayesuses a suitable risk function instead of likelihood.
Where does a risk function come from?
it might define parameter of interestor could be cooked-up by the user.
I’ll give examples of both types.
But risk functions do not come with an absolute scale:
only the shape matters for M-estimation,but scale is important in a Gibbs posterior.
Need to tune this scale for quality Gibbs posterior inference;my aim is to tune for calibration.
3 / 21
This talk
Gibbs posterior primer
Two examples:
minimum clinically important difference (MCID)image boundary detection
Calibrating the Gibbs posterior
Conclusions
4 / 21
Gibbs posterior primer
True interest parameter is defined as the minimizer of a riskfunction, that is, θ? = arg minR(θ).
risk function is given, e.g., median = arg min E|Y − θ|,or I can cook up a suitable risk function myself.
Apply Bayes’s formula but with the negative log-likelihoodreplaced by the empirical risk Rn(θ), i.e.,
Πn(dθ) ∝ e−ωnRn(θ) Π(dθ).
Direct attack on the quantity θ of interest:
potentially no nuisance parameters or marginalizationno priors for irrelevant quantitiesbasically no model assumptions
Choice of ω is crucial — data-driven?
5 / 21
Minimum clinically important difference
Data (Xi ,Yi ), i = 1, . . . , n, iid:
Xi ∈ R is a diagnostic measure on patient i ;Yi ∈ {−1,+1}, where “Yi = ±1” means patient i found thetreatment to be effective/ineffective.
Quantity of interest is the MCID:
θ? = θ?(P) = arg minθ
P{Y 6= sign(X − θ)}︸ ︷︷ ︸R(θ)
,
where P is the joint distribution of (X ,Y ).
MCID is the X -scale cutoff at which patients will tend to viewthe treatment as effective.
Statistical versus practical/clinical significance...
6 / 21
MCID, cont.
What model to use?
Standard binary regression models may not work, e.g.,sensitivity to the choice of link function, etc.
A fully nonparametric model seems like overkill.
M-estimation3 based on minimizing an empirical risk:
Rn(θ) =1
2n
n∑i=1
{1− Yi sign(Xi − θ)
}.
Gibbs posterior for θ makes sense and is readily available.
Is it any good?
3Hedayat et al (Biometrics, 2015)7 / 21
MCID, cont.
Compare Gibbs posterior (ω = 1) to two Bayes posteriors
logistic regressionbinary regression with nonparametric link
−1.0 −0.8 −0.6 −0.4 −0.2 0.0
02
46
8
MCID
Den
sity
GibbsNonpar.Logistic
(a) X ∼ 0.7N(−1, 1) + 0.3N(1, 1)
−1.0 −0.5 0.0 0.5 1.00
24
68
MCID
Den
sity
GibbsNonpar.Logistic
(b) X ∼ 0.7N(−1, 1) + 0.3N(3, 1)
8 / 21
MCID, cont.
Lack of smoothness makes the problem “non-regular.”
We4 showed that typical rate is n−1/3.
Advantages of Gibbs:
direct attack on MCID — I don’t have to model X or specify alink function;can easily incorporate available prior information;immediately gives credible regions.
Need to deal with ω, the risk scale — more later...
4Syring and M. (JSPI, 2017), arXiv:1501.018409 / 21
Image boundary detection
Data (Xi ,Yi ) are pixel locations and intensities, i = 1, . . . , n.
Intensities tend to be stronger inside a region Γ compared tooutside — a stochastic ordering condition.
Goal is to make inference on Γ...
10 / 21
Boundary, cont.
A fully Bayesian approach5 requires modeling intensities;
introduces nuisance parametersand possible model misspecification bias.
Gibbs can avoid modeling intensities, but what Rn...?
Need to define R(Γ) so that true Γ? satisfies
R(Γ) > R(Γ?), ∀ Γ 6= Γ?.
Roughly, we6 showed that a twist on missclassification errorprobability will do the job; this idea is fairly general.
Then take Rn the empirical version of R.
5Li and Ghosal (Annals, 2017), arXiv:1508.058476Syring and M. (Annals, to appear) arXiv:1606.08400
11 / 21
Boundary, cont.
Characterize Γ by its boundary γ = ∂Γ, treated as a function
Prior for γ is a mixture of b-splines.
For theory:
True Γ? in class H(α) with α-Holder smooth boundary.Optimal rate on H(α) is εn = {log(n)/n}α/(α+1).
For any Mn →∞, Gibbs Πn satisfies
supΓ?∈H(α)
EΓ?Πn({Γ : λ(Γ4Γ?) > Mnεn})→ 0, n→∞.
Adaptively attains optimal rate!
Robust because there’s no model for pixel intensities.
12 / 21
Boundary curve, cont.
Estimates with incorrectly specified intensity models.
Left is the observed image, middle is Bayes, right is Gibbs.
13 / 21
Gibbs calibration
Recall the Gibbs posterior:
Πn(dθ) ∝ e−ω nRn(θ) Π(dθ).
If Rn is minimized near θ? = arg minR(θ), then Gibbsposterior Πn is roughly centered there.
In that case, ω controls the spread of Πn around θ?.
Can we tune ω so that Πn credible regions are calibrated, e.g.,95% credible regions are 95% confidence regions?7
7Syring and M. (Biometrika, 2019) arXiv:1509.0092214 / 21
Calibration, cont.
Simple Gaussian process model with two parameters (µ, τ).
Compare several posteriors:
full Bayes posteriorcomposite Bayes posteriorour calibrated composite posterior
−0.2 −0.1 0.0 0.1 0.2 0.3
01
02
03
04
05
0
µ
De
nsity
(c) µ marginal
0.80 0.85 0.90 0.95 1.00 1.05 1.10
01
02
03
04
0
τ
De
nsity
(d) τ marginal
−0.2 −0.1 0.0 0.1 0.2
0.7
0.8
0.9
1.0
1.1
1.2
1.3
µ
τ
++
+++++++++++ ++++++++
+++++
++++++
++
++++++++++
++++++++
++++++
+++++++++
+++++++ +
++++++
++++++ +
+ +++
+++++++++++++++++++
+++ +++++++ ++
++++++ ++
+++++++++
+++++++++++++
++++++++
++++++++++++++++
+++++++++++
++
+++++++++++++++++
++++++
+++++++++++++
++++++++++++++++++
+++++++++++++++
+++ +++++++++++
++++++++++++++++++ ++
++++++++++++ +++ ++++++
+++++++++++++++ +++++
+++++++++
+++
++++++++++++ ++++++
++++++++
+++++++
+++++
+++++
+++++++++++++++++++++++
+++++ ++++
+
+++
++++++ ++++
++++++
+++ ++
++ +++++++++++ ++
++++
++++++++++++++++++ ++++
+++
++++++++++++ ++++++++++
+++++++++ ++++++++++++++++++++++++++++++++++++++
+++ ++++++++
++
+++++ +++++++
++++++++ +++
+ ++++
+ ++ ++++++++ +++++++++++++ ++++++ ++++++++++++++++++
+ +++++++++ ++++++ ++++++++ +++ +++ ++++
++++
++++ +++++++++++++++++++
++++++++++++++++
+ ++++++++++++++
++++
+++++++
+++++++++++++
+ ++++++++++++
+ +++++++++ +++++++++++++++++++
+++++++++++++++
+++++
+++++++ +
+++++++ ++
+++++
+++++++
+++++ +++++
++++++++++++++++
++++++
++++
+++ +++++++
+++++ ++
++
++++++++++
+++++++++
++++++++ ++++
++++++++
++++
+++++++++++
++
++++++ ++++++++++
+++++++++++++++ +++++++++++
+++++
+++ +++++++++++++
+++++++++++ +++++++
+++++++++
+++ ++++++++
+ ++ +++
+++++++
+++++ ++
++++++
+++++++++++++
+++ ++++++++ +++
++++
++++++++++++++++++++++++
+++++++++++++++
++
++++ ++++++
+++++++++++++++++++++
++++
+++
++
++++++
++++ ++++++++
++++++++++++++++ +++++++++++++ ++
+++++++++ ++++++++ ++
+++++
+++++++++++++
++++++++++++++ +++++++ +++
++++++++ +++++
+ ++++ +++++++++++
++++++++ ++++
++++++++++++++
++ +++++ +++++++++++++++++++
+++++++++ ++++++++++++++++++++
+++
+ +++++++++++
+
+++
+++++++++++++
+++++++++
++++++++++++++++++++++++++
++++++++++++++++++
+++++++++++++++++
++ +++
+++ ++++ +++++++++
++++++++++
++++++++++
++++++++++
+++++++++++++++++++++
+++ +
+++
+++++++ ++++++++++++++++ ++++++
++++++++++++++++ +++++++++ +++++ + +++++
+++++++++++
+ ++++++ ++++++ ++++
+++++++++++
+++++++++++++++ +++++++
+++++++
+ +++
++++++++
+++
+++++++++++++++
+++++++++++++++++++++ ++++++++++
++++++++++++ +++++++++
+++
++++++++++++++++++
+++ ++
+++++++++++++++ +++++++++++++
+
++++++++++
++++++
+++++
++ +++++++
+++++++++
++++++++++++++++++++++++++
++++++ ++
++++ ++ ++ ++++
+++++++ +++++
++++++++++++
++
++++++
+++++++
+ +++ ++++++++++++++
++++++++
+ ++++++++++
+++
++++
++++++++
+++++++++
+++++
+++++++++ +++++++
+++
++++ +++++++
++
++++++++ +
+++++++++++
+++++++++
+++++
++++++++++++ +++++++++
++++++++++++++++++++++++
+++
+++
+++
++++++++++++
++++++++++++++++++++++ +
+++++++++++++ ++++++ +++++++
+++
+++++
+++++++++
++++++ ++++++++++
+++++++++++++++++++
+++
++++++++++++++
++++++ ++++ ++++++ ++++++++++
+++++++++++++++++++++
++++
++++++++ +++++ +++++
+ +++++++++++++++++
+++++++++ ++ +++++
++++++++++
+++ ++++++++
+++++++++
+
++++ ++
+++++++++ +++ ++
++ ++++++++++
++
++++++
+++++++++++ +
+++++++++ +++++++++++++++++++++
+++++
+ +++++
++++++++++++++++++
++++++ ++++++++++ ++++++++
+++++
+++++++
+++++++++
+++++++
++++++++++++++++++++++++++
++++++
++++++ +++
++++++++++++++
++++++
++ +++++++++++++++
+++++++
+++++++++++++++++++ +++++++++
++ ++++++++++++++++++++++++
+++ +++++++++++++++ ++ +++
+ ++++++++++ ++++++++++++++++++++++++
+++++ ++++++++++ ++++
+
++ ++++++ +++++++
++++
++++++++++++ +++++++++++ +++++++++++
++
+++++++++++++++
++++++
++++++++++++++++++++ +
+++ ++++++++++++++++ ++++++++++++++
++++++
++++
++++++++++++++ ++++++++
++++++++
++
+++++++++ ++++++++++
+++++++++++++++++ ++++++++
++++++++++++++++++
++++++
++
++++
++++++++++++++++++++
++ +++++++++++++++++++++++
++
+ ++++++++++
++++++
+++ +++++++++++++++
++++ ++++++++ ++++
++
++++++++++++
+++++
+++ +++++++++++++++++++++++++++ +++++++++
++++
++ ++++++
++++++
++++++++++
++++++++++++
+++++++
++++++ +++++++++++
++++++++ +++++++
+++++++++++ ++++++++++
+++
++
+++
++
++++++ +++++++++++
++++
++++++++++++++++++++
++++++++
+++++
++++
++++++
++++++++++ ++
+++++++++++
++++++++++++++ ++++++ +++++
+
+ ++++++++ ++++++
++++++++++++++++++
++++++++++++++++ ++
++++++
+++++++ ++++
++++++
+++++
+++++++
+
+++++
+++++++
++ +++++
++ +++
+++++++++++++ ++
+++++
++++ ++++++++++++
++++++ ++++++++++++++++++
+ +++++++++++++++++++ +++++ +++++ ++++++++++++++
+++++
+++++++++
++++++++++
++++++ ++++++++++++++
++++
++++++
+++++++++++++++++
++++++
+ +++
++
+++++++++
++++++++++++++++
+++++ +++++++
+++++++++++++++++++++
+ +++++++++ ++++++++++++
+++
+++
+++++
++++++
+++++++++++++++++
++++++
+++++++++++++++++ +++++++
++++
++++++++++++++++
++
+++++
++++
+++++++++++++++++++++++++
++++++++
+++++ +++++
++++++++
++++++++++++++++++
+++++++++++++++++++++++
+++++ +++++++
++++++++++++++
+
++
+++++
++++++++
+++++++ ++++++++
++ ++++++ ++ ++++
++++++
+++++++++++++++ +++++
+++++++
+ ++++++ +++++++++++++++++++++
+ ++++
+++ ++
++ +++++++++++++++
+ ++++++++
++++++++++
+++++++++++
+++++ +++++ ++
+++++
+++++
++++ +++
++ +++++++++++
+++++++ +++ +++++++++++++++++++ ++++++
+++
++++++++
++++++++++
++ +++++++++++++++ +++
+ ++
++++
+++++++++++++++++++++ +++
++++++
++
++++++++
+
++ +++++
+++ ++
+++
++
++ +
++++ ++++++++++++++
+++
+++ +++++ +++++++++++++
+++++++++
++
+++ ++++++++++++ +++++++++++++++ ++++
++ +++++++++++++++++++
+ +++++++++ +++++++++++++++
+++++++
++ ++ +++
++
+++
+++++++
+++++++
+
++++++++
++
++++ ++++++ ++++++++
++++++++
+++++ ++++++++
++ +++++++++++++
++++
+++
++++++++++++++++++
++++++++ +++++++++++
++
++++++ +
+ ++++++
++++++ +++++++
+++
++++ +++++++++
+++++++++
++++
+++++++++++++++ +++++++++++
++++ ++++ ++++++++++++++++++++++++ +
++++++++++++++++++++++++++++++++++++++++++ +++++ ++
+++++
++++++
++++++++++
++++++++++++++++ ++
+++++++++++++
+++++
+++++++++ ++++++++
++++++++++++ +++ +
++++++++++++++++
++++++
+ ++++++++++++ +++ +++++++++++++ +++++
++ +++++++++
+++++++
+++++++++
+++++
++++++ +
++++++++
+++
++++++ +++++++++
+ ++++++
+++++++++ +++++++++++ +++++
++ +++++ ++
++++++++++++++
++++++++
+++++++++++++++ +
+
++++++++++ +
++++++++++++++ ++++++
++++++++ +
++++++++++++ +++++++++
++ ++++++
+++++++++++
+++++++
++++++++++++++++++++
+++++++++++++
++++ ++++
++++++++++
++++
+++++++++++++++++++++++++++
++++
++ +++++++
+++++ +++
+++++++++++++++++++++++++
++++++++++++++++++++++++++++
++
+ ++++++++++++
+++
++++++
++++++++++ +++++++++++
++++++++++++++++++++ ++++
+ +++++++++
++++++++
+++++
+ +++++++++++++++
+++++++
++++++++
+++++++
++++++++
+++ ++++++++ +
+++++++ ++++++++++
++++++++++++ +
+++++
+++++++++
++++++++ +
++++++
++++++++++++++ ++++++
+++++++++ ++++++++
++++++
++++++++++++
++++
++ +++++++++++++
+++++ ++++++
++
++++
++++
++
+++ ++++
+++++ ++++++
++++++ ++++++++++++
++ +++++
+ ++++++++++
++ ++++++++++++++++ ++
++++++
++ ++++++
+++++++++++ +++
++++++++++++++++++
++
+
+++++++
+ +++++++++++++++++++++++
+++ +++++++++++++++++++
+++++++++++++++++++
+++++++++
++
++++++ ++++
++ +++++++++++++++
++++++++++ ++
+++ ++++ +
+++
+++++++++ ++++ +++++++++++++ +++++++++++++++ ++
++++++ ++++++
+++++++++++ ++++++++
+++
+++ +
++++++++
+
++
+++ ++++++++++++++++++
++++
+++++++ ++++++ +++
+ ++ +++++
++++++++++++++++++++
++++++
+++++++++
+++
++++++++++
+++++++++++++ ++++++
++ ++++++++ ++++++++
++ ++++++++++++++ +++++ +
++++++++++++
++++++
+
++ ++++++++++++++
+++
+++++++++++
+++++++++++++
++++++++
++++++++ ++ ++
++++++++
++ ++
++
+++
+++++ ++
++++++++ ++++++++++++
+ ++++++++++++++++ ++++++++++++++++
+++ ++++++++++++++++++++++++
++ +++++++++++++++
++ ++++++ +++++++++++++++++
++++++++++
++++++
++++++ ++++++
+ ++++++++++++++++++++++++++++ +
+++++++++
+++ ++
+++
+++++++++
+++++++++++++++++
+++++
++
++++
+++ +++++
++++++
++ +++++ ++++
+++ ++++++++
++++++++++++++++++++++ ++
++ ++++++
+++++++
+++
++++
+++++++++
++
++++++
++++++++++
++++++++
++
++++
+
+++++++++++++++++++
++++ ++++
+++ +++++++++++++
+++ ++++++++ +++++++++++++
++++++++++++
+++++++++++ +++++++++++
+++
++++ +++++++ ++++
+++ ++++ +++ +++++++
++
+++++++++++++++++
+++++++++ +++++++++
++++++ +++++ ++++++
+++
++++
++++++++ +++++++++++
+++++++ ++ +++++++++++++++++++++ ++++
+ +++++++++++++
+++++++++++++++++++++ ++
+++ +++++++++++++++
++++ +++
+++++++
+++++++++++ ++ ++ +++
+
++++++++++
++++++
+++++++
++++ +++
++++++++++
+++++++
++++++
++++++++++
+++
+++ ++ +++++ +++
+++++
++ ++++++
+++++++++
+ +++ +++++++
+++++++ ++++ +++++
+++++++++
+
+++++++++++++
++++++
++++++++++++++ +++++++
+++ ++++++++++++++++ +++ ++++++
++++++
+++++++ +++++++ +++ ++ ++++++++++
+++++++++
+++++++++ ++++
+++++++
+++++++
+++++++++
++++++++++++
+++
++++++++++
+++++++++++++++++++++++++++++++++
++++++ +++++++++++++++++++
+++++++++
+++++++++++++++++++++++++
++++++++
++++++++++++
+++++++++++++++++++++
++++ +++++++++
++++
++++++++ +++
+++ ++
+++++++
++++++++
++++++++ ++++++++
+++++ ++++++++++++++++++
+++
+++++++
+++++++++++++++++
+++++
+++++
++++++ ++++++++++++++++++++++++
+ +++++ +++++++
++++++++ +
++++++++++++++++++++++++++++++++++ ++++
+++++
++++
++++++++++++++++++
++++++++++++++
+++++++
++ ++++++++ +
+ +++++++
++++++++++++++
++++++++++++++
+++++++
++++++++++++
+++++++
++++
+++++++ ++++ ++++++
++++++++++
++++
+++++++++++ +++++++ +++
+++++++ +++ ++
+ +++++
+++++++++++++++
+++ +++++
+++++++++++
++++++++++++++++++
+++++ ++++ ++++++++++++++++++++++++++++++
++
++++++++++++++++++++
+++++
+++++++++++++++++++++++++ ++++++++++++++++
+++++++++++++++++++++++++++++++++++++++
+++ +++++++++++ +++++ ++ +++++++++++ +++++
++++++
+++++ ++++++++++ ++
++++++++++++ ++++
+++++++++
++ ++++ ++++++
++++ +++ ++++++++ +++++
+++ +++++++ ++++
+++++ +++
+++++++++++
+++++ +++++++
+++++
++++++++++++ ++++ ++
+
+ +++++++++++++++++++++
++++ +
++++++ +++++++++
++++++++
+
++++
+++ ++++++ +++
++++
+++++++++++++
+++++ ++ +++ +++++
+++++++
+++++++++++++++++++
++++++++++++++
++
++++++++++++
+ +++
+++++++++ ++ +
++++++++++++++++
++++++++++++ +++
+++++++
+++++ +++++++ ++++
++++
++++ ++++++++++++++++++++++ ++++++++++++++
++++++++++
+++++++++++
+++++
+++ +++++++++++++++++++++
+++ +++++++++++++++ ++++++ ++
+ +++ +++
+++++
+
+++++++++
++++++++++++++++++++
+++ ++++++++
+++++ ++++
+++++++++
++
++++++++
+++++++++++
+
+++ +++++++++++++++++++++++++++++++++++
++++
+++++
++++++++ ++ +++
+ ++++ +++++
+ ++++ ++++++ +++++++
+++++++
++++ ++
++++++++ ++++++++++++ ++
+++
+
++++++++++++++++++
++++++++++++ ++++++
++++++
++++++ ++
++++ +++++++++
+++
++++++++++++
++++++
++++++++++++ +++++++++++++++++++++ +
++ +++ +++++++
+ ++++++
+++++++
+++++
++++
++++ +++++++++++++
+++++ ++++++ ++
+++
+++++++++++
+
+++++++++
++++++++++++++++++++
++++++
+++++++++
+++++++ ++++++++++++++
++++++++++++
++++++++++
++
++++
++++++++ ++++++
++++++++++
+ ++++ +++++
+++
++++++++ +++
++++ ++++++++++++
++ ++
++++++++++ ++++
+++++++++++++++++++++++++++++++++++++++
+++++++++++
+++++++++++
++++++++
+++++++++
++++
+++
+++++++++++++++++++++++++
+ ++++++++++++++++++++++
++
++++++++ ++ +++
+++++++++ ++++++
+++
+++
++++++++++++
++ +++++++++ +++++++ +++
+++++
++ +++ ++++++++++++ ++++++++
++++++++++++++
+++
+ ++++ +++
+++++++ +++++
+++++
+++++++++
+
++++++++++++++++++
++++
+ ++++++++ +++++
++++++++++++++
+++++++
++++++
++
+++++++++++++
+++++
+++++++++++
+++++++++++
+ +++++++++ +++
++++++ +++++++++
++++++++++++++++++++
+++++++++++++++ ++++ +++++++++++
+++++ ++++++++++++++++ +++++
++++++++++++++
++++++++++++++
++++++++++++ ++++
++++++++
++++++ +++++
++
++++ +++++
+++
++
+ ++
++++
+++
++
++++++++++++++++++++++
++++++++ ++++++++++++
++++++
++++++
+++++
++++++++ +
+++++++++++++++++
+++++ ++++++
+ +++++++++++ +++++++++++++++++++++
+ +++
++++++ ++
++++
++++++
+++++++++
+++++
+++++++ +++
++++++++ +
+++ ++++++++++++ ++++++++
++++++
+++++++
++
+++++++
+++++++++
+++
++++ ++++++++++++++++
(e) (µ, τ) joint
15 / 21
Calibration, cont.
Is it even possible to find a ω that tunes coverage?
Not obvious, but it seems the answer is YES...
Example: Model says X1, . . . ,Xniid∼ N(θ, σ2)
Fixed σ2 but the true variance is different, say, ψ2.Flat prior Gibbs posterior for θ is N(X , ω−1σ2n−1).Calibration achieved with ω = σ2/ψ2.
Similar asymptotic results can be obtained more generally,involving 2nd derivative of Rn and usual sandwich covariance.
These results are similar to those obtained by other efforts attuning the “learning rate” ω — more later...
16 / 21
Calibration, cont.
We developed an algorithm8 to tune toward coverage.
For simplicity, assume distribution P is known; if unknown,then replace it by the empirical distribution Pn.
General Posterior Calibration Algorithm:
0 Initialize ω0 and set t = 0
1 Sample lots of data sets from P to approximate the coverageprob of Gibbs credible region Cωt ,α.
requires Gibbs posterior computation for each data setcheck coverage using θ(P)
2 If coverage prob is close to 1− α, then stop; otherwise,update ωt → ωt+1 via stochastic approximation, increment t,and go back to Step 1.
8This algorithm is naive, can be made more efficient...17 / 21
Calibration, cont.
Two-dim quantile regression example
Several methods:
BEL.s Bayesian empirical likelihood9
Normal asymptotic normalityGPC Gibbs with GPC
GPC gives exact coverage and shorter length!
Coverage Prob ×100 Avg Length ×100n BEL.s Normal GPC BEL.s Normal GPC100 θ0 97 95 95 106 100 91
θ1 98 98 95 58 55 47400 θ0 95 95 95 50 50 46
θ1 97 97 95 26 25 231600 θ0 96 96 95 25 25 23
θ1 96 96 95 13 12 11
9Yang and He (Annals, 2012)18 / 21
Conclusions
Gibbs approach makes direct posterior inference possible:
no nuisance parametersno marginalization“real” priors
Nice theory and good practical performance.
Choice of scale, ω, is very important.
new algorithm designed for calibrationmore can still be done
Connections to other learning rate tuning methods?
More Gibbs stuff to do:
More applicationsGPC in high-dim problemsGeneral Gibbs concentration rate theoremsConnecting CS/ML/Stat literature on this
19 / 21
Commercial
The peer review system is broken in various ways.
Successful reform requires new ideas.
Harry Crane and I developed a new open-access publicationplatform, featuring an author-driven peer review process.
For details, check us out at
www.researchers.one
www.twitter.com/@ResearchersOne
20 / 21
