Construction, concentration, and calibration ofGibbs posteriors12

Ryan MartinNorth Carolina State University

www4.stat.ncsu.edu/~rmartin

6th African International Conference on StatisticsArsi University, Ethiopia

May 27th, 2019

1Some joint work with my former student, Nick Syring2Partially supported by NSF DMS–1811802; previously by the U.S. Army

1 / 21

Introduction

Model misspecification is usually viewed as a bad thing.

But there are cases where working with a misspecified modelhas certain advantages, e.g., quantile regression.

A “correctly specified model” can be too complex

lots of (potentially unnecessary) parametersmarginalization

A strategically misspecified model can:

reduce computational burdensimplify prior specificationeliminate the need for marginalization

But there’s a risk of misspecification bias.

How to balance benefits and risks?

2 / 21

Intro, cont.

I’m going to focus on Gibbs posterior distributions

similar to Bayesuses a suitable risk function instead of likelihood.

Where does a risk function come from?

it might define parameter of interestor could be cooked-up by the user.

I’ll give examples of both types.

But risk functions do not come with an absolute scale:

only the shape matters for M-estimation,but scale is important in a Gibbs posterior.

Need to tune this scale for quality Gibbs posterior inference;my aim is to tune for calibration.

3 / 21

This talk

Gibbs posterior primer

Two examples:

minimum clinically important difference (MCID)image boundary detection

Calibrating the Gibbs posterior

Conclusions

4 / 21

Gibbs posterior primer

True interest parameter is defined as the minimizer of a riskfunction, that is, θ? = arg minR(θ).

risk function is given, e.g., median = arg min E|Y − θ|,or I can cook up a suitable risk function myself.

Apply Bayes’s formula but with the negative log-likelihoodreplaced by the empirical risk Rn(θ), i.e.,

Πn(dθ) ∝ e−ωnRn(θ) Π(dθ).

Direct attack on the quantity θ of interest:

potentially no nuisance parameters or marginalizationno priors for irrelevant quantitiesbasically no model assumptions

Choice of ω is crucial — data-driven?

5 / 21

Minimum clinically important difference

Data (Xi ,Yi ), i = 1, . . . , n, iid:

Xi ∈ R is a diagnostic measure on patient i ;Yi ∈ {−1,+1}, where “Yi = ±1” means patient i found thetreatment to be effective/ineffective.

Quantity of interest is the MCID:

θ? = θ?(P) = arg minθ

P{Y 6= sign(X − θ)}︸ ︷︷ ︸R(θ)

,

where P is the joint distribution of (X ,Y ).

MCID is the X -scale cutoff at which patients will tend to viewthe treatment as effective.

Statistical versus practical/clinical significance...

6 / 21

MCID, cont.

What model to use?

Standard binary regression models may not work, e.g.,sensitivity to the choice of link function, etc.

A fully nonparametric model seems like overkill.

M-estimation3 based on minimizing an empirical risk:

Rn(θ) =1

2n

n∑i=1

{1− Yi sign(Xi − θ)

}.

Gibbs posterior for θ makes sense and is readily available.

Is it any good?

3Hedayat et al (Biometrics, 2015)7 / 21

MCID, cont.

Compare Gibbs posterior (ω = 1) to two Bayes posteriors

logistic regressionbinary regression with nonparametric link

−1.0 −0.8 −0.6 −0.4 −0.2 0.0

02

46

8

MCID

Den

sity

GibbsNonpar.Logistic

(a) X ∼ 0.7N(−1, 1) + 0.3N(1, 1)

−1.0 −0.5 0.0 0.5 1.00

24

68

MCID

Den

sity

GibbsNonpar.Logistic

(b) X ∼ 0.7N(−1, 1) + 0.3N(3, 1)

8 / 21

MCID, cont.

Lack of smoothness makes the problem “non-regular.”

We4 showed that typical rate is n−1/3.

Advantages of Gibbs:

direct attack on MCID — I don’t have to model X or specify alink function;can easily incorporate available prior information;immediately gives credible regions.

Need to deal with ω, the risk scale — more later...

4Syring and M. (JSPI, 2017), arXiv:1501.018409 / 21

Image boundary detection

Data (Xi ,Yi ) are pixel locations and intensities, i = 1, . . . , n.

Intensities tend to be stronger inside a region Γ compared tooutside — a stochastic ordering condition.

Goal is to make inference on Γ...

10 / 21

Boundary, cont.

A fully Bayesian approach5 requires modeling intensities;

introduces nuisance parametersand possible model misspecification bias.

Gibbs can avoid modeling intensities, but what Rn...?

Need to define R(Γ) so that true Γ? satisfies

R(Γ) > R(Γ?), ∀ Γ 6= Γ?.

Roughly, we6 showed that a twist on missclassification errorprobability will do the job; this idea is fairly general.

Then take Rn the empirical version of R.

5Li and Ghosal (Annals, 2017), arXiv:1508.058476Syring and M. (Annals, to appear) arXiv:1606.08400

11 / 21

Boundary, cont.

Characterize Γ by its boundary γ = ∂Γ, treated as a function

Prior for γ is a mixture of b-splines.

For theory:

True Γ? in class H(α) with α-Holder smooth boundary.Optimal rate on H(α) is εn = {log(n)/n}α/(α+1).

For any Mn →∞, Gibbs Πn satisfies

supΓ?∈H(α)

EΓ?Πn({Γ : λ(Γ4Γ?) > Mnεn})→ 0, n→∞.

Adaptively attains optimal rate!

Robust because there’s no model for pixel intensities.

12 / 21

Boundary curve, cont.

Estimates with incorrectly specified intensity models.

Left is the observed image, middle is Bayes, right is Gibbs.

13 / 21

Gibbs calibration

Recall the Gibbs posterior:

Πn(dθ) ∝ e−ω nRn(θ) Π(dθ).

If Rn is minimized near θ? = arg minR(θ), then Gibbsposterior Πn is roughly centered there.

In that case, ω controls the spread of Πn around θ?.

Can we tune ω so that Πn credible regions are calibrated, e.g.,95% credible regions are 95% confidence regions?7

7Syring and M. (Biometrika, 2019) arXiv:1509.0092214 / 21

Calibration, cont.

Simple Gaussian process model with two parameters (µ, τ).

Compare several posteriors:

full Bayes posteriorcomposite Bayes posteriorour calibrated composite posterior

−0.2 −0.1 0.0 0.1 0.2 0.3

01

02

03

04

05

0

µ

De

nsity

(c) µ marginal

0.80 0.85 0.90 0.95 1.00 1.05 1.10

01

02

03

04

0

τ

De

nsity

(d) τ marginal

−0.2 −0.1 0.0 0.1 0.2

0.7

0.8

0.9

1.0

1.1

1.2

1.3

µ

τ

++

+++++++++++ ++++++++

+++++

++++++

++

++++++++++

++++++++

++++++

+++++++++

+++++++ +

++++++

++++++ +

+ +++

+++++++++++++++++++

+++ +++++++ ++

++++++ ++

+++++++++

+++++++++++++

++++++++

++++++++++++++++

+++++++++++

++

+++++++++++++++++

++++++

+++++++++++++

++++++++++++++++++

+++++++++++++++

+++ +++++++++++

++++++++++++++++++ ++

++++++++++++ +++ ++++++

+++++++++++++++ +++++

+++++++++

+++

++++++++++++ ++++++

++++++++

+++++++

+++++

+++++

+++++++++++++++++++++++

+++++ ++++

+

+++

++++++ ++++

++++++

+++ ++

++ +++++++++++ ++

++++

++++++++++++++++++ ++++

+++

++++++++++++ ++++++++++

+++++++++ ++++++++++++++++++++++++++++++++++++++

+++ ++++++++

++

+++++ +++++++

++++++++ +++

+ ++++

+ ++ ++++++++ +++++++++++++ ++++++ ++++++++++++++++++

+ +++++++++ ++++++ ++++++++ +++ +++ ++++

++++

++++ +++++++++++++++++++

++++++++++++++++

+ ++++++++++++++

++++

+++++++

+++++++++++++

+ ++++++++++++

+ +++++++++ +++++++++++++++++++

+++++++++++++++

+++++

+++++++ +

+++++++ ++

+++++

+++++++

+++++ +++++

++++++++++++++++

++++++

++++

+++ +++++++

+++++ ++

++

++++++++++

+++++++++

++++++++ ++++

++++++++

++++

+++++++++++

++

++++++ ++++++++++

+++++++++++++++ +++++++++++

+++++

+++ +++++++++++++

+++++++++++ +++++++

+++++++++

+++ ++++++++

+ ++ +++

+++++++

+++++ ++

++++++

+++++++++++++

+++ ++++++++ +++

++++

++++++++++++++++++++++++

+++++++++++++++

++

++++ ++++++

+++++++++++++++++++++

++++

+++

++

++++++

++++ ++++++++

++++++++++++++++ +++++++++++++ ++

+++++++++ ++++++++ ++

+++++

+++++++++++++

++++++++++++++ +++++++ +++

++++++++ +++++

+ ++++ +++++++++++

++++++++ ++++

++++++++++++++

++ +++++ +++++++++++++++++++

+++++++++ ++++++++++++++++++++

+++

+ +++++++++++

+

+++

+++++++++++++

+++++++++

++++++++++++++++++++++++++

++++++++++++++++++

+++++++++++++++++

++ +++

+++ ++++ +++++++++

++++++++++

++++++++++

++++++++++

+++++++++++++++++++++

+++ +

+++

+++++++ ++++++++++++++++ ++++++

++++++++++++++++ +++++++++ +++++ + +++++

+++++++++++

+ ++++++ ++++++ ++++

+++++++++++

+++++++++++++++ +++++++

+++++++

+ +++

++++++++

+++

+++++++++++++++

+++++++++++++++++++++ ++++++++++

++++++++++++ +++++++++

+++

++++++++++++++++++

+++ ++

+++++++++++++++ +++++++++++++

+

++++++++++

++++++

+++++

++ +++++++

+++++++++

++++++++++++++++++++++++++

++++++ ++

++++ ++ ++ ++++

+++++++ +++++

++++++++++++

++

++++++

+++++++

+ +++ ++++++++++++++

++++++++

+ ++++++++++

+++

++++

++++++++

+++++++++

+++++

+++++++++ +++++++

+++

++++ +++++++

++

++++++++ +

+++++++++++

+++++++++

+++++

++++++++++++ +++++++++

++++++++++++++++++++++++

+++

+++

+++

++++++++++++

++++++++++++++++++++++ +

+++++++++++++ ++++++ +++++++

+++

+++++

+++++++++

++++++ ++++++++++

+++++++++++++++++++

+++

++++++++++++++

++++++ ++++ ++++++ ++++++++++

+++++++++++++++++++++

++++

++++++++ +++++ +++++

+ +++++++++++++++++

+++++++++ ++ +++++

++++++++++

+++ ++++++++

+++++++++

+

++++ ++

+++++++++ +++ ++

++ ++++++++++

++

++++++

+++++++++++ +

+++++++++ +++++++++++++++++++++

+++++

+ +++++

++++++++++++++++++

++++++ ++++++++++ ++++++++

+++++

+++++++

+++++++++

+++++++

++++++++++++++++++++++++++

++++++

++++++ +++

++++++++++++++

++++++

++ +++++++++++++++

+++++++

+++++++++++++++++++ +++++++++

++ ++++++++++++++++++++++++

+++ +++++++++++++++ ++ +++

+ ++++++++++ ++++++++++++++++++++++++

+++++ ++++++++++ ++++

+

++ ++++++ +++++++

++++

++++++++++++ +++++++++++ +++++++++++

++

+++++++++++++++

++++++

++++++++++++++++++++ +

+++ ++++++++++++++++ ++++++++++++++

++++++

++++

++++++++++++++ ++++++++

++++++++

++

+++++++++ ++++++++++

+++++++++++++++++ ++++++++

++++++++++++++++++

++++++

++

++++

++++++++++++++++++++

++ +++++++++++++++++++++++

++

+ ++++++++++

++++++

+++ +++++++++++++++

++++ ++++++++ ++++

++

++++++++++++

+++++

+++ +++++++++++++++++++++++++++ +++++++++

++++

++ ++++++

++++++

++++++++++

++++++++++++

+++++++

++++++ +++++++++++

++++++++ +++++++

+++++++++++ ++++++++++

+++

++

+++

++

++++++ +++++++++++

++++

++++++++++++++++++++

++++++++

+++++

++++

++++++

++++++++++ ++

+++++++++++

++++++++++++++ ++++++ +++++

+

+ ++++++++ ++++++

++++++++++++++++++

++++++++++++++++ ++

++++++

+++++++ ++++

++++++

+++++

+++++++

+

+++++

+++++++

++ +++++

++ +++

+++++++++++++ ++

+++++

++++ ++++++++++++

++++++ ++++++++++++++++++

+ +++++++++++++++++++ +++++ +++++ ++++++++++++++

+++++

+++++++++

++++++++++

++++++ ++++++++++++++

++++

++++++

+++++++++++++++++

++++++

+ +++

++

+++++++++

++++++++++++++++

+++++ +++++++

+++++++++++++++++++++

+ +++++++++ ++++++++++++

+++

+++

+++++

++++++

+++++++++++++++++

++++++

+++++++++++++++++ +++++++

++++

++++++++++++++++

++

+++++

++++

+++++++++++++++++++++++++

++++++++

+++++ +++++

++++++++

++++++++++++++++++

+++++++++++++++++++++++

+++++ +++++++

++++++++++++++

+

++

+++++

++++++++

+++++++ ++++++++

++ ++++++ ++ ++++

++++++

+++++++++++++++ +++++

+++++++

+ ++++++ +++++++++++++++++++++

+ ++++

+++ ++

++ +++++++++++++++

+ ++++++++

++++++++++

+++++++++++

+++++ +++++ ++

+++++

+++++

++++ +++

++ +++++++++++

+++++++ +++ +++++++++++++++++++ ++++++

+++

++++++++

++++++++++

++ +++++++++++++++ +++

+ ++

++++

+++++++++++++++++++++ +++

++++++

++

++++++++

+

++ +++++

+++ ++

+++

++

++ +

++++ ++++++++++++++

+++

+++ +++++ +++++++++++++

+++++++++

++

+++ ++++++++++++ +++++++++++++++ ++++

++ +++++++++++++++++++

+ +++++++++ +++++++++++++++

+++++++

++ ++ +++

++

+++

+++++++

+++++++

+

++++++++

++

++++ ++++++ ++++++++

++++++++

+++++ ++++++++

++ +++++++++++++

++++

+++

++++++++++++++++++

++++++++ +++++++++++

++

++++++ +

+ ++++++

++++++ +++++++

+++

++++ +++++++++

+++++++++

++++

+++++++++++++++ +++++++++++

++++ ++++ ++++++++++++++++++++++++ +

++++++++++++++++++++++++++++++++++++++++++ +++++ ++

+++++

++++++

++++++++++

++++++++++++++++ ++

+++++++++++++

+++++

+++++++++ ++++++++

++++++++++++ +++ +

++++++++++++++++

++++++

+ ++++++++++++ +++ +++++++++++++ +++++

++ +++++++++

+++++++

+++++++++

+++++

++++++ +

++++++++

+++

++++++ +++++++++

+ ++++++

+++++++++ +++++++++++ +++++

++ +++++ ++

++++++++++++++

++++++++

+++++++++++++++ +

+

++++++++++ +

++++++++++++++ ++++++

++++++++ +

++++++++++++ +++++++++

++ ++++++

+++++++++++

+++++++

++++++++++++++++++++

+++++++++++++

++++ ++++

++++++++++

++++

+++++++++++++++++++++++++++

++++

++ +++++++

+++++ +++

+++++++++++++++++++++++++

++++++++++++++++++++++++++++

++

+ ++++++++++++

+++

++++++

++++++++++ +++++++++++

++++++++++++++++++++ ++++

+ +++++++++

++++++++

+++++

+ +++++++++++++++

+++++++

++++++++

+++++++

++++++++

+++ ++++++++ +

+++++++ ++++++++++

++++++++++++ +

+++++

+++++++++

++++++++ +

++++++

++++++++++++++ ++++++

+++++++++ ++++++++

++++++

++++++++++++

++++

++ +++++++++++++

+++++ ++++++

++

++++

++++

++

+++ ++++

+++++ ++++++

++++++ ++++++++++++

++ +++++

+ ++++++++++

++ ++++++++++++++++ ++

++++++

++ ++++++

+++++++++++ +++

++++++++++++++++++

++

+

+++++++

+ +++++++++++++++++++++++

+++ +++++++++++++++++++

+++++++++++++++++++

+++++++++

++

++++++ ++++

++ +++++++++++++++

++++++++++ ++

+++ ++++ +

+++

+++++++++ ++++ +++++++++++++ +++++++++++++++ ++

++++++ ++++++

+++++++++++ ++++++++

+++

+++ +

++++++++

+

++

+++ ++++++++++++++++++

++++

+++++++ ++++++ +++

+ ++ +++++

++++++++++++++++++++

++++++

+++++++++

+++

++++++++++

+++++++++++++ ++++++

++ ++++++++ ++++++++

++ ++++++++++++++ +++++ +

++++++++++++

++++++

+

++ ++++++++++++++

+++

+++++++++++

+++++++++++++

++++++++

++++++++ ++ ++

++++++++

++ ++

++

+++

+++++ ++

++++++++ ++++++++++++

+ ++++++++++++++++ ++++++++++++++++

+++ ++++++++++++++++++++++++

++ +++++++++++++++

++ ++++++ +++++++++++++++++

++++++++++

++++++

++++++ ++++++

+ ++++++++++++++++++++++++++++ +

+++++++++

+++ ++

+++

+++++++++

+++++++++++++++++

+++++

++

++++

+++ +++++

++++++

++ +++++ ++++

+++ ++++++++

++++++++++++++++++++++ ++

++ ++++++

+++++++

+++

++++

+++++++++

++

++++++

++++++++++

++++++++

++

++++

+

+++++++++++++++++++

++++ ++++

+++ +++++++++++++

+++ ++++++++ +++++++++++++

++++++++++++

+++++++++++ +++++++++++

+++

++++ +++++++ ++++

+++ ++++ +++ +++++++

++

+++++++++++++++++

+++++++++ +++++++++

++++++ +++++ ++++++

+++

++++

++++++++ +++++++++++

+++++++ ++ +++++++++++++++++++++ ++++

+ +++++++++++++

+++++++++++++++++++++ ++

+++ +++++++++++++++

++++ +++

+++++++

+++++++++++ ++ ++ +++

+

++++++++++

++++++

+++++++

++++ +++

++++++++++

+++++++

++++++

++++++++++

+++

+++ ++ +++++ +++

+++++

++ ++++++

+++++++++

+ +++ +++++++

+++++++ ++++ +++++

+++++++++

+

+++++++++++++

++++++

++++++++++++++ +++++++

+++ ++++++++++++++++ +++ ++++++

++++++

+++++++ +++++++ +++ ++ ++++++++++

+++++++++

+++++++++ ++++

+++++++

+++++++

+++++++++

++++++++++++

+++

++++++++++

+++++++++++++++++++++++++++++++++

++++++ +++++++++++++++++++

+++++++++

+++++++++++++++++++++++++

++++++++

++++++++++++

+++++++++++++++++++++

++++ +++++++++

++++

++++++++ +++

+++ ++

+++++++

++++++++

++++++++ ++++++++

+++++ ++++++++++++++++++

+++

+++++++

+++++++++++++++++

+++++

+++++

++++++ ++++++++++++++++++++++++

+ +++++ +++++++

++++++++ +

++++++++++++++++++++++++++++++++++ ++++

+++++

++++

++++++++++++++++++

++++++++++++++

+++++++

++ ++++++++ +

+ +++++++

++++++++++++++

++++++++++++++

+++++++

++++++++++++

+++++++

++++

+++++++ ++++ ++++++

++++++++++

++++

+++++++++++ +++++++ +++

+++++++ +++ ++

+ +++++

+++++++++++++++

+++ +++++

+++++++++++

++++++++++++++++++

+++++ ++++ ++++++++++++++++++++++++++++++

++

++++++++++++++++++++

+++++

+++++++++++++++++++++++++ ++++++++++++++++

+++++++++++++++++++++++++++++++++++++++

+++ +++++++++++ +++++ ++ +++++++++++ +++++

++++++

+++++ ++++++++++ ++

++++++++++++ ++++

+++++++++

++ ++++ ++++++

++++ +++ ++++++++ +++++

+++ +++++++ ++++

+++++ +++

+++++++++++

+++++ +++++++

+++++

++++++++++++ ++++ ++

+

+ +++++++++++++++++++++

++++ +

++++++ +++++++++

++++++++

+

++++

+++ ++++++ +++

++++

+++++++++++++

+++++ ++ +++ +++++

+++++++

+++++++++++++++++++

++++++++++++++

++

++++++++++++

+ +++

+++++++++ ++ +

++++++++++++++++

++++++++++++ +++

+++++++

+++++ +++++++ ++++

++++

++++ ++++++++++++++++++++++ ++++++++++++++

++++++++++

+++++++++++

+++++

+++ +++++++++++++++++++++

+++ +++++++++++++++ ++++++ ++

+ +++ +++

+++++

+

+++++++++

++++++++++++++++++++

+++ ++++++++

+++++ ++++

+++++++++

++

++++++++

+++++++++++

+

+++ +++++++++++++++++++++++++++++++++++

++++

+++++

++++++++ ++ +++

+ ++++ +++++

+ ++++ ++++++ +++++++

+++++++

++++ ++

++++++++ ++++++++++++ ++

+++

+

++++++++++++++++++

++++++++++++ ++++++

++++++

++++++ ++

++++ +++++++++

+++

++++++++++++

++++++

++++++++++++ +++++++++++++++++++++ +

++ +++ +++++++

+ ++++++

+++++++

+++++

++++

++++ +++++++++++++

+++++ ++++++ ++

+++

+++++++++++

+

+++++++++

++++++++++++++++++++

++++++

+++++++++

+++++++ ++++++++++++++

++++++++++++

++++++++++

++

++++

++++++++ ++++++

++++++++++

+ ++++ +++++

+++

++++++++ +++

++++ ++++++++++++

++ ++

++++++++++ ++++

+++++++++++++++++++++++++++++++++++++++

+++++++++++

+++++++++++

++++++++

+++++++++

++++

+++

+++++++++++++++++++++++++

+ ++++++++++++++++++++++

++

++++++++ ++ +++

+++++++++ ++++++

+++

+++

++++++++++++

++ +++++++++ +++++++ +++

+++++

++ +++ ++++++++++++ ++++++++

++++++++++++++

+++

+ ++++ +++

+++++++ +++++

+++++

+++++++++

+

++++++++++++++++++

++++

+ ++++++++ +++++

++++++++++++++

+++++++

++++++

++

+++++++++++++

+++++

+++++++++++

+++++++++++

+ +++++++++ +++

++++++ +++++++++

++++++++++++++++++++

+++++++++++++++ ++++ +++++++++++

+++++ ++++++++++++++++ +++++

++++++++++++++

++++++++++++++

++++++++++++ ++++

++++++++

++++++ +++++

++

++++ +++++

+++

++

+ ++

++++

+++

++

++++++++++++++++++++++

++++++++ ++++++++++++

++++++

++++++

+++++

++++++++ +

+++++++++++++++++

+++++ ++++++

+ +++++++++++ +++++++++++++++++++++

+ +++

++++++ ++

++++

++++++

+++++++++

+++++

+++++++ +++

++++++++ +

+++ ++++++++++++ ++++++++

++++++

+++++++

++

+++++++

+++++++++

+++

++++ ++++++++++++++++

(e) (µ, τ) joint

15 / 21

Calibration, cont.

Is it even possible to find a ω that tunes coverage?

Not obvious, but it seems the answer is YES...

Example: Model says X1, . . . ,Xniid∼ N(θ, σ2)

Fixed σ2 but the true variance is different, say, ψ2.Flat prior Gibbs posterior for θ is N(X , ω−1σ2n−1).Calibration achieved with ω = σ2/ψ2.

Similar asymptotic results can be obtained more generally,involving 2nd derivative of Rn and usual sandwich covariance.

These results are similar to those obtained by other efforts attuning the “learning rate” ω — more later...

16 / 21

Calibration, cont.

We developed an algorithm8 to tune toward coverage.

For simplicity, assume distribution P is known; if unknown,then replace it by the empirical distribution Pn.

General Posterior Calibration Algorithm:

0 Initialize ω0 and set t = 0

1 Sample lots of data sets from P to approximate the coverageprob of Gibbs credible region Cωt ,α.

requires Gibbs posterior computation for each data setcheck coverage using θ(P)

2 If coverage prob is close to 1− α, then stop; otherwise,update ωt → ωt+1 via stochastic approximation, increment t,and go back to Step 1.

8This algorithm is naive, can be made more efficient...17 / 21

Calibration, cont.

Two-dim quantile regression example

Several methods:

BEL.s Bayesian empirical likelihood9

Normal asymptotic normalityGPC Gibbs with GPC

GPC gives exact coverage and shorter length!

Coverage Prob ×100 Avg Length ×100n BEL.s Normal GPC BEL.s Normal GPC100 θ0 97 95 95 106 100 91

θ1 98 98 95 58 55 47400 θ0 95 95 95 50 50 46

θ1 97 97 95 26 25 231600 θ0 96 96 95 25 25 23

θ1 96 96 95 13 12 11

9Yang and He (Annals, 2012)18 / 21

Conclusions

Gibbs approach makes direct posterior inference possible:

no nuisance parametersno marginalization“real” priors

Nice theory and good practical performance.

Choice of scale, ω, is very important.

new algorithm designed for calibrationmore can still be done

Connections to other learning rate tuning methods?

More Gibbs stuff to do:

More applicationsGPC in high-dim problemsGeneral Gibbs concentration rate theoremsConnecting CS/ML/Stat literature on this

19 / 21

Commercial

The peer review system is broken in various ways.

Successful reform requires new ideas.

Harry Crane and I developed a new open-access publicationplatform, featuring an author-driven peer review process.

For details, check us out at

www.researchers.one

www.twitter.com/@ResearchersOne

20 / 21

The end

Thank you!rgmarti3@ncsu.edu

www4.stat.ncsu.edu/~rmartin

21 / 21