2.4. The 95% interval of Gamma(2,1)

(Optional question)Jungho Ryu

The situation

• We need to find the 95% confidence interval of Gamma distribution f(x) with one of these conditions satisfied:1. Choose two points x0 and x1 with x0 < x1 such that

I. f(x0) = f(x1)II. For all (x’ < x0) or (x’ >x1) , f(x’) < f(x0).

2. (equivalently) Choose two points x0 and x1 with x0 < x1 such that

I. f(x0) = f(x1)II. For all possible x0’ and x1’ with x0’ < x1’ and f(x0’) = f(x1’)

and F(x0) – F(x1) = 0.95, x1’ – x0’ > x1 – x0.

The situation

• We need to find the 95% confidence interval of Gamma distribution f(x) with one of these conditions satisfied:1. Choose two points x0 and x1 with x0 < x1 such that

I. f(x0) = f(x1)II. For all (x’ < x0) or (x’ >x1) , f(x’) < f(x0).

2. (equivalently) Choose two points x0 and x1 with x0 < x1 such that

I. f(x0) = f(x1)II. For all possible x0’ and x1’ with x0’ < x1’ and f(x0’) = f(x1’)

and F(x0) – F(x1) = 0.95, x1’ – x0’ > x1 – x0.

Pictorically SpeakingCase 1: all points not belonging to the interval have lower f(x)

biggest f(x) are in the 95% interval

All x’s that belong to 95% interval

f(x0) f(x1)

Conceptually Speaking

• Notice that x0 and x1 are on the opposite side of the peak (global maximum)– So we need to find the global maximum

• To always include only the biggest f(x)’s, we can start x0 and x1 from the top and go down– while f(x0) and f(x1) keeping the same!

• But the sides look different (function is not symmetrical!)

Into action!

• First we need to find that global maximum (easy!: Find it analytically).Gamma Distribution (scale = 2, shape = 1)f(x) = x e-x.We take the derivative and set it to 0.f’(x) = e –x – xe-x = 0 e –x = xe-x x = 1

The global maximum is at x =1.So, x0 < 1 and x1 > 1.

Ideally,

• We would update (decrease) x0 and then find the inverse f-1 to get x1. – Not doable analytically.

• Then we do it numerically:– Update (decrease) x0 calculate f(x0) slowly

update (increase) x1 so that f(x1) ≈ f(x0)• Actually, f(x1) + ɛ = f(x0)

We lead with x0 and update with x1 because f(x1) decreases slower (therefore more accurate).

Describe the Update in PictureStart with x0 and x1 close to 1.

Possible 95% interval candidate

f(x0) f(x1)

1

Describe the Update in PictureMove the x0 to a new position

Possible 95% interval candidate

f(x0)

f(x1)1 1. Lower x0 lower f(x0)

Describe the Update in PictureIncrease x1 little by little

biggest f(x) only

Possible 95% interval candidate

f(x0)

f(x1)2

1. Lower x0 lower f(x0)2. Increase x1 slowly so

that f(x1) ≈ f(x0)

Describe the Update in Picturef(x1) ≈ f(x0)

biggest f(x) only

Possible 95% interval candidate

f(x0)f(x1)

1. Lower x0 lower f(x0)2. Increase x1 slowly so

that f(x1) ≈ f(x0)

Describe the Update in PictureCheck to see the interval represents 95%

Possible 95% interval candidate

f(x0)f(x1)

1. Lower x0 lower f(x0)2. Increase x1 slowly so

that f(x1) ≈ f(x0)3. Check whether the

interval F(x1)-F(x0) is big enough.

3

Describe the Update in PictureAll together

biggest f(x) only

Possible 95% interval candidate (check)

f(x0)

f(x1)1 2

3

1. Lower x0 lower f(x0)2. Increase x1 slowly so

that f(x1) ≈ f(x0)3. Check whether the

interval F(x1)-F(x0) is big enough.

the Algorithm

• To start the algorithm: f(x0) > f(x1) F(x1) – F(x0) < 0.95.

Iterates by: Update x0, and then update x1. Check for F(x1) – F(x0) > 0.95. Print result of each iteration

Algorithm Ends when: F(x1) – F(x0) ≈ 0.95

(actually F(x1) – F(x0)= 0.95 + ɛ’)

Setting up BEFORE the algorithmPositions of f(x0) and f(x1)

Definitely smaller than 95%

f(x0)

f(x1) • f(x0) < 1 < f(x1)• f(x0) < f(x1)• F(x1) – F(x0) < 0.95

Our Algorithm Conceptually

Initiate x0 = 0.7, x1 = 1.3( f(x0) = 0.3476 and f(x1) = 0.3543. F(x1) – F(x0) = 0.21737)

while (F(x1) – F(x0) < 0.95)

Our Algorithm Conceptually

Initiate x0 = 0.7; x1 = 1.3; define f(x)while (F(x1) – F(x0) < 0.95)

x0 = x0 - 0.001while ( f(x1) – f(x0) > 0)

x1 = x1 + 0.001 (UPDATE)

Our Algorithm Conceptually

Initiate x0 = 0.7; x1 = 1.3; define f(x)while (F(x1) – F(x0) < 0.95)

interval = F(x1) – F(x0)print x0, x1, f(x0), f(x1), interval

x0 = x0 - 0.001while ( f(x1) – f(x0) > 0)

x1 = x1 + 0.001 (UPDATE)

Everything Correct (Final)x0 = 0.7; x1 = 1.3; f = function(x){p = x*exp(-x)

return(p)}while (pgamma(x1, shape =2) –

pgamma(x0, shape=2) < 0.95){

x0 = x0 – 0.001while (f(x1) – f(x0) > 0){

x1 = x1 + 0.001}

interval = pgamma(x1, shape =2) – pgamma(x0, shape=2)

print(c(x0, x1, f(x0), f(x1), interval))}

The Output

…

x0, x1, f(x0), f(x1), interval

Optional: Improving the algorithm

• We saw epilson ɛ twice.• This implies that the algorithm is biased!

The algorithm will always have the right point a little too far out and the interval slightly too big!

Ideally,

• We would update (decrease) x0 and then find the inverse f-1 to get x1. – Not doable analytically.

• Then we do it numerically:– Update (decrease) x0 calculate f(x0) slowly

update (increase) x1 so that f(x1) ≈ f(x0)• Actually, f(x1) + ɛ = f(x0)

We lead with x0 and update with x1 because f(x1) decreases slower (therefore more accurate).

the Algorithm

• To start the algorithm: f(x0) > f(x1) F(x1) – F(x0) < 0.95.

Iterates by: Update x0, and then update x1. Check for F(x1) – F(x0) > 0.95. Print result of each iteration

Algorithm Ends when: F(x1) – F(x0) ≈ 0.95

(actually F(x1) – F(x0)= 0.95 + ɛ’)

Correcting Bias

• We can correct the bias by adding or subtracting a small amount to negate ɛ.

• How much? Look at the output and take the average

• Where? Right after the updates are done

BUT ON NEXT TIME!