Es#ma#ng  the  costs  of  scaling  up  immuniza#on:  evidence  from  the  EPIC  studies

Nicolas  Menzies,  Kyu  Eun  Lee,  Zach  Ward,  Chris9an  Suharlim,  Stephen  Resch    

Outline  

• Mo9va9on  •  EPIC  Study  background  •  Issues  with  pooling  data  • Basic  data  descrip9on  • Analy9c  approach  • Results  • Next  steps  • Conclusions  

2  

Mo#va#on

• Mul9-­‐country  study  using  a  common  approach  

• Useful  to  have  beMer  informa9on  on  costs  of  suppor9ng  immuniza9on  services:  • What  is  the  average  cost  per  fully  immunized  child  (as  compared  to  historical  benchmarks)?  •  How  do  costs  differ  across  countries  and  what  explains  these  differences?  • What  are  the  site-­‐level  determinants  of  the  cost  per  outcome?  • What  are  the  costs  of  scaling  up?  

3  

Purpose  of  EPIC  Study:  to  update  evidence  on  rou9ne  immuniza9on  program  costs  and  financing  based  on  detailed  facility-­‐level  analysis  

EPIC  Study  Country  Characteris#cs  (Year  2011) Country   GNI  per  capita   Infant  Popula8on   DTP3  Coverage  

Level  (1)  Uganda   $470   1,326,826     80%  Benin   $720   348,577   85%  Zambia   $1,180   567,320   81%  Ghana   $1,420   1,011,012   91%  Moldova   $1,980   47,537   93%  

Honduras   $2,030   177,733   91%  

1 WUENIC Estimates, 2011. DTP3 = Diptheria, Tetanus, Pertussis Vaccine, 3rd dose  

Facility  Unit  Costs  for  the  EPIC  Studies

Country   Weighted  Average  Cost/  

Dose  

Weighted  Average  Cost/  Fully  

Immunized  Child  (i.e.  DTP3)  

Share  of  Above  Facility  Costs  

Government  Share  of  Total  Financing  of  Immuniza8on  

Program  Benin   $2   $25   6%   45%  Uganda   $5   $44   16%   45%  Ghana   $5   $51   31%   85%  Zambia   $7   $66    26%   82%  Honduras   $9    $128    14%   64%  Moldova   $18   $332   18%   95%  

2011  US  Dollars  

Cost  Profiles  for  the  EPIC  Studies  –  Facility  Level

16%  37%   47%   47%  

61%   71%  65%   34%  

32%   43%   19%  14%  

19%   28%   21%   10%   20%   15%  

BENIN   UGANDA   ZAMBIA   HONDURAS   GHANA   MOLDOVA  

Labor   Vaccine   Other  

Pooling  data  I

• Data  from  6  country  studies  available  in  a  (mostly)  standardized  format.  •  Some  ‘real’  differences  between  country  studies  –  ques9ons  asked  in  different  ways,  some  ques9ons  not  asked,  some  concepts  inherently  local  (e.g.  defini9ons  of  HS  level)  • Addi9onal  differences  in  data  coding  and  format.  • Consistency  checks  with  communica9on  with  country  teams  to  resolve  apparent  inconsistencies  • Goal  to  develop  a  publicly  accessible  common  dataset  with  standardised  defini9ons,  missing  data  chased  where  possible,  links  between  summary  and  details  informa9on.  

7  

Pooling  data  II

• Where  up  to  so  far:  •  S9ll  working  on  preparing  full  dataset  •  Current  cost  informa9on  don’t  include  regional/na9onal  level  costs  •  Don’t  yet  have  a  full  division  of  cost  categories  for  all  countries  •  Some  explanatory  variables  s9ll  ‘in  progress’  

•  For  this  analysis:  •  Any  missing  data  imputed  by  mean  regression  imputa9on  (will  underes9mate  uncertainty,  though  only  minor  missingness)  •  Avoided  variables  with  country-­‐level  missingness  •  Avoided  variables  with  known  measurement  error  (this  comes  back  later)  

8  

Descrip#ve  sta#s#cs  I •  Sites  per  country  /total   •  Major  cost/outcome  variables  

9  

County   N  

Benin   45  

Ghana   50  

Honduras   71  

Moldova   50  

Uganda   49  

Zambia   51  

TOTAL     316  

Variable   Mean  (sd)  

Total  cost  (USD)   21,700    (25,600)  

Total  Doses   5,993    (10,056)  

Total  DTP3   439    (774)  

                 …  facility   289    (514)  

                 …  outreach   150    (325)  

Total  Measles   450    (781)  

                 …  facility   303    (545)  

                 …  outreach   147    (316)  

Descrip#ve  sta#s#cs  II

• Distribu9on  of  costs/outcomes:  

10  

Total Cost (ln scale)

1,000 10,000 100,000

No. zeros = 0

Total Doses (ln scale)

10 1,000 100,000

No. zeros = 0

DTP3 (ln scale)

1 100 10,000

No. zeros = 0

Measles (ln scale)

1 100 10,000

No. zeros = 0

Dtp3_Coverage

0.0 0.5 1.0 1.5 2.0

Wealth_Ratio

0.3 0.5 0.7 0.9

ANC_4

0.3 0.5 0.7 0.9

Per Cap GDP (ln scale)

500 1,000 2,000 4,000

Descrip#ve  sta#s#cs  III

ln_Tot_Cost

0 2 4 6 8 4 6 8 10

789101112

02468

ln_Dtp3

ln_Measles2468

7 8 9 10 12

46810

2 4 6 8

ln_Doses

11  

Descrip#ve  sta#s#cs  IV

ln_Tot_Cost

0.0 1.0 2.0 0.4 0.6 0.8

789101112

0.00.51.01.52.0

Dtp3_Coverage

Wealth_Ratio0.51.01.52.02.5

0.40.50.60.70.80.9

ANC_4

7 8 9 11 0.5 1.5 2.5 6.5 7.0 7.5

6.57.07.5

ln_GDP_2011

12  

Analy#c  approach  I •  Issues  to  deal  with:  •  In  each  country,  cluster  sampling  approach  for  site  selec9on  à  Analysis  adopted  mul9level  model  with  random  effects  for  cluster  levels    

•  Comparability  of  some  measures  across  countries  à  Presence/absence  of  beds  used  to  proxy  for  health  system  level  à  ‘peri-­‐urban’  and  ‘semi-­‐urban’  categories  pooled  into  ‘urban’  category    

•  Some  poten9ally  informa9ve  variables  s9ll  under  construc9on  à  Restricted  subset  of  parameters  used  in  these  analyses  

•  LiMle  price  varia9on  within  country  à  Limited  scope  to  inves9gate  impact  of  price  differen9als  

13  

Analy#c  approach  II

à  Regression  on  log  total  cost  –  implies  mul9plica9ve  rela9onship  between  costs  and  linear  predictor    

à  Predictors  include  logged  measures  of  service  provision  (e.g  total  doses),  other  characteris9cs  of  site  /  senng,  and  log  per  capita  GDP  (crude  country-­‐level  price  index)  

à  Random  effects  for  country  (δc)  and  province  (δp).  

à  Opera9onalized  in  a  Bayesian  framework,  weakly  informa9ve  priors  for  regression  coefficients  and  variance  terms  

14  

!" !"! = !!! + !!!!"!!"

+ !!!! + !!! + !! !

Results  I

15  

Variable*)Model)specification)

1) 2) 3) 4) 5) 6)Intercept) 9.48))(0.06)) 9.48))(0.31)) 9.46))(0.27)) 9.47))(0.27)) 9.52))(0.19)) 9.55))(0.25))

Output)volume) ) ) ) ) ) )ln(Doses)) ) ) 1.03))(0.03)) 1.03))(0.03)) 1.05))(0.04)) 0.98))(0.19))

ln(Doses))sq) ) ) ) G0.03))(0.02)) G0.02))(0.02)) 0.07))(0.03))Other)predictors) ) ) ) ) ) )

Govt)Owned) ) ) ) ) G0.09))(0.11)) G0.11))(0.10))Urban) ) ) ) ) 0.09))(0.10)) 0.08))(0.10))

ln(pre)capita)GDP)) ) ) ) ) 0.26))(0.18)) 0.35))(0.25))ANC4) ) ) ) ) 0.13))(0.08)) 0.05))(0.07))

Facility)Delivery) ) ) ) ) G0.13))(0.07)) 0.03))(0.07))Wealth)Ratio) ) ) ) ) G0.04))(0.07)) G0.11))(0.07))

Country)random)effects)for)intercept) ) Yes) Yes) Yes) Yes) Yes)Country)random)effects)for)ln(Doses)) ) ) ) ) ) Yes)Variance)parameters) ) ) ) ) ) )

Error)term) 1.06))(0.04)) 0.86))(0.04)) 0.44))(0.02)) 0.44))(0.02)) 0.43))(0.02)) 0.41))(0.02))WAIC*) 932.5) 833.4) 399.8) 400.8) 399.4) 361.3)

*)WatanabeGAkaike)information)criterion)approximates)outGofGsample)prediction)error)for)the)fitted)model.)Lower)values)suggest)better)model)fit.))

First  Differences

16  

Comparison*+Percentage+difference+in++average+cost+per+dose**+

Each+country,+vs.+overall+mean++ Benin+(incl.+per<capita+GDP+effects):+ Ghana(( Honduras(( Moldova(( Uganda((( Zambia(

<39%+++(<67%,+0.0%)+3.7%+++(<41%,+66%)+<7.0%+++(<48%,+48%)+98%+++(11%,+231%)+<41%+++(<67%,+<2.5%)+56%+++(<13%,+160%)+

Government<owned+sites,+vs+non<government<owned+sites.+ <10%+++(<27%,+9.7%)+Urban+sites,+vs+rural+sites.+ 8.9%+++(<9.9%,+32%)+ANC+coverage+20%+higher.+ 3.8%+++(<5.8%,+14%)+Facility+delivery+rates+20%+higher.+ <1.3%+++(<4.3%,+7.2%)+Wealth+ratio+20%+higher.+ <4.0%+++(<8.6%,+0.07%)+GDP+20%+higher.+ 13%+++(<5.6%,+34%)+Service+delivery+volume+20%+higher+ Benin+(vs+country+median):+ Ghana(( Honduras(( Moldova(( Uganda(

(( Zambia(OVERALL+

<4.9%+++(<7.8%,+<1.9%)+<9.9%+++(<12%,+<7.9%)+<5.1%+++(<6.4%,+<3.8%)+<3.1%+++(<4.9%,+<1.4%)+<4.9%+++(<6.7%,+<3.0%)+<4.9%+++(<7.8%,+<1.9%)+<8.7%+++(<11%,+<6.5%)+

*+Controlling+for+all+other+model+parameters+except+for+those+described+in+the+comparison.++

Total  costs  with  increasing  service  volume

17  

DoseVals

colM

eans

(TC

1b)

●

●

●

●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

● ●●

●

●●

●

●

●●

●●

●

● ●

●

●

●

●●

●

●

●

●

0 5 10 15 20 25

0

10

20

30

40

50

60

Benin

DoseVals

colM

eans

(TC

1b)

●

●●

●

●

●●

●

●

●

●

●●

● ●

●

●

●

●

●●

●

●

●

●

●

●●

●●

●●● ●

●

●●

●

●

●●

●

●

●

●●●

●

●

●

0 5 10 15 20

0

10

20

30

40

50

60 Ghana

DoseVals

colM

eans

(TC

1b)

●●●●●

●

●●●

●

●

●●●

●

●●●●

●●●●●

●

●

●

●●

●●

●●

●

●●●

●

●●●●

●

●●

●

●●●●●

●

●

●

●

●

●●●

●

●

● ●

●

● ●●●

●

●

●

0 10 20 30 40 50 60 70

0

50

100

150

200

250 Honduras

DoseVals

colM

eans

(TC

1b)

●●●●●

●●●●

●

●

● ●●●●●

●

●

●

●●

●●● ●●

●

●●●●●

●●●●●

●●

●

●●●●●

●

●●

●

0 2 4 6 8

0

50

100

150 Moldova

DoseVals

colM

eans

(TC

1b)

●

●

●●●

●

●●●

●●●

●

●

●

●●●

●

●

●

●●●

●

●

●

●●●

●

●●

●

●●●

●

●●●

●●

●

●

●●

●●

0 10 20 30 40 50 60 70

0

50

100

150Uganda

DoseVals

colM

eans

(TC

1b)

●

●●●●

●●●

●

●●

●●●

●

●

●●●

●

●

●

●

●●

●●●●

●

●

●●

●

●

●● ●●

● ●●●●

●

●●●

● ●●

0 10 20 30 40 50

0

50

100

150 Zambia

●Mean estimate 95% interval Study data Overall study average

Tota

l Cos

t (U

SD, 0

00s)

Doses (000s)

Total  costs  with  increasing  service  volume  (log-­‐log)

18  

DoseVals

colM

eans

(TC

1b)

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●●

●

●●

●

●

●●

●●

●

●●

●

●

●

●

●

●

●

●

●

2 5 10 20

5

10

20

50Benin

DoseVals

colM

eans

(TC

1b)

●

●

●

●

●

●●

●

●

●

●

●●

● ●

●

●●

●

●

●

●

●

●

●

●

●●

●●

●

●● ●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

0.2 0.5 1 2 5 10 20

5

10

20

50 Ghana

DoseVals

colM

eans

(TC

1b)

●

● ●

●●

●

●

●

●

●

●

●●

●

●

●●

●●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

● ●

●●

●

●

●

0.2 0.5 1 2 5 10 20 50

2

5

10

20

50

100

200Honduras

DoseVals

colM

eans

(TC

1b)

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●●

●

●

●

●

●

●●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

0.05 0.1 0.2 0.5 1 2 5 10

0.512

51020

50100200

Moldova

DoseVals

colM

eans

(TC

1b)

●

●

●

●

●

●

●

●●

●●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

0.2 0.5 1 2 5 10 20 50

1

2

5

10

20

50

100

200

Uganda

DoseVals

colM

eans

(TC

1b)

●

●

●

●●

●●

●

●

●●

●

●●

●

●

●

●●

●

●

●●

●

●

●

●

●

●

●

●

●●

●

●

●

● ●●

●●

●●●

●

●

●●

● ●●

1 2 5 10 20 50

10

20

50

100 Zambia

●Mean estimate 95% interval Study data Overall study average

Tota

l Cos

t (U

SD, 0

00s)

Doses (000s)

Average  costs  with  increasing  service  volume

DoseVals

colM

eans

(AC

1b)

●

●

●

●

●●

●

●

●●

●

●

●

●● ●

●

●

●

●

●

●

●●

●●

●

●

●

●

●

●● ●

●

●●

● ●

●

● ●

●

●

●

0 5 10 15 20 25

0

1

2

3

4

5Benin

DoseVals

colM

eans

(AC

1b)

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

● ●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

0 5 10 15 20

0

5

10

15Ghana

DoseVals

colM

eans

(AC

1b)

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●●

●

●

●

●

●

●

●

0 10 20 30 40 50 60 70

0

5

10

15 Honduras

DoseVals

colM

eans

(AC

1b)

●

●

●

●

●

●

●

●● ●

●

●

●

●

●

●

●

●●

●●

●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●

●

●●

●

●

0 2 4 6 8

0

5

10

15

20

25

30Moldova

DoseVals

colM

eans

(AC

1b)

●●

●

●

● ●

●

●●

●●

●

●

●

●

●

●●

●●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●

●●●

●

●●

●

●

●

●

●

●●

0 10 20 30 40 50 60 70

0

2

4

6

8

10

12Uganda

DoseVals

colM

eans

(AC

1b)

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●

●●

●

●● ● ●

●

●●

●

●

●

●

●

●

●

●

●

●

●

●●

●

● ●●

●

●

●

●

●●

●●

0 10 20 30 40 50

0

5

10

15 Zambia

●Mean estimate 95% interval Study data Overall study average

Aver

age

Cos

t Per

Dos

e (U

SD)

Doses (000s)19  

Marginal  costs  with  increasing  service  volume?

20  

xx

colM

eans

(MC

1b)[−

1]

0 5 10 15 20 25

0

1

2

3

4

5

6Benin

xx

colM

eans

(MC

1b)[−

1]

0 5 10 15 20

0

2

4

6

8

10

Ghana

xx

colM

eans

(MC

1b)[−

1]

0 10 20 30 40 50 60 70

0

1

2

3

4

5

6

7

Honduras

xx

colM

eans

(MC

1b)[−

1]

0 2 4 6 8

0

5

10

15

20 Moldova

xx

colM

eans

(MC

1b)[−

1]

0 10 20 30 40 50 60 70

0

1

2

3

4

5

6 Uganda

xx

colM

eans

(MC

1b)[−

1]0 10 20 30 40 50

0

1

2

3

4

5

6 Zambia

Mean estimate 95% interval Overall study average

Mar

gina

l Cos

t Per

Dos

e (U

SD)

Doses (000s)

no.  

What  we  expect:  

•  Possibly:  expecta9ons  wrong,  or  observing  sites  not  yet  near  full  coverage  • More  likely:  es9mates  biased  without  controlling  for  catchment  pop  

21  

Big  economies  of  scale  as  sites  get  up  on  their  feet  

Some  stuff  happens  in  the  middle  

Increasingly  hard  to  get  to  that  last  kid  

Margina

l  Cost  

Coverage  0%   100%  

Conclusions

•  Service  volume  a  clear  cost  determinant  à  strong  nega9ve  rela9onship  with  average  costs  

• Rela9onship  of  costs  to  service  volume  different  by  country,    though  qualita9ve  rela9onship  the  same  

• Other  determinants  (those  assessed  so  far  )  have  rel.  weak  rela9onship  to  total  costs  

•  Strong  conclusions  on  costs  of  scaling  up:  not  there  yet.    

22  

Next  steps

•  Finish  data  processing  –  cleaner  data,  inclusion  of  regional/na9onal  overheads  

• Reconsider  specifica9on  with  full  range  of  covariates  available  

•  Es9ma9ng  marginal  costs:  can  we  make  use  of  noisy  measure  of  catchment  popula9on?  

• Approaches  for  projec9ng  future  na9onal  costs  under  various  scenarios.    

• Project  website  forthcoming:  

23  

immunizationcosting.org