Measuring Pneumococcal Conjugate Vaccine (PCV) Impact: September 2013 WHO Meeting Conclusions

Cancun December 10, 2013 Claire Broome, MD

cvbroome@gmail.com

Talk Objective

• Summarize discussion and conclusions from September 2013 meeting of global technical experts and WHO Geneva and all Regional Offices

• Meeting had sessions with presentations and discussions on two major PCV impact endpoints – Naso-pharyngeal carriage

– Pneumonia, including WHO chest x-ray standardization

• Meeting also considered disease burden models and communication challenges

• Identify information that may be useful to country plans to use surveillance to monitor PCV impact

PCV Impact Evaluation • Are more impact studies needed?

– Scientific questions remain:

• Impact on pneumonia,

• serotype replacement

• effectiveness of particular vaccine serotypes

• measurement and definition of invasiveness

• indirect effects including among older children and adults

• Does every country need to have an impact evaluation of some kind?

– No---Bridging of data from one setting to another, models acceptable

What is the role of Naso-pharyngeal (NP) carriage studies in PCV impact evaluation?

• Carriage studies alone inadequate to document PCV impact. – NP data can serve as proxy for impact on Vaccine Type (VT) disease due to

“typical” VTs.

– NP carriage data are NOT adequate proxy for trends in epidemic types (serotypes 1 and 5) and NVT disease (i.e., replacement disease)

• Carriage studies may be useful in non-vaccine target groups, such as adults, or those too young to be immunized (i.e. neonates)

• What information is still needed to better define the role of NP studies?

– Stability of invasiveness across various settings and over time; use of colonization in pneumonia cases as an impact monitoring tool; behavior of outbreak/invasive serotypes in colonization and their relationship to disease (e.g. 1, 5, 12F), influence of vaccine on outbreak types, understanding complex carriage dynamics, eg carriage of multiple strains, role of subdominant strains & strain density

– Logistic issues for NP studies in lower income countries (lab support )

– Substantial methodological issues with current analytic models ( NP carriage; trying to predict disease impact from NP carriage data)

Evaluation of PCV Impact on Pneumonia • Vaccine endpoint-- the syndrome of pneumonia

– Important to policy makers—major morbidity and mortality

– NOT pneumonia due to pneumococcus, or VT pneumococcus

• Secular trend analysis ( pre-post analysis one example) usual approach—meeting identified caveats, and strategies to improve studies

• Burden of disease baseline data—pre-PCV introduction-- of sufficient duration important—preferably 2 years or more

• Secular trends in identified pneumonia disease burden unrelated to vaccine introduction need monitoring----eg malaria control, HIV and HAART, changes in area population, health services access

Pre-post analyses---addressing risks of method

• If phased introduction necessary or appropriate, can use areas with later introductions as controls

• Multiple years, sufficient sample size, broad geographic area, denominator

• Negative control condition—for example dehydration – Choose condition NOT affected by vaccine

• Catch-up campaign may help—easier to see rapid, obvious impact – Regardless of actual vaccine roll-out strategy, data on

vaccine distribution is necessary for impact results • Essential to have consistent, unchanging study methods and

case identification methods over the full time-period of observation----avoid increasing sensitivity of surveillance especially at the time of PCV introduction

• Duration of monitoring----5+ years after introduction, PCV effects are still being seen in the whole population

Evaluation of PCV Impact on Pneumonia

• Case definition of pneumonia----central issue

– Focus on groups where higher proportion of pneumonia likely due to pneumococcus (specificity)

– CXR documented; hospitalized; age <2 years

– Other possible adjuncts: NP, urine (serotype specific—effectiveness yet to be established ), CRP, other biomarkers, hypoxia

– Efforts to increase specificity must be balanced with needs for sensitivity – will under-estimate the total burden of disease and vaccine impact if definition is highly specific (eg bacteremic pneumococcal pneumonia )

Utility of randomized controlled trials (RCT) to better understand pneumonia syndrome

• Use Northern California Kaiser individually RCT of PCV7 efficacy as an example

• WHO commissioned reanalysis of chest x-rays (CXR) from this RCT. – Possible to compare original routine point of care

reading in 23 clinical sites with WHO standardized protocol (reading by two experts in pediatric CXR’s with discrepancies evaluated by reference panel)

• With RCT data, can compare different sub-groups without concern for bias since vaccine assigned randomly

Effect of age on PCV pneumonia impact: Kaiser USA Randomized Controlled Trial of PCV 7

CXR positive, original point of care reading

Age Per protocol efficacy

95% confidence interval

<12 months 32.2% 3.3, 52

12-23 months 18.9%

>24 months 9.1% -30.9, 36.8

Data from Kaiser RCT PCV 7 Black et al PIDJ 2002

Effect of CXR case definition on PCV impact: Kaiser RCT

CXR reading method

Per protocol efficacy

95% confidence interval

Original point of care

20.5 4.4, 34

WHO protocol-radiology panel

30.3 10.7, 45.7

Hansen et al PIDJ 2006

Difficulty of CXR reading standardization, with WHO protocol and experienced

readers, Kaiser RCT CXR Results by Readers A & B

Number of CXR

# Positive by reference

panel

% “true” positive

Positive by A & B 250 20/25 80%

Positive by A only 129 45 34.9%

Positive by B only 142 66 46.5%

Negative by A & B 1925 0/25

Hansen et al PIDJ 2006

Trade-offs in designing pneumonia impact studies– not single right answer

Routinely collected data Specially collected data

Positive Extensive retrospective baseline possible if system stable

Can add key data eg-CRP Can implement standardized data collection eg CXR reading by panel

Negative Reliability of data collection must be assessed eg CXR reading may be variable

Expensive May not be time to establish baseline if PCV introduction is soon

Sensitive case definition Specific case definition

Positive Measure more complete impact Easier to detect impact with higher portion of pneumococcal disease

Negative May include smaller proportion of pneumococcal disease, pre-post difference harder to detect

Likely to underestimate total impact

Chest Radiography— standardization discussion

• Core support needed for sites conducting standardized CXR based pneumonia impact – Teaching package – New set of films, new software training – Training on technical quality of CXRs – Ongoing QA – Safety assessments – Standard interpretation and methodologies – Radiology reference group

• Is a revision of the WHO CXR guidance document needed? And if so what are the

key missing elements to revise? No consensus on any changes to definitions or procedures—definition of density, size, silhouette sign, requirement that readers agree not only on presence of pneumonia but also side

• If definitions are revised, how can we maintain a bridge to studies using the current definitions ( eg trials)? Discussed but no consensus on any changes

• Does it matter if films are taken by analogue or digital? Prefer digital

Use of models for estimating PCV impact on disease burden

• Country Cost Effectiveness (CE) analysis likely to become KEY for sustainability of vaccine investment

• Simplicity of models important and transparency is critical

• Need to use existing models where appropriate (e.g., TRIVAC, PCV online CE model)

• Differentiation between what is measured in ‘special studies’ (e.g.pneumonia with WHO standardized CXR readings ) and data from routine hospital data

– Bridging information between the impact studies and the CE models

PCV Impact: Sharing the evidence- communication needs and approaches

• Risk mitigation and planning:

– expect increase in number of PCV impact study results, and expect heterogeneity of results---some of which will result from methods---significant risk of misinterpretation and therefore risk to PCV programs

– critical to link back to impact on IPD surveillance data where these exist so that MOH can respond to data requests

– critical to develop strategies for responding to anti-vaccine messages

• creating key message documents, briefing documents, other strategic communication tools with input from expert epidemiologist who are able to frame and contextualize results

ADDITIONAL INFORMATION

Case control assessment of PCV impact on pneumonia

• Trade-offs

– Can be done in absence of pre-introduction data

– Allows for adjustment of likely confounders-eg host risk factors (see background document for additional design issues)

– Will not provide information about overall disease reduction impact

– Concern about point estimate variability for vaccine effectiveness in pneumonia case control studies in LMIC

– Expected effectiveness 30-40% gives wide spread of point estimates in small studies, due to chance alone

Precision of Effectiveness Estimates by True Efficacy Level: A Monte Carlo Simulation

Ward et al Pediatrics 1988

Distribution of results of five independent VE studies, each with 75 cases and 150 controls; vaccine coverage in controls 25%

Impact of case definition on HibCV PRPT effectiveness against pneumonia

Case Definition Vaccine PRP-T

Control Effectiveness

Hib sterile site isolate

0 10 100%

CXR positive pneumonia

132 170 22.4%

WHO case definition-chest indrawing, neg

CXR

394 400 1.5%

data from Gambia RCT

Mulholland et al Lancet ‘97

Pre-post analyses----optimization approaches

• Rate reduction rather than percent reduction is the preferred measure of impact

• Select sites such that pneumococcal cases would be representative of pneumococcal cases in community

• In addition to measurement of changes in rates of pneumonia outcomes, also evaluate shifts in mix of pneumonia case severity