Air Quality Health Risk Assessment –Methodological Issues and Needs

Presented to SAMSI

September 19, 2007Research Triangle Park, NC

Anne E. Smith, Ph.D.Vice President

asmith@crai.com(202) 662-3872

2

Risk Assessment for Ambient Air Pollutants

• A policy analysis primarily performed by environmental protection agencies to assess benefits of air quality regulations

– The analysis step where clinical and epidemiological evidence of “whether” there is a health effect shifts to asking “how much” is the public health being affected.

• Focus of this presentation is on issues in performing quantitative portion of an air quality risk assessment

– Important not to lose sight of the qualitative elements of RA

• Typical quantitative risk assessment questions:

– How much does ambient pollution affect public health outcomes?

– How much would public health improve if we changed ambient air quality standards?

3

Ambient Air Environmental Risk Assessment

• Part of formal USEPA air quality standards-setting process

• Frequently in the headlines

4

Quantitative Risk Assessment Requires Extrapolations

• From epi associations “dose-response” function

• From observed current ambient air changes in “dose”

# Health Events

= f( ΔExposure* Toxicity)

MultipleEpi-derived

Ests of Toxicity

Current AmbientConcentrationDistributions

Changes dueto Policy

Policy“Benefits”

Analysis of the added uncertainties is paramount…and a substantial challenge for analystsAnalysis of the added uncertainties is paramount…and a substantial challenge for analysts

What is true toxicity value? What is true shape of f(.)?

How will policy actuallyaffect concentration distribution?

5

Discussion of Uncertainties in the “Dose-Response” Function Portion of a Risk Assessment

Epidemiological associations are quantitative, but that does not mean they should be interpreted at face value as “exposure-response functions”.

6

Uncertainties in Using Epidemiological Associations for Extrapolating to Public Health Outcomes

• Are the associations evidence of a causal relationship?

– Causality with respect to the specific ambient pollutant that the air quality standard applies to is especially uncertain

– How should uncertainty on causality enter into risk estimates?

• Assuming causality:

– Impact of exposure error on quantitative estimate of toxicity

– Impact of unknown missing explanatory variables

– Toxicity estimates from “multi-pollutant” vs. “single-pollutant” models

– Unknown “correct” lag structure

– Very limited capability of epidemiological estimates to identify any non-linearities in underlying concentration-response function

Particularly important for extrapolation to much lower ambient concentration levels than studied

7

Example: Multiple Estimates in Epidemiological Literature on Chronic Exposure Mortality Risk from Fine Particulate Matter (PM2.5)

3 Cohorts Studied

“Am Cancer Institute” “6 Cities” “Veteran’s Admin”

“1-pollutant” formulation

“2-pollutant” formulation

“1-pollutant” formulation

“2-pollutant” formulation

“1-pollutant” formulation

, (σ) .005 (.001).006 (.002).006 (.003).006 (.004).004 (.002).002 (.001)

, (σ) .003 (.002).012 (.004).004 (.004).002 (.004)

, (σ) .013 (.004).008 (.004)

For details of analysis: Anne Smith, Comments to USEPA, 3/31/07

, (σ) <0 (signif)<0 (signif)<0 (signif) <0 (insignif)

, (σ) <0 (signif)

Single estimate used in USEPA’s deterministic risk assessment

8

0-2%

2-4%

4-6%

6-8%

8-10

%

10-1

2%

12-1

4%

14-1

6%

16-1

8%

18-2

0%

20-2

2%

22-2

4%

24-2

6%

NoPM2.5riskat all

Estimates of percent of long-term mortality incidence attributed to PM2.5

-- Los Angeles for 2003 ambient levels –

USEPA RA’s 95% confidence intervalbased on single regression

Example of Results of Integrated Uncertainty Analysis for PM2.5 Mortality, Compared to Deterministic Results

For documentation of Integrated Uncertainty Analysis assumptions, see “Appendix C” of Anne Smith’s Comments on the Second Draft of EPA’s PM 2.5 Risk Analysis, 3/31/05 (available in PMDocket or on request from author). EPA RA estimates are from Final Risk Assessment (July 2005)

EPA DISTRIBUTION

DISTIBUTION FROM ILLUSTRATIVE INTEGRATED

UNCERTAINTY ANALYSIS

Probability of No Effect 0% 37% Median Effect

7% 3% Mean Effect

7% 5% 95% C.I.

2% - 11% 05 – 20%

= 3684 deaths per year in Los Angelesdue to 2003 PM2.5 concentrations

9

Example: Multiple Estimates in Epidemiological Literature on Chronic Exposure Mortality Risk from Fine Particulate Matter (PM2.5)

3 Cohorts Studied

“Am Cancer Institute” “6 Cities” “Veteran’s Admin”

“1-pollutant” formulation

“2-pollutant” formulation

“1-pollutant” formulation

“2-pollutant” formulation

“1-pollutant” formulation

, (σ) .005 (.001).006 (.002).006 (.003).006 (.004).004 (.002).002 (.001)

, (σ) .003 (.002).012 (.004).004 (.004).002 (.004)

, (σ) .013 (.004).008 (.004)

For details of analysis: Anne Smith, Comments to USEPA, 3/31/07

, (σ) <0 (signif)<0 (signif)<0 (signif) <0 (insignif)

.33.33 .33

, (σ) <0 (signif)

Single estimate used in USEPA’s deterministic risk assessment

.5 .5 .5 .5

Illustrativeassignments of weights for uncertainty analysis

10

EPA

DISTRIBUTION

DISTIBUTION FROM ILLUSTRATIVE INTEGRATED

UNCERTAINTY ANALYSIS

Probability of No Effect 0% 37% Median Effect

7% 3% Mean Effect

7% 5% 95% C.I.

2% - 11% 0% – 20%

0%

10%

20%

30%

40%

50%

60%

0-2%

2-4%

4-6%

6-8%

8-10

%

10-1

2%

12-1

4%

14-1

6%

16-1

8%

18-2

0%

20-2

2%

22-2

4%

24-2

6%

NoPM2.5riskat all

Estimates of percent of long-term mortality incidence attributed to PM2.5

-- Los Angeles for 2003 ambient levels –

USEPA RA’s 95% confidence intervalbased on single regression

Example of Results of Integrated Uncertainty Analysis for PM2.5 Mortality, Compared to Deterministic Results

For documentation of Integrated Uncertainty Analysis assumptions, see “Appendix C” of Anne Smith’s Comments on the Second Draft of EPA’s PM 2.5 Risk Analysis, 3/31/05 (available in PMDocket or on request from author). EPA RA estimates are from Final Risk Assessment (July 2005)

Histogram of pdf from Illustrative Integrated Uncertainty Analysis

11

Discussion of Uncertainties in the Exposure Portionof a Risk Assessment

12

Uncertainties in Pollutant Exposure Changes due to Policy: The Pollutant “Rollback” Assumption

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

Daily 8-Hour Maxima (PPB)

Cu

mu

lati

ve F

req

uen

cy

Key RA assumption: how each part of the pollutant exposure distribution would be affected by a change in the ambient standard

Large share of change assumed to occur on days with relatively low pollutant concentration (e.g., well below the standard)

-- Rollback Alternative 1 --

13

Uncertainties in Pollutant Exposure Changes due to Policy: The Pollutant “Rollback” Assumption

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%0 3 6 9 12 15 18 21 24 27 30 33 36 39 42 45 48 51 54 57 60 63 66 69 72

Daily 8-Hour Maxima (PPB)

Cu

mu

lati

ve F

req

uen

cy

-- Rollback Alternative 2 --

Larger share of change assumed to occur on days with relatively high pollutant levels

Rollback if there is a substantial “background” or uncontrollable portion of pollutant along the distribution

14

Needs for Methodological Development

15

Bottom Line of these Examples & Research Needs

• Health risk assessment for air quality standards is subject to enormous uncertainty

• Key uncertainties are not statistical

– Model selection and model shape for exposure-response

– Modeling of air quality distributional changes as result of policy

• Initial examples of methods to incorporate the above forms of uncertainty into the standard risk assessments produce very different characterization of risk and of benefits from air quality standards

• The challenge to the risk analysis community is in developing workable approaches for

– Integrating multiple sources of uncertainty

– Dealing with the highly judgmental aspects required to represent the key sources of these uncertainties

16

Coda: Estimated Annual “Lives Saved” from Revised PM2.5 Ambient Standard (as published with Final Rule)

Some use of uncertainty analysis as rule was finalized -- based on direct elicitation -- did not address any uncertainties in exposure (“rollbacks”) -- still many issues to be addressed …and other pollutants (e.g., ozone)

EPA’s Regulatory Impact Assessment, p. ES-8:

17

Top-down method “Direct Elicitation”: Ask “experts” what they believe to be likelihood of different levels of pollutant’s risk, including their personal uncertainty.

Bottom-up method, “Evidence-Driven Elicitation”: Ask experts to assign weights to all relevant epidemiological model results, based on their personal views about the relative quality of each model formulation/study design, etc.

Two Approaches for Characterizing Probability Distributions on Concentration-Response Relationships

• Disadvantages in the epi-based risk analysis situation: – The vast majority of “experts” are those who wrote the epidemiological papers (motivational

bias)

– By the time risk analysis is starting, most potential experts are viewed as having a “position” on the decisions that the risk analysis will inform

• Advantages in the epi-based risk analysis situation:– Experts can be disinterested outsiders to the study of the pollutant in question; need to be experts only in their ability to interpret regression-based studies generally

– “Evidence-driven”: creates explicit linkage of weights to subjective opinions about relative quality of underlying studies

– Explicit criteria for “quality” in a model can be articulated

– It is possible to “blind” the experts to the quantitative estimates in each study (minimization of potential motivational bias)