SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT National Research Council Committee on Improving...

SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT

National Research CouncilCommittee on Improving Risk Analysis Approaches Used by EPA

Board on Environmental Studies and Toxicology

Society for Risk Analysis - New England ChapterJanuary 29, 2009

STUDY MOTIVATION

Risk assessment is at a crossroads, and its credibility is being challenged.

Science is increasingly complex.

Risk assessment is being extended to address broader environmental questions, such as life-cycle analysis and issues of costs, benefits, and risk-risk tradeoffs.

Stakeholders are often disengaged from the risk-assessment process at a time when risk assessment is increasingly intertwined with societal concerns.

Disconnects between the available scientific data and the information needs of decision-makers hinder the use of risk assessment as a decision-making tool.

COMMITTEE’S CHARGE

Develop scientific and technical recommendations for

improving risk analysis approaches used by EPA, including practical improvements that EPA could make in the near term (2-5 years) and in the longer term (10-20 years).

Focus primarily on human health risk assessment but also consider broad implications of findings and recommendations for ecologic risk analysis.

COMMITTEE

Thomas Burke (Chair), Johns Hopkins Bloomberg School of Public Health

A. John Bailer, Miami University

John M. Balbus, Environmental Defense

Joshua T. Cohen, Tufts Medical Center

Adam M. Finkel, University of Medicine and Dentistry of New Jersey

Gary Ginsberg, Connecticut Department of Public Health

Bruce K. Hope, Oregon Department of Environmental Health

Jonathan I. Levy, Harvard School of Public Health

Thomas E. McKone, University of California

Gregory M. Paoli, Risk Sciences International

Charles Poole, University of North Carolina School of Public Health

Joseph V. Rodricks, ENVIRON International Corporation

Bailus Walker Jr., Howard University Medical Center

Terry F. Yosie, World Environment Center

Lauren Zeise, California Environmental Protection Agency

EVALUATION

The committee focused on two broad elements:

Improving technical analysis Entails the development and use of scientific knowledge

and information to improve characterizations of risk.

Improving utility Entails making risk assessment more relevant and useful

for risk-management decisions.

KEY MESSAGES

Enhanced framework Formative focus Four steps still core Matching analysis to decisions Clearer estimates of population risk Advancing cumulative assessments People and capacity building

CONCLUSIONS AND RECOMMENDATIONS

Design of risk assessment Uncertainty and variability Selection and use of defaults Unified approach to dose-response assessment Cumulative risk assessment Improving the utility of risk assessment Stakeholder involvement Capacity-building

DESIGN OF RISK ASSESSMENT

Design The process of planning a risk assessment and ensuring that its level and complexity are consistent with the needs to inform decision-making.

Recommendations Increased attention to the design of risk assessment in its formative stages. Planning and scoping and problem formulation (as in EPA ecological and cumulative risk

guidance) should be formalized and implemented.

VALUE OF INFORMATION

Value of information (VOI) cannot be ascertained without comprehensive uncertainty characterization, well-framed decision context Formal quantitative VOI analysis may not be feasible in many settings, but essential reasoning behind VOI can be more widely utilized

UNCERTAINTY

There have been substantial differences among EPA’s approaches to and guidance for addressing uncertainty in exposure and dose-response assessment.

The level of detail for characterizing uncertainty is appropriate only to the extent that it is needed to inform specific risk-management decisions appropriately.

Inconsistency in the treatment of uncertainty among components of a risk assessment can make the communication of uncertainty difficult and sometimes misleading.

VARIABILITY

Variability in human susceptibility has not received sufficient or consistent attention in many EPA health risk assessments (exceptions include lead and ozone)

Committee encourages EPA to move toward the long-term goal of quantifying population variability more explicitly in exposure assessment and dose-response relationships.

UNCERTAINTY AND VARIABILITY

Recommendations

EPA should encourage risk assessments to characterize and communicate uncertainty and variability in all key computational

steps.

Uncertainty and variability analysis should be planned and managed to reflect the needs for comparative evaluation of the

risk management options.

In the short term, EPA should adopt a “tiered” approach for selecting the level of detail in uncertainty and variability assessments

This should be made explicit in the planning stage.

EPA should develop guidance on the appropriate level of detail needed in uncertainty and variability analyses Provide clear definitions and methods for identifying and addressing different sources of uncertainty and variability.

SELECTION AND USE OF DEFAULTS

Uncertainty is inherent in all stages of risk assessment EPA typically relies on assumptions when chemical-specific data are not available.

Much scientific controversy and delay in completion of some risk assessments

In part (clearly not in total) related to the long debates on adequacy of the data to support use of a default or an alternative approach.

1983 Red Book recommended guidelines to select defaults To ensure consistency and to avoid manipulations in the risk-assessment process.

Committee acknowledges EPA’s efforts to examine scientific data related to defaults, but changes are needed to improve the agency’s use of them.


Established defaults need to be maintained for risk assessment steps that require inferences EPA, for the most part, has not yet published clear, general guidance on what level of evidence is needed to justify use of agent-specific data instead of a default. Clear criteria should be available for judging whether, in specific cases, data are adequate for direct use or to support an inference in place of a default.

There are a number of defaults (missing or implicit defaults) that are engrained in EPA risk-assessment practice but are absent from its risk-assessment guidelines.

EPA does not quantify uncertainty when default assumptions are used.

SELECTION AND USE OF DEFAULTSRecommendations

EPA should continue and expand use of the best, most current science to support and revise default assumptions. develop clear, general standards for the level of evidence needed to justify the use of alternative assumptions in place of defaults. work toward the development of explicitly stated defaults to take the place of implicit defaults.


Evidentiary Standard for Replacing Default

“The committee recommends that EPA adopt an alternative assumption in place of a default when it determines that the alternative is ‘clearly superior’, that its plausibility clearly exceeds the plausibility of the default.”

SELECTION AND USE OF DEFAULTS Showing Uncertainty When Using

Defaults To the extent feasible, EPA should move beyond

qualitative description of uncertainty when default assumptions are used

Long term – improved probabilistic description of uncertainty commensurate with risk management needs

Short term, EPA should develop criteria for listing alternative values.

Goal – Provide sensitivity analysis to illustrate impact of alternative assumptions and hence characterize robustness of risk estimates Limit attention to assumptions with plausibility comparable to the default Goal is not an exhaustive presentation of plausible estimates

UNIFICATION APPROACH TO DOSE-RESPONSE ASSESSMENT

Historically, dose-response assessments at EPA conducted differently for cancer and noncancer effects

Methods have been criticized for not providing the most useful results.

A consistent approach to risk assessment for cancer and noncancer effects is scientifically feasible and needs to be implemented.

Current Approach to Dose-Response Assessment

Hazard AssessmentSort by Cancer or Non-Cancer Endpoints

Non-Cancer Risk Assessment Cancer Risk Assessment

Identify NOAEL or derive POD

Select Uncertainty/Adjustment Factors- Cross-species (U1)- Human interindividual variability (U2)- Other (U3)

Derive Reference Dose RfD = POD / (U1

x U2 x U3)

Risk Characterization: Hazard Index (HI) or MOEHI = Σi (Exposure / RfDi)MOE = POD / Exposure

Limitations and Issues

1. Possibility for low dose linearity (for example, due to background exposure) not assessed

2. No risk measure produced. HI, RfD, and MOEsof limited utility for risk/benefit analyses

3. Uncertainty not distinguished from variability or other adjustments

Evaluate Mode of Action (MOA)

Animal to human dose conversion:mg/kg3/4-d scaling or pharmacokinetic modeling with pharmacodynamic adjustment


1. Inter-human variability in risk either not addressed at all (animal based) or incompletely (epidemiology based)

2. For low dose non-linear carcinogens, no risk measure produced. HI, RfD, and MOEs of limited utility for risk/benefit analysis

3. Uncertainty is not characterized

Derive POD (for example, LED01) and Slope FactorFor example, Slope Factor = 0.01 / POD

Risk Characterization: Low Dose RiskExtra Risk = Slope Factor x Exposure

MOA not established Linear MOA

Low Dose “Non-Linear”





Identify NOAEL or derive POD

Select Uncertainty/Adjustment Factors- Cross-species (U1)- Human interindividual variability (U2)- Other (U3)

Derive Reference Dose RfD = POD / (U1

x U2 x U3)

Risk Characterization: Hazard Index (HI) or MOEHI = Σi (Exposure / RfDi)MOE = POD / Exposure


1. Possibility for low dose linearity (for example, due to background exposure) not assessed

2. No risk measure produced. HI, RfD, and MOEsof limited utility for risk/benefit analyses

3. Uncertainty not distinguished from variability or other adjustments

Evaluate Mode of Action (MOA)

Animal to human dose conversion:mg/kg3/4-d scaling or pharmacokinetic modeling with pharmacodynamic adjustment


1. Inter-human variability in risk either not addressed at all (animal based) or incompletely (epidemiology based)

2. For low dose non-linear carcinogens, no risk measure produced. HI, RfD, and MOEs of limited utility for risk/benefit analysis

3. Uncertainty is not characterized

Derive POD (for example, LED01) and Slope FactorFor example, Slope Factor = 0.01 / POD

Risk Characterization: Low Dose RiskExtra Risk = Slope Factor x Exposure

MOA not established Linear MOA

Low Dose “Non-Linear”

Current EPA Dose-Response Approach

EPA has taken important steps to harmonize cancer and noncancer approaches, but with many scientific and operational

limitations: Noncancer effects do not necessarily have threshold or low-dose nonlinearity The mode of action of carcinogens varies. Background exposures and underlying disease processes contribute to population background risk

can lead to linearity at the population doses of concern. RfDs and RfCs do not quantify risk for different magnitudes of exposure but rather provide a bright line

Their use in risk-management decision-making is limited. Cancer risk assessments usually do not account for human differences in cancer susceptibility (other than possible differences in early-life).

Dose-response relationship is dependent on heterogeneity in:■ Background exposure (endogenous and xenobiotic) ■ Biological susceptibility

UNIFICATION OF DOSE-RESPONSE ASSESSMENTRecommendations

A consistent, unified approach for dose-response modeling that includes formal, systematic assessment of

background disease processes and exposures possible vulnerable populations modes of action that may affect a chemical’s dose-response in humans.

Redefine the RfD or RfC as a risk-specific dose provides information on the percentage of the population expected above or below a defined acceptable risk

(with specific degree of confidence).

Formal introduction of variability into cancer dose-response modeling Will require implementation and development

As new chemicals are assessed or old chemicals are reassessed Of test cases to demonstrate proof of concept.

Unification Approach to Dose-Response Assessment

Assemble Health Effects Data

Endpoint Assessment

• Identify adverse effects, focusing on those of concern for exposed populations

• Identify precursors and other upstream indicators of toxicity

• Identify gaps – for example, endpoints or lifestages under assessed or not assessed

MOA Assessment

(for each endpoint of concern)

• Research MOAs for endpoints observed in animals and humans

• Evaluate the sufficiency of

the MOA evidence

• Evaluate endogenous processes contributing to MOA

Vulnerable Populations Assessment

Identify potentially vulnerable groups and individuals, considering endpoints, the potential MOA, background rate of health effect, and other risk factors

Background Exposure Assessment

• Identify possible background exogenous and endogenous exposures

• Conduct screening level exposures and analysis focusing on high end exposure groups

Conceptual Model Selection

Develop or select conceptual model:

• From linear conceptual models unless data sufficient to reject low dose linearity

• From non-linear conceptual models otherwise

Dose Response Method Selection

Select dose response model and method based on:

• Conceptual model

• Data availability

• Risk management needs for form of risk characterization

Dose-Response Modeling and Results Reporting

Assemble Health Effects Data

Endpoint Assessment

• Identify adverse effects, focusing on those of concern for exposed populations

• Identify precursors and other upstream indicators of toxicity

• Identify gaps – for example, endpoints or lifestages under assessed or not assessed

MOA Assessment

(for each endpoint of concern)

• Research MOAs for endpoints observed in animals and humans

• Evaluate the sufficiency of

the MOA evidence

• Evaluate endogenous processes contributing to MOA

Vulnerable Populations Assessment

Identify potentially vulnerable groups and individuals, considering endpoints, the potential MOA, background rate of health effect, and other risk factors

Background Exposure Assessment

• Identify possible background exogenous and endogenous exposures

• Conduct screening level exposures and analysis focusing on high end exposure groups

Conceptual Model Selection

Develop or select conceptual model:

• From linear conceptual models unless data sufficient to reject low dose linearity

• From non-linear conceptual models otherwise

Dose Response Method Selection

Select dose response model and method based on:

• Conceptual model

• Data availability

• Risk management needs for form of risk characterization

Dose-Response Modeling and Results Reporting

Diagnostic Questions to Aid Dose-Response Assessment

What is known or suspected to be the chemical’s MOA?

What underlying degenerative or disease processes might the toxicant affect?

What are the background incidences and population distributions of these processes?

Are there identified sensitive populations?

What environmental contaminants in air, drinking water, food or in consumer products (e.g., in cosmetics) or endogenous chemicals (e.g., natural hormones) are similar to the chemical?

Could they potentially operate by MOAs similar to that of the chemical in question?

What chemicals might operate by a different MOA but have the potential to affect the same toxic process as the chemical under study?

Can subgroups with particularly high exposures be identified?

…more

Conceptual Dose-Response Model – Based on MOA, Background Exposure and Disease Processes

CUMULATIVE RISK ASSESSMENT

EPA is increasingly asked to address broad public-health and environmental-health issues that stakeholder groups often consider inadequately captured by current risk assessments

multiple exposures complex mixtures vulnerability of exposed populations

There is a need for cumulative risk assessments as defined by EPA that include

combined risks posed by exposure to multiple agents or stressors aggregate exposure to a given agent or stressor

all routes, pathways, and sources of exposure

Chemical, biologic, radiologic, physical, and psychologic stressors are considered.


Committee applauds the agency’s move toward the broader definition, making risk assessment more informative and relevant to decisions and stakeholders.

However, in practice, EPA risk assessments often fall short of what is possible and supported by agency guidelines.

Little consideration of nonchemical stressors, vulnerability, and background risk factors.

Because of the complexity of considering so many factors simultaneously, there is a need for:

Simplified risk assessment tools Orientation around pertinent risk management options to limit the

number of stressors under formal consideration

Step 1: -Develop a conceptual model for the stressors of primary interest for the analysis (stressors that would be significantly influenced by any of the risk-management options under study).

- MOA assessment, assessment of background exposures to chemical and nonchemical stressors that may affect the same health outcome, vulnerability assessment that takes into account underlying disease processes in the population

-Identify the receptors and end points affected by these stressors. -Review the conceptual model and stressors, receptors, and end points of interest with stakeholders in initial planning and scoping.

Step 2: -Use epidemiologic and toxicologic evidence and screening-level benefit calculations to provide an initial evaluation of which stressors should be included in the cumulative risk assessment. Gather stakeholder feedback and review and re-evaluate planning and scoping for the analysis. -Focus the assessment only on stressors that contribute to end points of interest for risk-management options and are either differentially affected by different control strategies or influence the benefits of stressors that are differentially affected.

Step 3: -Evaluate benefits of different risk-management options with appropriate characterization of uncertainty, including quantification of the effects of individual stressors and bounding calculations of any possible interaction effects.

Step 4: -If Step 3 is sufficient to discriminate among risk-management options given other economic, social, and political factors, conclude the analysis; otherwise, sequentially refine the analysis as needed, taking into account potential interactions.


RecommendationsEPA should Draw on other approaches to incorporate interactions between chemical

and non-chemical stressors in assessments, including those from ecologic risk assessment and social epidemiology Develop guidelines and methods for simpler analytical tools

to support cumulative risk assessment to provide for greater involvement of stakeholders.

In short-term, develop databases and default approaches to allow for incorporation of key non-chemical stressors in the absence of population-specific data, considering

exposure patterns contributions to relevant background processes interactions with chemical stressors.

In long-term, invest in research programs related to interactions between chemical and non-chemical stressors, including epidemiologic investigations and physiologically-based pharmacokinetic modeling.

IMPROVING THE UTILITY OF RISK ASSESSMENT

Committee proposes a framework for risk-based decision-making.

At core is risk assessment paradigm from 1983 Red Book Differs from Red Book primarily in its initial and final steps

Framework asks implicitly: What options are there to reduce the hazards or exposures that have been identified, and How can risk assessment be used to evaluate the merits of the various options?


Under this framework, the questions posed arise from

early and careful planning of the types of assessments (including risks, costs, and technical feasibility) and the required level of scientific depth needed to evaluate the relative merits of the options being considered.

Risk management choosing among the options after the appropriate assessments have been undertaken and evaluated.

• What are the relative health or environmental benefits of the proposed options?

• How are other decision-making factors (technologies, costs) affected by the proposed options?

• What is the decision, and its justification, in light of benefits, costs, and uncertainties in each?

• How should the decision be communicated?

• Is it necessary to evaluate the effectiveness of the decision?

• If so, how should this be done?

Stage 1: Planning

• For the given decision-context, what are the attributes of assessments necessary to characterize risks of existing conditions and the effects on risk of proposed options? What level of uncertainty and variability analysis is appropriate?

Stage 3: Confirmation of Utility

• Does the assessment have the attributes called for in planning?

• Does the assessment provide sufficient information to discriminate among risk management options?

• Has the assessment been satisfactorily peer reviewed?

FORMAL PROVISIONS FOR INTERNAL AND EXTERNAL STAKEHOLDER INVOLVEMENT AT ALL STAGES

• The involvement of decision-makers, technical specialists, and other stakeholders in all phases of the processes leading to decisions should in no way compromise the technical assessment of risk, which is carried out under its own standards and guidelines.

• What problem(s) are associated with existing environmental conditions?

• If existing conditions appear to pose a threat to human or environmental health, what options exist for altering those conditions?

• Under the given decision context, what risk and other technical assessments are necessary to evaluate the possible risk management options?

• Hazard Identification

What adverse health or environmental effects are associated with the agents of concern?

• Dose-Response Assessment

For each determining adverse effect, what is the relationship between dose and the probability of the occurrence of the adverse effects in the range of doses identified in the exposure assessment?

• Risk Characterization

What is the nature andmagnitude of risk associated with existing conditions?

What risk decreases (benefits) are associated with each of the options?

Are any risks increased? What are the significant uncertainties?

• Exposure Assessment

What exposures/doses are incurred by each population of interest under existing conditions?

How does each option affect existing conditions and resulting exposures/doses?

Stage 2: Risk Assessment

NO YES

PHASE I: PROBLEM FORMULATION

AND SCOPING

PHASE II: PLANNING AND CONDUCT

OF RISK ASSESSMENT

PHASE III: RISK MANAGEMENT

IMPROVING THE UTILITY OF RISK ASSESSMENT Phase I: Problem Formulation and Scoping

a. What is the problem to be investigated, and what is its source?

b. What are the possible opportunities for managing risks associated with the problem? Has a full array of possible options been considered, including legislative requirements?

c. What types of risk assessments and other technical and cost assessments are necessary to evaluate existing conditions and how the various risk-management options alter the conditions?

d. What impacts other than health and ecosystem threats will be considered?

e. How can the assessments be used to support decisions?

f. What is the required timeframe for completion of assessments?

g. What resources are needed to undertake the assessments?

Phase IIPlanning and Conduct of Risk Assessment

Stage 1: Planning

■ For the given decision-context, what are the attributes of assessments necessary to characterize risks of existing conditions and the effects on risk of proposed options?

■ What level of uncertainty and variability analysis is appropriate?

Stage 2: Risk Assessment

Stage 3: Confirmation of the Utility

■ Does the assessment have the attributes called for in planning?

■ Does the assessment provide sufficient information to discriminate among risk-management options?

■ Has the assessment been satisfactorily peer reviewed?

Phase III Risk Management

■ What are the relevant health or environmental benefits of the proposed risk-management options?

■ How are other decision-making factors (technologies, costs) affected by the proposed options?

■ What is the decision, and its justification, in light of benefits, costs, and uncertainties in each?

■ How should the decision be communicated?

■ Is it necessary to evaluate the effectiveness of the decision? If so, how should this be done?


The framework… Systematically identifies problems and options that risk assessors should evaluate at the earliest stages of decision-making.

Expands the array of impacts assessed beyond individual effects (e.g., respiratory effects) to include broader questions (e.g., health status and ecosystem protection).

Provides a formal process for stakeholder involvement.

Increases understanding of the strengths and limitations of risk assessment by decision-makers at all levels, e.g., by making uncertainties and choices more transparent.

Maintains the conceptual distinction between risk assessment and risk management articulated in the Red Book.


Recommendation

EPA adopt a framework for risk-based decision-making that embeds the Red Book risk assessment paradigm into a process with

initial problem formulation and scoping, upfront identification of risk-management options, and use of risk assessment to discriminate among these options.

STAKEHOLDER INVOLVEMENT

Many stakeholders believe that the current process for

developing and applying risk assessments lacks credibility and transparency.

Greater stakeholder involvement is necessary to ensure that the process is transparent and to ensure risk-based decision-making proceeds effectively, efficiently, and credibly.

Stakeholder involvement needs to be an integral part of the risk-based decision-making framework.

It is important that EPA adhere to its own guidance on stakeholder involvement

particularly in the context of cumulative risk assessment, in which communities often have not been adequately involved.

STAKEHOLDER INVOLVEMENT

Recommendation

EPA should establish a formal process for stakeholder involvement in the framework for risk-based decision-making with:

time limits to ensure that decision-making schedules are met incentives to allow for balanced participation of stakeholders including impacted communities and less advantaged stakeholders.

CONCLUDING REMARKS

Risk assessment remains essential to the agency’s mission to ensure protection of public health and the environment.

The committee hopes that the recommendations and the proposed framework for risk-based decision-making will provide a template for the future of risk assessment in EPA and strengthen the scientific basis, credibility, and effectiveness of future risk-management decisions.

SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT National Research Council Committee on Improving...

Documents

Transcript of SCIENCE AND DECISIONS: ADVANCING RISK ASSESSMENT National Research Council Committee on Improving...