Michael D. Sohn, Randy L. Maddalena, Thomas E. McKoneIndoor Environment DepartmentLawrence Berkeley National Laboratory

Value of Information: Concepts and Potential Applications

Exposure-Based Chemical Prioritization Workshop: Exploring Opportunities for CollaborationApril 6-7, Research Triangle Park, North Carolina

The research discussed in this presentation was funded in parts by the S&T Directorate of the US Dept. of Homeland Security, the Defense Threat Reduction Agency, the US Environmental Protection Agency, and LBNL’s Laboratory Directed Research and Development Fund, and was performed by LBNL under US DOE Contract No. DE-AC03-76SF00098.

Goal of this Talk

• Introduce the overarching concepts of value of information analysis

• Emphasize that it can be a natural extension of existing EPA analysis procedures

• Suggest possible places where it may be applied

Not a Goal of this Talk

• Show you how to perform a value of information analysis:– literature is becoming rich (past 10 years)– gaining acceptance as an effective decision analysis tool– experts in engineering and finance fields exist, though fewer in

environmental quality, public health, and chemical hazard fields

Value of Information Introductory Concepts

“Risk assessors attempting to use probabilistic approaches to describe uncertainty often find themselves in a data-sparse situation ... In determining whether or not to collect additional data ... it is useful to consider the expected value of the information.” (Raiffa 1968, NRC 1996, Hammitt and Shlyakhter 1999).

• Key points in the above statement:– framed as a decision analysis question– uncertainty is the prime-driver– analysis is quantitative and prescriptive: conduct analysis before acting– additional data may not be needed– implies an approach that merges models and data in a hypothesis-based

study

Risk Characterization& Communication

Energy use

Industry

Agriculture

Buildings

Transportation

Consumer products

Sources & emissions

air

water

soil layerssediment

plants

Air (moving phase)

Non-moving phases (floor, carpet, walls, dust, surface films)

Air

Soil tracking

Cleaning

Reactions

Intake and uptake

Exposure events

liver

lung

fatskin

muscle

saliva

Biokinetics & toxicology

Personal air

dermal

oral

nasal

Risk Characterization& Communication

Sources & emissions

air

water

soil layerssediment

plants Air (moving phase)

Non-moving phases (floor, carpet, walls, dust, surface films)

Air

Soil tracking

Cleaning

Reactions

Intake and uptake

Exposure events

liver

lung

fatskin

muscle

saliva

Biokinetics & toxicology

= monitoring

= experiments = modeling

E

M

µ

µ µ

M

µ

µM

E

E

µ

E

M

E

Energy use

Industry

Agriculture

Buildings

Transportation

Consumer products

M

Personal air

µ

dermal

oral

nasal

Science and Decisions (NRC, 2009)

The Need to Broaden the Utility of Risk Assessment (NRC, 2009)

In “Science and Decisions” the National Research Council focused on the need for

– Treatment of uncertainty, variability, vulnerability

– Consistent approach to carcinogens and non-carcinogens

– A focus on disease burden and community health

– Cumulative exposure and aggregate risk

– Focus on solutions

Variability, Susceptibility, Vulnerability

Modernizing Risk Assessment, February 2010

Overall variability in risk relative to a median or baseline risk for a population

Variability in exposure

Variability in susceptibility (endogenous factors)

Age, genderGeneticsPre-existing disease

Variability in susceptibility (exogenous factors)

Exposures to other agents

Variation in vulnerability

What is VOI?

• Approach or framework for comparing decisions or consequences where uncertainty affects end results

• Often employs Bayesian inference and event tree methods– easy to propagate uncertainties– tools exist for data-poor analyses– communicating results is straight-forward

• Can be used to combined different types of limited observations and data to confirm or refute hypotheses– biomarker data– monitoring data– modeling results

How does VOI Fit into EPA Programs

• Prioritize research efforts– What data or modeling study is expected to refute beliefs?– What data or modeling study is expected to reduce uncertainties?– Is a reduction in uncertainty expected to alter decisions/rankings?– How much “resources” should we expend/demand, and why?– What are the key unknowns?

Morgan, and Henrion, 1990

VOI applied in a Chemical Screening

(excerpted from OPPTS, 2009)

decision based on a priori data

VOI applied in a Chemical Screening (cont.)

• What are the consequences of the decision?P[safe] , P[not safe]

E[loss in benefit due to false decision]

• What data can refute the decision?

Information has value if it can lead to a change in the decision option preferred a priori and that the preferred decision changes depending on the outcome of the experiment or study [Small MJ, 2010]

broad distributionof source and

exposure conditions

apply models and

beliefs a priori estimate of hazard and

exposure

propose data (µ,

apply Bayes’

ruleUpdate Input and Output

Uncertainties

estimate benefits

Metrics

• Expected Value of Perfect Information:– “If I had perfect information, would it change my decision?”– “What are the limits to reducing uncertainty/unknowns?”

• Expected Value of Sample Information:– “Given the expected noise/error/variability in data/modeling/etc,

will more studies change my decision?”– “What is the likely reduction in uncertainty/unknowns?”

• Expected Value of Consequences

Defining Some Jargon

P(Y|O) =P(O|Y)P(Y)

P(O)Bayes’ Rule

Likelihood Function

Prior

Normalizing Function

P(O|Y) = f(distribution, error assessment)

Other Possible Applications for VOI Analysis

• Grade relative rankings in exposure-based prioritization by comparing the relative certainty in exposure pathways/assessment

µChemA < µChemB

ChemA > ChemB

• Extrapolate from known chemicals to unknown chemicals using a precautionary principle (esp. for data-poor chemicals)

Concluding Remarks

• The point of this talk is not to promote another “tool.” VOI is not a magic bullet for Exposure-Based Chemical Prioritization

• If EPA is to include Exposure in its chemical managing then uncertainty and the consequences of uncertainty must be included in the analysis.

• VOI or some other forms decision analysis methods are suitable tools, with a long list of successful applications

• Move from ad hoc methods of decision analysis to transparent and quantitative method to prioritize data-gathering and further modeling studies

Important References

• Taylor et al., The Value of Animal Test Information in Environmental Control Decisions, Risk Analysis 13(4): 403-412, 1993. VOI used to evaluate the benefit of performing animal bioasses to assess cancer potency.

• Dakins, The Value of the Value of Information, Human and Ecolog. Risk Assess., 5(2):281-289, 1999. Easy description of key concepts.

• Yokota & Thompson, Value of Information Analysis in Environmental Health Risk Management Decisions: Past, Present, and Future, Risk Analysis (24(4):635-650, 2004. Evaluation and recommendation for health risk management.

•