RAF PRA FAQ FINAL

Office of the Science AdvisorRisk Assessment Forum

EPA/100/R-14/003 July 2014 www.epa.gov/raf

U.S. Environmental Protection Agency Office of Research and Development Washington, DC 20460

Official Business Penalty for Private Use $300

PRESORTED STANDARDPOSTAGE & FEES PAID

EPAPERMIT NO. G-35

lmarti03

Typewritten Text

MBRODER

Typewritten Text

MBRODER

Typewritten Text

MBRODER

Typewritten Text

MBRODER

Typewritten Text

MBRODER

Typewritten Text

Probabilistic Risk Assessment to Inform Decision Making: Frequently Asked Questions

MBRODER

Typewritten Text

MBRODER

Typewritten Text

MBRODER

Typewritten Text

jfitzpat

Typewritten Text

JFitzpat

Typewritten Text

JFitzpat

Typewritten Text

JFitzpat

Typewritten Text

JFitzpat

Typewritten Text

JFitzpat

Typewritten Text

jfitzpat

Typewritten Text

jfitzpat

Typewritten Text

EPA/100/R‐14/003

ProbabilisticRiskAssessmenttoInformDecisionMaking:

FrequentlyAskedQuestions

July25,2014

U.S.EnvironmentalProtectionAgencyOfficeoftheScienceAdvisor

RiskAssessmentForumProbabilisticRiskAnalysisTechnicalPanel

Washington,D.C.20460

ii

DISCLAIMER ThisdocumenthasbeenreviewedinaccordancewithU.S.EnvironmentalProtectionAgencypolicy.Mentionoftradenamesorcommercialproductsdoesnotconstituteendorsementorrecommendationforuse.

PreferredCitation:

U.S.EnvironmentalProtectionAgency(USEPA).2014.ProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions.EPA/100/R‐09/001B.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://epa.gov/raf/prawhitepaper/index.htm.

iii

ThisdocumentwasproducedbyaTechnicalPaneloftheRiskAssessmentForumtosummarizetheRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudiesformanagersandagencyscientisttogainabetterunderstandingoftheprinciplesofprobabilisticriskassessment.Theauthorsdrewontheirexperienceinperformingandinterpretingprobabilisticassessmentstoimprovethemanagementofenvironmentalandhealthhazards.Interviews,presentationsanddialogueswithriskmanagersconductedbytheTechnicalPanelhavecontributedtotheinsightsandrecommendationsinthisFAQdocumentandtheassociatedwhitepaper.

AuthorsRobertHetes,EPAOfficeofResearchandDevelopment(DocumentLead)KathrynGallagher,EPAOfficeofWater(formerlywiththeOfficeoftheScienceAdvisor)MarianOlsen,EPARegion2RitaSchoeny,EPAOfficeofResearchandDevelopment(formerlywiththeOfficeofWater)CynthiaStahl,EPARegion3

ProbabilisticRiskAnalysisTechnicalPanelHalûkÖzkaynak,EPAOfficeofResearchandDevelopment(Co‐Chair)RobertHetes,EPAOfficeofResearchandDevelopment(Co‐Chair)KathrynGallagher,EPAOfficeofWater(Co‐Chair)ChristopherFrey,NorthCarolinaStateUniversity(whileontemporaryassignmentatEPA)

(WhitePaperCo‐Lead)JohnPaul,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)PaskyPascual,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)MarianOlsen,EPARegion2(CaseStudyLead)MichaelClipper,EPAOfficeofSolidWasteandEmergencyResponseMichaelMessner,EPAOfficeofWaterKeeveNachman,currentlyatTheJohnsHopkinsBloombergSchoolofPublicHealthZacharyPekar,EPAOfficeofAirandRadiationRitaSchoeny,EPAOfficeofResearchandDevelopmentCynthiaStahl,EPARegion3DavidHrdy,EPAOfficeofPesticideProgramsJohnLangstaff,EPAOfficeofAirandRadiationElizabethMargosches,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)AudreyGalizia,EPAOfficeofResearchandDevelopmentNancyRios‐Jafolla,EPARegion3DonnaRandall,EPAOfficeofChemicalSafetyandPollutionPreventionKhoanDinh,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)HarveyRichmond,EPAOfficeofAirandRadiation(retired)

RiskAssessmentForumScienceCoordinator

JulieFitzpatrick,EPAOfficeoftheScienceAdvisor

ExternalPeerReviewersScottFerson(Chair),AppliedBiomathematicsAnnetteGuiseppi‐Elie,DuPontEngineeringDaleHattis,ClarkUniversityIgorLinkov,U.S.ArmyEngineerResearchandDevelopmentCenterJohnToll,WindwardEnvironmentalLLC

iv

TABLE OF CONTENTS EXECUTIVESUMMARY.........................................................................................................................................1

FREQUENTLYASKEDQUESTIONS....................................................................................................................2

WhyshouldIcareaboutPRA?Whyisitimportanttoriskmanagers?.......................................................2 Asamanager,whatdoIneedtoknowaboutPRA?.............................................................................................2 HowcouldariskmanagerapproachaPRA?Whatwouldamanagerconsider?....................................2 WhatisPRA?Howdoesitcomparewithcurrentapproaches?......................................................................3 WhatarecommonchallengesfacingEPAdecisionmakers?...........................................................................3 Whatarevariabilityanduncertainty?Whatistheirrelevanceinriskassessmentandrisk

management?............................................................................................................................................................4 HowisriskassessmentconductedcurrentlyatEPA?........................................................................................7 Dohumanhealthriskassessmentandecologicalriskassessmenttreatuncertaintyand

variabilitydifferently?...........................................................................................................................................7 HowdoesEPAtypicallyaddressvariabilityanduncertainty?........................................................................7 HowdoesPRAaddressvariabilityanduncertainty?..........................................................................................9 Howcanenhancedcharacterizationofuncertaintyandvariabilityhelpinformdecisions?...........10 Whatistheimpactofvariabilityanduncertaintyonmakingdecisions?.................................................11 Isuncertaintyandvariabilitylimitedonlytoriskassessment?...................................................................11 Whatkeyquestionsrelatedtouncertaintyandvariabilityareaskedorconsideredbydecision

makers?......................................................................................................................................................................12 Whatisthedesiredpercentileofthepopulationtobeprotected?Bychoosingthispercentile,

whomaynotbeprotected?HowcanaddressinguncertaintyandvariabilityusingPRAhelpinformdecisions?...................................................................................................................................................12

WhataresomeofthelimitationsofandchallengesforPRA?.......................................................................14 WhatisEPA’sexperiencewithPRA?.......................................................................................................................14 WhatareotherEPApublicationsonPRA?............................................................................................................17 WhenshouldoneconsiderusingPRA?...................................................................................................................17 Whatistherightlevelofanalysis?............................................................................................................................18 IfoneisgoingtousePRA,whatfactorsshouldbeconsidered?...................................................................20 WhataretheresourcesneededtoconductaPRA?............................................................................................20 DoesPRArequiremoredatathanconventionalapproaches?......................................................................20 CommunicationofPRAresultstothemanagerandcommunity:doesthepresentationofresults

matter?.......................................................................................................................................................................21 HowcanIgetmoreinformationonPRA?..............................................................................................................21

GLOSSARY..............................................................................................................................................................22

REFERENCES.........................................................................................................................................................26

v

LIST OF FIGURES, TABLES AND EQUATIONS Figures Figure1.Source‐to‐EffectContinuum.............................................................................................................................5

Figure2.IngestionofDrinkingWaterforAdults.......................................................................................................6

Figure3.GeneralPhasesoftheRiskAssessmentProcess.....................................................................................8

Figure4.WhatQuestionsCanPRAAddress?.............................................................................................................13

Figure5.TieredApproachforRiskAssessment.......................................................................................................19

Tables Table1.SelectedExamplesofEPAApplicationsofRiskAssessmentTechniquesfromDeterministic

tomoreComplexAnalysis..................................................................................................................................15

Equations Equation1.DeterministicRiskAssessment.................................................................................................................7 Equation2.ProbabilisticRiskAssessment...................................................................................................................9

1

EXECUTIVE SUMMARY Probabilisticriskassessment(PRA),initssimplestform,isagroupoftechniquesthatincorporatevariabilityanduncertaintyintoriskassessments.Variabilityreferstotheinherentnaturalvariation,diversityandheterogeneityacrosstime,spaceorindividualswithinapopulationorlifestage,whileuncertaintyreferstoimperfectknowledgeoralackofpreciseknowledgeofthephysicalworld,eitherforspecificvaluesofinterestorinthedescriptionofthesystem(USEPA2011c).Variabilityanduncertaintyhavethepotentialtoresultinoverestimatesorunderestimatesofthepredictedrisk.

PRAprovidesestimatesoftherangeandlikelihoodofahazard,exposureorrisk,ratherthanasinglepointestimate.StakeholdersinsideandoutsideoftheAgencyhaverecommendedamorecompletecharacterizationofrisks,includinguncertaintiesandvariability,inprotectingmoresensitiveorvulnerablepopulationsandlifestages.PRAcanbeusedtosupportdecision‐makingriskmanagementbyassessmentofimpactsofuncertaintiesoneachofthepotentialdecisionalternatives.TheaimofthisisdocumentistosummarizetheRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a)formanagersandagencyscientiststogainabetterunderstandingoftheprinciplesofPRAwithoutthemoredetaileddiscussionpresentedintheWhitePaper.

Numerousadvisorybodies,suchastheScienceAdvisoryBoard(SAB)andtheNationalResearchCouncil(NRC)oftheNationalAcademyofSciences(NAS),havealsorecommendedthatEPAincorporateprobabilisticanalysesintotheAgency’sdecision‐makingprocess.EPA’sRiskAssessmentForum(RAF)formedaTechnicalPanel,consistingofrepresentativesfromtheAgency’sprogramandregionaloffices,todevelopthisFrequentlyAskedQuestions(FAQ)documentanditscompanionpublicationtitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a).TheRAFisalsorecommendingthedevelopmentofAgencyresources,suchasaclearinghouseofPRAcasestudies,bestpractices,resourcesandseminars,toraisegeneralknowledgeabouthowtheseprobabilistictoolscanbeused.

ThepurposeofthisFAQdocumentistopresentgeneralconceptsandprinciplesofPRA,describehowPRAcanimprovethebasesofAgencydecisions,andprovideillustrationsofhowPRAhasbeenusedinriskestimationandindescribingtheuncertaintyindecisionmaking.Thisdocumentisacompaniontothemoredetailedwhitepaper,whichalsoincludesacompendiumof16relevantcasestudiesrangingintheirlevelofcomplexity.Bothdocumentsdescribethepotentialandactualusesofprobabilistictoolsintheriskdecisionprocessandfacilitatetheirimplementationinhuman,ecologicalandenvironmentalriskanalysisandrelateddecision‐makingatEPA.Thedocumentsdonotprescribeaspecificapproachtoriskassessment;rather,theydescribethevariousstagesandaspectsofanassessmentordecisionprocessinwhichprobabilistictoolsmayaddvalue.Thesearenotregulatoryguidancedocuments,buttheyaredesignedtobehelpfulhandbooksand/orreferencesforriskassessors,riskmanagersanddecision‐makersinterestedinimprovingtheriskassessmentanddecision‐makingprocess.

Overall,thisFAQdocumentanswersmorethan20questionscommonlyaskedaboutPRA,beginningwith“WhyshouldIcareaboutPRA,andwhyisitimportanttoriskmanagers?”andendingwith“HowcanIgetmoreinformationonPRA?”ThesequestionscoverawiderangeofPRAissues,andriskmanagers,decision‐makersandotherinterestedreaderswillfindaplethoraofPRAinformationinthisdocument.Furtherdetailedinformation,includingrecommendationsforfutureactivities,areprovidedinthecompanionwhitepaper(USEPA2014a).

2

FREQUENTLY ASKED QUESTIONS

Why should I care about PRA? Why is it important to risk managers? Uncertaintyisunavoidable,evenwhenusinghighlyaccuratedatawiththemostsophisticatedmodels.EPAmakesdecisionsinthepresenceofuncertainty.Therefore,understandingthedecisioncontextiscriticalforAgencydecisionmaking.VariousstakeholdersinsideandoutsideoftheAgencyhaverequestedamorecomprehensivecharacterizationofrisks,includinguncertainties,inprotectingmoresensitiveorvulnerablepopulationsandlifestages.Probabilisticriskassessment(PRA)isonewaytocharacterizetheuncertaintyassociatedwithanyriskassessment.Aspartofadecisionanalysis,theenhanceduseofPRAandcharacterizationofuncertaintywouldallowEPAdecision‐makersopportunitiestoassessuncertaintypertainingtoitseffectondecisionsandexplorethedefensibilityoftheavailableriskmanagementoptionsthroughamorerobustandtransparentprocess.Mostoften,riskmanagerswanttoknowifbetterunderstandingofuncertaintiesmightsupportadifferentdecisionalternativeorprovidefurthersupportfortheselecteddecision.

TheuseofPRAinaddressinguncertaintyandvariabilityatEPAisnotsystematicallypracticed,andthelackthereofisoftenamajorrecommendationfrombothinternalandexternalreviewsofEPAproductsandprocedures(e.g.,theSABreviewofEPApracticesin2007[USEPA2007b],theNRCreviewoftheDioxinReassessment[NRC2006],theOfficeofManagementandBudget’sCircularA‐4[OMB2003]andUpdatedPrinciplesforRiskAnalysis[OMB2007]).TheAgencyhaspublishedbasicguidanceaswellassomeprogram‐specificproceduresandapplicationsforPRA(USEPA2001).AsthescienceofPRAcontinuestoevolve,theenhanceduseofthisapproachwillfacilitatebettercharacterizationofuncertaintyandimprovetheoveralltransparencyandqualityofEPAassessments.PRAapproachesprovideadditionaltoolstoaddressspecificchallengesfacedbymanagersandimproveconfidenceinAgencydecisions.PRAcaninformdecision‐makersaboutspecificsegmentsofthepopulationwhoareatrisk,asopposedtothepopulationasawhole.APRAalsocanaddconfidencetotheconclusionsofadeterministicriskestimate.Thisinformationcanbeimportanttomanagerswhenadifferentdecisionmightbemadeiftheupperorlowerlimitsoftherangeofestimatedexposures,dosesorriskswerepresentedinananalysisasdescribedmorefullyinRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(EPA2014a).

As a manager, what do I need to know about PRA? Thereareprobability,uncertaintyandvariabilityinmanyfields,suchasweatherforecasting,politicalpollsorclimatechangepredictions.PRAcanbeusedineachoftheseexamplesandothers,atmanylevelsordegreesofsophistication,tosupportorimprovedecisionsinavarietyofdifferentdecisioncontexts.ThefollowingsectionsdescribethebasicsofPRAinmoredetail,howitcanbeusedtosupportdecisions,andwhattoconsiderwhenpursuingaPRAorinterpretingPRAresults.

How could a risk manager approach a PRA? What would a manager consider? PRAresultsarelikelyjustsomeoftheinformationthatriskmanagersmightusetoinformtheirdecision‐making.Iftheriskmanageristryingtoansweraquestionthatrequirestheuseofmanydifferentcriteria(whereriskmightbejustonecriterion),thentheymightwanttocontemplatetheuseofformaldecisionanalytictechniques.Withdecisionanalytictechniques,riskmanagerscanexploretherelativeimportanceofriskinformationcomparedtootherinformationinmakingthedecision,andexplorehowuncertaintyaffectspotentialdecisions.PRAprovidesinformationorinputtodecisionanalysis.Decisionanalysisisalogicalprocedureforthebalancingoffactorsthatinfluenceadecision.Theprocedureincorporatesuncertainties,values,andpreferencesinabasic

3

structurethatmodelsthedecision(Howard,1966).DecisionanalysisisabroaderactivitythanPRA,andformaldecisionanalytictechniquesareoutsideofthescopeofthisFAQdocumentandcompanionPRAwhitepaper.ThereisarangeofPRAtechniquesthatmightbeusefultosupportenvironmentaldecisions,andtheyvaryincomplexityandresourcerequirements.Communicationbetweentheriskmanagersandriskassessorsiscriticaltoclearlydefinethespecificneedsofthedecision‐makerandthequestionstobeaddressedbytheriskassessment,sothatonecanfocusthePRAontheinformationmostlikelytoinformthedecision.AspartofPlanningandScopingoftheriskassessmenttheriskassessorandriskmanagershouldevaluatethetypesoftechniquesappropriatetomeetthegoalsoftheassessmentandestablishaprocessforcompletingandreviewingthePRAinacost‐effectiveandtimelymanner.Thedialogueshouldcontinuethroughouttheprocess(U.S.EPA2014b).

What is PRA? How does it compare with current approaches? Currentapproachestoriskassessmenttypicallyrelyonsinglepoint(oftencalled“deterministic”)estimatesofrisk,withbriefqualitativedescriptionsofuncertainty.PRAprovidesriskmanagerswithinformationabouttheuncertaintiesinthedata,models,assumptionsandresults.AbasiccharacteristicofPRAisthatitdoesnotgenerateasinglepointestimate(likeconventionalapproaches),butratherproducesalikelihoodandrangethataparticularexposure,doseoreffectwilloccur.PRAisagroupofcomputationaltechniquesthatallowstheanalysisofvariabilityanduncertaintytobeincorporatedintoexposureand/orriskassessments.PRAalsocanbeusedtosupportriskmanagementusingsensitivityanduncertaintyanalysestoassesstheimpactofuncertaintiesoneachofasetofpotentialdecisionalternatives.Asmentionedabove,thistypeofinformationcanbecriticalindecisions,especiallyindeterminingthelikelihoodofexposureorriskinaspecificportionofthepopulation.Unlikeconventionalapproaches,PRAalsoissuitedtoprovidingdecision‐makerswithabetterunderstandingoftheimpactofuncertaintiesontherelativemeritsofeachofthedecisionalternatives,providingtheriskmanagerwithaclearerrationaleforhisorherdecisionchoice.

What are common challenges facing EPA decision makers? ThechallengesfacingEPAdecision‐makersincludetheneedtomakecomplex,science‐baseddecisions,informedbyconsideringtheimpactofuncertainties,whereconflictingstakeholderperspectivesandshorttimeframesareinvolved.Attributescriticaltodecision‐makinginclude:(1)understandingwhomorwhatwillbeprotected;and(2)knowingtheappropriatedegreeofconfidenceintheestimatedprotectionprovidedbyaparticulardecision.Decisionsareoftentime‐sensitiveandneedtobemadebasedonthecurrentstateofknowledge.Theymayinvolvethecomparisonofseveralalternatives.Inmostcases,thehealthandenvironmentalimpactsofenvironmentalexposurescannotbeisolatedandmeasureddirectly;therefore,riskassessmentmethodstoestimatethehealthandecologicalrisksbasedonavailabledataandinformationcanbeused.BecauseEPAdecisionmakersmayfacechallengesindecisionmakingthatarenotcompletelyrelatedtodecidingwhetherornottousePRA,theymustachieveabalanceinthedecision‐makingprocess,tradingoffthecostsandbenefitsofmoredetailedanalyseslikePRAwiththeiroveralldecisiongoals.

Makingariskdecisionmayalsoconsiderotherfactors,suchastheeconomicimpactsorcostsofalternativestoaddressdifferentialrisksinavulnerablepopulation.EPAdecisionmakersneedtoconsidermultipledecisioncriteriainformedbyvaryingdegreesofconfidenceintheunderlyinginformation,understandingtherelationshipbetweenandamongthosedecisioncriteria(includingmulti‐pollutantandmulti‐mediaeffects)anddecisionalternatives.

4

What are variability and uncertainty? What is their relevance in risk assessment and risk management? Variabilityreferstotheinherentnaturalvariation,diversityandheterogeneityacrosstime,spaceorindividualswithinapopulationorlifestage,whileuncertaintyreferstoimperfectknowledgeoralackofpreciseknowledgeofthephysicalworld,eitherforspecificvaluesofinterestorinthedescriptionofthesystem(USEPA2011c).Variabilityanduncertaintyhavethepotentialtoresultinoverestimatesorunderestimatesofthepredictedrisk.

Variabilityreferstonatural,inherentvariation,variabilityisunavoidable,andcannotbereducedinthewayuncertaintycan.Dataandinformationcanbeusedtoprovideabetterdescriptionandunderstandingofvariabilityintheworldoraparticularsystemtherein.Variabilityispresentinallaspectsofthesource‐to‐effectcontinuum(Figure1),includingthefollowing:

Howpollutantsarereleased(e.g.,theeffectivenessofemissioncontrols).

Howpollutantsareinfluencedbyenvironmentalconditionsoncereleased(e.g.,meteorology—temperature,windandprecipitation).

Theexposureofpollutantstoreceptors(e.g.,inhalationoringestionrates).

Theeffectsofpollutants(e.g.,endpoint,healthstatus,geneticsusceptibility).

Anexampleofvariabilityistheamountofwaterconsumedbyanadultpopulation.Forexample,asurveyof1,000peopleondailywaterconsumptionmightresultinthedistributiondepictedinFigure2A.AnalternativepresentationofthesamedistributionusingcumulativeprobabilityisgivenbyFigure2B.Bothfiguresshowthat5percentoftheadultpopulationconsumes2.96litersofwaterperdayormore,whereasthemean(average)individualconsumptionis1.04litersofwaterperday(basedonstudiesidentifiedinthe1997editionoftheExposureFactorsHandbook[USEPA1997a],whichwasupdatedin2011[USEPA2011b]).

Uncertaintyisthelackofunderstandingoftheworld;whileunavoidableitcanbereducedthroughadditionalinvestigationorcollectionofbetterinformation.Numerousschemesforclassifyinguncertaintyhavebeenproposed,mostfocusonthreebroadtechnicalcategories:

Inputorparameteruncertaintyreferstouncertaintiesinspecificestimatesorvaluesusedinamodel,suchastheaveragedrinkingwaterintakerate.

Modeluncertaintyreferstogapsinthescientificknowledgeortheorythatisrequiredtomakeaccuratepredictions,suchashowtwocorrectlyspecifyexposurestowater.

Scenariouncertaintyreferstoerrors,typicallyofomission,resultingfromincorrectorincompletespecificationoftheriskscenariotobeevaluatedsuchaserrorsintheproblemformulationomittingindirectwateringestionexposurepathways.Theriskscenarioisasetofassumptionsforthesituationtobeevaluated.

Adecisionanalysiswouldincludenotonlytheimpactoftheinput,modelandscenariouncertaintiesontherelativeattractivenessofpotentialdecisionalternatives,butalsowouldincludethedegreetowhichspecificchoices(suchasselectinginputdata,modelsandscenarios,andevenhowtheproblemordecisionanalysisisframed)alsomayimpacttherelativeattractivenessofpotentialdecisionalternatives.

Fromariskmanager’sperspective,understandingbothuncertaintyandvariabilityareimportant.Variabilityrelatestoourunderstandingofwhomorwhatweareprotecting,anduncertaintyrelates

5

Figure 1. Source‐to‐Effect Continuum. Variability exists within each aspect of the source‐to‐effect continuum, the path from an environmental source to resulting exposures and ultimate health outcomes (adapted from NRC 2009, Figure 4‐1, page 95).

toourconfidenceintheestimateandthelevelofprotectionaffordedbytheoptions.TheAppendixinthecompaniondocumenttothisFAQdocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2013),describesingreaterdetailsomespecificissuesrelatedtothemethodologyforPRA.

WhetheritisappropriateorusefultoconsidervariabilityanduncertaintyindecisionsmadebyEPAprogramsandregionsmaybeaffectedbystatutes,precedentsandinputprovidedbystakeholders.CongressestablisheslegalrequirementsthatgenerallydescribethelevelofprotectivenessthatEPAregulationsmustachieve.Infrequently,Congressimposesspecificriskassessmentorotherrequirementsthatmayminimizetheregulatorynecessitytoaddressuncertaintyorvariability.Individualstatutesidentifyvaryingriskstoevaluateandprotectagainst(e.g.,establishamarginofsafety,protectsensitiveresources,orreduceoverallrisks).Inaddition,theymaymandatedifferentlevelsofprotection(e.g.,protectpublicwelfare,preventunreasonablerisk,reduceoverallrisks,orfunctionwithoutadverseeffects)(USEPA2004b).Thesedifferenceshaveimplicationsforwhetherorhowoneanalyzesorconsidersuncertainty(e.g.,thedegreeofconfidenceinprotection)orvariabilitywithinapopulationorlifestage(e.g.,evaluatingthe“expectedriskorcentralestimateofriskforthespecificpopulations”1vs.risksto“sensitiveorsusceptibleindividualsorgroups”2or“individualmostexposed”3).

The2004EPAstaffpapertitledAnExaminationofEPARiskAssessmentPrinciplesandPractices(USEPA2004b)andthePRAwhitepapertitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a;companionpublicationtothisFAQdocument)describeuncertaintyandvariabilityinsomedetailandcanbereferredtoformoreinformation.

1SafeDrinkingWaterActAmendmentsof1996.42U.S.C.§300g‐1(b)(3)(1996).2CleanAirActAmendmentsof1990.42U.S.C.§108(f)(1)(C)(1990).3CleanAirActAmendmentsof1990.42U.S.C.§112(c)(9)(B)(i)(1990).

6

Figure 2. Ingestion of Drinking Water for Adults. Variability can be demonstrated by the amount of water consumed by a population. In this example, the distribution of daily water consumption for adults (ages > 21 years) is depicted in part A, while the cumulative probability is presented in part B.

7

How is risk assessment conducted currently at EPA? GeneralphasesoftheriskassessmentprocessareillustratedinFigure3.Therearemanysimilaritiesbetweenecologicalandhumanhealthriskassessment,andEPAisworkingtoincreaseintegrationbetweenthesetwoprocesses.

Do human health risk assessment and ecological risk assessment treat uncertainty and variability differently? Humanhealthriskassessment(HHRA)addressestheprotectionofonespecies.Ecologicalriskassessment(ERA)generallyaddressestheprotectionofalargenumberofspecies,usingafewspeciesassurrogatesforhundredstothousandsofspeciespresentintheenvironment.HHRAusuallyaddressestheevaluationofrisktoindividualsorgroupsofsusceptiblehumanpopulationsorlifestages.ERAgenerallyaddressestheevaluationofrisktopopulations(notindividuals,exceptinthecaseofendangeredspecies),communitiesoforganismsorecosystems.HHRAgenerallyfocusesonsublethaleffectsorendpoints.ERAfrequentlyfocusesonlethality,reproductionandgrowthasmajorassessmentendpoints.

How does EPA typically address variability and uncertainty? BecauseEPAcannotperformatime‐andresource‐intensiveriskassessmentforeverysituationanddecision,theAgencymustbestrategicindeterminingwhenmoreintensiveassessmentsareneeded.WhenEPAdoesnotexplicitlyquantifythedegreeofconfidenceinariskestimate,theAgencyattemptstoincreaseconfidencethatriskisnotbeingunderestimatedbyapplyingvariousassumptionstoaddressuncertaintyandvariability.AsdepictedinEquation1,methodsthatrelyonacombinationofpointestimates—someconservative(highparametervaluesthataremorelikelytooverestimaterisk)andsomethataretypicaloraverage—usuallymultiplythepointestimatestoachieveasingleestimate(e.g.,conservativepointestimatesyieldaconservativeestimate,averagepointestimatesyieldanestimateforanaverageortypicalindividual).

Equation 1. Deterministic Risk Assessment

= RISK Concentrationin environment

Exposure Duration

Ingestion or Inhalation Rate

Toxicity Factor

[C] [E] [I] [T]

[c] [e] [i] [t]Central tendency (average) values for all parameters

High-end values for some or all parameters

Risk. .

Yields a reasonable estimate for average or typical individual

Yields estimate that is likely biased high (conservative)

[ [] [ [] ] ]X X X

=

X

X X X

XX =

8

Figure 3. General Phases of the Risk Assessment Process. Risk assessment is an iterative process comprised of planning, scoping and problem formulation; analysis (e.g., hazard identification, dose‐response assessment and exposure assessment); interpretation and risk characterization; and risk communication.

9

Thisapproachtypicallyproducesasingleestimateofrisk(e.g.,a1x10‐6excesscancerrisk,whichisanincreasedriskofonein1millionadditionalcancers)alongwithaqualitativediscussionofuncertainty.Thesearegenerallyreferredtoas“deterministicassessments”or“pointestimates”ofriskbecausealltheelementsofthecalculationareassumedtobedeterminedwithoutvariationatsinglevalues.Pointestimatesareuseful,particularlyinscreeningassessmentswitheithercancerornoncancerendpoints,buttheinherentvariabilityanduncertaintiesarenotquantifiedfully.TheseinherentlimitationscanaffectEPAdecisionsinthefollowingways:

TheinabilitytocharacterizeexplicitlythebasicelementsofEPAdecisions—whomorwhatisbeingprotectedorwithwhatdegreeofconfidence.

Theinabilitytocapturemorerealisticallytheriskrangeinsteadofapointestimate.

Theinabilitytocompareaccuratelytheoptions(e.g.,cleanuplevels,permitlevels,regulations,actions)duetothedifferinglevelsofconservatismimplicitintheindividualestimates.

Thedecreasedabilitytomaketradeoffsorreachanappropriatebalancebetweenbenefitsandcosts.

Useofapointestimatedrawscriticismandengendersdebateforbeingconservativeandunrealisticorforprovidinginadequateprotection.Thesecriticismsmaynotbeansweredreadilyduetolimitationsineitherthemethodsordataused.

Pointestimatesdonotpermitmeasurementofuncertaintybecausetheyarepresentedasasingle“true”value,ratherthanarangeofpossiblevalues.

How does PRA address variability and uncertainty? PRAusesdistributionsofvaluesthat,forvariability,reflectvariationsintherealworld,andforuncertainty,reflectlackofknowledgeregardingparameters,modelsand/orscenarios.Theresultisanoverallprobabilitystatementoftherisk;forexample,whattherisksaretotheaverageormeanindividualandthehigh‐endindividual,suchasthe95thpercentile,illustratedinEquation2.Thehypotheticalexampleillustratestheapproachforestimatingindividualrisk.Insomecases,exposureandhealthriskareestimatedforanentirepopulationoramongsusceptiblepopulationsandlifestages.

Equation 2. Probabilistic Risk Assessment

95th % population -level risk

% p

opul

atio

n

risk

Mean risk95th %

population -level risk

% p

opul

atio

n

risk

Mean risk95th %

population -level risk

% p

opul

atio

n

risk

Mean risk= RISK Concentration

in environment

Exposure Duration

Ingestion or Inhalation Rate

Toxicity Factor[ [] [ [] ] ]X X X

X X X =

10

PRAcouldbeappliedtooneormorephasesoftheriskassessmentparadigm,includinghazardcharacterizationaswellasexposure,toxicityand/orriskassessment.InthepastEPAhasfocusedprobabilistictechniquesonpredictionsofexposures.Probabilistictechniquescanalsobeusedduringtheeffectsassessmentphasetoaddressvariability/uncertaintyindataandactivitiesthatincludethefollowing:

Hazardidentification;

Assessmentofdose‐responserelationships(e.g.,aprobabilisticreferencedose(RfD)couldhelpreducepotentialinaccuraciesintheestimationofriskclosetoorbelowtheRfD);

Evaluationofhealtheffectsdata;and

Considerationofdifferentialvulnerabilitiesandsensitivitieswithinapopulation.

How can enhanced characterization of uncertainty and variability help inform decisions? Pertainingtoinputparameter,modelandscenariouncertainties,thereareseveralwaysbywhichvarioustypesofPRAcaninformriskmanagementdecisionmaking.First,asensitivityanalysiscandetermineifmorerefinedinformationaboutthedistributionandrangeofdatawouldhaveasubstantialeffectontheassessmentofriskandmanagementoptions.Thisanalysiswillhelpdeterminehowrobustadecisionmaybewithcurrentinformationand/oridentifywhichparametershavingfurtherrefinedinformationwouldmostbenefitthedecision.Second,PRAcanbeusedtodescribeuncertaintyintheriskassessment,increasethetransparencyoftheinputstothedecisionandassistinselectingamongvariousmanagementoptions.Inaddition,PRAcandothefollowing:

EnhanceEPAdecisionsbyprovidingmoreinformationaboutthepossibleimpactsofalternativedecisions.

Provideclarityonwhomorwhatweareprotectingandthedegreeofconfidencethatcanbeplacedontheestimatesofprotectionprovidedbyagivendecision.

Allowforamoredetailedcomparisonofalternativeriskmanagementoptionsintermsoftheestimatedimpactsonbothprotectionandcosts.

Characterizetheinherentuncertaintiesandimpactofthoseuncertaintiesonthedecisionandimprovetheoverallconfidenceinspecificdecisions.

Supporttheunderstandingofthecost,timeandfeasibilityofobtainingimprovedinformationoncriticaluncertainties.

TheNRCrecommendedthatEPAadopta“tiered”approachtoselectthelevelofdetailusedinuncertaintyanalysisandvariabilityassessments(NRC2009).Furthermore,itrecommendedthatadiscussionofthelevelofdetailneededforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationphaseoftheriskassessmentprocess.ThecompaniondocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a)providesadetaileddiscussionofPRAandEPA’sexperiencewithriskassessments,highlightingthestepsintheprocessduringwhichPRAcanbeappliedtocharacterizeuncertaintyandvariability,including:

Planning,scopingandproblemformulation(e.g.,developinganenvironmentalsamplingplanthatconsiderscost,timelinessandtheeliminationofsamplingbias).

Analysis(e.g.,consideringuncertaintyandvariabilityinexposurefactors,exposurescenarios,environmentalconcentrationsofagentsanddose‐responserelationships;

11

assessingwhoisexposed;modelingandevaluatingwhetherfurtherplanning,scopingandproblemformulationisrequired).

Interpretationandriskcharacterization(e.g.,describingvariabilityinriskestimatesbecauseofvariabilityofgeographicconditionsandexposurewithinapopulation,conductinganuncertaintyanalysisbyperformingasensitivityanalysis,estimatinguncertaintyandidentifyingsourcesofuncertainty).

Riskcommunication(e.g.,beingexplicitaboutvariabilityanduncertainty;evaluatingwhetherfurtherriskcharacterizationisrequired).

Decisionmaking(e.g.,prioritizingdataneeds,evaluatingwhetherfurtherriskcommunicationoranalysisisrequired).

What is the impact of variability and uncertainty on making decisions? Variabilityanduncertaintyareinherentinriskassessments.Wheredataandinformationareincompleteorareinadequate,makinginformeddecisionsismoredifficult,andwithoutconsideringuncertaintyinthestateofknowledgeortheimpactofchoicesmade(suchasinselectingdata,models,scenariosorframingtheproblem),thereexistsgreaterpotentialformakingpoorlyinformeddecisionchoices.Inthecaseofenvironmentalregulations,specificdecisionsmayleadtoover‐orunder‐regulationcomparedwithdecisionsthatcouldbemadewithperfectinformation.Settinganenvironmentalstandardthatistoolaxmaythreatenpublichealth,whereasastandardthatisunnecessarilystringentmayimposeasignificanteconomiccostforamarginalgaininpublichealthandenvironmentalprotection.

Theuseofpointestimatesindecisionmakingmightconveyanunfoundedhighdegreeofprecision.Aprobabilisticapproachcanbeappliedratherthandefiningasinglevalueandmightbelesslikelytoimplyundueprecision.Thedisplayofresultsasadistributioncanconveythatnoparticularvaluecarriesspecificrisksofadverseeffects;italsocansupporttheunderstandingofthemagnitudeandlikelihoodofriskofadverseimpactforarangeofexposures,notjustanindividualpointestimateofdoseorexposure.

Is uncertainty and variability limited only to risk assessment? PRAcanalsohelptoinformdecisionsbyforcingdelineationoftheuncertaintypresentandprovidingevaluationmeasuresofoutcomes(oftenreferredtoasadecisionanalysis).Uncertaintiesoftenarerepresentedasprobabilitiesorprobabilitydistributionsingraphsornumericalformats.Thiscanbeveryusefulinevaluatingandcommunicatingtheriskmanagementalternativesavailabletothedecisionmaker.

Indecisionmaking,uncertaintyandvariabilityarenotlimitedtodata,modelsandscenarioconstructionandhowthesemayaffectthepreferenceofonedecisionalternativeoveranother.Adecisionmakermaybefacedwithcombiningmanydifferentdecisioncriteriarequiringtheuseofweightsorvalues(e.g.,whatismoreimportantandbyhowmuch?)tochooseadecisionalternative.Theuseandcombinationofdecisioncriteriamayintroduceotheruncertaintiesandvariability.Thecombinedimpactofdata/model/scenarioselection,expertjudgmentandweighting/valuesonthepreferenceofdecisionalternativesfallsunderthebroadumbrellaofthefieldofdecisionanalysis.

Byfullyunderstandingthewaysinwhichalltypesofuncertaintyinfluencetheinformationusedindevelopingadecision,managerscanbemoreconfidentthattheirdecisionwillproducetheresultsthattheyseekaswellasbebetterabletodefendtheirdecisionsandexplainhowthechosenalternativemeetsAgencyandstakeholdergoals.Thissummaryisfocusedprimarilyonaddressinguncertaintyandvariabilityrelatedtodata/inputs,modelsandscenarios.Uncertaintyand

12

associatedwithdecisioncriteriavaluesandweightsarebeyonditsscope.AlthoughworkinthisaspectofdecisionmakingatEPAhasbeenlimited,someexamplesareavailableatEPA’sMulti‐CriteriaIntegratedResourceAssessment(MIRA)website(http://www.epa.gov/reg3esd1/data/mira.htm).

What key questions related to uncertainty and variability are asked or considered by decision makers? ThePRATechnicalPanelconductedseveraldialogueswithEPAdecisionmakers,askingthemwhatquestionsarisewhentheyarefacedwiththetaskofmakingdecisionsinthepresenceofuncertainty.Thefollowingquestionsrepresenttypicalconcernswhenaddressingdifferenttypesofuncertaintyandvariability:

Wouldmydecisionbedifferentifthedataweredifferent,improvedorexpanded?Wouldadditionaldatacollectionandresearchlikelyleadtoadifferentdecision?Howlongwillittaketocollecttheinformation,howmuchwoulditcost,andwouldtheresultingdecisionbesignificantlyaltered?

Whataretheliabilitiesandconsequencesofmakingadecisionunderthecurrentlevelofknowledgeanduncertainty?

Howdothealternativesandtheirassociateduncertaintyandvariabilityaffectthetargetpopulationorlifestage?

Howrepresentativeorconservativeistheestimateduetodata,methodorscenariouncertainty?

Whatarethemajorgapsinknowledge,andwhatarethemajorassumptionsusedintheassessment?Howreasonablearetheassumptions?

Canaprobabilisticapproach(e.g.,tobettercharacterizevariabilityanduncertainty)beaccomplishedinatimelymanner?

What is the desired percentile of the population to be protected? By choosing this percentile, who may not be protected? How can addressing uncertainty and variability using PRA help inform decisions? PRAcanprovideinformationtodecisionmakersonspecificquestionsrelatedtovariabilityanduncertainty.

Characterizationoftheuncertaintyinestimates(i.e.,whatisthedegreeofconfidenceinanestimate?).Couldtheestimatebeoffbyafactorof2,afactorof10orafactorof1,000?

Criticalparametersandassumptionsthatmostaffectorinfluenceadecisionandtheriskassessment.

“Tippingpoints”wheretheoptionschosenwouldbealterediftheriskestimatesweredifferent,orifadifferentassumptionwasvalid.

Estimatethelikelihoodthatvaluesforcriticalparameterswilloccur.

Testthevalidityofassumptions.

Estimatethedegreeofconfidenceinaparticulardecisionand/orthelikelihoodofspecificdecisionerrors.

Thepossibilityofalternativeoutcomeswithadditionalinformation,orestimatetradeoffsrelatedtodifferentrisksordecisions.

13

PRAcanprovideinformationontheimpactofadditionalinformationondecisionmaking,consideringthecostandtimetoobtaintheinformationandthepotentialforchangeinthedecision(i.e.,thevalueoftheinformation).Forconsiderationofvariability,PRAcanprovidethefollowingtypesofinformationforexposures:

Explicitlydefinedexposuresforvariouspopulationsorlifestages(i.e.,whoarewetryingtoprotect?).Thatis,willtheregulatoryactionkeep50%,90%,99.9%orsomeotherfractionofthepopulationbelowaspecifiedexposure,doseorrisktarget?

Variabilityintheexposures,amongvariouspopulationsorlifestages,andinformationonthepercentileofthepopulationthatisbeingevaluatedintheriskassessment(e.g.,variationsinthenumberoflitersofwaterperkilogram[kg]bodyweightperdayconsumedbythepopulation).Thisinformationishelpfulinaddressing:

TheconservatismofEPA’sriskassessments;

Concernsaboutwhetherparticularexposureswereevaluatedintheriskassessment;

Whomorwhatisbeingprotectedbyamanagementaction;and

Whetherandwhatadditionalresearchmaybeneededtoreduceuncertainty.

AfewhypotheticalexamplesofthetypesofriskassessmentquestionsthatcanbeaddressedexplicitlythroughPRAareillustratedinFigure4.

Figure 4. What Questions Can PRA Address? PRA can be used to determine the likelihood of exposure or risk in a specific fraction of the population.

14

What are some of the limitations of and challenges for PRA? Dataand/orresourcesmaynotbeavailabletosupportprobabilistictechniquesatallstagesinthesameassessment,requiringtheriskassessortocontinuetoapplysomedeterministicscience‐policyassumptionsandconversions.Ifscience‐policyassumptionsordefaultvaluesforparametersareappliedtoanassessmentthatincludesPRA,theyshouldbearticulatedclearlyinthedisseminationofresults.PRAtypicallyrequiresmoretimetodevelopthanadeterministicassessment,butthesetechniquescanfitintoagraduatedortieredapproachtoriskanalysis.

Additionallimitations/challengesincludethefollowing:

PRAmaybemoredataintensivethandeterministicapproaches,requiringadditionaltimeandfinancialandanalyticresourcestoobtainthenecessarystatisticaldistributioninputdataforeachaspectoftheriskassessment.Moreroutineincorporationofprobabilisticdesignsinriskassessmentanditssupportingresearchcouldreducethiscostdifferential.

Todate,PRAtechniqueshavebeenusedbyEPAscientistsmostsuccessfullyintheexposureaspectofhumanhealthriskassessment.AlthoughPRAcanbeusedtocharacterizetheuncertaintyandvariabilityinsituationswithlimiteddata,experienceisneededtousePRAtocharacterizetherangeofeffectsordose‐responserelationshipsforpopulations,includingsensitivepopulationsandlifestages.Moreworkisrequiredtodevelopthesemethodsforothertypesofanalyses.

Thedisseminationofastatisticaldistributionorprobabilityoutputnumbershouldberelatedcarefullytothequalityandcoverageoftheinputdata;otherwise,thePRAresultscouldleadtoafalsesenseofprecision.

PRAassessmentsthatarenottransparentmaybeusedinappropriatelytomanipulateresultsandobfuscatethebasesofdecisions.Itshouldbenotedthatthismayoccurwithdeterministicassessmentsaswell.

What is EPA’s experience with PRA? RecentEPAexperiencewithPRAisprimarilylimitedtoanalyzingdata,modelandscenariouncertaintiesandvariability.Inthepast,however,EPAusually—butnotalways—reliedondeterministicorpointestimatestoevaluaterisk(e.g.,cancerriskof1x10‐6excesscancersoronein1million).TheuseofPRAtoevaluateuncertaintyandvariabilityinriskassessmentsisincreasing.Theseeffortsarevariedacrossprogramofficesandregions,bothincomplexityandinapplications.ManyPRAapplicationsfocusedonspecificelementsofariskassessment(e.g.,exposure),variabilityoruncertainty.ExamplesareprovidedintheAppendixoftheRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a),thecompanionpublicationtothisFAQdocument.Thewhitepapercontains16casestudies(11HHRAsand5ERAs)thatdetailPRAsthathavebeenconductedtosupportregulatorydecisionsand/orregulatoryimpactanalyses.ExamplesofEPA’suseofdeterministicriskassessmentandPRAareillustratedinTable1,whichhighlights6ofthe16casestudiesinthecompaniondocument.TheseexamplesapplydifferentPRAtoolsinriskassessment.Thecasestudiesrangefromlessresource‐intensiveanalysestomoredetailedandresource‐intensiveapproaches.

U.S.EPAInternalReviewDraft 15 DoNotCite–Quote–orDistribute

Table 1. Selected Examples of EPA Applications of Risk Assessment Techniques from Deterministic to More Complex Analysis

Case Study No. Description Type of Analysis

(Application) Factors Informing

Decisions Type of Risk Assessment

Program/ Region

2*

Atmospheric Deposition to Watershed Contamination: EPA’s Office of Research and Development (ORD) developed an analysis of nitrogen, mercury and polynuclear aromatic hydrocarbons (PAHs) in the Casco Bay Estuary in southwestern Maine.

Deterministic Analysis (Nitrogen Deposition Rate)

Cost-Effectiveness, Timeliness

Ecological ORD

5†

Hudson River Polychlorinated Biphenyl (PCB)-Contaminated Sediment Site: EPA Region 2 evaluated the variability in risks to anglers who consume recreationally caught fish contaminated with PCBs from sediments in the Hudson River.

1-Dimensional Monte Carlo

Analysis (Exposure Factors)

Exposure Variability Within a Population

Human Health

Superfund/

Region 2

(New York)

7†

Environmental Monitoring and Assessment Program (EMAP): ORD developed and the Office of Water (OW) applied probabilistic sampling techniques to evaluate the nation’s aquatic resources under the Clean Water Act (CWA) Section 305(b).

Probabilistic Sensitivity Analysis

(Environmental Sampling Plan)

Representative Results, Cost-Effectiveness, Elimination of Sampling Bias

Ecological ORD/OW

9†

Chromated Copper Arsenate (CCA) Risk Assessment: ORD and the Office of Pesticide Programs (OPP) conducted a probabilistic exposure assessment of children’s exposure (addressing both variability and uncertainty) to arsenic and chromium from contact with CCA-treated wood play sets and decks.

2-Dimensional Monte Carlo

Analysis (Exposure Scenarios)

Exposure Variability, Sensitivity Analysis, Prioritization of Data

Needs

Human Health ORD/OPP


Table 1. Selected Examples of EPA Applications of Risk Assessment Techniques from Deterministic to More Complex Analysis

Case Study No. Description Type of Analysis

(Application) Factors Informing

Decisions Type of Risk Assessment

Program/ Region

13†

Evaluating Ecological Effects of Pesticide Uses: OPP developed a probabilistic model that evaluates acute mortality levels in generic and specific ecological species for user-defined pesticide uses and exposures.

Probabilistic Analysis (Exposure,

Dose-Response)

Variability of Geographic Conditions,

Prioritization of Data Needs

Ecological OPP

14†

Fine Particulate Matter (PM2.5) Health Impacts: ORD and the Office of Air and Radiation (OAR) used expert elicitation to characterize more completely, both qualitatively and quantitatively, the uncertainties associated with the relationship between the reduction in particulate matter (PM2.5) and benefits of reduced PM2.5-related mortality.

Expert Elicitation (Dose-Response)

Estimation of Uncertainty,

Identification of Sources of Uncertainty

Human Health ORD/OAR

* Deterministic risk assessment † Probabilistic risk assessment


What are other EPA publications on PRA? EPA’sexperiencewithPRAincludesnotonlyindividualassessmentsorapplications,butalsothedevelopmentofgeneralguidanceandpoliciesrelatedtodata,modelandscenariouncertainty,suchasthefollowingdocuments:

PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency(USEPA1997c);

GuidingPrinciplesforMonteCarloAnalysis(USEPA1997b);and

RiskAssessmentGuidanceforSuperfund,VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment(USEPA2001).

When should one consider using PRA? ItisappropriatetoconsiderPRAanytimethereisanexpectationthatthequantificationofuncertaintyand/orvariabilitymightinfluenceorimpactadecision.AsensitivityanalysisforthedecisioncanhelpmanagersdeterminewhetherhavingsuchinformationiscriticalandwhetherthetimeandresourcesneededforPRAarewarranted.PRAmaynotbeneededwhenthedecisionisroutine,legislativelymandatedorifastandardmethodologyisprescribed.Furthermore,PRAmaynotbeneededwhenthereishighconfidenceinthedataandmodelsusedtosupportthedecision.Planningandscopingdiscussionsareneededbetweentheriskassessorsandtheriskmanagers;thesearedescribedinEPA’sFrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking(USEPA,2014b),EcologicalRiskAssessmentFramework(USEPA1992),GuidelinesforEcologicalRiskAssessment(USEPA1998),andFrameworkforCumulativeRiskAssessment(USEPA2003).AnanalysisplandevelopedduringtheplanningandscopingphasemayindicatethatinformationfromaPRAcouldsignificantlyinformthedecision.Someexamplesare:

Aspecifiedtargetlevelofprotectioninapopulationisidentifiedbythemanager(e.g.,the95thpercentile),anditisnecessarytodemonstratethatthisgoalismet.

Significantequityorenvironmentaljusticeissuesareraisedbyvariationinrisksamongtheexposedpopulationofconcern.

Screening‐levelpointestimatesofriskarehigherthananacceptedlevelofconcern.

Uncertaintyinsomeaspectoftheriskassessmentishigh,anddecisionsarecontentiousorhavelargeresourceimplications.

Specificcriticalriskestimatesandassumptionspointtodifferentmanagementoptions.

ThescientificrigorandqualityoftheassessmentiscriticaltothecredibilityoftheEPAdecision.

Whenascreening‐levelDRAindicatesthatrisksarepossiblyhigherthanalevelofconcernandamorerefinedassessmentisneeded.

Whentheconsequencesofusingpointestimatesofriskareunacceptablyhigh.

Whensignificantequityorenvironmentaljusticeissuesareraisedbyinterindividualvariability.

Whenexploringtheimpactoftheprobabilitydistributionsofthedata,modelandscenariouncertaintiesaswellasvariabilitytogethertocomparepotentialdecisionalternatives.


PRAisusefulespeciallyifthedecisionprocessisiterativeandadaptive,allowingfordetailedandtargetedanalysisovertime.Ifthedecisionhasasingledecisionpointorseparatesinglestepsinmakingadecisionthatwillnotberepeatedorrevised,however,PRAshouldbeusedwithcautionasitmaybeusedtomanipulateresultsifitisnotdesignedproperlyandtransparently.

PRAisnotnecessarilyappropriatewhenregulatoryandlegislativemandatesdefinetheassessment(e.g.,“individualmostexposed”),whichmayminimizestheissueofvariability.Anuncertaintyanalysiswouldstillbeappropriate.

What is the right level of analysis? TheNRC(2009)recommendsatieredapproachtoriskassessmentusingbothqualitativeandquantitative(deterministicandprobabilistic)tools,withthecomplexityoftheanalysisincreasingasprogressismadethroughthetiers.ThewaythatPRAfitsintoagraduatedhierarchical(tiered)approachisdescribedmorefullybelowandisillustratedinFigure5.ThethreetiersinthefigureroughlycorrespondtothethreegroupsofEPAcasestudiesthatusePRAtools,describedinTableA‐1oftheAppendixofthispublication’scompaniondocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a).

Thereisnosingle“right”levelofanalysis.Instead,theappropriatelevelofanalysisisguidedbythenatureofthedecisionandapplicableregulations.Asisthecaseforriskassessmentingeneral,approachestoPRAandspecificanalyticalmethodsmayvarydramaticallyintheircomplexityandresourceimplications.Theconceptofiterativeortieredanalyses,depictedinFigure5,iswidelyacceptedinriskassessment,andthisappliestoPRAaswell.Highertierscouldreflectincreasingcomplexityandgenerallywillrequiremoretimeandresources.Ananalysistypicallymightstartatalowertier(suchasscreeningforprioritization)andonlyprogressifthereisaneedforamoresophisticatedassessmentcommensuratewiththeimportanceoftheproblem.Highertiersalsoreflectincreasingcharacterizationofvariabilityand/oruncertaintyintheriskestimate,whichmaybeimportantfordecisionmaking.ThereisawiderangeofPRAmethodsandapproachesofvaryingcomplexityandrigorthatcanbeappliedfordifferentpurposes,rangingfromsensitivityanalysistointegratedanalysisofuncertaintyandvariability.Thegoalistochoosealevelofdetailandrefinementforananalysisappropriatetotheoverallobjectivesofthedecisionandtypesofavailabledataandanalysesneededtosupportdecisions.Intheplanningandscopingandproblemformulationphases,itiscriticaltohaveearlyandcontinueddialoguebetweenthemanagerandriskassessortodevelopaclearunderstandingoftheoverallprojectobjectives,theneedsofthedecisionmaker,thetimingofthedecisionandhowPRAmayplayaroleinthedecision‐makingprocess.Theuseofformaldecisionanalyticapproaches,althoughnotoftenusedinthepast,canprovideinsighttodecisionmakersaboutthetypeandlevelofanalysesrequiredtoanswertheirquestions.


Figure 5. Tiered Approach for Risk Assessment. The applicability of a probabilistic approach depends on the needs of decision makers and potential stakeholders. Assessments that are high in complexity and regulatory significance benefit from the application of probabilistic techniques. Source: Adapted from USEPA 2004a and WHO 2008.

Someoftheconsiderationsindeterminingthetypeandlevelofanalysesthatshouldbeperformedinclude:

1. Whetherornottheriskassessmentinitscurrentstateissufficienttodiscriminateamongoptions(i.e.,aclearpathtoexitingtheprocessisavailable).

2. Iftheassessmentisdeterminedtobeinsufficient,whetherornotprogressiontoahigherlevelofcomplexitywouldprovideasufficientbenefittowarranttheadditionaleffortofperformingaPRA.

3. Whethertherearesignificantdifferencesincostsand/orbenefitsbetweenthealternatives.

SomeoftheEPAcasestudiesprovidedintheAppendixofthissummary’scompaniondocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies (USEPA2014a)offerexamplesofhowPRAtoolscanbeusedinaniterativeortieredapproachina


riskassessment.BothCaseStudies1and9dealwithchildren’sexposuretoarsenicinchromatedcopperarsenate(CCA)pressure‐treatedwood.CaseStudy1providesanexampleoftheapplicationofpointestimatesensitivityanalysis(Tier1)toidentifyimportantvariablesforpopulationexposurevariabilityforCaseStudy9.InCaseStudy9,atwo‐dimensionalMonteCarloanalysis(2‐DMCA)wasconductedtoaddressthevariabilityanduncertaintyintheexposureassessment.Twostagesofrandomsamplingwereconductedtodistinguishbetweenvariabilityanduncertaintyinexposureandtoxicityvariables.

If one is going to use PRA, what factors should be considered? IfonedecidesthattheuseofPRAwouldprovidevaluableinformationinsupportofadecision,someotherthingstoconsiderinmovingforwardincludethefollowing:

ResourcesneededtodevelopthePRAandreviewthedocument,including:

ExpertiseofEPAstafftodevelopaPRAorreviewaPRAsubmittedbyacontractorormemberoftheregulatedcommunity.

Timeneededforthedevelopmentandreviewoftheanalyses.

Funding,eitherintramuralorextramural,thatmaybenecessaryfordevelopmentandreviewofthePRA.

Dataavailabilityandformat(e.g.,electronicorpapercopy)fordevelopingdistributionstoincludeinthePRA.

Peerreview,includingeitherinternaland/orexternalreview,whichhastimeandcostimplications.

Communicationofresultstothescientificcommunity,Agencyexecutives,stakeholdersandthepublic.PRAmayoftenincreasethecomplexityoftheriskinformationtobecommunicatedtovariousaudiences.

What are the resources needed to conduct a PRA? PRAgenerallyrequiresmoretime,effortandresourcesthandeterministicassessmentswithstandardscience‐policyassumptions.Theresourceswillvarydependingonthetoolorapproachthatisselected.ThereisacontinuumofPRAmethodologiestochoosefrom,rangingfromsimplesensitivityanalysestocomplexapproachessuchas2‐DMonteCarloAnalysis.Insomecases,simplesensitivityanalyses,whichmayrequirelimitedtimeandriskassessorresources,canbeconducted“in‐house.”MoresophisticatedanalysesmayrequirespecificexpertiseortheuseofspecifictoolsormodelsnotavailableatEPA;maintainingtransparencywillbeimportantintheseefforts.Properapplicationofprobabilisticmethodsrequiresnotonlysoftwareanddata,butalsoguidanceandtrainingforanalystsusingthetoolsaswellasformanagersanddecisionmakerstaskedwithinterpretingandcommunicatingtheresults.Thedevelopmentofstandardizedapproachesand/ormethodscanleadtotheroutineincorporationofPRAinAgencyapproachesandgreatlyreducecostsinfutureapplications.GreaterdevelopmentandunderstandingofthedecisioncontexttowhichpotentialPRAmightcontributewillallowdecisionmakerstomakeamoreinformeddecisionaboutthebenefitsversusthecostsofconductingaPRArelativetotheirdecision‐makinggoals.

Does PRA require more data than conventional approaches? Ingeneral,PRArequiresmoredatathanconventionalapproachesbecausedistributionsofvaluesratherthansinglevaluesareused.Minimumdatarequirementscurrentlyareatopicofdebateinthebroaderriskassessmentcommunity.Minimumdataneedsvarydependingontheanalytical


approachused;empirical‐based(observationalorfrequentist)methodshavesignificantdatarequirementscomparedwithsubjectivemethods.Someofthedatathatwouldbeappliedinafrequentistapproachmaybeavailablealreadyaspartoftheunderlyingdatasetusedinstandarddeterministicanalyses.Asaresult,PRAcanbeappliedinmostcasesprovidedthatthemethodsusedareappropriatefortheavailablebodyofevidenceanddata.

Communication of PRA results to the manager and community: does the presentation of results matter? ThelackoffamiliaritywithPRApresentsachallengeineffectivelypresentingresultstodecisionmakers,stakeholdersandthepublic.ManyviewPRAasahighlytechnicaldisciplinethatusessophisticatedmathematicsandrequiresextensivetrainingtoapplyandunderstand.Singlepointestimatesareeasiertograspformostpeople,basedinpartonfamiliaritywiththisapproachoverthehistoryofEPA.Althoughsomepeopleinitiallyhavedifficultyinterpretingprobabilitydistributionsofvalues,everyonehasacommonbaselineexperiencewithprobability,uncertaintyandvariabilityfromeverydaylife(e.g.,weatherforecasting,oddsofwinningalottery),andthisexperiencecouldbeusedtoframethediscussionofresults.Itisnotnecessarytounderstandtheunderlyingmathematicsoreventoincluderesultsasfulldistributions.Resultscanbedistilleddowntothecriticalessenceordecision‐meaningfulinputofinterest.

Thedifferentaudiencesandtheirrangeofknowledgeandexpertisemustbeconsideredindevelopingmaterialsforeffectivecommunication.WhenadecisionismadetoconductaPRA,itishelpfultoconsiderearlyorientationofthecommunity,managersandothersinthebasicprinciplesbeforethefinaldecisionispresented.Alternatively,itmaybehelpfultopresenttheresultsofthePRAalongwiththepointestimatetoprovideacontextualframefortheresults.

How can I get more information on PRA? ThisdocumentprovidesageneraloverviewandbasicconceptstoestablishsomefamiliarityandafoundationforfurthereducationonPRA.ThecompanionpublicationtothisFAQdocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a)providesadetaileddiscussionofPRAandEPA’sexperiencewithit.TherearenumerousadditionalresourcestoprovidemoredetailaboutPRA,includingEPA’sRiskAssessmentwebsite(www.epa.gov/risk),whichistheAgency’ssourceforlinkstoriskassessmentmethodsandpolicies.

TheRAFcontinuestohaveongoingactivitiesaddressingtheuncertaintyandvariabilityneedsoftheAgency;detailscanbefoundontheRAF’swebsite(www.epa.gov/raf).


GLOSSARY Analysis.Examinationofanythingcomplextounderstanditsnatureortodetermineitsessentialfeatures(WHO2004).

Assessment.InterpretationandevaluationofEnvironmentalMonitoringandAssessmentProgram(EMAP)resultsforthepurposeofansweringpolicy‐relevantquestionsaboutecologicalresources,including(1)determinationofthefractionofthepopulationthatmeetsasociallydefinedvalue,and(2)associationamongindicatorsofecologicalconditionsandstressors(USEPA2010a).

Assessmentendpoint.Anexplicitexpressionoftheenvironmentalvaluethatistobeprotected,operationallydefinedbyanecologicalentityanditsattributes.Forexample,salmonarevaluedecologicalentities;reproductionandageclassstructurearesomeoftheirimportantattributes.Together,salmon“reproductionandageclassstructure”formanassessmentendpoint(USEPA1998).

CumulativeDistributionFunction.Inprobabilitytheoryandstatistics,thecumulativedistributionfunction(CDF)describestheprobabilitythatareal‐valuedrandomvariableXwithagivenprobabilitydistributionwillbefoundtohaveavaluelessthanorequaltox.

Deterministic.Amethodologyrelyingonpoint(i.e.,exact)valuesasinputstoestimaterisk;thisobviatesquantitativeestimatesofuncertaintyandvariability.Resultsalsoarepresentedaspointvalues.Uncertaintyandvariabilitymaybediscussedqualitativelyorsemi‐quantitativelybymultipledeterministicriskestimates(USEPA2006b).

Deterministicriskassessment(DRA).Riskevaluationinvolvingthecalculationandexpressionofriskasasinglenumericalvalueor“singlepoint”estimateofrisk,withuncertaintyandvariabilitydiscussedqualitatively(USEPA2012).

Ecologicalriskassessment.Theprocessthatevaluatesthelikelihoodthatadverseecologicaleffectsmayoccurorareoccurringasaresultofexposuretooneormorestressors(USEPA1998).

Ecosystem.Thebioticcommunityandabioticenvironmentwithinaspecifiedlocationinspaceandtime(USEPA1998).

Environment.Thesumofallexternalconditionsaffectingthelife,developmentandsurvivalofanorganism(USEPA2010a).

Expertelicitation.Asystematicprocessofformalizingandquantifying,typicallyinprobabilisticterms,expertjudgmentsaboutuncertainquantities(USEPA2011a).

Frequentist(orfrequency)probability.Aviewofprobabilitythatconcernsitselfwiththefrequencywith which an event occurs givenalongsequenceofidenticalandindependenttrials(USEPA1997b).

Hazardidentification.Theriskassessmentprocessofdeterminingwhetherexposuretoastressorcancauseanincreaseintheincidenceorseverityofaparticularadverseeffect,andwhetheranadverseeffectislikelytooccur(USEPA2012).

Humanhealthriskassessment(HHRA).1.Theprocesstoestimatethenatureandprobabilityofadversehealtheffectsinhumanswhomaybeexposedtochemicalsincontaminatedenvironmentalmedia,noworinthefuture(USEPA2010b).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment).Theproductoftheriskassessmentisastatementregardingtheprobability


thatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization)(USEPA2006a).

Inputs.Quantitiesthatareappliedtoamodel(WHO2008).

Model.Amathematicalrepresentationofanaturalsystemintendedtomimicthebehavioroftherealsystem,allowingdescriptionofempiricaldata,andpredictionsaboutuntestedstatesofthesystem(USEPA2006b).

Modeling.Developmentofamathematicalorphysicalrepresentationofasystemortheorythataccountsforallorsomeofitsknownproperties.Modelsoftenareusedtotesttheeffectofchangesofcomponentsontheoverallperformanceofthesystem(USEPA2010a).

MonteCarloanalysis(MCA)orsimulation(MCS).Arepeatedrandomsamplingfromthedistributionofvaluesforeachoftheparametersinagenericexposureorriskequationtoderiveanestimateofthedistributionofexposuresorrisksinthepopulation(USEPA2006b).

One‐dimensionalMonteCarloanalysis(1‐DMCA).Anumericalmethodofsimulatingadistributionforanendpointofconcernasafunctionofprobabilitydistributionsthatcharacterizevariabilityoruncertainty.Distributionsusedtocharacterizevariabilityaredistinguishedfromdistributionsusedtocharacterizeuncertainty(WHO2008).

Parameter.Aquantityusedtocalibrateorspecifyamodel,suchas“parameters”ofaprobabilitymodel(e.g.,meanandstandarddeviationforanormaldistribution).Parametervaluesoftenareselectedbyfittingamodeltoacalibrationdataset(WHO2008).

Probability.Afrequentistapproachconsidersthefrequencywithwhichsamplesareobtainedwithinaspecifiedrangeorforaspecifiedcategory(e.g.,theprobabilitythatanaverageindividualwithaparticularmeandosewilldevelopanillness)(WHO2008).

ProbabilityDensityFunction.Inprobabilitytheory,aprobabilitydensityfunction(pdf)ofacontinuousrandomvariableisafunction,oftendenotedasf(x),thatdescribestherelativelikelihoodforthisrandomvariabletotakeonagivenvalue.

Probabilisticriskanalysis(PRA).Calculationandexpressionofhealthrisksusingmultipleriskdescriptorstoprovidethelikelihoodofvariousrisklevels.Probabilisticriskresultsapproximateafullrangeofpossibleoutcomesandthelikelihoodofeach,whichoftenispresentedasafrequencydistributiongraph,thusallowinguncertaintyorvariabilitytobeexpressedquantitatively(USEPA2012).

Problemformulation.Theinitialstageofariskassessmentwherethepurposeoftheassessmentisarticulated,exposureandriskscenariosareconsidered,aconceptualmodelisdeveloped,andaplanforanalyzingandcharacterizingriskisdetermined(USEPA2004a).

Referenceconcentration(RfC).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofacontinuousinhalationexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNo‐Observed‐Adverse‐EffectLevel(NOAEL),Lowest‐Observed‐Adverse‐EffectLevel(LOAEL),orbenchmarkconcentration,withuncertaintyfactorsgenerallyappliedtoreflectlimitationsofthedataused.ItgenerallyisusedinEPA’snoncancerhealthassessments(USEPA2007a).

Referencedose(RfD).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofadailyoralexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNOAEL,LOAELorbenchmarkdose,withuncertaintyfactorsgenerallyappliedtoreflect


limitationsofthedataused.IttypicallyisusedinEPA’snoncancerhealthassessments(USEPA2011c).

Risk.1.Riskincludesconsiderationofexposuretothepossibilityofanadverseoutcome,thefrequencywithwhichoneormoretypesofadverseoutcomesmayoccur,andtheseverityorconsequencesoftheadverseoutcomesifsuchoccur.2.Thepotentialforrealizationofunwanted,adverseconsequencestohumanlife,health,propertyortheenvironment.3.Theprobabilityofadverseeffectsresultingfromexposuretoanenvironmentalagentormixtureofagents.4.Thecombinedanswersto:Whatcangowrong?Howlikelyisit?Whataretheconsequences?(USEPA2011c).

Riskanalysis.Aprocessforidentifying,characterizing,controllingandcommunicatingrisksinsituationswhereanorganism,system,subpopulationorpopulationcouldbeexposedtoahazard.Riskanalysisisaprocessthatincludesriskassessment,riskmanagementandriskcommunication(WHO2008).

Riskassessment.1.Aprocessintendedtocalculateorestimatetherisktoagiventargetorganism,system,subpopulationorpopulation,includingtheidentificationofattendantuncertaintiesfollowingexposuretoaparticularagent,takingintoaccounttheinherentcharacteristicsoftheagentofconcern,aswellasthecharacteristicsofthespecifictargetsystem(WHO2008).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment)(NRC1983).Theproductoftheriskassessmentisastatementregardingtheprobabilitythatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization;USEPA2000).3.Qualitativeandquantitativeevaluationoftheriskposedtohumanhealthortheenvironmentbytheactualorpotentialpresenceoruseofspecificpollutants(USEPA2012).

Risk‐baseddecisionmaking.Aprocessthroughwhichdecisionsaremadeaccordingtotheriskeachposedtohumanhealthandtheenvironment(USEPA2012).

Riskmanagement.Adecision‐makingprocessthattakesintoaccountenvironmentallaws,regulations,andpolitical,social,economic,engineeringandscientificinformation,includingariskassessment,toweighpolicyalternativesassociatedwithahazard(USEPA2011c).

Scenario.Asetoffacts,assumptionsandinferencesabouthowexposuretakesplacethataidstheexposureassessorinevaluating,estimatingorquantifyingexposures(USEPA1992).Scenariosmightincludeidentificationofpollutants,pathways,exposureroutesandmodesofaction,amongothers.

Sensitivityanalysis.Theprocessofchangingonevariablewhileleavingtheothersconstanttodetermineitseffectontheoutput.Thisprocedurefixeseachuncertainquantityatitscrediblelowerandupperbounds(holdingallothersattheirnominalvalues,suchasmedians)andcomputestheresultsofeachcombinationofvalues.Theresultshelptoidentifythevariablesthathavethegreatesteffectonexposureestimatesandhelpfocusfurtherinformation‐gatheringefforts(USEPA2011b).

Tieredapproach.Referstovarioushierarchicaltiers(levels)ofcomplexityandrefinementfordifferenttypesofmodelingapproachesthatcanbeusedinriskassessment.Adeterministicriskassessmentwithconservativeassumptionsisanexampleofalowerleveltypeofanalysis(Tier0)thatcanbeusedtodeterminewhetherexposuresandrisksarebelowlevelsofconcern.Examplesofprogressivelyhigherlevelsincludetheuseofdeterministicriskassessmentcoupledwithsensitivityanalysis(Tier1),theuseofprobabilistictechniquestocharacterizeeithervariabilityoruncertaintyonly(Tier2),andtheuseoftwo‐dimensionalprobabilistictechniquestodistinguish


betweenbutsimultaneouslycharacterizebothvariabilityanduncertainty(Tier3)(USEPA2004aandWHO2008).

Two‐dimensionalMonteCarloanalysis(2‐DMCA).Anadvancednumericalmodelingtechniquethatusestwostagesofrandomsampling,alsocallednestedloops,todistinguishbetweenvariabilityanduncertaintyinexposureandtoxicityvariables.Thefirststage,oftencalledtheinnerloop,involvesacomplete1‐DMCAsimulationofvariabilityinrisk.Inthesecondstage,oftencalledtheouterloop,parametersoftheprobabilitydistributionsareredefinedtoreflectuncertainty.Theseloopsarerepeatedmanytimesresultinginmultipleriskdistributions,fromwhichconfidenceintervalsarecalculatedtorepresentuncertaintyinthepopulationdistributionofrisk.(WHO2008).

Uncertainty.Uncertaintyoccursbecauseofalackofknowledge.Itisnotthesameasvariability.Forexample,ariskassessormaybeverycertainthatdifferentpeopledrinkdifferentamountsofwaterbutmaybeuncertainabouthowmuchvariabilitythereisinwaterintakeswithinthepopulation.Uncertaintyoftencanbereducedbycollectingmoreandbetterdata,whereasvariabilityisaninherentpropertyofthepopulationbeingevaluated.Variabilitycanbebettercharacterizedwithmoredatabutitcannotbereducedoreliminated.Effortstoclearlydistinguishbetweenvariabilityanduncertaintyareimportantforbothriskassessmentandriskcharacterization,althoughtheybothmaybeincorporatedintoanassessment(USEPA2011c).

Uncertaintyanalysis.Adetailedexaminationofthesystematicandrandomerrorsofameasurementorestimate;ananalyticalprocesstoprovideinformationregardinguncertainty(USEPA2006b).

Valueofinformation.Ananalysisthatinvolvesestimatingthevaluethatnewinformationcanhavetoariskmanagerbeforetheinformationisactuallyobtained.Itisameasureoftheimportanceofuncertaintyintermsoftheexpectedimprovementinariskmanagementdecisionthatmightcomefrombetterinformation(USEPA2001).

Variability.Referstotrueheterogeneityordiversity,asexemplifiedinnaturalvariation.Forexample,amongapopulationthatdrinkswaterfromthesamesourceandwiththesamecontaminantconcentration,therisksfromconsumingthewatermayvary.Thismayresultfromdifferencesinexposure(e.g.,differentpeopledrinkingdifferentamountsofwaterandhavingdifferentbodyweights,exposurefrequenciesandexposuredurations),aswellasdifferencesinresponse(e.g.,geneticdifferencesinresistancetoachemicaldose).Thoseinherentdifferencesarereferredtoasvariability.Differencesamongindividualsinapopulationarereferredtoasinter‐individualvariability,anddifferencesforoneindividualovertimearereferredtoasintra‐individualvariability(USEPA2011c).


REFERENCES4 Howard,R.A.(1966).DecisionAnalysis:AppliedDecisionTheory,ProceedingsoftheFourthInternationalConferenceonOperationalResearch,pp.55‐71.Wiley‐Interscience.

NRC(NationalResearchCouncil).1983.RiskAssessmentintheFederalGovernment:ManagingtheProcess.Washington,D.C.:TheNationalAcademiesPress.

NRC.2006.HealthRisksfromDioxinandRelatedCompounds:EvaluationoftheEPAReassessment.Washington,D.C.:TheNationalAcademiesPress.

NRC.2009.ScienceandDecisions:AdvancingRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.

OMB(OfficeofManagementandBudget).2003.CircularA‐4.Washington,D.C.:OfficeofManagementandBudget.http://www.whitehouse.gov/omb/circulars_a004_a‐4/.

OMB.2007.MemorandumfortheHeadsofExecutiveDepartmentsandAgenciesonUpdatedPrinciplesforRiskAnalysis.September19,2007.M‐07‐24.http://www.whitehouse.gov/sites/default/files/omb/memoranda/fy2007/m07‐24.pdf.

USEPA(U.S.EnvironmentalProtectionAgency).1992.FrameworkforEcologicalRiskAssessment.EPA/630/R‐92/001.Washington,D.C.:USEPA.http://www.epa.gov/raf/publications/framework‐eco‐risk‐assessment.htm.

USEPA.1997a.ExposureFactorsHandbook.EPA/600/P‐95/002Fa‐c.Washington,D.C.:NationalCenterforEnvironmentalAssessment,OfficeofResearchandDevelopment,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12464.

USEPA.1997b.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/ncea/pdfs/montcarl.pdf.

USEPA.1997c.PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency.MemorandumfromFredHansen,DeputyAdministrator.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/probpol.pdf.

USEPA.1998.GuidelinesforEcologicalRiskAssessment.EPA/630/R‐95/002F.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF.

USEPA.2000.SciencePolicyCouncilHandbook:RiskCharacterization.EPA/100/B‐00/002.Washington,D.C.:OfficeofSciencePolicy,OfficeofResearchandDevelopment,USEPA.http://www.epa.gov/spc/pdfs/rchandbk.pdf.

USEPA.2001.RiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment.EPA/540/R‐02/002.Washington,D.C.:OfficeofEmergencyandRemedialResponse,USEPA.http://www.epa.gov/oswer/riskassessment/rags3adt/.

USEPA.2003.FrameworkforCumulativeRiskAssessment.EPA/630/P‐02/001F.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.www.epa.gov/raf/publications/pdfs/frmwrk_cum_risk_assmnt.pdf

4AllInternetlinkswerecheckedatthetimeofpublicationofthisdocument.


USEPA.2004a.AirToxicsRiskAssessmentReferenceLibrary.Volume1:TechnicalResourceManual.EPA/453/K‐04/001A.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/fera/data/risk/vol_1/title_page_volume_1.pdf.

USEPA.2004b.AnExaminationofEPARiskAssessmentPrinciplesandPractices.EPA/100/B‐04/001.Washington,D.C.:OfficeoftheScienceAdvisor,OfficeofResearchandDevelopment,USEPA.http://www.epa.gov/osa/pdfs/ratf‐final.pdf.

USEPA.2006a.AFrameworkforAssessingHealthRisksofEnvironmentalExposurestoChildren.EPA/600/R‐05/093F.Washington,D.C.:USEPA.http://ofmpub.epa.gov/eims/eimscomm.getfile?p_download_id=459047.

USEPA.2006b.AirToxicsRiskAssessmentReferenceLibrary,Volume3:Community‐ScaleAssessment.EPA/452/K‐06/001C.Washington,D.C.:USEPA.http://www.epa.gov/ttn/fera/risk_atra_vol3.html.

USEPA.2007a.Concepts,MethodsandDataSourcesforCumulativeHealthRiskAssessmentofMultipleChemicals,ExposuresandEffects:AResourceDocument.EPA/600/R‐06/013F.Washington,D.C.:USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=190187.

USEPA.2007b.LetterFromM.G.MorganandR.T.Parkin,ScienceAdvisoryBoard,toS.Johnson,USEPA.February28,2007.EPA/SAB‐07/003.Washington,D.C.:ScienceAdvisoryBoard,OfficeoftheAdministrator,USEPA.http://yosemite.epa.gov/sab/sabproduct.nsf/55E1B2C78C6085EB8525729C00573A3E/$File/sab‐07‐003.pdf.

USEPA.2010a.EnvironmentalMonitoringandAssessmentProgramMasterGlossary.LastmodifiedNovember8.http://www.epa.gov/emap2/html/pubs/docs/resdocs/mglossary.html#ee.

USEPA.2010b.HumanHealthRiskAssessmentOnondagaLake,LakeBottomSubsite:SedimentConsolidationArea,Camillus,NY.Washington,D.C.:USEPA.http://www.epa.gov/region2/superfund/npl/onondagalake/healthriskassess.pdf.

USEPA.2011a.ExpertElicitationTaskForceWhitePaper.Washington,D.C.:USEPA.http://www.epa.gov/stpc/pdfs/ee‐white‐paper‐final.pdf.

USEPA.2011b.ExposureFactorsHandbook:2011Edition.EPA/600/R‐09/052F.Washington,D.C.:NationalCenterforEnvironmentalAssessment,OfficeofResearchandDevelopment,USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=236252.

USEPA.2011c.IntegratedRiskInformationSystemGlossary.LastmodifiedAugust21.http://ofmpub.epa.gov/sor_internet/registry/termreg/searchandretrieve/glossariesandkeywordlists/search.do?details=&glossaryName=IRIS%20Glossary.

USEPA.2012.WasteandCleanupRiskAssessmentGlossary.OfficeofSolidWasteandEmergencyResponse,USEPA.LastmodifiedDecember24.http://www.epa.gov/oswer/riskassessment/glossary.htm.

USEPA.2014a.RiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies.EPA/100/R‐09/001A.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.

USEPA.2014b.FrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking.EPA/100/R‐14/001.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/raf/frameworkhhra.htm.


WHO(WorldHealthOrganization).2004.InternationalProgrammeonChemicalSafetyRiskAssessmentTerminology.Geneva:WHO.http://www.who.int/ipcs/methods/harmonization/areas/ipcsterminologyparts1and2.pdf.

WHO.2008.UncertaintyandDataQualityinExposureAssessment.Geneva:TheInternationalProgrammeonChemicalSafety,WHO.http://www.who.int/ipcs/publications/methods/harmonization/exposure_assessment.pdf.

RAF PRA FAQ FINAL

Documents

Transcript of RAF PRA FAQ FINAL