CFSAN’s Peer Review for Risk Assessments

Robert L. Buchanan, Sherri Robert L. Buchanan, Sherri Dennis, and Marianne Dennis, and Marianne

MiliotisMiliotis

BackgroundBackground

Our mission is to protect public Our mission is to protect public healthhealth

We must develop and use efficient We must develop and use efficient means to collect, organize, review means to collect, organize, review and share information used in and share information used in regulatory decisionsregulatory decisions

Risk assessment is an important Risk assessment is an important tool used to guide risk tool used to guide risk management decisionsmanagement decisions

CFSAN Risk Analysis Working CFSAN Risk Analysis Working Group: RecommendationsGroup: Recommendations

1. “Formalize a peer review process 1. “Formalize a peer review process that will encourage critical review that will encourage critical review and evaluation of CFSAN’s risk and evaluation of CFSAN’s risk assessments by government and assessments by government and non-government experts in a manner non-government experts in a manner that improves the science and that improves the science and acceptance of complex risk acceptance of complex risk assessments”assessments”

CFSAN’s Risk Assessment Framework Report, March 2002CFSAN’s Risk Assessment Framework Report, March 2002

CFSAN Risk Analysis Working CFSAN Risk Analysis Working Group: RecommendationsGroup: Recommendations

2. “Develop criteria to evaluate the 2. “Develop criteria to evaluate the quality of data used for risk quality of data used for risk assessment and specify what assessment and specify what information is needed to information is needed to scientifically evaluate the usefulness scientifically evaluate the usefulness of a study or data set used for risk of a study or data set used for risk assessment”assessment”3. “Develop guidelines to evaluate 3. “Develop guidelines to evaluate risk assessments and supporting risk assessments and supporting data developed by stakeholders and data developed by stakeholders and submitted to the Center”submitted to the Center”

Peer Review Peer Review Requirements for FDARequirements for FDA

FDA’s Data Quality Guidelines:FDA’s Data Quality Guidelines:

““Solicit advice and opinions of Solicit advice and opinions of advisory committees as well as advisory committees as well as

peer review from experts peer review from experts within and outside of the within and outside of the

agency”agency”

CFSAN’s Current Peer Review CFSAN’s Current Peer Review ProcessProcess

Specific activities include:Specific activities include:Announce intention in the Federal Register Announce intention in the Federal Register and request submission of dataand request submission of dataHold public meetings: key assumptions, Hold public meetings: key assumptions, data, and modeling approachesdata, and modeling approachesSeek input from stakeholders, scientific Seek input from stakeholders, scientific experts and advisory committeesexperts and advisory committeesConduct extensive review of report/ model Conduct extensive review of report/ model by government employeesby government employeesIssue draft document and model for public Issue draft document and model for public commentcommentPublish revised risk assessmentPublish revised risk assessment

CFSAN’s Goal for Peer CFSAN’s Goal for Peer ReviewReview

Formalize and implement a Formalize and implement a comprehensive peer review comprehensive peer review process that:process that:

Encourages critical review and Encourages critical review and evaluation of the risk assessment evaluation of the risk assessment in a manner that ultimately leads in a manner that ultimately leads to improvements in both the to improvements in both the science and acceptability of science and acceptability of complex risk assessment and complex risk assessment and modelsmodels

CFSAN’s Peer Review CFSAN’s Peer Review ProcessProcess

For Food Safety Risk For Food Safety Risk Assessments includes:Assessments includes:

Periodic solicitation of input from Periodic solicitation of input from scientific experts and our scientific experts and our stakeholders throughout the stakeholders throughout the process of conducting the risk process of conducting the risk assessmentassessment

Terminal reviews only are too Terminal reviews only are too latelate

What Is NeededWhat Is Needed

A description of the process A description of the process (activities) by which peer review is (activities) by which peer review is to be conductedto be conducted

Specific elements that a reviewer Specific elements that a reviewer should consider in evaluating food should consider in evaluating food safety risk assessmentsafety risk assessment

Evaluation of benefits/ limitations Evaluation of benefits/ limitations of the peer review elementsof the peer review elements

Sufficient numbers of reviewersSufficient numbers of reviewers

Peer Review ElementsPeer Review ElementsElements of the Elements of the review should review should include a means include a means to evaluate:to evaluate:

Data (used, not Data (used, not used, used, weighting)weighting)

AssumptionsAssumptions

Modeling Modeling techniquestechniques

Peer Review ElementsPeer Review ElementsExamples:Examples:

CompletenessCompletenessAccuracyAccuracyTransparencyTransparency

Reviewers should be given a complete Reviewers should be given a complete description of the goal/scope of the risk description of the goal/scope of the risk assessment including the risk assessment including the risk management issue and the risk management issue and the risk assessment question. Ideally, the peer assessment question. Ideally, the peer reviewer should be asked to respond to reviewer should be asked to respond to specific questions and provide an overall specific questions and provide an overall evaluation of the workevaluation of the work

Characteristics of Peer Characteristics of Peer ReviewersReviewers

Independent/ objectiveIndependent/ objectiveCan maintain confidentialityCan maintain confidentialitySubject-matter expertSubject-matter expertUnderstanding of science, research, Understanding of science, research, and policy issues related to food and policy issues related to food safety problemssafety problemsHas the time to review the Has the time to review the document, model or specific document, model or specific portions within the timeframe portions within the timeframe neededneeded

Picking the “Right” Picking the “Right” PeoplePeople

Weigh the timeframes requiredWeigh the timeframes required

Avoid only talking to the same Avoid only talking to the same old peopleold people

Go out of your way not to only Go out of your way not to only pick your “friends”pick your “friends”

Confidentiality of the Confidentiality of the informationinformation

Picking the “Right” Picking the “Right” PeoplePeople

CFSAN believes CFSAN believes strongly in strongly in seeking public seeking public commentcomment

True test of True test of transparencytransparency

Responding to Peer Responding to Peer ReviewsReviews

Need to establish Need to establish how to respond how to respond to peer reviews to peer reviews ahead of timeahead of time

Criteria for Criteria for conflicting conflicting recommendationsrecommendations

““Publish” reviewsPublish” reviews