Tips & Tricks for Developing or Improving Your Auditing Function
description
Transcript of Tips & Tricks for Developing or Improving Your Auditing Function
![Page 1: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/1.jpg)
Tips & Tricks for Developing or Improving Your Auditing Function
Leslie M. Howes, MPH, CIPDirector, Office of Human Research Administration
Harvard School of Public HealthHarvard Medical School & Dental School
![Page 2: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/2.jpg)
Today’s Outline
• Audit Basics• Key Elements• Audit Process • Preparation• Onsite Engagement• Report & Resolution• Follow-Up
• Metrics & Trends• Questions
2
![Page 3: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/3.jpg)
Auditing Basics
One working definition…A systematic and independent examination of
study- related activities and regulatory documentation to evaluate protocol adherence and compliance with regulation, institutional policy, ethical guidance, and applicable guidelines
Audit types• Not for cause (“Routine”)• For cause (“Directed”)
3
![Page 4: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/4.jpg)
Creating a Strong Audit Function: Key Elements
4
![Page 5: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/5.jpg)
Key Elements
• Understand your “client”• Define Scope and Authority• Obtain Institutional Support• Obtain Research Community Buy-in• Establish Goals/Focus • Develop SOPs and Templates
5
![Page 6: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/6.jpg)
Who’s Your Client
• Institutional culture• Principal Investigators’ involvement in their studies• Organizational/Departmental structure/hierarchy• Who’s likely to take advantage
• Research portfolio• FDA vs non-FDA regulated• Biomedical vs Socio-behavioral• Full Board vs Expedited review• Study populations• Domestic vs International
6
![Page 7: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/7.jpg)
Scope & Authority
• Auditing (for cause/not for cause)• Investigators only• IRB only• Both Investigators and IRB
• Auditing + education• Auditing + education + human research support• IRB submission assistance• External audit prep• Study consultation• On-call research coordinator, etc. 7
![Page 8: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/8.jpg)
Scope & Authority, cont.
• Reportable Information, Noncompliance, Suspensions & Terminations• Identify• Refer/Report to IRB (with recommendation)• Make official determination• Reprimand Investigators
• Conduct further investigation• Assist PI in reporting
8
Tip: If the IRB is responsible for making official determinations, develop an escalation plan –be transparent with your research community
![Page 9: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/9.jpg)
Institutional Support• Identify Key Constituents• Institutional Official, VP of Research, Department
Chairs/Chiefs, IRB Director/Chairs• Convince the leadership with data• Specific “business plan” for your audit function• If available, your own audit findings• Relevant FDA Warning Letters, OHRP Determination Letters• Borrow “data” from other/neighboring institution
• Less is more• Don’t ask for “too much” in terms of resources (in the
beginning) 9
![Page 10: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/10.jpg)
Research Community Buy-In
• Provide Incentives• Human research training credit• IRB Submission Assistance support• Not for cause audit “free pass” for defined time period
• Pilot different approaches• Voluntary • 360o Review
• Seek feedback from current “clients” and share with the research community• Share relevant “common findings” with investigators
10
![Page 11: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/11.jpg)
Establish Goals
• S.M.A.R.T • Specific• Measurable• Attainable• Realistic • Time-related
• Flexible/Creative• Anything is better than nothing
• Regular adjustment• Maximizing efficiency and effectiveness 11
Consider the following goals, what’s more realistic?A. Offer 3 departments the opportunity for an onsite review
vs B. Conduct 30 onsite reviews
![Page 12: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/12.jpg)
SOPs and Templates
• Develop Standard Operating Procedures • Easy to create and update• User-friendly• Develop templates • Audit report • Standard communications, e.g., letters/emails• Compile Common Observations
12
![Page 13: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/13.jpg)
An Overview of the Audit Process
13
![Page 14: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/14.jpg)
Preparation Phase
14
![Page 15: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/15.jpg)
Preparation
• Identify Protocol (not for cause)• Notify Investigator et al., as appropriate• When to communicate, if at all (unannounced visit)• What to communicate (What, Who, How)• How to communicate, e.g., email, phone, website
• Schedule Date/Time• All at once vs separate components, e.g., interview,
onsite review• How much time -for investigators, support staff• Secure private space 15
![Page 16: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/16.jpg)
Preparation, cont.
• Review IRB File, etc.• Protocol• Funding source• Study tools, e.g., survey• Consent documents (all versions)• External/Ancillary reviews, if any• Modifications to date• Reportable Information to date, e.g., AEs, Safety Reports, etc.• IRB determinations and concerns, if any• Directed/For-cause/Area of focus, e.g., participant payment
• Prepare Tools, e.g., notes vs audit worksheet/checklist16
![Page 17: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/17.jpg)
Onsite Engagement Phase
17
![Page 18: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/18.jpg)
Onsite Engagement
• Introduction• Set the tone• QA/I Program (Scope, Authority) • Overall process, purpose• Type of audit, e.g., for-cause vs not-for-cause• Questions
• Discussion vs Interview• PI to share challenges with study implementation, IRB
18
Not For Cause Audit Tip: Ease into the review; develop rapport with the PI/study team. This will set the stage for an educational, not adversarial approach.
![Page 19: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/19.jpg)
Onsite Engagement, cont.
• Orientation to organization system • Ensure access to all materials
• Review of Regulatory Documentation• Document observations• Make photocopies, when necessary
• Review of Participant files• File selection –how to determine sample size?• Be flexible –you may need to expand the sample
19
![Page 20: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/20.jpg)
How to Determine Sample Size
• Consider the goal of the QA/I Program• 100% Compliance vs “Sampling” of
Noncompliance• May depend on…• Specific institutional concerns• Audit trigger, e.g., Routine vs For-Cause audit;
General vs Directed/Focused audit
20
![Page 21: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/21.jpg)
Sample Size, cont.
• 100% Compliance – no sample plan needed• “Sampling” of Noncompliance – must identify a
representative sample• Random – selection process occurs at random, with
equally likelihood that one protocol would be identified vs another; lacks any pattern
• E.g., http://www.random.org/sequences/ • Online tool that generates a sequence
of numbers that can be matched to a protocol number
21
![Page 22: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/22.jpg)
Sample Size, cont. • Non-Random – You decide!• Informed by…
• Institutional Environment/Policy• Prior Noncompliance/Trends• Other Industry Standards, e.g., 10% of total
sample• Key stakeholders,
e.g., IO, IRB, or investigators
22
Tip: Whatever you chose, be transparent with the research community about your selection process
![Page 23: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/23.jpg)
Onsite Engagement, cont.
• Exit Interview• What is the PI/study team doing well• What does the PI/study team need to work on• Summary of noncompliance• Explore “root cause”
23
![Page 24: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/24.jpg)
Report & Resolution Phase
24
![Page 25: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/25.jpg)
Report & Resolution
• Compile Report • Provide in a timely manner, e.g., 5 business days• Recommended elements• Introduction (who, what, when)• Observations• Provide examples, frequencies• Corrective Actions• Best Practice Recommendations• Regulatory citations
• Summary, e.g., “…regulatory document well organized…/…could use improvement…”
25
TIP: Be concise, keep it factual, make it easy for investigators to
understand what you found and
how to correct it, use tables & bullets (avoid
narrative)
![Page 26: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/26.jpg)
Report & Resolution
• Create Report Template • Create common observations
• Improves efficiency and consistency• Facilitates tracking and trending• Consider including• General Observations• Best Practice Recommendations• Corrective Actions• Regulatory citations
26
TIP: Consider your institutional research portfolio, e.g., SBER,
IND/IDE, etc., and seek input from
stakeholders, e.g., IRB
![Page 27: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/27.jpg)
Report Distribution: A Case Study
27
![Page 28: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/28.jpg)
Report Distribution Example
28
Not for Cause/Routine
For Cause/Directed
Potential serious or continuing noncompliance
observed
No potential serious or continuing noncompliance
observed
• PI; key study staff• IRB• Institutional
Official • Others, e.g., VP of
Research, department chairs, Associate Dean of Students
PI; key study staff
TIP: Share aggregate findings from not for cause audits with key stakeholders
![Page 29: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/29.jpg)
Follow-Up Phase
29
![Page 30: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/30.jpg)
Follow-Up Options
• Obtain confirmation that correction actions have been implemented• QA/I staff conduct 2nd, 3rd, etc., audit• PI attestation, e.g., formal letter, eIRB system verification
• Additional audits of investigator/department• QA/I staff provide support to PI to implement corrective
actions/best practice recommendations• Education• General education offerings• Focused in-serve for PI and study team
• PI evaluation of audit 30
TIP: Consider your resources; conduct a risk-benefit analysis
![Page 31: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/31.jpg)
Metrics & Trends
31
![Page 32: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/32.jpg)
Audit Metrics & Trends
• Collect data• Not-for-cause/for-cause audits• Common observations• Post-audit evaluation data• Basic demographics/characteristics
• Analyze data • Track frequencies• Identify trends, e.g., isolated, pattern, systemic• Inform general education offerings
• Share findings with key stakeholders 32
![Page 33: Tips & Tricks for Developing or Improving Your Auditing Function](https://reader036.fdocuments.in/reader036/viewer/2022062501/568161ae550346895dd16e4c/html5/thumbnails/33.jpg)
Questions?
33