Failure Modes and Effects Analysis (FMEA) for Partial ...€¦ · Failure Modes and Effects...

Failure Modes and Effects Analysis (FMEA) for Accelerated Partial Breast Irradiation via High Dose‐Rate Intracavitary Brachytherapy

Jeffrey F. Williamson1, Bruce R. Thomadsen2 , Geoffrey S. Ibbott 3, and Sasa Mutic 4

1Virginia Commonwealth University, 2University of Wisconsin; 3MD Anderson Cancer Center; 4Washington

University

Disclosures and Acknowledgments

• Disclosures:– JFW has research/service contracts with Varian Medical Systems, NIH, and AAPM

– JFW has never led formulation of FMEA in real‐life clinical application

• Acknowledgments– Chapter coauthors: Ibbott, Thomadsen, and Mutic– Clinical input and advice from Douglas Arthur, M.D.– Statistics expertise from Nitai Mukhopadhyay, Ph.D.– Slides from Saiful Huq, Ph.D.

Learning Objectives• Illustrate the application of TG‐100 Risk analysis techniques to balloon‐applicator HDR breast brachytherapy

• Review failure modes rated to be highest risk• Review QA/QC strategies for mitigating high risk failures

4

Example: Risk-Informed QM Formulation for Brachytherapy

• Scenario: Accelerated partial breast irradiation using multi-catheter balloon HDR brachytherapy applicator

– CT-based evaluation and planning– Multicatheter balloon applicator, e.g., Contura– Automated plan transfer but not full EMR charting

• History of this project– 2005: TG-100 assigns 4 coauthors to devise a

brachytherapy risk analysis case study– 2008: dropped from TG-100 report– 2010 - 2013: JFW revives project. FMs and scenarios

revised and extended with feedback from BRT.– 2013: Chapter authors ranked revised FMs.

5

Image-Guided Balloon Catheter PlacementAccelerated Partial Breast: MammoSite HDR BTx

• gg

Intraoperative Ultrasound

Visualize LumpectomyCavity: Select Approach

Assess conformality

Intraop/PostOp CT

Assess conformality

Assess SymmetryD. Arthur, VCU

Assess Skin Distance

6

Multi-Cath Balloon Applicator Mismatch errors

• ee

Asymmetric loading to spare skin

and chestwall

TG-100 Risk Analysis Steps• Steps

1. Define process by creating a process map2. Failure modes and effects analysis (FMEA): Identify

threats to success (failure modes) and rank according to risk

3. Fault-tree Analysis (FTA): Propagation of failures through system and placement of QM interventions

4. Develop QA or QC interventions to mitigate risk

7

TG-100 Risk Analysis Steps• Process Map: Step 1

– Delineate and then understand the steps in the process to be evaluated

– Visual illustration of the physical and temporal relationships between the different steps of a process

– Demonstrates the flow of these steps from process start to end

• prospective risk analysis for hypothetical clinical process modeled on VCU and UW-Madison processes

– Assumes NO QA or QC checks– Partial automation of EMR and data transfer

8

9

Breast Brachytherapy Process Map

TG-100 Risk AnalysisStep 2 FMEA

• Step 2a: For each process step, ask the following questions

– What could possibly go wrong ? (enumerate/ describe failure modes)

– How could that happen? (what are possible causes of FM?)

– What effect would such an undetected failure have? (Potential impact on quality)

• Step 2b: Assess risk of FM by estimating O, S, and P

• Present analysis: 96 Failure Modes 10

Process tree

Sub-process #1

Sub-process #17

Sub-process #19

Step #4Step #1 Step #j

Failure mode #2Failure mode #1 Failure mode #k

Causes of failure #6Causes of failure #1 Causes of failure #m

Effects of failure #4Effects of failure #1 Effects of failure #n

Step 2a: enumerate FMEA Failure Modes

12

Assess Risk Posed by Each FMStep 2b

• For each subprocess, enumerate the possible scenarios, i.e., Failure Modes (FM), that could lead an unsuccessful treatment: 96 FMs

– Identify causes and effect on process outcome• Assess risk to successful outcome posed by

each FM assuming no QA

– Assign O,S, and P a value from 1-10– 4 Observers: Ibbott, Mutic, Williamson, Thomadsen– Significant additions/modifications by JFW

• Reorder list in terms of descending RPN

Likelihood of Severity of Likelihood ErrorRisk occurrence consequences Detected

O S P Risk Proba

No

bility Number R S P

t

PN O

TG-100 FMEA Rating Scales

• ff

13

14

Combined

ItemNumber Major Processes Step Potential Failure Modes

Potential Causes of Failure JFW Comments and descriptive scenario

Potential Effects of Failure

AVG O AVG S AVG D Avg RPN

1 Imaging and diagnosis

RO reviews EMR prior to RO consult

Med Onc or Surgeon consultation mis interprets or mis represents primary clinical findings ( imaging studies, path reports, etc); incorrectly stages patient, and recommends BCT and APBI for patient that is not appropriate candidate

RO bases Tx recommendation on secondary MD report rather than reviewing primary clinical findings

and discovering the ups tream error

Upstream phys ician error potentially discoverable by Rad

Onc s ince primary c linical data is available We should recommend that the RO performs their dut ies

diligently .

wrong/very wrong dose distribut ion 5.00 8.25 5.50 269.3

2 Imaging and diagnosis

RO reviews EMR prior to RO consult

path or biomarker reports is incorrect due to mis labeling of surgical spec imen or biomarker report . Hence pat ient is unders taged and inappropriately offered BCS by Med Onc and Surgeon

RO recommendation for APBI is fully consistent with prior EMR

An error not eas ily discoverable by Rad Onc Based on the worse case.

Very wrong dose 4.25 8.75 8.25 309.5

3Patient database

information

Entry of patient data in RO EMR or writ ten

chart

Incorrect patient ID data Documentation error

Wrong patient ID leading misfiling of demographic and c linical data from hospital DB; identification of wrong patient

Very wrong dose 3.00 8.75 2.75 70.0

4 Patient Database Information

Entry of patient data in RO EMR or writ ten

chart

Correct patient ID data but clinical

findings/ images from wrong pat ient loaded

into RO EMR

Omission in entry, incomplete pat ient history

Incorrect clinical findings leads to faulty decision to t reat or downstream peer-review correc tion

Very wrong dose 5.00 7.75 3.75 154.5

5 Consultat ion and decision to t reat

Decision of treatment technique and protocol

Clinically inappropriate patient selected for

APBI

misinterpretating of clinical findings incomplete H&P

Even though upstream clinical data are correct, Error by RO assessing indicat ions and contraindications to APBI., e.g. , SLN+ with Surg untreated axilla. RO mis represents or neglects key finding and offers inappropriate treatment plan to patient

Very wrong dose 4.25 7.75 7.75 252.8

6Consultation and

dec ision to treat or imaging/diagnos is

Decision of treatment technique and protocol or imaging/diagnosis

patient with radiographically too

large or closed seroma cavity selected

RO error in interpreting imaging studies; inappropriate imaging used; or poor imaging quality

JFW: New failure mode

Very wrong dose if not detec ted; more likely

major inconvenience or infection from

unnecessary invas ive procedure

4.75 6.25 4.75 140.3

7 Consultat ion and decision to t reat

Decision of treatment technique and protocol

Clinically inappropriate patient selected for

APBI

Upstream error in document ing clinical findings . Various scenarios with differing levels of discoverability

Because of upstream Imaging and Diagnosis errors, RO offers inappropriate treatment plan to the patient

Very wrong dose 4.00 7.50 5.50 180.0

First 7 Failure Modes

15

8 Highest Risk FMs

• Red entries: New JFW FMs or modified RPN

16• Red entries: New JFW FMs or modified RPN

e r M ajor Processes Step Pote ntial Fa ilure Modes

Potential Causes of Failure JF W Comments and descriptive scenario

Potential Effects o f Failu re

AVG O AVG S AVG D Avg RPN

Init ial treatment Run treatment Incorrec t balloon rott ion

Inattent ion,.miss ing required check , poor documentation

Failure to check that applicator rotat ionmatches plan, so that wrong dwell posit ion sequence is opposing sk in bridge.

Wrong dose distribut ion

5.25 7.25 7.25 302.0

Plan approval MD reviews planPlan fails to sat isfy

c linical intentions , bad plan approved

Inadequate or incomplete review, failure to reoptimize inadequate plan, miscom munication, inat tent ion, lack of s tandardized procedures

Error is that MD approves a plan that violates cons traints same phys ician previously approved and subopt imal plan is loaded into EMR in prep for treatment

Subopt imal plan, poor treatment used

5.75 5.75 8.25 299.0

Treatment Planning Optim izat ion sett ings

Specification of optimization m ethod, dose-point locations,

prescribed dose protocol not followed

accurately

Inadequately trained personnel, inat tent ion, poor inter-disc iplinary comm unicat ion

Planners following different protocol for sett ing constraints and goals

Very wrong dose 5.25 8.00 6.75 293.0

Treatment planning Dwell pos ition cons truc t ion

Distal-mos t dwell location inaccurately

digit ized

Inadequately trained, inattent ion, poor inter-disciplinary comm unicat ion

E,g,, offset from tip to dwell 1 used as s tandard pract ice in c linic not known


4.50 7.25 7.75 287.5

Treatment planningCatheter

localizat ion/labeling: s ingle catheter

Catheter t rajectory inaccurately localized

W rong catheter s lice images, inadequately trained personnel, poor inter-disciplinary comm unicat ion, inattent ion

Many possible scenarios : operator ignorance of (a) which vis ible s truc ture denotes inner catheter tip, (b) offset from dis talmos t source center and catheter tip. Also s loppy contouring or import ing wrong plan template


4.25 8.00 7.75 285.5

Treatment planning Dwell pos ition cons truc t ion

Random error: Treatment length incorrect (wrong

transfer tube length, wrong sounding

information, wrong dwell spac ing)

Inadequately trained personnel, inat tent ion, poor inter-disc iplinary comm unicat ion, default distances, equipment failure

Mult iple error pathways: (1) Inaccurate sounding measurements from pos t-op imaging; (2) wrong spac ing and/or offsets; (3) afterloader spec ific calibrat ion correc tions , e.g. , Varian QuickConnect 14 mm correct ion, ignored; (4) arithmetic error

Very wrong/wrong dose dis tribution

5.00 8.50 6.25 284.0

Next 6 Highest Risk FMs

Fault Tree Analysis and Designing QM interventionsSteps 3 and 4

• Step 3: Create Fault Trees (optional)– Time consuming: Limit FTA to selected FMs– Visualize interactions between FMs possibly in different

process tree branches– JFW: helped me refine list of FMs and scenarios

• Step 4: Design QM intervention– Rank FMs according decreasing risk and severity– Mark high RPN/S FMs on fault and process trees– FTA guides optimal placement of intervention– Design intervention: balance cost, specificity,

sensitivity and benefit

17

Fault Tree AnalysisStep 3: TG100 risk analysis methodology

• FTA compliments process tree• Leftmost box is the failure (error)

• Each daughter node is a FM that could cause the error

• Works backwards in time (to the right) until root cause is reached

• Models propagation of error through system

18

• ‘OR’ means error occurs if any one of antecedent FMs occurs

• ‘AND’ means all antecedent FM’s must be realized for error to occur

• No QA/QC assumed

• Relevant FMs scattered across at least 4 PT branches

• Interactions

19

Program treatment unit

failure

Wrong data file imported

Inconsistency between treatment

program and default operating

parameters

Software failure

InattentionOr

Or

Dose in wrong location due to source position

Units length distance is incorrect

Unit step size inaccurate

Applicator in wrong location

Or

Single or multiple catheter failure

Catheter trajectory inaccurately

localized

Incorrect catheter number asssigned

Wrong catheter slice images

Inadequately trained personal

Poor labeling on photographs

Poor image quality

OrOr

Or

Dwell position construction

failure

Distal-most dwell location

inaccurately digitized

Treatment length incorrect (wrong

transfer tube length, wrong

sounding information, wrong

dwell spacing)


Poor images

Default distances used

Equipment failure

Or

Or

Or

48

Or

Systematic offset Commissioning

failure

Catheter localization

failure

Wrong catheter position Marked

Catheter indicators not inserted fully

Or

Or

Non-Positional Failure Modes

Post-procedure CT imaging

error

Sounding measurement error

Channel numbering error: marking or

recording

Channel and applicator numbers not matching when connecting transfer tubes to applicator

Wrong length transfer tube

OrConnect transfer

tubes to applicator failure


Poor quality/incomplete

images

Initial treatment failure

Or

Treatment planning

Error

Channel Mismatch

Or


Incorrect Catheter Polar rotation

26

25

26

25

23,24

51

49

50

46

52

52 53

73

75

72, 70, 71

86

76

Source Positioning ErrorFault Tree

• Error types– Channel mismatch– Incorrect Tx length– Incorrect step length

• Top level causes– Post procedure

imaging error– Tx Planning error– Error in treatment

setup or device programming

20

Dose in wrong location due to source position




Or

Single or multiplcatheter failure

Dwell positionconstruction

failure

O

Or



error


Treatment planning

Error

Channel Mismatch

Or

Source Positioning ErrorFault Tree

• Incorrect information /poor images Dwell position programming error

– Channel numbering or documentation– Catheter length measurement– Imaging performed with incorrect marker position

• QM interventions– QC: second therapist assists with measurements– QA: Independent check of localization data before patient leaves

imaging suite

21


failure



Or

Or



error



recording


images

And Physicist/dosimetrist

check images failure

Adequate QM program for planning and afterloader

systems

And

Assisting therapist misses errors

26

25

26

25

23,24

Post-procedure CT imaging Localization Errors

22

Post-Procedure ImagingLocalization steps

23

Localization FMs:Treatment Planning

• Catheter trajectory delineation error– Dwell 1 length error

» Systematic positional offset error– Dwell position digitization error

• channel mismatch error



localized





Poor image quality

Or Or

Or


failure






dwell spacing)


Poor images


Equipment failure

Or

Or

Or

48

Or


failure



Or

Or




images

Initial treatment Operator check

And

Physicist check plan failure


systems

Treatment planning

Error

26

25

26

23,24

51

49

50

46

52

52 53

Treatment length

Dwell positions

24

Example: Systematic Offset Error• Systematic source

positioning error: caused by invalid treatment length estimation protocol

• Varian “quick connect” indexer interface

– 14 mm offset compared to standard transfer tube connector with usual transfer tube-applicator combination length measurement

– No Software offset or hardware interlock initially provided

25

TG-56 Structured HDR Positional Accuracy Tests

dwell 1 (1500 mm) position

indexer reference L1 = 0.0

transfer tube

Tube Length: L (1218 mm)

t

Fully Inserted Radiographic Marker

LRM

transfer tube L1 (programmed length) = 1500

mm

Tube Length: L (1218 mm)

t

Radioactive Source Programmed to 1500 mm

PDR Head

Compare for

Coincidence

0"Odcalibrated dummy ribbon

d : Dummy Insertion Depth

iApplicator Orfice

26

Mitigating RTP Localization FMs

• Implement well-defined, rigidly followed procedures: – Adequate patient volume

• QC: use only one transfer tube length & use equi-length catheters

• QA: Final physics plan review focus on dwell position



localized





Poor image quality

Or Or

Or


failure






dwell spacing)


Poor images


Equipment failure

Or

Or

Or

48

Or


failure



Or

Or




images

Initial treatment Operator check

And



systems

Treatment planning

Error

26

25

26

23,24

51

49

50

46

52

52 53

• Adequate device QA: – Maintain image quality– Eliminate offsets and

incorrect default parameters

– Consistency of procedure with device function

• QA/QC in red• Adequate device

QA protocol• Written procedures

– Redundancy– Uniformity– Patient volume

• Physics Checks – Simulation– Tx plan– Setup/RAL

programming

27

Program treatment unit

failure

Wrong data file imported

Inconsistency between treatment

program and default operating

parameters

Software failure

InnatentionOr

Or




Or



localized





Poor image quality

Or Or

Or


failure






dwell spacing)


Poor images


Equipment failure

Or

Or

Or

48

Or


failure


failure



Or

Or



error



recording

Channel and applicator numbers not matching when connecting transfer tubes to applicator

Wrong length transfer tube

OrConnect transfer

tubes to applicator failure



images


Or

And

Operator check failure: imported parameters vs.

plan

And


And Physicist/dosimetrist

check images failure


systems

Treatment planning

Error

Failure: Physicist & MD

final setup check

And

Channel Mismatch

Or


And

Assisting therapist misses errors

Incorrect Catheter Polar rotation

26

25

26

25

23,24

51

49

50

46

52

52 53

73

75

72, 70, 71

86

76

Mitigating Source Positioning Errors

28

Patient Selection Error FTA

with QA

– Upstream medic error in EMR “Imaging-Dx”

» Med or surg Onc error: recommends APBI

» Upstream histopath /biomarker error

• Two major FMs– Rad Onc misses

or ignores API clinical or technical contraindication

Are Pathology Errors Common?• ASTRO guidelines for APBI

– T1 uni-focal/centric GTV < 2 cm with margins > 2 mm– Negative SLN biopsy or ALND– Favorable histology/biomarkers

• Pignol: outside specimens of 77 patients referred for APBI reviewed inhouse

• 18.6% of patients deemed suitable for APBI on initial path, reclassified as “unsuitable”.

29Pignol, Clin Oncol 2012

30

Mitigating Patient Selection Errors• QM interventions and process redesign

– QC checklist to guide MD consultation– QA: Peer- and technical-review of MD decisions– QC: National electronic medical record

• Second review of pathology and biomarkers

31

TG-100 Site-Specific Treatment Schemas• Clinical parameters

– Pre-Tx eligibility criteria (Site, stage, histology, etc.) for this care pathway

– Overall plan, including sequencing of chemo, surgery, external and brachytherapy courses

– Pre-Tx Simulation and clinical instructions, e.g., fiducial marker placement, MR or PET imaging, dental extractions

• Planning parameters– GTV, CTV, normal tissue delineation guidelines;

image selection and registration– Prescribed dose, fractionation, IMRT class

solution, planning goals, and constraints. – Motion and uncertainty management techniques

32

Conclusions-I: Major lessons• FTA/FMEA is not rocket science: reinforces

decade-old TG-59 philosophy– Adequate device QC is essential to avoid systematic

errors– Planning & delivery processes must have built-in

catastrophic error detection and correction mechanisms– QA/QC must emphasize

» Redundancy (QC) when capturing key data» Compensating for weaknesses of devices, i.e., key actions where

interlocks don’t “force” desired outcomes

• QM of physician decision making is essential– Upstream histopath/biomarker error not uncommon– Fundamental departure from “MD command/control”

model: ancillary staff must be trained and empowered to question MD actions

33

Conclusions-II• Process mapping and FMEA advantages

– Focuses attention on process as well as device failures– Provides a vehicle for team to work collaboratively to

» better understand the process » Appreciate each other’s vulnerabilities » buy into core QM/QI values

– Most expert member gets to fix FM– Promotes clinical process uniformity so that desired

process-step outcomes get internalized– Better understanding of device-process interactions

helps physicist prioritize device QA• Downsides

– Resource intensive to build/use FMEA expertise– Not a mechanical, one-size-fits-all prescriptive

approach: requires judgment and individualization

Failure Modes and Effects Analysis (FMEA) for Partial ...€¦ · Failure Modes and Effects...

Documents

Transcript of Failure Modes and Effects Analysis (FMEA) for Partial ...€¦ · Failure Modes and Effects...