Tools and Techniques for process based...

Recommendations of TG100

M. Saiful Huq - Recommendations of TG100 1

Tools and Techniques for process based safety

M. Saiful Huq, Ph.D., FAAPM, FInstP Professor and DirectorDepartment of Radiation Oncology

University of Pittsburgh Cancer Institute ; UPMC CancerCenterPittsburgh, Pennsylvania, USA

6%

29%

16%16%

5%

9%

6%7% 6%

Procedures

Decision

Professional errorComminication

Instrument

OtherInterpre tation

Training

Supervision

Source: IAEA - Safety Report Series No 17

90 % human errors

Main causes of radiotherapy accidents

2Saiful Huq - Tools and techniques for process based safety 2012 AAPM annual meeting, July 29 - August 2

3

Human failure18%

Lack of standardized procedures

18%Inadequate

training15%

Inadequate resources

13%

Hardware/software failures

13%

Lack of communication

11%

Design failure4%

Commissioning 4%

Others4%

TG-100 analysis of causes of failure for IMRT treatment

Saiful Huq - Tools and techniques for process based safety 2012 AAPM annual meeting, July 29 - August 2



Challenges with current QA paradigm….

4

• Current RT QA guidance is focused on equipment performance even though most RT events have resulted from human performance failures rather than equipment failure

• Different from a process centric QM approach which should be designed to mitigate all failures with detectable impact on patients, not just the ones resulting from equipment failure


2008

2008

2009

Recent reports

5

2000


6

• As technology & processes changeØ Retrospective approaches are not

sufficientØ All-inclusive QC checks may not be

feasibleØ Develop proactive approaches to

anticipation of failure modesØ Evaluate risks from each failure mode

ICRP 112 Conclusions and recommendations


• QM program should be based on risk assessment



Risk assessment

7

• Many risk assessment and analysis tools/techniques exist in industry

• These tools can be easily adapted to RT to enhance safety and quality of treatment process

• Risk assessment is the process of analyzing the hazards involved in a process


Risk assessment tools

8

• Failure Modes and Effects Analysis (FMEA)

• Fault Tree Analysis (FTA)

• Process Tree (Mapping)

• Establishment of a risk based QM program


9

Process tree


• Graphically illustrates the relationship between the different steps in a process

• Delineate and then understand the steps in the process to be evaluated

• Demonstrates the flow of steps from process start to end



10

Patient database information entered

Immobilization and positioning

CT simulation

Other pre-treatment imaging

Transfer images and other DICOM data

Initial treatment planning directive

RTP anatomy contouring

Treatment planning

Plan approval

Plan preparation

Initial tx(Day 1)

Subsequent tx(Day N)

End of txStart of tx

Example: IMRT process tree


Verify patientsetup

Register EPID andpseudo radiograph

LoadEPID

LoadPseudo-

radiograph

Determine patientShifts and rotations

Reimage if necessary a

Verify beam outlinesSelect beam in record & verify

Image

SetparametersVerify clearance

and achievabilityRegister beamoutline c plan

Repeat for each beam b

b

Approval to treat

Review setup images

Review beam images

Approve treatmentif good

Day 1 imaging verification

EPID imagingfor localization

Place patient on table

Align mold marks

Pt in mold

Align allmarks

Make AP imageSetmu

Set gantryMakeexposure Set field size

Set machineMake lateral image

Setmu

Set gantryMakeexposure

Set machineVerify images are adequate

Approve patient position

a

11

Process Tree

11




Align mold marks

Pt in mold

Align allmarks

Make AP imageSetmu



Setmu




a

11

Process Tree





Align mold marks

Pt in mold

Align allmarks

Make AP imageSet

mu



Setmu



Verify patientsetup

Register EPID andpseudo radiograph

LoadEPID

LoadPseudo-

radiograph

Determine patientShifts and rotations

Reimage if necessary


a

a

Verify beam outlinesSelect beam in record & verify

Image

SetparametersVerify clearance

and achievabilityRegister beamoutline c plan

Repeat for each beam b

b

Approval to treat

Review setup images

Review beam images

Approve treatmentif good

11

Process Tree




A Process Tree

21%

21%

18%

22%

18%

Saiful Huq - Tools and techniques for process based safety

1. Cannot be used as a risk assessment tool2. Is a visual display of various stages of a

process3. Cannot be used to develop a QM program in

radiation therapy

4. Is a tool used by industrial engineers only5. Cannot represent the radiation treatment

process of a patient

Correct answer: 2

2. Is a visual display of various stages of a process


15

• Assess potential risks involved in the process

• FMEA assesses the likelihood of errors in a process and considers the effects of such error

FMEA




Ø What could possibly go wrong (potential failure mode)

Ø How could that happen (what are the causes that result in a failure mode)

Ø What effects would such a failure produce (potential effects of failure)

Ø The overall risk of each identified failure mode is then scored and prioritized according to RPN

Ø A good FMEA then identifies corrective actions to prevent failure from reaching the patient

16

• FMEA looks at each process and at each step asks the questions:


FMEA

• First day of Tx: Patient maybe positioned incorrectly relative to the isocenter

Example of FMEA

17

Failure mode:

• Tx machine or simulator laser misalignment• Therapist error due to inattention• Poor instruction or setup documentation in the chart• Inadequate immobilization• Organ motion or tumor growth

Causes of failure:

• The magnitude of the displacement from planned isocenter• Tx technique (stereotactic, 3D conformal or large fields)• Proximity of critical structures and when in the course of tx the

error was detected

Consequences: severity depends on:


Steps to perform a quantitative FMEA

18

• Identify all of the potential causes for each FM

• Determine the impact of each FM on the outcome of the process assuming the failure is not detected and corrected in subsequent steps

• Identify all possible failure modes (FM)

• Establish a risk based QM program




Step Potential failure modes

Potential causes of failure

Potential effects

of failure

O S D RPN Comment

FMEAFor a given sub-process:

RPN = O x S x D [ 1 ≤ RPN ≤ 1000 ]

Examples of FMEA

Step Potential Failure Modes

Potential Cause of Failure

Potential Effects of Failure

O S D RPN Comments

Import images into RTP system data base

Wrong patient’s images

MiscommunicationUser error

Wrong dose distributionWrong volume

3 9 5 135

Wrong imaging study (correct patient)Viz.; wrong phase of 4D CT selected for planning; wrong MR for target volume delineation

Ignorance of available imaging studiesAmbiguous labeling of image setsInadequate training MiscommunicationUser error


7 8 7 392

File(s) corrupted

Network problem Lost imagesWrong dose distributionWrong volume

43

39

24

24108

File probably would not open

Sub-process: RTP planning


Examples of FMEA




O S D RPN Comments





3 9 5 135




7 8 7 392

File(s) corrupted


43

39

24

24108






Examples of FMEA




O S D RPN Comments





3 9 5 135




7 8 7 392

File(s) corrupted


43

39

24

24108




Examples of FMEA




O S D RPN Comments





3 9 5 135




7 8 7 392

File(s) corrupted


43

39

24

24108




What are O, S and D’s?

• O : Probability that a specific cause will result in a failure mode

• S: Severity of the effects resulting from a specific failure mode if it is not detected or corrected

• D: Probability that the failure will not be detected

• Risk priority number RPN = O*S*D

• For O, S and D one assigns values from 1 to 10.

• TG100 has developed scales for O,S, and D indices that are tied to radiotherapy outcomes and observations




TG-100 O, S, and D values used for FMEARank Occurrence Severity Detectability

Qualitative Frequency Qualitative Categorization Estimated Probability of

going undetected in %

1 Failure unlikely 1/10,000 No effect 0.012 2/10,000 Inconvenience Inconvenience 0.23 Relatively few

failures5/10,000 0.5

4 1/1,000 Minor dosimetric error

Suboptimal plan or treatment

1.0

5 <0.2% Limited toxicity or under-dose

Wrong dose, dose distribution,

location or volume

2.06 Occasional

failures<0.5% 5.0

7 <1% Potentially serious toxicity or under-

dose

108 Repeated

failures<2% 15

9 <5% Possible very serious toxicity

Very wrong dose, dose distribution, location or volume

20

10 Failures inevitable

>5% Catastrophic >20


FMEA- mark process map

26

• Shows if potential failures are uniformly distributed through the processes or clustered

• If clustered, should consider the major step as a hazard

• Prioritize the potential failure modes based on RPN and severity function

• Mark the highest rank steps on the process map


27

Major Processes Step Potential Failure

Modes Potential Causes of Failure Potential Effects of Failure AVG O AVG S AVG D AVG RPN

4 - Other pretreatment imaging for

CTV localization

6. Images correctly

interpreted (e.g. windowing for

FDG PET)

Incorrect interpretation of tumor or normal

tissue.

User not familiar with modality or inadequately

trained)(Poor inter-disciplinary

communication)

Wrong volume 6.50 7.44 8.00 387.75

7 - RTP Anatomy

Delineate GTV/CTV (MD)

and other structures for planning and optimization

>3*sigma error contouring

errors: wrong organ, wrong site,

wrong expansions

User error

Inattention, lack of time, failure to review own work

Very wrong dose distributionsVery wrong volumes.

5.29 8.43 7.86 366.00

12 - Day N Treatment

Treatment delivered

LINAC hardware failures/wrong dose per MU;

MLC leaf motions inaccurate,

flatness/symmetry, energy – all the

things that standard physical

QA is meant to prevent.

Poor hardware design Poor hardware maintenance

Inadequate department policy (weak physics QA

process)Poorly trained personnel

Wrong dose Wrong dose distribution

Wrong locationWrong volume

5.44 8.22 7.22 354.00

FMEA by RPN (TG-100)




28

Process map marking I wouldn’t try to read it…will hurt your eyes


FMEA

20%

21%

21%

19%

19%


1. Identifies unknown potential failure modes in a process

2. Prioritizes the identified failure modes according to RPN only

3. Identifies the causes and effects of each failure mode

4. Provides only qualitative means to evaluates the risks associated with a process

5. Cannot provide for problem follow-up and corrective action

Correct answer: 3

3. Identifies the causes and effects of each failure mode




Fault tree (FT)

31

• Evaluate propagation of failures using a FT analysis

• FT gives a visual representation of the propagation of failure in the procedure

• It helps identify intervention strategies to mitigate the risks which have been identified


32

Errorin data

Errorin QC

Error in data input

Errorin QC

Error in Calculation algorithm

Errorin QC

Error in prescription

Errorin QC

Error in calculation

Errorin QA

Error in Calculated value for patient

• Fault tree compliments process tree

• Begins on the left with something that could possibly go wrong (failure mode)

• Works backwards in time (to the right) to study what could possibly cause that error

Fault tree


Design of QM program

• Use the FTA and FMEA risk and process oriented information to design a QM program for the process being investigated

• Example (From TG-100 IMRT analysis)Ø Sub-process: RTP anatomyØ Step: Delineate GTV/CTV and other structuresØ FM: › 3σ contouring error, wrong organ, site, or expansionsØ RPN = 366Ø Rank: #2




34

Rank RPN Step# Process Step

#2 366 58 7. RTP Anatomy Delineate GTV/CTV (MD) and other structures

FM: > 3σ contouring error, wrong organ, site, or expansions

2nd ranked Failure Mode


35

Segment of RTP anatomy Fault Tree w/o QM placement

Step Failure Mode

Causes of failure

Effect of failure


36

Segment of RTP anatomy Fault Tree with QM placement




37

Segment of RTP anatomy Fault Tree with QM placement


A Fault Tree analysis

21%

18%

21%

19%

20%


1. Focuses on identifying the root factors that could cause a failure

2. Is unnecessary to design a process based QM program in radiation therapy

3. Diagram uses only “AND gate” and never any “OR gate”

4. Cannot help determine corrective action required to prevent or control the causes

5. Is only useful for designing a QM program

Correct answer: 1

1.Focuses on identifying the root factors that could cause a failure




Summary

40

• QA/QM should be more process centric

• Should be based on rigorous sensitive analyses of all components of radiotherapy process

• Be based on industrial engineering approaches of risk analysis and mitigation

• Will be infrastructure dependent and may shed light on how much QA is enough for a given institution

• Risk analysis gives guidance for developing a QM program


41

It is useful to report all accidents before consequences appear

Our job is not to prevent errors, but to keep the errors from injuring the patients.

Lucian Leape

It is impossible to make anything foolproof because fools are so ingenious.

Artur Bloch, Murphy’s law

Courtesy: Josef Novotny


Major Processes Step Potential

Failure ModesPotential Causes of

FailurePotential Effects

of Failure AVG O AVG S AVG D AVG RPN

12 - Day N Treatment

Treatment delivered

Gantry or other LINAC

hardware (e.g. OBI) collides with patient

Carelessness Poor departmental

policy

Non-radiation-related physical

injury3.33 9.11 5.44 162.78

10 - Plan Preparation

Prepare e-chart

1. Incorrect Txinfo, 2. Wrong

Rx, 3. Wrong

patient/plan

Manual entry: Large error

Inadequate standard/ procedure / practice

Used incorrectly procedure/practice

Inadequate training/orientation

Very wrong dose Very wrong dose

distributionVery wrong

locationVery wrong

volume

5.44 8.89 5.56 272.89

11 - Day 1 Treatment

Gather patient

treatment information

3. Incorrect treatment data Hardware failure

Very wrong dose Very wrong

volume2.33 8.89 4.78 102.89

FMEA by Severity (S) (TG-100)




43

Process tree (map) I wouldn’t try to read it…will hurt your eyes


44

Process tree (map) I wouldn’t try to read it…will hurt your eyes


ICRP 112 definition of risk

45

• Risk can be regarded as some function of the probability of an event’s occurring and the severity of the consequences for the patient should the event occur




Top 6 RPN (TG-100)Major Sub-Processes Step Potential Failure Modes Potential Causes of Failure

1

Other pretreatment imaging for CTV localization

Images correctlyinterpreted (e.g.

windowing for FDG PET)

Incorrect interpretation of tumor or normal tissue

Inadequate training (user not familiar with modality

Lack of communication (inter-disciplinary)

2 RTP Anatomy

Delineate GTV/CTV (MD) and other structures for planning and optimization

Contouring errors: wrong organ, wrong site, wrong expansions

Lack of standardized procedures; Hardware failure (Defective materials/tools/equipment); Human failure(Inattention; failure to review work); Lack of staff

3 Day N treatment Treatment delivered

Linac hardware failure/wrong dose per MU; MLC leaf motion inaccurate; flatness/symmetry, energy – all the things that standard physical QA is meant to prevent

Poor hardware design; Inadequate maintenance; software failure; Lack of standardized procedures; Human failure; standard linac performance QM failure; Inadequate training

4Initial treatment planning directive (from MD)

Retreatment, previous treatment, brachy etc

Wrong summary of other treatments, Other treatments not documented

Lack of staff; Human failure (inattention; reconstructing previous treatment; wrong info obtained); information not available; Lack of communication

5 RTP anatomy

Delineate GTV/CTV (MD) and other structures for planning and optimization

Excessive delineation error resulting in ˂3σ segmentation errors

Lack of standardized procedures; Availability of defective materials/tools/equipment; Human failure (inattention; Failure to review work; etc); Inadequate training

6 RTP Anatomy PTV construction

Margin width protocol for PTV construction is inconsistent with actual distribution of patient setup errors

Lack of standardized procedures; Lack of communication; Inadequate training; Human failure (inattention; failure to review work; etc)

46

Tools and Techniques for process based...

Documents

Transcript of Tools and Techniques for process based...