Risk Assessment Tools for IVD Assays

Risk Assessment Tools for Identifying Hazards and

Evaluating Risks Associated with

IVD AssaysRobert C. Menson, PhD

AACC Expert AccessLive on Line06 July 2004

© 2004 Menson & Associates, IncAACC Expert Access

Live on Line 2

What Risks Must Be Managed?

Business

Product liability

Regulatory

Risk to safety of patients, users, handlers


Live on Line 3

Elements of the Risk Management Process

Risk Analysis

Risk Evaluation

Risk Control

Post-Production Information

Ass

essm

ent

Man

agem

ent


Live on Line 4

Risk Assessment Tools

Risk MatrixPHA= Preliminary Hazard AnalysisFTA=Fault Tree AnalysisFME(C)A=Failure Mode Effects (Criticality) AnalysisHAZOP=Hazard Operability AnalysisHACCP=Hazard Analysis and Critical Control Point


Live on Line 5

Applications of Risk Analysis

DesignProductionPre-market NotificationsComplaintsMDRChange ControlFailure AnalysisEtc.


Live on Line 6

Risk ManagementLife Cycle

Concept & Feasibility Development Scale-Up & Transfer Production

System Level Assessment

Design Assessment

Process Assessment

Customer FeedbackAssessment


Live on Line 7

Risk Management& Design Control

Scale-Up & Transfer

DesignControl

RiskAssessment

Requirements Plan SpecificationsTest

Methods &Results

ProductionsMethods

Change Records

RiskManagement

Plan

PreliminaryHazard

Analysis

Detailed Analysis(FMEA, FTA, HACCP, HAZOP)

RiskManagement

Report

Risk Reviews

ProductionDevelopmentConcept& Feasibility Planning


Live on Line 8

Key Concepts of Risk

The frequency of the potential harm;How often the loss may occur;

The consequences of that loss;How large the loss might be;

The perception of the loss;How seriously the stakeholders view the risk that might affect them.


Live on Line 9

Risk Management Terms

Intended Use/PurposeHarmHazardRiskResidual RiskTolerable Risk


Live on Line 10

Intended Use/Purpose

Use of a Product, Process or Service in accordance with the specifications, instructions and information provided by the manufacturer

ANSI/AAMI/ISO 14971:2000, definition 2.5


Live on Line 11

Harm

Physical injury or damage to health of people, or damage to property or the environment.

ISO/IEC Guide 51:1999, definition 3.3“Guidelines for inclusion of safety aspects in

standards.”


Live on Line 12

Hazard

Potential source of Harm

ISO/IEC Guide 51:1999, definition 3.5


Live on Line 13

Risk

Combination of the probability of occurrence of harm and the severity of harm.



Live on Line 14

Residual Risk

Risk remaining after protective measures have been taken.



Live on Line 15

Tolerable Risk

Risk which is accepted in a given context based on the current values of society.



Live on Line 16

Hazard Identification

HazardIdentifcation

BrainstormingPHAFTA

FMEA

LawsCodes

Standards

EventsMDRs

AccidentsEtc

Listof

Hazards


Live on Line 17

Risk Matrix

HIITOccasional

IILTRemote

HHILProbable

HHILFrequent

SevereMajorMinorNegligible

SeverityFrequency


Live on Line 18

Severity of Consequence Codes

Severity Nuclear Medical Generic

Severe Virtually complete loss of plant.

-Many fatalities

Potential death or Permanent injury

Product must be scrapped.Loss of customer or repeat business.

Compliance in jeopardy.

Major Extensive damage to plant.

Few fatalities

Potential serious injury.

Not permanent

Major service or downtime.Significant rework.

Customer will notice.May cause injury.

Loss of customer confidence

Minor Significant damage to plant. Severe injury or

occupational illness

Potential minor injury.

Not permanent

Reduced service.Minor rework or re-inspection.

Customer may notice.

Negligible Minor systems damage.

Minor injury or occupational

illness

Potential minor discomfort.

Not permanent.

Minor nuisance.No effect on service.

Customer won’t notice.


Live on Line 19

Frequency Codes

Frequency Nuclear Medical Generic

Frequent > Once/yr >50% of time Continual occurrences

Probable 0.1-1/yr 10-50% of time Occurrences are frequentHappens, probable reoccurrence

Occasional 0.01-0.1/yr 1-9.9% of time Isolate occurrences happen

Remote 0.0001-0.01/yr 0.1-0.99% of time Isolated occurrences possibleDon’t expect reoccurrence

Improbable 0.000001-0.0001/yr 0.0001-0.1% of time

Not likely

Incredible <0.000001/yr<1 in 1,000,000

yrs

<.0001% of time<1 in 10,000

Incredible


Live on Line 20

Estimation of Risk Codes

H: High I: Intermediate

L: Low

T: Trivial

Risk must be reducedReduced to ALARP-cost a minor factor

Reduce to ALARP-consider cost/benefitBroadly acceptable

ALARP=As Low As Reasonably Possible


Live on Line 21

PHA

IOMaQSR TrainingSOPsInternal Software Checks

Installation of Incorrect Software Version

TRMiBattery life certificationDrain testingPreventive maintenance

Loss of bits on SRAM due to weak battery

IOMiSoftware Qualification TestingBackup Sample

Software bug/failure

LRMiReadily apparent to operatorSample can be reprocessedBackup sample

Pump, Valve or Liquid Sensor Malfunction

TRNStress/Simulated Life Cycle Testing8-Hour “burn in”

System failure due to deterioration over life cycle

Impact

Freq.Sev.Investigation/ControlsHazardHazards Arising From Functional Equipment Failure


Live on Line 22

ALARP Curve

Increasing Severity of Harm

Incr

easi

ng P

roba

bilit

y of

Occ

urre

nce

Intolerable Region

ALARP

BroadlyAcceptable

Region

MaximumTolerable

Risk


Live on Line 23

FMEA vs FTA

Assumes failure of the functionality of a productIdentifies part/module failure as cause of functional failure

Assumes component or part failureIdentifies functional failure as a result of part failure

FTAFMEA


Live on Line 24

FMEA vs. FTADone for entire design

Systematic way to predict new problems

A bottoms-up analysis

People expect the same results from FTA which is not true

Too difficult to do for entire designSystematic way to predict causes for problems

A top down analysis

People do not expect the same results from and FMEAOften a fault tree is used for a problem or an accident.

FMEA FTA


Live on Line 25

FTAAssumes fault and analyzes possible causesConnection tool for PHA* to subsystems or modulesTop downDeductiveEvaluate system (or subsystem) failuresConsidered more structured than FMEAGraphical presentation--visual picture*Preliminary Hazard Analysis


Live on Line 26

FailuresPrimary--Due to internal causes that include poor design or use of inappropriate materialsSecondary--Due to failures in the operation that include equipment failureControl--Due to failures in the systems that are in place to protect the quality and safety

e.g. raw material outside specificationfailure of safety switchfailure of test method


Live on Line 27

FTA Basic SymbolsBasic Flow

FAULT

AND

OR

Fault in a box indicates that it is a result of subsequent faults

Connects a preceding fault with a subsequent fault that could cause a failure

Connects two or more faults that must occur simultaneously to cause the preceding fault


Live on Line 28

FTA During Design P O O R F ITIN H A N D

D O E S N O TR E S T R IC TS P IL L A G E

O R

O R

A S P E C T R A T IOT O O H IG H

M A J O RD IA M E T E R

T O O L A R G E

A S P E C T R A T IOT O S M A L L

B A S EF L A T N E S S

B A S E S IZ ET O O S M A L L

R IM T O F IL LIN S U F F C IE N T

CompetingDesign Criteria


Live on Line 29

FTA Lab Failure

Patient Death OR

Other

Lab Error OR

Outliers

Systematic

Random

OR

CalibrationError

Interference

Other


Live on Line 30

FTA During ReliabilityHAZARD

SYSTEMFAILURE

DRIFT>LIMIT

CMPT A FAILS CMPT B FAILS CMPT C DRIFTS REFERENCEDRIFTS

AN

D

OR

OR 4. x 10-9

4. x 10-91. x 10-16

3. x 10-9 1. x 10 -92. X 10 -85. x 10 -9


Live on Line 31

What is FMEA?What is FMEA?

FMEAFailure Mode and Effects Analysis

FMECAFailure Mode Effects and Criticality Analysis

FMEAFailure Mode and Effects Analysis

FMECAFailure Mode Effects and Criticality Analysis


Live on Line 32

What is FMEA?

Powerful prioritization toolInductiveHigh effective tool for identifying critical quality attributesHigh structuredMethodicalBreaks large complex designs into manageable steps


Live on Line 33

FMEA

Bottom up approachEvaluates specific failuresDetailed analysis tool

Use in conjunction with PHA and FTA Complements FTA

May lead to different failure results


Live on Line 34

Advantages of FMEA

Less analyst dependent than FTAAllows direct criticality assessment of componentsValuable troubleshooting aidIdentifies areas of weak designIdentifies areas of high riskPrevention planningIdentifies change requirements


Live on Line 35

Disadvantages of FMEADoes not consider operator errorTediousMay not apply to all systems--especially softwareMay require extensive testing to gain informationMay miss some failure modes

Time pressuresInformation missing


Live on Line 36

DefinitionsCriticality -- Weighting of hazard severity with the probability of failureSeverity -- Seriousness of effect through its impact of the system functionOccurrence -- Likelihood a specific failure will be caused by a specific cause under current controls, i.e. the probabilityVerification -- Ability of the current evaluation technique to detect potential failure during designDetection -- Ability of the current manufacturing controls to detect potential failure before shipping


Live on Line 37

DefinitionsDefinitions

Risk Priority Number (RPN)

= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)

Risk Priority Number (RPN)

= (S) x (O) x (D) or (V)Severity (S)Likelihood of occurrence (O)Likelihood of detection (D)Likelihood of verification (V)


Live on Line 38

FMEA ExampleP O T E N T IA L F A IL U R E M O D E A N D E F F E C T S A N A L Y S IS

Im u n n o a s s a y D e v ic e F M E A N u m b e r S u b s y s te m P a g e o f X C o m p o n e n t _ S u p p o rt_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ R e s p o n s ib ility P re p a re d B y P ro c e s s F M E A D a te (O rig .) (re v .) C o re T e a m : _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

D e vic e / P o te n tia l P o te n tia l S P o te n tia l O C u rre n t D R R e c o m m e n d e d R e s p o n s ib ilit

y A c tio n R e s u lts

F u n c tio n F a ilu re E ffe c t(s ) C a u s e (s ) C o n tro ls P A c tio n (s ) a n d T a rg e t A c tio n s R M o d e o f F a ilu re o f F a ilu re N C o m p le te D a te T a k e n S O D P

N S u p p o rt S y s te m

H o ld s A n tib o d y

-

S e p a ra te s A n a ly te

F a ils to S e p a ra te a n a ly te

N o A n s w e r E rro n e o u s R e s u lt F a ls e H i F a ls e L o

4

79

W ro n g M a te ria l

3

33

5

55

6 0

1 0 5 1 3 5

D e v e lo p s C o lo r


Live on Line 39

Process FMEA Process FMEA Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for

reducing occurrence or increasing detection of the failure conditions

Identifies potential product-related process failure modes Assesses the potential customer effects of the failuresIdentifies the potential internal and external manufacturing or assembly process causesIdentifies process variables on which to focus controls for

reducing occurrence or increasing detection of the failure conditions


Live on Line 40

Sources of Process Defects

Sources of Process Defects

Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts

Omitted processingProcessing errorsErrors setting up work piecesMissing partsWrong parts

Adjustment errorAdjustment errorProcessing wrong Processing wrong work piecework pieceMisMis--operationoperationEquipment not set Equipment not set up properlyup properlyTools and fixtures Tools and fixtures improperly preparedimproperly prepared


Live on Line 41

FMEA Summary

Powerful tool for summarizing:Important modes of failureFactors causing these failuresEffects of these failuresRisk prioritizationIdentifying plan to control and monitorCataloging risk reduction activities


Live on Line 42

HAZOP

Haz ard and Op erability StudyBottom up analysisDeviations from design intentionsSystematic brainstorming based on guide words


Live on Line 43

HAZOPGuide Words

No/NotMoreLessAs well asOther than


Live on Line 44

Hazop ModelDesign Statement

Activity Material Destination

Transfer Powder Hopper


Live on Line 45

HAZOP

LiquidWrong powder

Other than

Larger tankInaccurate gage

Pump fastMore

Valve closedHopper full

Tank emptyValve closedLine blockedPump broken

NoHopperPowderTransfer


Live on Line 46

HAZOP Plan

Interlock

Operator trainingPM

Low

Med

Med

Valve closedLine blockedPump broken

Powder flow

NO

WhoActionRiskCausesDeviationGuide


Live on Line 47

HACCP

Risk Management SystemBiological HazardsChemical HazardsPhysical Hazards

RequiresPrerequisite Quality System Program Traditionally GMPs


Live on Line 48

HACCP Steps1. Conduct hazard analysis and identify

preventive measures2. Identify Critical Control Points.3. Establish critical limits4. Monitor each critical control point5. Establish corrective action to be taken when

deviation occurs6. Establish verification procedures7. Establish record-keeping system


Live on Line 49

HACCP Decision TreeP re v e n tiv eM e a s u re s

E x is t

S te p E lim in a teo r R e d u c e

C ritic a lC o n tro l P o in t

Is c o n tro lN e c e ss a ry fo r

S a fe ty

S to pN o t a C C P

M o d ify S te p

N o

N o

Y e sY e s

Y e s

E x ce s sC o n ta m in a tio n

P o s s ib le

N o

N o

S u b s eq u en tS te p s e lim in a te

o r R e d u c eY e s

Y e s

N o


Live on Line 50

HACCP Worksheet

Biological

Chemical

Physical

Is this step a critical control point? (Y/N)

What preventative measures can be applied to prevent

the significant hazards?

Justify your

decisions for column

3.

Are any potential safety

hazards significant?

(Y/N)

Identify potential hazards

introduced, controlled or

enhanced at this step(1)

Material/processin

g step

654321

Product Description:

Method of Storage and Distribution

Intended Use and Consumer

Firm Name:

Firm Address:


Live on Line 51

HACCP Plan

(7) Who

(6)Frequency

(5)How

(4)What

(10)Verificat-

ion

(9)Record

s

(8)Correct-

iveActions

Monitoring

(3)Critical

Limits for each

Action

(2)Significant Hazards

(1)Critical Control Point

Product Description:

Method of Storage and Distribution:

Intended use and Consumer:

Firm Name:

Firm Address


Live on Line 52

Contact Information

Robert C. Menson, PhD

Menson & Associates, IncQuality Systems and Compliance

For FDA Regulated IndustriesPO Box 480

Newport, RI 02840Phone: 1.401.847.6320

Email: [email protected]

Risk Assessment Tools for IVD Assays

Documents

Transcript of Risk Assessment Tools for IVD Assays