CLSI EP23™—Laboratory Quality Control Based on

Risk ManagementJames H. Nichols, PhD, DABCC, FACB

ChairholderEP23 Document Development Committee

Professor of Clinical Pathology, Microbiology and ImmunologyVanderbilt University School of Medicine

Medical Director, Clinical ChemistryNashville, Tennessee, USA

2

Objectives

• Review key aspects of risk management.• Describe the various types of control processes.• Identify CLSI document EP23 as a resource for

developing a laboratory quality control (QC) plan based on risk management.

• Use CLSI document EP23 to develop a quality control plan (QCP) based on risk management for a simple, moderate complexity device.

3

Risk

Would you walk underneath this piano?

4

Risk Management

• Risk management is not a new concept; laboratories:Evaluate the performance of new devices.Troubleshoot instrument problems.Respond to physician complaints.Estimate harm to a patient from incorrect results.Take actions to prevent errors.

• Risk management is a formal term for what clinical laboratories are already doing every day.

5

Risk Management Definition

• Systematic application of management policies, procedures, and practices to the tasks of analyzing, evaluating, controlling, and monitoring risk (ISO 14971)

6

Risk Definition

• Risk – the chance of suffering or encountering harm or loss (Webster's Dictionary and Thesaurus. Ashland, OH: Landall, Inc.; 1993).

• Risk can be estimated through a combination of the probability of occurrence of harm and the severity of that harm (ISO/IEC Guide 51).

• Risk, essentially, is the potential for an error to occur that could lead to patient/staff harm.

7

What could go wrong?

8

Sources of Laboratory Error

• Environmental:– – Temperature– – Humidity– – Light intensity– – Altitude

• Operator:– – Improper specimen preparation, handling– – Incorrect test interpretation– – Failure to follow test system instructions

• Specimen:– – Bubbles– – Clots– – Incorrect tube additive

• Analysis:– – Calibration factor incorrect– – Mechanical failure

9

Managing Risk With a Quality Control Process

10

Quality Control

• Advantages– QC monitors the end product (result) of the entire test system.– QC has target values: if assay recovers the target, then everything is

assumed stable (ie, instrument, reagent, operator, sample).

• Disadvantages– When a problem is detected, one must go back and reanalyze patients

since the last “good” QC.– If results are released, then results may need to be corrected.

• Need to get to fully automated analyzers that eliminate errors up front– Until that time, need a robust QC plan (QCP)

11

Types of Quality Control

• “On-Board” or Analyzer QC – built-in device controls or system checks

• Internal QC – laboratory-analyzed surrogate sample controls

• External QC – blind proficiency survey

• Other types of QC – control processes either engineered by a manufacturer or enacted by a laboratory to ensure result reliability

12

13

Quality Control Limitations

• No single QC procedure can cover all devices, because the devices may differ.

• QC practices developed over the years have provided laboratories with some degree of assurance that results are valid.

• Newer devices have built-in electronic controls, and “on-board” chemical and biological controls.

• QC information from the manufacturer increases the user’s understanding of device’s overall quality assurance requirements.

ISO. Clinical laboratory medicine – In vitro diagnostic medical devices – Validation of user quality control procedures by the manufacturer. ISO 15198. Geneva, Switzerland: International Organization for Standardization; 2004.

14

Laboratory-Manufacturer Partnership• Developing a quality plan surrounding a laboratory device

requires a partnership between the manufacturer and the laboratory.

• Some sources of error may be detected automatically by the device and prevented, while others may require the laboratory to take action, such as analyzing surrogate sample QC on receipt of new lots of reagents.

• Clear communication of potential sources of error and delineation of laboratory and manufacturer roles for how to

detect and prevent those risks is necessary.

15

Quality Control

16

CLSI Document EP23

• Laboratory Quality Control Based on Risk Management; Approved Guideline (EP23-A™)

• James H. Nichols, PhD, DABCC, FACB, Chairholder of the document development committee

• EP23 describes good laboratory practice for developing a QCP based on the manufacturer’s risk mitigation information, applicable regulatory and accreditation requirements, and the individual health care and laboratory setting.

17

The Scenario

• CLSI document EP23 provides guidance on developing an appropriate QCP that will: – Save time and money.– Use electronic and/or integrated QC features.– Use other sources of QC information.– Conform to one’s laboratory and clinical use of the

test.

18

Developing a QCP

19

The Quality Control Toolbox

• Every QC tool has its strengths and weaknesses (there is no perfect QC tool).

• Implement a combination of tools in order to properly control a test.

• EP23 explains the strengths and weaknesses of the different QC processes.

20

Examples of Quality Control Tools• Intralaboratory QC• Interlaboratory QC• Integrated (built-in) QC• Measuring system function checks• Electronic system checks• Calibration checks• Repeat testing of patient samples• Monitoring aggregated patient results• Implausible values• Delta checks• Correlation of multiple analytes in same sample

21

What Could Go Wrong?

22

Gather Information for the Risk Assessment

• Gather information from several sources:– Regulatory and accreditation requirements

• Clinical Laboratory Improvement Amendments Test/test system information

• User’s manual, reagent package insert, literature

– Health care and testing site settings• Temperature conditions, operator training programs

– Medical requirements for the test results• Allowable performance specifications via physicians

23

Developing a Process Map

• Break down all phases of the test or test system into steps, so that weak points can be identified.

• Each step can be analyzed to find potential failure modes that could present significant risk to patients.

• Process can then be further analyzed to see if controls can be put into place to avoid the failures.

24

Process Map

2Operator training and

compentency

3Startup/maintenance/

calibration

Reagents/calibrators/parts procurement and

storage

Sample acceptability is evaluated

Samples are loaded and tested (retested)

5Measuring system error message or

malfunction?

Troubleshooting performed and

corrective action taken

Performanceverified?

Results are evaluated

Results are reported

Repeat examination

required?

Process MapHigh-Level

Measurement Process

4Laboratory

environment

Measuring systemExamination ready

1Samples received at measuring system

Yes

Start

Samples are unloaded and stored

End

No

Yes

YesNo

25

Key Process Steps

• View the preexamination (preanalytical), examination (analytical), and postexamination (postanalytical) areas of the laboratory.

• Think about what steps can be taken to reduce potential errors “unrelated” to the actual testing of the sample.

26

Where Is the Risk in the Process?

What could possibly go wrong?

27

Identify the Risks

Incorrect Test Result

1 Samples

2 Operator

3 Reagents

5Measuring

System

4Laboratory Environment

Sample Integrity

Sample Presentation

- Lipemia- Hemolysis- Interfering subtances- Clotting- Incorrect tube

- Bubbles- Inadequate volume

Operator Capacity

Operator staffing

Atmospheric Environment

Utility Environment

- Training- Competency

- Short staffing- Correct staffing

- Dust- Temperature- Humidity

- Electrical- Water quality- Pressure

Reagent Degradation- Shipping- Storage- Used past expiration- Preparation

Quality Control Material Degradation- Shipping- Storage- Used past expiration- Preparation

Calibrator Degradation- Shipping- Storage- Used past expiration- Preparation

Instrument Failure

Inadequate Instrument Maintenance

- Software failure- Optics drift- Electronic instability

- Dirty optics- Contamination- Scratches

Identify Potential Hazards

28

• Some areas to consider for weaknesses in the process:

– Testing personnel training and competency

– Reagent/calibrator/parts procurement and storage

– Patient sample acceptability

– System startup– System calibration– Loading and testing of

patient samples– Proper device function– Test result review

Hazard or Risk Identification

29

Risk Assessment

30

Perform the Risk Assessment

• Identify the potential failures and their causes.– Review the process map, fishbone diagram,

manufacturer’s instructions, etc.

• Assess each potential failure.• Where a failure could occur, add an element to

the QCP that will reduce the possibility of that failure, making residual risk acceptable.– For some types of failures, the manufacturer’s

information may already have a quality check in place.

31

• Construct a table; see which types of errors are detected and which ones are not. – If not detected, it must be included in the QCP.

• For each possible failure, assess the possibility of that failure.– Do this for each identified failure.– Use all of the information gathered in order to make

these assessments.

Perform the Risk Assessment (cont’d)

32

Assemble the Quality Control Plan

• Use the information gathered earlier to assess all of the identified risks and their control measures.

• Construct the QCP.

• Include each of the identified QCP actions in the QCP.

33

What could go wrong?

34

Monitor Quality Control Plan for Effectiveness

• Verify that the QCP that is put in place actually works.

• Continue to monitor errors and control failures.

• If an error occurs:– Take the appropriate corrective action.– Investigate the cause of the error.– Once the cause is understood, evaluate whether any

changes need to be made in the QCP.

35

• Review any complaints that the laboratory receives from health care providers. – These complaints may include pointing out

another source of QC “failure” that must be addressed.

• For patient safety, the QCP should be reviewed and monitored on an ongoing basis to ensure that the QCP is optimal.

Monitor Quality Control Plan for Effectiveness (cont’d)

36

Don’t Be Discouraged—Risk Management Is Documenting Much of What We Already Do!

37

Polling Question #1

What is a QCP?

1) The frequency of liquid controls for a test2) A form that defines required specifications of

materials from suppliers3) A document that describes the practices,

processes, and sequences of specified activities to control the quality of a particular measuring system

4) None of the above

38

The Scenario

• A laboratory director wants to develop a QCP – Incorporates the right QC processes for the specific

test– Uses adequate QC to control for their potential

error sources– Follows manufacturer’s instructions

• A unit-use blood gas analyzer will be used as the example.

39

Developing a Quality Control Plan

40

Polling Question #2

Raise your hand if you are responsible for blood gas or electrolyte testing in your facility, or if you are a manufacturer or distributor of blood gas/electrolyte test systems.

41

Gather the Information

42

Blood Gas and Electrolytes

• Generic unit-use blood gas/lytes analyzer in a same-day surgical center

• Low volume: 0 - 5 tests/day

• Need for daily liquid QC uses two kits ($10 each) and adds to turnaround time (TAT).

• Adoption of nontraditional QC through EP23 would improve cost, test, and labor efficiency.

43

Blood Gas and Electrolytes

• Portable clinical analyzer for in vitro quantification of various analytes in whole blood

• Analyzers and cartridges should be used by healthcare professionals trained to use the system according to the facility’s policies and procedures.

44

• System consists of:– Portable clinical analyzer– Test cartridges sealed in foil

pouch for protection during storage

– Quality assurance materials• Control solutions• Calibration verification set

– Data Management System• Server class computer• Data management software• Wireless connectivity and

LIS/HIS interfaces

Blood Gas and Electrolytes

Device

Device

Device

Device

45

Blood Gas and Electrolytes

• Unit use cartridge contains all components to perform the testing – Calibrating solution – Reagents– Sample handling system– Sensors

• Analyzer automatically controls all steps of the testing process:– Fluid movement– Calibration– Reagent mixing– Thermal control

46

Cartridge Operations

• Cartridges are standardized to plasma core lab methods using multi-point calibration curves stored in the device memory that are stable over many lots

• Upon insertion, a calibrant solution in the cartridge is passed across the sensors.

• Signals produced by the sensor’s responses to calibrant are measured – A one-point calibration adjusts the offset of stored multi-point calibration curve.

• Analyzer then moves sample over sensors, and the signal of the sensor responses to the sample are measured off the adjusted calibration curve

47

Polling Question #3

What types of quality control processes can help laboratories manage their risk of errors for a blood gas/electrolyte test system?

1) Liquid quality control samples

2) Manufacturer checks and simulated internal electronic controls

3) Staff training and competency

4) All of the above

48

Internal Control Processes

• Simulated internal QC - diagnostic check of the edge connector, internal electronics, and analyte circuitry.

• Internal QC simulates electronic signals produced by the sensors during a cartridge test.

• An isolated region of the internal circuit board sends a range of simulated sensor signals through the cartridge measurement channels

• Range of signals encompasses entire linear range expected from blood analytes

• Next, conductivity out of the connector pins is measured, insuring no contamination is present in the edge connector which would interfere with the test.

• Signal measurements must fall within strict predetermined thresholds to pass.

49

Quality Control Recommendations

• Internal Simulated QC:– Automatically performed by device every 8 hrs– If significant change in analyzer temp (cold to hot)– Whenever performance of device in question

• Liquid QC:– Each shipment of cartridges– New lots of cartridges– If cartridges experience temperature shift >8°C (15°F)– Periodically as required by facility policies

• Temperature Verification– Monitored continuously during each patient test, but

verification cartridge available and recommended annually, or as required by facility policy

50

Identify Weak Steps forHazards or Risk of Error

Create a Process Map

51

Polling Question #4

What are some common sources of error for a blood gas/electrolyte test system?

1) Shipping and storage temperature of cartridges2) Operator technique3) Device failure4) All of the above

52

Finding the Failure Points Hazard Identification

Incorrect Test Result

1 Samples

2 Operator

3 Reagents

5Measuring

System

4Laboratory Environment

Sample Integrity

Sample Presentation

- Lipemia- Hemolysis- Interfering subtances- Clotting- Incorrect tube

- Bubbles- Inadequate volume

Operator Capacity

Operator staffing

Atmospheric Environment

Utility Environment

- Training- Competency

- Short staffing- Correct staffing

- Dust- Temperature- Humidity

- Electrical- Water quality- Pressure

Reagent Degradation- Shipping- Storage- Used past expiration- Preparation

Quality Control Material Degradation- Shipping- Storage- Used past expiration- Preparation

Calibrator Degradation- Shipping- Storage- Used past expiration- Preparation

Instrument Failure

Inadequate Instrument Maintenance

- Software failure- Optics drift- Electronic instability

- Dirty optics- Contamination- Scratches

Identify Potential Hazards

53

Process Map: Finding the Failure Points• Work from the current package insert• Test order – electronic or hardcopy • Test collection

– Incorrect collection – bubbles, sample exposure to air– Wrong tube type – calcium titrated, heparinized BG tubes– Indirect phlebotomy – line draw contamination– Undermixing/overmixing – sample clots, hemolysis– Analytic delay – glucose, BG, pH, iCa, etc.

• Analysis– Wrong sample volume loaded onto cartridge– Incorrect procedure, timing, result interpretation – Expired reagent– Reagent exposure during shipment – Degradation during storage

• Infection Control• Result reporting errors

54

Identify Control Processes for Each Hazard That

Maintain Risk at a Clinically Acceptable Level

Conduct a Risk Assessment

55

Blood Gas and ElectrolytesRisk Assessment

• Refer to Appendix A, CLSI document EP18 for a more comprehensive list of error sources.

• Work from the manufacturer’s current package insert.

• Samples– Physician order – POCT possible w/o order, need written

or electronic physician order before commencing test.– Wrong tube type – train to use BG syringes/tubes– Line contamination – train on preferred collection and

techniques if catheter collection is only option– Sample mixing – analyzer has clot detection, but will not

detect hemolysis - train on proper mixing technique– Analytic delay – YES – train to analyze immediately, no

longer than 15-30 mins of collection

56

• Operator– Operator lock-out – prevents use of analyzer by untrained

operators– Sample volume detection – Analyzer detects inadequate

sample volume and prevents overloading.– Incorrect procedure, timing, result interpretation – analysis and

result interpretation fully automated, clotted sample or bubbles will be detected by analyzer.

– Expired reagents – cartridges are bar-coded with lot number and expiration date, analyzer prevents use past expiration

– Wireless Connectivity – data management automates reporting of result provided patient properly identified, train on proper patient ID, use barcoded wristbands

– Infection control – train to clean and disinfect after each use

Blood Gas and ElectrolytesRisk Assessment (cont’d)

57

Blood Gas and ElectrolytesRisk Assessment

• Reagents– Test exposure outside specifications (eg, temperature,

humidity) during shipment – analyze liquid QC with each shipment

– Lot-to-lot variability – analyze liquid QC with each lot– Liquid QC degradation – monitor refrigerator (2 to 8°C), bring

to room temperature at least 30 minutes before use, discard within 30 days of opening bottle

– Degradation during storage – monitor storage conditions, (stable @ room temp for 2 years). If refrigerated, bring to room temperature at least 30 minutes before use, (analyze liquid QC due to temp change). What about other QC frequency?

58

Polling Question #5

How should laboratories determine the optimal frequency of liquid quality controls?

1) Refer to the manufacturer’s package insert recommendations

2) Identify local and regional regulatory requirements

3) Conduct a risk assessment4) All of the above

59

Liquid Quality Control Frequency

• Minimum – follow manufacturer recommendations and regulatory requirements (CLIA for BG analysis – one QC sample q 8 hr, two levels q 24 hrs, one QC w/ each pt sample unless calibration every 30 mins)

• Manufacturer recommends liquid QC with each– Shipment– New lot– Significant change in cartridge temperature (>8°C)– Whenever question of test system performance

• Options for determining liquid QC frequency– Peer publications – verify what others are already doing– Develop QC rules based on six-sigma of test system– Verify in your facility:

– analyze 2 levels each day for several weeks, then reduce to every few days, weekly or monthly after more experience with test system

• QC-lockout assists with compliance

60

• Environment– Incorrect collection – train staff anaerobic phlebotomy– Compliance with documentation – risk bases on prior issues

with other testing noted at this location (refrigerator monitoring, QC documentation, etc.)

• Clinical Application– Immediate medical decisions – test results used to manage

critical patients, higher risk since only one chance to get right result!

– Sample not stable – analyze immediately, presents higher risk since can’t be repeated!

Blood Gas and ElectrolytesRisk Assessment (cont’d)

61

Summarize the Quality Control Plan

62

Blood Gas and ElectrolytesQuality Control Plan

• Analyze liquid QC.– Each new shipment*– Start of a new lot*– After significant change in cartridge temperature (>8°C) *– Whenever uncertainty about analyzer performance*– Monthly (based on facility verification and experience with test)– Note: Simulated multi-level quality control automatic every 8 hours and

internal calibration with each test cartridge **

• Use checklist to document training/competency.– Test only when electronic or written physician order– Proper patient identification– Use BG syringes/tubes for specimen collection– Arterial BG collection (preferred), or line draws as required– Use anaerobic technique– Mix specimens appropriately and analyze immediately– Monitor refrigerator and room temperatures– Clean and disinfect analyzers after each use * (* Manufacturer recommendations) (** Mandated by accreditation regulations)

63

Polling Question #6

What should a laboratory do once a QCP is drafted?

1) Have a party; the laboratory has now proved the quality of its test.

2) Nothing; once developed, the QCP is the end of the risk management process.

3) Monitor its QCP for effectiveness, and modify the plan as needed.

4) None of the above.

64

Monitor for Failure/Errors and Modify Quality

Control Plan as Needed

Implement the Quality Control Plan

65

Quality Control Plan Implementation: BG/Electrolyte Monitors

• Frequency of analyzer error codes• Liquid QC failure rates• Frequency of specimen issues – hemolysis,

clots, or other problems• Number of physician complaints: results

that don’t match clinical situation• Any other unexpected error

66

Risk Management will help you learn about your processes and weaknesses

67

Summary

• Risk management is something laboratories are already doing. EP23 simply formalizes this.

• A QCP is necessary for result quality, and each QCP is unique.

• A QCP is the industry standard. It depends upon the extent to which the device’s features achieve their intended purpose in union with the laboratory’s expectation for ensuring quality results.

• Once implemented, the QCP is monitored for effectiveness and modified as needed to maintain risk at a clinically acceptable level.

68

Now that you’ve survived this…