Proficiency Testing : A guide to Improve Laboratory Standard

Sasiwimol UbolyamHIV-NAT Research Laboratory/Chula CRL

After this training, participants should

1. Discuss 4 GLPs that can help maintain successful proficiency test performance

2. Discuss 5 proficiency testing practices that can help avoid proficiency test failures

3. List 5 steps to follow when investigating an unsatisfactory proficiency test result

External Quality Assurance (EQA)

EQA is a means of evaluating a laboratory's performance through analysis of unknown samples

provided by an external source.

•Blinded samples sent on a regular schedule •Run as patient samples•Results are compared to peers

PT History

• The first PT event occurred in 1946• 1967 , the Clinical Laboratory Improvement

mandated PT for all laboratories • CLIA’ 88 specifies performance standards

laboratories must achieve.

EQA cycle

Creationor

Acquisition

Quality Control and

Validation

SampleSend-out

LaboratoryProcessing

Results Receipt

ResultsAnalysis

ClientReports

ChallengeSelection

Pre-Analytic

Post-Analytic

Purchase or Produce

Purchase Easier Less expensive Import and Transport issues May not address regional

issues

Produce Meet specific needs Meet regional needs Ensure quality Need Infrastructure Requires larger staff budget Need special location and

equipment

Sometimes, the best plan is a blended co-share solution.

Regulations

Satisfactory performance as a score of 80% of results within acceptable limits (4 acceptable results out of 5 samples) in a single proficiency testing event for most analytes

For ABO/Rh blood groups and compatibility samples, 100% of results must be within acceptable limits.

Regulations

Unsatisfactory performance occurs when a laboratory scores less than 80% (or less than 100% for ABO/Rh and compatibility samples) in a single proficiency testing event.

Unsuccessful proficiency test performance occurs when a laboratory fails to achieve a satisfactory score on 2 consecutive PT events, or 2 out of 3 consecutive PT test events

Regulations

Laboratories designated as unsuccessful must resolve the

problem and successfully complete 2 PT events before continuing to

test patient samples

Preventing Proficiency Testing FailuresGood Laboratory Practices can help achieve successful

performance on PT events

Ensure the staff are adequately trained Use quality control methods that will regulate

the instrument within the performance range specified by the manufacturer

Validate the instrument’s analytical measurement range as specified by CLIA’88 regulations, or more often if recommended by the manufacturer.

Follow consensus guidelines issued by scientific panels

Preventing Proficiency Testing Failures

Proficiency Testing Tips : 5 commonsense PT practices can also help avoid errors

1. Confirm your laboratory is assigned to the correct peer group

2. Mark the shipping dates for PT samples on calendar

3. Avoid specimen handling and clerical errors.

4. Submits results by due date

5. Review standard deviation index (SDI) data on the evaluation supplied by the PT provider

CAP Evaluation What Should You Review?

• Evaluate each analyte and specimen for negative/positive bias, trends or shifts

• Evaluate ungraded challengesIf an evaluation criteria is not listed in the

Participant Summary you can use information provided by manufacturer

Review Results to Prevent Failure

Education Challenges Code 26

POC.03275 Phase II N/A YES NOIs there evidence of evaluation and, if indicated, corrective action in response to "unacceptable" results on the proficiency testing reports and results of the alternative performance assessment system?

NOTE: The evaluation must document the specific reason(s) for the "unacceptable" result(s) and actions taken to reduce the likelihood of recurrence. This must be done within one month after the POCT program receives its evaluation. In addition, each ungraded challenge, each educational challenge, and each episode of nonparticipation must be reviewed and corrective action instituted as appropriate.

SDI algorithm 1. If no more than 1 of the 5 SDIs exceeds the

same (+1 or – 1) SDI limit, significant error is unlikely and further scrutiny is not needed

2. If 2 or more SDIs exceed the same (+1 or -1) SDI limit, calculate the average SDI. If the average SDI is greater than 1.5, a significant systemic error is possible.

3. If the average SDI is less than 1.5, check whether 1 observation exceeds 3 SDI or the difference between the largest and smallest SDI exceeds 4.0 if either of these conditions exists, a significant random error is likely.

Reasons for Failure

Troubleshooting Proficiency Test Failures

Each unsatisfactory result should be investigated even if the laboratory’s overall performance is successful

Because this will help detect and correct problems before they progress to unsuccessful performance in the future

Troubleshooting Proficiency Test Failures

To find the reason for an unacceptable result, you may need to examine the result form or retest specimen.

Therefore , you should always keep a copy of the result form and instrument printouts or result logs and save the PT samples (usually frozen)

Investigate Unsatisfactory Results Thoroughly

1. Look for clerical errors : Check result for evaluation is the same as result form, selected the correct instrument, reagent, or kits and transcribed results correctly, verify units

2. Investigate possible specimen handling problems : verify specimen arrived in acceptable condition

3. stored according to instructions, reconstitute correctly

4. tested within time allow for specimen stability, examine specimen hemolysis.

5. Notify PT provider if you suspect that a problem with specimen handling occurred during manufacture or transit

Investigate Unsatisfactory Results Thoroughly

3. Investigate possible operator errors : such as mishandling dilutions, misinterpretation instrument codes, and in correctly loading or sampling the specimen.

• Dilution errors were the most common operator in some study

Investigate Unsatisfactory Results Thoroughly

4. Review reagent logs, quality control logs and calibration records : • to look for evidence of instrument or kit malfunction, check

expire date , examine refrigerator and room temperature records.

• Verify that controls were within acceptable limits and that there were no indications of shifts or trends.

• Make sure that the instrument had been properly calibrated with in time frame recommended by the manufacturer. Calibration verification had been performed as recommended by manufacturer.

• Also check whether the instrument was calibrated during the interval between the PT test event

Investigate Unsatisfactory Results Thoroughly

5. Review instrument maintenance records. : Look for the indications that the instrument may not have been performing optimally on the day samples were tested. Also look for changes that could have

impacted performance. Check for maintenance performed after the

PT event which may have corrected a problem that had not been detected on the day samples were tested.

Example: Investigation Form

Root cause analysis

Conclusion Since the 1st PT Test event in 1946, PT has become a

critical tool to evaluate performance and judge whether a laboratory is sufficiently proficient to reliable test patient samples.

No longer voluntary, successful participation in a PT program is now required of all laboratories that test patient specimens

The best strategy for maintaining successful performance is to use good laboratory practices to prevent or correct problems early, pay attention to proficiency testing details to avoid clerical and sample handling errors, and investigate unsatisfactory results thoroughly and methodically

Proficiency Testing Exception Summary (PTES)

• Unsatisfactory PT performance Failure to attain the minimum satisfactory score for an analyte, test,

subspecialty, or specialty for a testing event.

Unsuccessful PT performance (2 of 3 testing events) Failure to attain the minimum satisfactory score for an analyte, test,

subspecialty, or specialty for two consecutive or two of three consecutive testing events.

Critical PT performance (3 of 4 testing events) Failure to attain the minimum satisfactory score for an analyte, test,

subspecialty, or specialty for three consecutive or three of four consecutive testing events. A laboratory must immediately cease testing for that analyte or the discipline.

The Benefits of PT/EQA

Health Care SystemA valuable assessment tool

An indicator of laboratory performance

A mechanism to provide notice and education

The LaboratoryA valuable tool for: education

Self assessment

System confidence

Public confidence

The PatientKnows the system is working together to ensure the public receives

accurate and useful information

EQAEducation

Info

rmat

ion

Quality Management