Aacc Talk On Calibration Verification 2009

AACC Short Course 73214How to Fulfill the CLIA Requirements

for Calibration, Calibration Verification, and Reportable Range

Anthony Killeen, MD, PhDWilliam Castellani, MD

(in conjunction with CAP)

This PowerPoint Slide Set

• http://tinyurl.com/kmq8vu

Outline

• Calibration Verification• Linearity• CVL Program at the CAP• Examples and Troubleshooting• Cases• Discussion of Selected CAP Checklist

Questions

Calibration Verification

• Periodic verification that the calibration of the analytical system remains valid

• Typically assessed by comparing test results with appropriate target values

• If the calibration changes, then patient test result values will also change

Linearity

• From CLSI document EP6-A (2003)– A quantitative analytical method is linear when

there exists a mathematically verified straight-line relationship between the observed values and the true concentrations or activities of the analyte.

– …if the true amount of an analyte in a sample were to double, clinicians expect that would be reflected in a doubling of the measured value.

– (Notice that there’s no reference to accuracy)

Calibration Verification vs. Linearity

• Calibration verification is the process of verifying agreement between calibrators (or other materials of known analyte concentrations) and measured values

• Calibrators should be traceable to a reference method to ensure accuracy

• Linearity evaluation does not require knowledge of the “true” analyte concentration

• “Linearity” does not appear in CLIA

Analytical Measurement Range vs. Linearity

• AMR is the range of analyte values that a method can directly measure without any dilution, concentration, or other pretreatment of the specimen that’s not part of the usual assay process

• A method should be linear across the AMR

Linearity as Defined in EP6-A vs. Instrument Response

• Linearity typically refers to the final analytical result, not the raw instrument output. A plot of analyte concentration vs. the raw instrument output may not be linear (e.g., competitive immunoassay)

• Linearity as used here is a straight-line relationship between “true” analyte concentrations and measured concentrations

Formal Calibration Verification/Linearity Program

• College of American Pathologists– CVL Surveys

• Other programs e.g., Maine Standards

Advantages to Participating in a CVL Program

• Supplements proficiency testing (PT)• Linearity testing often has smaller absolute

limits for error based on medically or analytically relevant criteria than does PT

• Challenges the full calibration range• Can detect problems earlier than QC or PT• Calibration verification is required by CLIA,

CVL program provides the materials and data analysis

Linearity Study Materials

• Generally 5 samples that span the AMR• Preparation– Individually spiked samples– Serial dilutions of a high concentration sample– Admixtures of low and high concentrations (CAP)– At least 2 replicates at each level to assess

random error

CAP Linearity Assessment

• Uses a polynomial method described in EP6-A• For statistical approach see – EP6-A CLSI/NCCLS Document, Wayne, PA (2003)– Kroll MH et al., Arch Pathol Lab Med 124:1331-8, 2000

Calibration Verification Evaluation

• Results reported as “verified” or “different”• If “verified”, then differences between your

means and the peer-group means are within the allowable error limits in the range specified

• If “different”, then at least one of your values exceeds the peer-group allowable error limits

Tabular Data

Graphical Data

Peer GroupSummary

Result

Calibration Verification Report Example

Difference = 100 * (Your Mean – Peer Mean) / Peer Mean

Allowable Error = larger of: One half of goal for total error OR minimum detectable difference

100% 100%

Note that CLIA Allowable Error for PT is ± 10% for albumin

Calibration not verified

Need to re-calibrate

High biases

Proficiency Testing Data from Same Lab

CAP Linearity Evaluations

• Standard• Extended Range• Dilution• Results– Linear– Nonlinear– Imprecise

Result

Tabular Data

Graphical Data

Extended Range Linearity Evaluation

• Extended range evaluation is used if– Survey includes an additional high specimen that

is spiked and not part of the admixture set– If there is a large gap in the relative

concentrations of the admixture ratios such that a specimen(s) may have a large influence on the fitted regression line, the specimen(s) will be considered to be extended range

Scoring Extended Range Specimens

• Specimens evaluated for linearity excluding the extended range specimen

• If a range that includes the highest non-extended range specimen is linear, then the extended range specimen is evaluated

• If the mean of the extended range specimen value falls on the extrapolated line (within allowable error) then the linear range will include the extended range specimen

Scoring Extended Range Specimens (2)

• The extended range specimen is not used to calculate the best-fit line

Dilution Validation Linearity Evaluation

• Performed if– Lab reports that it diluted at least one specimen– Enough undiluted specimens (normally 5) are

reported for evaluation using the standard linearity evaluation

EXAMPLES AND TROUBLESHOOTING

Linearity demonstrated for LN-01 to LN-05 only

Linearity Example 1: Standard Evaluation, specimens excluded from the linear range

Linearity Example 1: Standard Evaluation, specimens excluded from the linear range

Example 1: Troubleshooting

• Does the linear range cover the AMR? If the high specimen is above the AMR, did you dilute? Was the dilution protocol followed?

• If the linear range does not cover the AMR, then there may be problems with reagents, specimen handling, or the test system

• Check QC, PT, calibration data• Address identified problems and re-run linearity• Consider adjusting AMR to cover the linear range

Example 2: Extended Range Evaluation, extended range sample not linear


• Confirm that the linear range covers the AMR• Check the dilution protocol. Did you dilute?

Dilution can introduce variability• If the linear range does not cover the AMR,

then there may be problems with reagents, specimen handling, or the test system

Example 3: Standard Evaluation, nonlinear data

(Best appreciated with graphical data)

Example 3: Standard Evaluation, nonlinear data


• Is the peer group generally linear?• If the peer group is generally linear then there

may be problems with specimen handling or the test system

• Review QC, calibration, PT data• Eliminate specimen or reagent handling errors• Diagnose fix any identified test system failures• Re-run linearity study

Example 4: Dilution Validation Evaluation, nonlinear trend in non-diluted samples

Example 4: Troubleshooting• (Because non-diluted samples are nonlinear, no

evaluation of the diluted sample is performed• Is the peer group generally linear?• If the peer group is generally linear then there

may be problems with specimen handling or the test system

• Review QC, calibration, PT data• Eliminate specimen or reagent handling errors• Diagnose fix any identified test system failures• Re-run linearity study

Example 4: Standard Evaluation, large replicate imprecision

Example 5: Standard Evaluation, large replicate imprecision


• Pattern suggests pipetting problems which should be carefully investigated

• Fix any identified test system failures• Re-run linearity study

Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5)

Looks OK?

Example 6: Standard Evaluation, only difference plot shows replicate imprecision (similar problem to Example 5)


• Pattern suggests pipetting problems which should be carefully investigated

• Fix any identified test system failures• Re-run linearity study

Example 7: Standard Evaluation, Imprecision from poor fit

Example 7: Standard Evaluation, Imprecision from poor fit

Here, the term “imprecision” refers to the inability to find a subset of data displayinglinearity with adequate precision. Points zigzag around the best-fit line.

Example 7:Troubleshooting

• Unusual pattern, may be difficult to diagnose• Is the peer group linear? If not, call Customer

Service• Highest value may be diluted and a dilution

error is masking loss of recovery at the high end of the AMR

• Specimen handling problems may cause this

Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated

Example 8: Extended Range Evaluation, Imprecise in non-extended range, extended range sample(s) not evaluated

Large gap indicates extended range

Imprecise. E.R. specimennot evaluated

LN-24

• Creatinine linearity survey• Uses human serum spiked with pure

crystalline creatinine• Accuracy assigned values (NIST) allow for

accuracy based grading in addition to calibration verification and linearity checks

Case 1. Calibration Verification (AST)

Linearity study show the data were linear

Case 2. Calibration Verification (calcium)

Case 3. Calibration Verification

Case 4. Linearity (testosterone)

Executive Summary Page from CVL Survey

SOME CAP CHECKLIST QUESTIONS

CHM. 13000

• Are calibration procedures for each method adequate, and are the calibration results documented?

CHM.13100

• Are high quality materials with method- and matrix-appropriate target values used for calibration and calibration verification whenever possible?

CHM.13125

• Are all calibration materials used for non-FDA cleared assays documented as to quality?

• NOTE: Standards used to prepare calibrators for non-FDA-cleared assays require certificates of purity from the vendor, or a check on purity as part of the initial assay validation process. The laboratory should document the accuracy of a new lot of calibrators by checking the new lot against the current lot.

CHM.13400• Are criteria established for frequency of recalibration or

calibration verification, and the acceptability of results?• • NOTE: Criteria typically include: • At changes of reagent lots for chemically or physically active or

critical components, unless the laboratory can demonstrate that the use of different lots does not affect the accuracy of patient/client test results and the range used to report patient/client test data

• QC fails to meet established criteria• After major maintenance or service• When recommended by the manufacturer• At least every 6 months

CHM.13500

• Is the method system recalibrated when calibration verification fails to meet the established criteria of the laboratory?

CHM.13600

• Is validation of the analytical measurement range (AMR) performed with matrix-appropriate materials which include the low, mid and high range of the AMR, and is the process documented?

CHM.13700

• Are criteria established for validating the analytical measurement range, and is compliance documented?

GEN.42085• Is the reportable range verified/established for each

analytic procedure before implementation?• NOTE: The reportable range includes all results that may be

reliably reported, and embraces two types of ranges: • The ANALYTICAL MEASUREMENT RANGE (AMR) is the

range of analyte values that a method can directly measure on the specimen without any dilution, concentration, or other pretreatment not part of the usual assay process

• The CLINICALLY REPORTABLE RANGE (CRR) is the range of analyte values that a method can measure, allowing for specimen dilution, concentration, or other pretreatment used to extend the direct analytical measurement range

Summary of Topics Covered

• Calibration Verification• Linearity• CVL Program at the CAP• Examples and Troubleshooting• Cases• Discussion of Selected CAP Checklist

Questions

Aacc Talk On Calibration Verification 2009

Technology

Transcript of Aacc Talk On Calibration Verification 2009