Implementing traceability

with EQAS

The risk management of trueness

Marc Thelen, SKML, Netherlands

10th anniversary of CIRME meetings, 17-18 November 2016

Risk management of trueness

1. ISO 15189:2012, 5.5.1.3Method validation, verification

– T=0; trueness and imprecision are fit for intended use

2. ISO 15189:2012, 5.6.2Internal QC

– Conditions as on T=0 are still true

– Or if not, still meet criteria of acceptance on t=0

3. ISO 15189:2012, 5.6.3External QC

– What can be wrong?

Risk of new bias, with stable IQC

Risk of new bias, with stable IQC

Risk of new bias, with stable IQC

False alarm

Goals for perfect EQAS

• Materials as intended by ISO17043:

Commutable, homogenous, stable

– Material cannot be blamed

• Value assignment in reference labs using

refererence methods

– Value assignment cannot be blamed

• Smart reporting

– You know what to do

SMART scoring of EQAS by SKML

• Specific:– Performance of lab, not of EQAS material

– Material and assigned values beyond discussion

– Support root cause analysis: method grouping

• Measurable:– Mathematical relation between statistical dispersion of results and

score

• Achievable:– if SA precision profile>Tea, then score in SA precision profile

• Realistic:– TEa based on EFLM performance goal criteria

• Time dependent– time weighed regression

Dif

fere

nce

mm

ol/

l

mmol/l

Example SMART scoring by SKML

10

Room for improvement?

Please, DO change appropriately

How good is good enough?

• EFLM second strategic conference on analytical performance specifications (APS), Milan 2014

• TFG-PSEQA“Performance specifications for EQAS”

• Chair: Graham Jones

• Members:– Stéphanie Albarède – France

– Gabriela Gutiérrez - Spain

– Mauro Panteghini - Italy

– Marc Thelen - The Netherlands

– Anne Vegard Stavelin - Norway

– Annette Thomas - UK

– Pat Twomey - UK

– Emma Ventura - Spain

Aim of the TGF-PSEQA

• Terms of Reference

– To define performance specifications for the most common measurands that should be used by EQAS organisers (for category I EQAS).

– This TFG will take into account the recommendation of the TFG-TE and TFG-DM.

• TFG-TE “Total error”

• TFG-DM “Allocation of laboratory tests to different models

for performance specifications”

• Deliverables

– A manuscript dealing with these recommendations.

Manuscript so far:

6 factor terminology PSEQA

Requirements for documenting EQA

1.EQA material commutability;

2.the method used to assign the target value; – Reference laboratory

– Certified reference materials

– Formulated (weighed in)

– statistical

3.the data set to which APS are applied;– Single measurements of single sample

– Multiple measurements of single sample

– Multiple samples

4.the applicable analytical property being assessed – Total error

– Bias

– Imprecision

6 factor terminology PSEQA (2)

5. the type of the Milan model(s) used to set the APS

– Outcome-based (Milan 1a)

– Based on clinical decision applications (Milan 1b)

– Derived from biological variation (Milan 2)

– State of the art, defined as the highest level of analytical

performance technically achievable in that moment (Milan 3).

6. the rationale for the selection of the APS

– Passable: everyone should theoretically pass

– Satisfactory: good performing laboratories should pass

– Favourable: no clinical benefit of further improvement

– Aspirational: aim to improve performance, educational.

PSEQA requirements

• All aspects communicated to (potential)

participants

• For every measurand

• In every scheme

• Possibly multiple APS for single measurand:

– Intended use !

Budget versus source

Total mu budgetAvailable mu budget for end-user

total mu budget smaller than mu

No mu budget available for end-user

IVD regulation 2016 annex 1

17.3. Information in the instructions for use

17.3.1. The instructions for use shall contain the following particulars:

•…

•XIX: The metrological traceability of values assigned to calibrators and trueness-control materials, including identification of applicable applied reference materials and/or reference measurement procedures of higher order, and information regarding maximum (self-allowed) batch to batch variation provided with relevant figuresand units of measure

•NB: this text is result of succesful lobby work by our EFLM IVD WG for 5 years ☺

•Implementation required within 5 years

How to divide the budget?

Task:

Formulate how to calculate and express the traceability and the uncertainty of value

assignment of a IVD product

Group:

ISO TC212 WG2, CEN TC140 IVD industry WG, EFLM WG accreditation and IVD

Task:

Discuss a better distribution of the total uncertainty budget between different steps in

chain

Group:

EFLM quality committee, WG accreditation and IVD

CEN TC140 IVD industry WG

laboratories IVD industry

Conclusions

• Class 1 EQC should have– Commutable samples

– Value assignment in reference methods

– Reports based multiple sample evaluation

• Class 1 EQC is needed for– Participating laboratories: local mistakes

– IVD industry: success of standardisation and harmonisation

• EFLM TFG PSEQA– Transparency of EQA APS and rationale

– Harmonisation of EQA

• New IVD regulation– Opportunity for transparency on IVD product uncertainty

– Opportunity for redistribution of total uncertainty budget