ALTERNATIVE APPROACH: FOR ESTIMATION OF

MEASUREMENT UNCERTAINTY IN

ANALYTICAL CHEMISTRY

D. Von Mollendorf

Midvaal Water Company

Overview• Background

The use of GUM proved to be very successful when studying or quantifying individual uncertainty componentsBut it was found that in our case many of the uncertainty estimates were found to be unrealistic – in most cases under estimatedE.g.* Standard uncertainty of 5% < long-term precision

(Standard deviation of a series of results)* Control limits of QC chart: 12%* Selectivity results of PT for different matrices/concentrations: 83 - 115%

(Excellent z-scores)* Intra-assay precision (%RSD): 92 - 106%

Also experience challenges using GUM mainly because - Large number of uncertainty sources in complex testing procedures arechallenging to quantify separately

- Experienced difficulty in determining a complete mathematical model todescribe method.

Overview• Objective

Finding a simplification of the GUM approachEmpirical approach? Since the use of empirical approaches for quantitative test procedures as an alternative to the modelling approach are increasingly being recognised.Empirical approaches are based on whole method-performance investigations, designed and conducted so as to compromise the effects fromas many relevant uncertainty sources as possible

• ResultsNordtest approach was implemented:* In most cases uncertainty values was found to be higher than previously

found using the modelling approach.* Proved to have been a better estimation of the real variation of our

methods, mainly due to maximising the probability of including alluncertainty contributions

* Higher uncertainties was in good agreement with inter laboratorycomparison results and quality control data.

• RecommendationsNordtest approach - holistic approach - minimise risk of excluding important uncertainty sources & easy to use already existing data

Uncertainty Approaches

Definition of the Measurand

Single laboratory Inter laboratory

Model-based ? One procedure ?

ModelingComponent by

Component EvaluationISO GUM

Single labValidationWithin-lab

reproducility & Bias

NordTest TR 537

Interlaboratoy Validation

Reproducibility and BiasISO 5725

ISO TS 21748

Proficiency Testing (PT)

Between Lab Variability

ISO Guide 43ISO 13528

Yes No Yes No

Single Laboratory Approaches

A) Model based approach (GUM, Bottom Up)

• Involves component-by-component evaluation of uncertainties

B) Single-lab validation approach

• Calculates a typical uncertainty based on whole method-performance and not directly linked to any particular sample

• Has been formalised in several guides, of which the NordtestUncertainty Guide is the best known

Inter laboratory approachesA) Proficiency Testing Approach

• Gives general uncertainty which can be expected from the same procedure in different laboratories

• Main use: Estimation of uncertainty for a procedure.

B) Inter laboratory Validation Approach

• Use of PT data for estimating measurement uncertainty still under debateand authoritative references are few.

• But, if a laboratory has successfully participated in an inter-laboratory PT, it may utilise the results for estimating the measurement uncertainty.

Comparison of the approachesModelling

•Advanced laboratories- Extra work usually required- Deep knowledge/competency required

•Danger to under estimations

•Promotes Thinking - more information

Single-lab Validation•Routine laboratories

- Less extra work required- Does not need such a high competency level

Realistic uncertainty estimations - usually does not lead to under estimations, sometimes toover estimations• Not as detailed - Does not learn as much• Takes into account more general uncertainty

contributions such as within-lab reproducibility& laboratory bias.

Inter-laboratory Validation•Uncertainty of results obtained using the same

procedure in different laboratories • Crude uncertainty estimates

PT Approach• Actually data from your lab not taken into

account - therefore not recommended•Very crude uncertainty estimation

• Should be used only as first approximation

Single Laboratory Validation ApproachMeasurement accuracy = precision + trueness

Measurement uncertainty= within-lab reproducibility + uncertainty of bias

Combine:

Standard deviation s & an estimate bias b

Measurement uncertainty is the estimated as a root sum of squares of:

Single Laboratory Validation Approach• Provided that measurements are under statistical control,

uncertainty estimates obtained using this approach are applicable for all measurements within the scope of the

measurement procedure.

• Application range of the uncertainty estimates is determined by the range covered in validation study & on-going quality

control.

• Therefore these investigations should include:Appropriate within-scope variations, e.g. different levels of the measurand and different types of test items.

The Nordtest Approach

Combination of Uncertainty components:

Reproducibility within the laboratory: Rw

(from control samples & other estimations)

Estimation of the method and laboratory bias: u(bias)

(from CRM, PT or Recovery tests)

Combined Standard Uncertainty (uc):Estimated as a root sum of squares of the two components

(All expressed as % relative uncertainty)

Cause and Effect diagramsGUM Nordtest

Identify potential uncertainty sources Determine the uncertaintyestimates contribute to:

Input quantities Bias Precision

Reproducibility within the Laboratory RwTwo different standard deviations can be obtained:

• srw - within-laboratory repeatability standard deviationNormally not suitable as uncertainty estimate, since it excludesmajor uncertainty contributions

• sRw - within-laboratory reproducibility standard deviation(long-time repetition).

Method 1:Control sample covering whole analytical

process• If control sample covers the whole analytical process and has a matrix

similar to the samples:* The reproducibility within the laboratory Rw at that concentration level

can simply be estimated from the analyses of the control sample* If the control sample analyses covered a wide concentration range it

should be used.

• * Combined uncertainty for Rw is calculated by the root sum of squares ofthe uncertainty estimates

Method 2: Control samples for different matrices and

concentrations If a synthetic control solution is used for quality control, and the matrix type of the control sample is not similar to the natural samples: • Mean Control Chart:

Long term reproducibility contribution - does not include uncertainties arising from different matrices & sample preparation procedures.

• Range Control chart:Estimate repeatability from different matrices & sample preparation processes.

• Combined uncertainty for Rw is calculated by the root sum of squares of the repeatability and reproducibility uncertainty estimates

Note: The repeatability component is included two times!!

Method 3:Unstable control samples

• If the laboratory does not have access to stable control samples (e.g. measurement of dissolved oxygen):* Estimate uncertainty components from repeatability

obtained from a range control chart * “long-term“ uncertainty component (from batch to batch)

needs expert judgement

Laboratory and Method Bias

• Possible bias - Not easy to determine the true bias of a procedure - it needs a lot of parallel measurements under carefully controlled conditions.

• In this procedure the best estimate of true value is taken into account - Include sample preparation!

Laboratory and Method Bias

Calculate bias by combining two uncertainty components

Because bias will in most cases vary with different matrix & concentration ranges* Ideally use several different reference materials/certified materials & several

PT results* To prevent that bias is grossly under estimated or grossly over estimated

Component accounts

for the average bias of

the laboratory results

Component accounts for

the average uncertainty

of the reference material

RMSbias

• Accounts for average bias of the laboratory:

• Every separate bias value (% difference) is calculated as the value found in the laboratory minus the reference value:biasi = Clabi - Crefi

Clabi - from repeated measurementsCrefi - preferably include several different reference materials or inter laboratory comparisons

u(Cref)

• Accounts for the average uncertainty of the reference values.

• u(Cref) - uncertainty of the nominal/certified value - from COA

If only one CRM is used - include additional term

• u(Cref) - Carried out by inter laboratory comparisonsEstimated from the standard deviation of the participants results

Laboratory and Method BiasFour different possibilities:A) Repeated analysis of the same sample with a reference

procedure - excellent way B) Repeated analysis of CRM (low uncertainty & high reliability) -

which is very much like your real sample. Very suitableC) Repeated inter laboratory comparison measurements

Care should be taken: consensus values cannot be called very reliable → over estimated bias. • Reproducibility standard deviation sR will be used.

D) Repeated recovery data/spiking results

Method 1A - Use of one certified reference material

Uncertainty of certified value u(Cref): 2.21%

Bias of CRM analysed 12 times: Mean: 11.9 100.(11.9 - 11.5)/11.5 = 3.48%

Standard deviation (sbias): 2.2%

Total standard uncertainty(Root mean square method):

Method 1B - Use of several certified reference materials

Mean uncertainty of certified value (Cref):

Total standard uncertainty(Root mean square method):

Bias of 3 CRM’s: 3,48%, -0,9% and 2,4%

Method 2 - Use of PT/Inter laboratory comparison data

Bias of 6 participations: 2%, 7%, -2%, 3%, 6% and 5%

Total standard uncertainty(Root means square method)::

Method 3 - From Recovery Tests

The spike of 0.5 mL was added with amicropipetteUse GUM-approach to quantify

Uncertainty of concentration of spike: 1.0%

Bias of recoveries (6 different matrices: 95 %, 98 %, 97 %, 96 %, 99 % and 96 %

Total standard uncertainty:

Nordtest Approach: Summary

2 Uncertainty components

Recommendations

• Use the Modeling approachIf you have - Competence and time

- Data on all important influencing quantities

• Use the Single-lab validation approachIf you have - Quality control data & results of participation in

ILC-s or CRM analysis

Simplification of GUM approach for Analytical Laboratories recommended!

Examples and literature

• NORDTEST “Handbook for calculation of measurementuncertainty in environmental laboratories“

• EUROLAB technical report contains:

* Detailed examples, relevant standards, guidelines,books, internet websites

* Handbook is available fromhttp://www.nordicinnovation.net/nordtest.cfm as technicalreport No. 537

Thank you!