Detection Limits and Measurements Uncertainties From · PDF fileDetection Limits and...
Transcript of Detection Limits and Measurements Uncertainties From · PDF fileDetection Limits and...
Detection Limits and
Measurements Uncertainties
From FTIR Instruments
Sylvie Bosch-Charpenay,
Peter Zemek, Barbara Marshik
MKS Instruments
On-Line Product Group
Terminology Detection Limit
EPA Method Detection Limit MDL– “Minimum change in concentration of analyte that the
method can detect with 99% confidence limit that a change has occurred”
ASTM D6348 Minimum Detection Limit MDC
Method 320 Detection limit MAU– “Lowest concentration limit for which its overall
fractional uncertainty is less than the analytical uncertainty (e.g. 5%) chosen”
Factors Influencing
Detection Limit
White noise
Biases (due to interferences, poor background)
NOT (or minimally) influenced by calibration,
instrument parameters, etc
MKS Confidential 3
How do we report
Detection Limits?
“Industry-accepted standard” is 3*stdev in N2– Does not account for effect of potential interferent
EPA Method 301??? MDL– Requires 7 low (non-zero) concentration samples +
Student’s t table for statistics
– Requires capability to create low level concentrations
ASTM D6348 MDC– 3 different definitions: which one to use ?
Method 320 MAU
MKS Confidential 4
EPA 40 CFR Part 63 Method 301 - Field Validation of Pollutant Measurements Methods From Various Waste Media
EPA 40 CFR Part 136 - Appendix B
Hubaux, A., Vos, G., Anal. Chem., 42, 8 (1970) 849-855
Terminology (NIST)
Repeatability = precision– “Closeness of agreement between the results of successive
measurements of the same measurand (e.g., cylinder) carried out under
the same conditions of measurement”
Reproducibility– “Closeness of agreement between the results of measurements of the
same measurand (e.g., cylinder) carried out under different conditions of
measurement (e.g., different instrument, different time, different location)”
Accuracy– “Closeness of the agreement between the result of a measurement and
the value of the measurand” (qualitative concept)
Error– “Result of a measurement minus the value of the measurand”
Confidence limit = Uncertainty– “A pair of numbers used to estimate a characteristic of a population, such
that the numbers can be stated with a specified probability that the
population characteristic is included between them”MKS Confidential 5
Terminology Uncertainty NIST
Total Uncertainty – sum of systematic error and random error
– Law of Propagation of Uncertainty: square root of the sum of the squares
Standard uncertainty u – 1-sigma
Expanded uncertainty U = u * k– k =coverage factor for stated level of confidence (k =
1.96 for 95%, k = 2.576 for 99%, k = 3 for 99.76%)
Confidence limit (or confidence interval) – Term used interchangeably with expanded
uncertainty
Random vs Systematic Error
MKS Confidential 7
8
Factors Influencing Uncertainty
Estimated uncertainty with
“good” spectral fit
Assuming “good” fit– i.e. no issues with interfering compounds, noise
much smaller than signal
Using released recipes
Assuming reading >> detection limit– i.e., within ~ 20-100 % of range
Combined effect of T, P, matrix broadening, resolution, frequency– 3% estimated uncertainty
After calibration span – 2% estimated uncertainty
Estimates do not hold at low level and poor spectral fit
What if the spectral fit is not
very good ?
How does goodness-of-fit
relate to overall uncertainty ?
One of the factors only
Loose correlation
Reading with a poor spectral fit might still have
high accuracy (low error)– but it is less likely
Reading with a great spectral fit might have a low accuracy (high error) if P, T, span factor etc… are not correct
However, if we assume all other factors are in spec (P, T, resolution, laser frequency, span factor, etc…), goodness-of-fit gives an indication of uncertainty
ASTM D6348, EPA m320
ASTM 6348 provides 3 different “Minimum
Detection Limits” calculations – MDC1, MDC2, MDC3
Method 320– MAU = detection limit
– FRU, FCU, FAU, FMU = fractional uncertainties
– Uses the largest as the overall uncertainty
Calculations across methods are in some cases similar, but not equivalent
How do ASTM D6348 and EPA
m320 compare ?
MDC1 and MAU are similar but not equivalent – MDC1 is using RMS and MAU is using the surface
area in the normalizing denominator
MDC2 has no equivalent
FRU, FCU, FAU have no equivalent– They are always smaller than MAU and FMU
MDC3 and FMU*reading are similar but not
equivalent – MDC3 is using RMS and FMU is using the surface
area in the normalizing denominator
14
EPA Method 320
Name What it does What it needs Level
MAU ??????
Analytical
Uncertainty
Goodness-of-fit parameter
accounting for noise
Noise spectrum Can be
large
FRU Fractional
Reproducibility
Uncertainty
Goodness-of-fit parameter
accounting for errors in
reproducibility of CTS spectra
CTS spectra taken
before and after
reference spectra
Small 1
FCU Fractional
Calibration
Uncertainty
Accounts for errors in Beer’s Law Reference Spectra Small 2
FAU Fractional
Analytical
Uncertainty
Accounts for errors due to different
pathlengths, T, P between CTS
spectra taken at under different
conditions
CTS done with
different PL, T, P
Small 3
FMU Fractional
Model
Uncertainty
Goodness-of-fit parameter
accounting for errors in the model
to extract multiple concentrations
from overlapping compounds
Sample Can be
large
OFU Overall
Fractional
Uncertainty
Maximum of all Uncertainties
(1) Because Multigas instruments have very similar alignment, and are very stable
(2) Because Multigas calibrations have multi points
(3) Because Multigas instruments have the same pathlength and are run at the same T, P as the reference spectra
15
ASTM D6348
Name What it does What it needs Level
MDC1 Noise-limited
Minimum
Detection Limit
Goodness-of-fit parameter
accounting for noise
Noise spectrum Can be
large
MDC2 Analytical
Algorithm Error
Calculated as 3 * stdev in
spectra with no analyte but with
interferents
Spectra with
interferents but no
analyte
Can be
large
MDC3 Analytical
Algorithm Error
Goodness-of-fit parameter
accounting for errors in the
model to extract multiple
concentrations from
overlapping compounds
Sample spectrum Can be
large
Graphical Goodness-of-fit
Representation
16
Black = calibration spectrum
Red = sample spectrum
Green = noise-only spectrum
Yellow filling:MDC3 or FMU*RS
Green filling:MDC1 or MAU
Calculations
17
Assume same pathlength, T and P between reference and sample
This assumption highlights absolute calculation differences
ASTM MDC1 and m320 MAU
18
NEAi = Noise Equivalent Absorbance at wavenumber i(spectrum in N2)
Aref i = Absorbance of reference spectrum at wavenumber i
Cref = Concentration of reference spectrum
N = number of points in analysis region
For constant Aref i = A, MDC1=MAU
refN
iref
N
i
C
A
NEA
MDC
0
2
0
2
)(
)(
1#refN
i
ref
N
i
C
A
NEA
NMAU
i
)(
)(0
2
NAA
NN 1)()(
00
2
MDC1 vs MAU
MKS Confidential 19
MDC1 = MAU
Absorbance
Wavenumber
Absorbance
MDC1 MAU
Wavenumber
ASTM MDC2
20
P spectra containing interferents but no analyte
P = number of measurements (spectra), minimum 8.
Cave = average concentration for analyte (= analytical bias)
Cp = measured concentration (reading) on spectrum p
P
pave CCP
MDC0
2)(1
32#
ASTM MDC3 and m320 FMU
21
REAi = Residual Equivalent Absorbance at wavenumber i
Aref i = Absorbance of reference spectrum at wavenumber i
Cref = Concentration of reference spectrum
N = number of points in analysis region
RS = reading of sample
For constant Aref i = A, MDC1=FMU*RS
refN
iref
N
i
C
A
REA
MDC
0
2
0
2
)(
)(
3#RS
C
A
REA
NFMUref
N
i
ref
N
i
i
)(
)(0
2
SEC = standard error of estimated
concentration
22
N
ref
ref
iA
CSEC
0
2)(
)1(
)(0
2
2
N
REAN
i
REAi = Residual Equivalent Absorbance at wavenumber i
Aref i = Absorbance of reference spectrum at wavenumber i
Cref = Concentration of reference spectrum
N = number of points in analysis region
refN
iref
N
i
C
A
REA
NSEC
0
2
0
2
)(
)(
)1(
1
Comparison ASTM MDC3 and SEC
23
refN
iref
N
i
C
A
REA
MDC
0
2
0
2
)(
)(
3# refN
iref
N
i
C
A
REA
NSEC
0
2
0
2
)(
)(
)1(
1
SEC calculation is similar to MDC3, except that it includes an additional factor of 1/sqrt(N-1), with N = number of points in analysis region
The additional factor is because the error is assumed to be random instead of systematic
SEC values are much smaller than MDC3, and are loosely correlated to the precision (standard deviation in N2)
Where does the +/- from MG2000
fit in?
24
Similar to FMU*RS except smaller by a factor or sqrt(N)
Similar in value to SEC (but not equivalent, again due to different normalizing denominator)
Closer to a precision value than an uncertainty value
refN
i
ref
N
i
C
A
REA
MG
i
)(
)(
/20000
2
RS
C
A
REA
NFMUref
N
i
ref
N
i
i
)(
)(0
2
What do I need for DL
calculation and if possible
uncertainty calculation ??
MKS Confidential 25
MKS Analysis Validation Utility
provides needed parameters
MDC1, MDC2 and MDC3 calculated
MAU and FMU calculated
FRU, FCU, FAU not calculated – They are smaller than MAU and FMU
– FRU small because Multigas instruments have very
similar alignment and are very stable
– FCU small because of multi-points calibrations
– FAU small because Multigas instruments have the
same pathlength and are run at the same T, P as the
reference spectra
SEC and +/- ??? MKS Confidential 26
27
Parameters in MKS’s Analysis
Validation Utility
MDC1, MAU MDC2 MDC3, FMU*Reading
Purpose Quantify
goodness-of-fit
for noise
Quantify detection
limit from precision
Quantify goodness-of-
fit for sample
Required
spectra
One noise-only
spectrum
At least eight
interference-only
spectra
Sample Spectrum
Comments on
required
spectra
Easy to obtain,
can be the first
spectrum in N2
taken after a
background
Can be difficult to
obtain for each
instrument
Available
Drawbacks Assumes all
error is
systematic
Combination of
mostly random
(noise) and some
systematic
(interferent) error
Assumes all error is
systematic
Example
28
similar similar
Typically lower than MDC3 or FMU*R
Which MDC to use for
Detection Limit?
MDC2 likely slightly lower than “real” DL – It does not include a measure of bias under sample
conditions
– Best option for DL
MDC3 likely significantly higher than “real” DL – It assumes all error is systematic.
– MDC3 provides more information on the uncertainty
than detection limit
MDC1 not directly related to “real” DL – Uses noise-only data
– Assumes all error is systematicMKS Confidential 29
Estimated Detection Limit
~ DL = MDC2 + bias– Provides “rule of thumb” values
– Value slightly more conservative than MDC2
– No goodness-of-fit parameters used in calculation
MDC2 – Estimate of mostly random error (due to noise) and
some systematic error (due to interferents) at zero
Bias – Average bias for analyte in the interferents-only spectra
used for MDC2)
– Estimate of systematic error due to interferents
MKS Confidential 30
Can any MDC be used for
Confidence Limit?
MDC2 likely lower than “real” CL – It does not include a measure of bias under sample
conditions
MDC3 likely higher than “real” CL – It assumes all the error from the goodness-of-fit yields
a systematic error in the reading
MDC1 not directly related to “real” CL – Uses noise-only data
MKS Confidential 31
Can we calculate a “Rule of
Thumb” Confidence Limit ?
Systematic Error = Max (MDC3-MDC1-C, bias) – MDC3-MDC1 is a goodness-of-fit estimate of sample
systematic error without the effect of noise
– C is typical value for goodness-of-fit for a “good fit”
– Bias is estimate of systematic error due to interferents
Random Error = MDC2
Best level for uncertainty is 2%– “Good fit”
– Readings in 20-100% of range
MKS Confidential 32
“Rule of Thumb” Confidence Limit
33
~CL = max [max (MDC3-MDC1-C, bias) + MDC2, 2%]
Estimated Systematic Error fromsample goodness-of-fit
Estimated Systematic Error frominterferentspectra
Estimated Systematic Error from Sample goodness-of-fit
Estimated Random (+ some Systematic) Error
Estimated Error With “good” goodness-of-fit (but errors in T, P,…)
Estimated Error from sample goodness-of-fit and interferents
Estimated CL and DL
MKS Confidential 34
Color-coded for faster evaluation
Recommendations
MDC2 is best value to use for detection limit– Choice of interference spectra is critical
– Nitrogen spectra would only give DL due to noise
A more conservative value for DL is MDC2 + bias
MDC3 is a rough estimate for uncertainty– Because it assumes that all error is systematic
– Does not account for significant reduction of random
error in spectral analysis
– Does not account for possible interferent bias
~CL is a more reliable estimate for uncertainty– Based on goodness-of-fit, precision, and typical
parameters uncertainties
– Truly only an estimate ! Could still be “off” MKS Confidential 35
Big Question:
which spectra for MDC2
should be acceptable ?
Best choice– Interferents at (multiple) levels covering levels seen in
sample
– Different interferents for each analyte
– Collect a minimum of 8 (consecutive, non-
consecutive??) spectra
Interference spectra taken on other instrument ?
H2O and CO2 interference spectra taken on other
instrument ?
N2 spectra taken on instrument?
MKS Confidential 36
Recommended Procedure to
Generate DLs
Setup instrument – Allow instrument to equilibrate
– Load recipe (calibrations + instrument parameters)
Run interferent gases (no analyte) – Interferents at levels same as in sample
– Collect a minimum of 8 (consecutive??) spectra
– Calculate stdev and bias (average concentration of
analyte, which should be close to 0)
Calculate DL=MDC2 (3 * stdev) and ~DL=MDC2
+ bias
Determine if DL and ~DL under those conditions
meets regulatory requirements MKS Confidential 37