An Introduction to Expressing Uncertainty in Measurement
PALCAN Team Leaders’ SeminarOttawa, Nov. 2002
Mike Ouellette, CLAS Technical Advisor
2002-10-25 M. Ouellette 1
National ResearchCouncil Canada
Conseil nationalde recherches Canada
Institute for NationalMeasurement Standards
Institut des étalonsnationaux de mesure
2002-10-25M. Ouellette 2
Overview
• Why the Big Deal about Uncertainty?
• Background Stats: The Basics -- Nothing Fancy- Minimal stats jargon- Minimal algebraic notations- No system modeling- No calculus- Promise!
• How to apply it (on the back of an envelope)
• A Worked Example
2002-10-25M. Ouellette 3
Why the Big Deal?
• 17025
• VIM (re. Traceability)
No stated uncertainty = No Traceability !
2002-10-25M. Ouellette 4
Don’t get Sucked in!
• ESTIMATE uncertainty only to the extent that you need, don’t make a science of it.
• Accept that you’ll never be certain about uncertainty.
• Focus mostly on the “biggies”; don’t sweat the small stuff (much).
2002-10-25M. Ouellette 5
The Background Stats:The Standard Deviation
• Repeat measurements aren’t identical.
• Stdev tells us how widely they are dispersed about the mean.
• Approx. 2/3 (i.e., 68%) of all readings fall within 1 stdev of the mean.
-1 S
tdev
+1
Std
ev
-3 -2 -1 0 1 2 3
Differences from the mean
68.3%
2002-10-25M. Ouellette 6
The Background Stats:The Standard Deviation
• Approx. 95.5% of all readings fall within 2 stdev of the mean.
+2S
tdev
-2S
tdev
-3 -2 -1 0 1 2 3
Differences from the mean
95.5%
1. Get average Reading,X X - average (X-average)^2
2. Deviations from 1,000,000,043 -7 49average 1,000,000,055 5 25
1,000,000,055 5 253. Square those 1,000,000,051 1 1deviations 1,000,000,058 8 64
1,000,000,043 -7 494. Sum the squares 1,000,000,045 -5 255. Divide by (n-1) 1,000,000,045 -5 256. Take sqr root 1,000,000,057 7 49
1,000,000,048 -2 4=average() 1,000,000,050 Sum()= 316
=Count(), n 10Excel Stdev Fnc:15.08 n - 1 9Nominal: Sum / (n-1) 35.111.00E+09 Stdev= sqrt[Sum/(n-1)] 5.93
To Calculate Standard Deviation
Step 1
Step 2 Step 3
Step 4
Step 5
Step 6
1. Get average Reading,X X - average (X-average)^2
2. Deviations from 1,000,000,043 -7 49average 1,000,000,055 5 25
1,000,000,055 5 253. Square those 1,000,000,051 1 1deviations 1,000,000,058 8 64
1,000,000,043 -7 494. Sum the squares 1,000,000,045 -5 255. Divide by (n-1) 1,000,000,045 -5 256. Take sqr root 1,000,000,057 7 49
1,000,000,048 -2 4=average() 1,000,000,050 Sum()= 316
=Count(), n 10Excel Stdev Fnc:15.08 n - 1 9Error in Stdev is: 155% Sum / (n-1) 35.11
Nominal = 1.00E+09 Stdev= sqrt[Sum/(n-1)] 5.93try 4.0e8, 4.2e9, 10e12, 11e12
Step 1
Step 2 Step 3
Step 4
Step 5
Step 6
2002-10-25M. Ouellette 9
Experimental Std Deviation of the Mean
• The stdev describes the spread of values in an individual set of measurements. What if we took several different sets of measurements?
• The mean of each set of measurements would vary.
• The spread of the means is given by the experimental std deviation of the mean (stdm).
• To predict stdm from a set of n replicate measurements, divide the stdev by (n).
Stdm = stdev / (n)
2002-10-25M. Ouellette 10
Experimental Std Deviation of the Mean: An Example
• If standard deviation, Stdev, is 5.93, and
• if number of replicate measurements, n, is 10, then
• experimental standard deviation of the mean, stdm, is
Stdm = Stdev / sqrt (n)
= 5.93 / sqrt (10)
= 1.88
2002-10-25M. Ouellette 11
Experimental Std Deviation of the Mean: So what?
• The stdm feeds directly into our uncertainty budget whenever we estimate uncertainty in the mean of a set of random repeated measurements.
Very Useful!More on that later
2002-10-25M. Ouellette 12
Distributions of Measurements
• The spread of a set of measurements can take on different frequency distributions. Examples:
2002-10-25M. Ouellette 13
Distributions of Measurements: Normal Distribution (Gaussian)
• Most values fall near the mean.
• Progressively fewer values falling further from the mean.
• Very common distribution in nature.
2002-10-25M. Ouellette 14
Distributions of Measurements: Uniform Distribution (Rectangular)
• Measurements distributed equally across the interval.
• Reasonably good choice when you don’t have a clue!
2002-10-25M. Ouellette 15
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Uncertainty in their calibration Sensitivity to transportation & handling
Long term driftDesign issues (such as unequal lengths of equal arm balances)
Stability during measurement ParallaxesResolution & quality of their scales Interpolation between calibration pointsLinearity Reading systemDigitization Transporation
Reference Standards & Measurement Equipment; e.g…..
Not a complete list. Not all necessarily apply. Ditto for subsequent slides.
Make a List of Contributors. Consider ...
2002-10-25M. Ouellette 16
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Absolute temperature ContaminationTime variance in temperature IlluminationSpatial gradiant in temperature Air pressure, composition, flowHeat radiation from operator, etc. GravityHumidity Electromagnetic interferenceNoise, Vibration Transients in power supply
Environmental Conditions; e.g…..
2002-10-25M. Ouellette 17
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Warming up Loading effectsCosine errors Cabling, shielding, filteringOptical aperture Stiffness/rigidity of mechanical systemsParasitic voltages Properties of measurement probesCurrent leakage
Measurement Setup; e.g…..
2002-10-25M. Ouellette 18
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Sensitivity to stresses of measurement CleanlinessSurface roughness Internal strengthConductivity Distortion during measurement
Weight, size, shapeSelf heating (e.g., current measurement)
Magnetism No. of terminalsStability Orientation
Measurement Object; e.g…..
2002-10-25M. Ouellette 19
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Repeatability, conditioning Choice of reference & apparatusNo. & order of measurements Clamping, fixturing, probingDuration of measurements Drift checkChoice of principle of measurements Reversal measurementsMagnetism Multiple redundancy checksAlignment Strategy
Measurement Process; e.g…..
2002-10-25M. Ouellette 20
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Rounding FilteringAlgorithms InterpolationNo. of signif. digits in calculat'n ExtrapolationSampling Outlier handling
Software & Calculations; e.g…..
2002-10-25M. Ouellette 21
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
E.g., definition of "Diameter" for dimensional measurements of objects that are not perfectly round.
Definition of the Measurement Characteristic; e.g…..
D= 0.5 (Max + Min) ?
D= Mean of many measurements?
D= Other? (e.g., Bullets --> Max diameter?)
2002-10-25M. Ouellette 22
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Uncertainty in knowledge of the physical values used
Various properties of the working, measuring instrument
Temperature coefficient Various properties of ambient airPower coefficient Local force of gravity
Physical Constants and Conversion Factors; e.g…..
2002-10-25M. Ouellette 23
Estimating Uncertainty Step 1:Identify the Contributors to Uncertainty
Differences in judgements of different operators. Metrologist Effects; e.g…..
Note: Shouldn’t need to include uncertainty due to “Operator Error.” This is an aberration that should be corrected before measurement.
Has System Stabilised?
0
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
0 5 10 15 20
Time of Measurement, Minutes
Me
as
ure
d V
alu
e
How would you integrate shoulder peaks, below?
2002-10-25M. Ouellette 24
Estimating Uncertainty Step 1:Summary of Some Contributors to Uncertainty
• Reference Standards & Measurement Equipment
• Environmental Conditions
• Measurement Setup
• Measurement Object
• Measurement Process
• Software & Calculations
• Definition of the Measurement Characteristic
• Physical Constant & Conversion Factors
• Metrologist Effects
2002-10-25M. Ouellette 25
Step 2: Decide on the Uncertainty Units
Units may be:
• ppm or % of result
• units of measurement
• other
Generally doesn’t matter what you pick; just be consistent: Apples + Apples = Apples.
Everything gets boiled down to a standard deviation of the mean or equivalent (called standard uncertainty).
2002-10-25M. Ouellette 26
Step 3: Estimate the Magnitude of the Uncertainty Contributors 1) Type A, Normal
Type A contributors to uncertainty are those that you have statistical data for. Use this data if you have it.
E.g., For Standard Uncertainty in the mean of repeated measurements (preferably 10 or more) use stdm, the experimental standard deviation of the mean.
2002-10-25M. Ouellette 27
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B
Type B contributors to uncertainty are those that you have no statistical data for; e.g.,
• Manufacturer’s specification
• Professional judgement
• Uncertainty in cal certificate for your reference standard
GUM: There’s “no substitute for critical thinking, intellectual honesty, and professional skill.”
2002-10-25M. Ouellette 28
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B
Gather information from wherever we can.
Boil down the uncertainty info to a standard uncertainty.
But info doesn’t always come at level of confidence of 68.3%. It usually comes at a higher level of confidence. It’s called an Expanded Uncertainty when coming from a normal distribution.
2002-10-25M. Ouellette 29
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Normal
For a normal distribution, no problem!
Divisor Level ofConfidence
0.676 50%
1 68.27%
1.645 90%
1.960 95%
2 95.45%
2.576 99%
3 99.73%
Simply divide the Expanded Uncertainty by a Divisor, depending upon the level of confidence at which the Expanded Uncertainty is given.
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Normal
E.g., Cal certificate for our mass standard states:
“The uncertainty in the reported mass is 26 mg at a level of confidence of 95% assuming a normal distribution.”
The standard uncertainty is….
26 mg 1.960 = 13 mg
Divisor Level ofConfidence
1.960 95%
2 95.45%
2.576 99%
3 99.73%
2002-10-25M. Ouellette 31
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Normal
E.g., Cal certificate for our resistor states:
“The expanded uncertainty in the reported resistance is 30 with a coverage factor, k, of 3.”
The standard uncertainty is….
30 3 = 10 Here, the “coverage factor” is another word for “divisor”.
2002-10-25M. Ouellette 32
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Uniform
But not all distributions are normal.
Equal probability that true value is anywhere between interval a- to a+ .
Rectangular (I.e., Uniform) distributions also very common.
2002-10-25M. Ouellette 33
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Uniform
For Rectangular (I.e., Uniform) distributions , the Standard Uncertainty, u, is the area shaded in grey.
Standard uncertainty, u, is
the complete interval, a, divided by 3 or ~1.73.
2002-10-25M. Ouellette 34
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Uniform
E.g., Gauge readability (resolution).
If smallest subdivision is 5 measurement units, andif we can “eyeball” the gauge to within 1/2 a subdivision, then the gauge’s true value can be anywhere in the interval a = ± 5 / 2 = ± 2.5 with equal probability.
Standard Uncertainty, u, = ± 2.5 / 3 = ± 1.4 meas. unitsor, if you prefer , u = ± 5 / 12 = ± 1.4 meas. units.
0 105
2002-10-25M. Ouellette 35
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Uniform
E.g., Ambient temperature
Approx. equal probability that temperature is anywhere between control limits a+ = 24°C and a- = 22°C.
Standard Uncertainty, u, = ± 1°C / 3 = ± 0.58°C
Control Limits = 23°C ± 1°C
a-
a+
TIME
TEM
PERA
TURE
2002-10-25M. Ouellette 36
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Uniform
u = ± 0.58°C x (0.5 ppm per °C ) = 0.29 ppm of reading.
But remember that we need to express all uncertainties in the same units!
Control Limits = 23°C ± 1°C
a-
a+
TIME
TE
MP
ER
AT
UR
E
If we’re expressing uncertainty in ppm of reading of a resistor, and if temperature coefficient of resistor is 0.5 ppm/°C, then std uncertainty becomes:
u = ± 0.58°C
2002-10-25M. Ouellette 37
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Unknown Distrib’n
Clueless about the distribution?
Standard Uncertainty, u, = ± (Max Interval width) / 3
Usually safe to treat it as a Rectangular
(Uniform) distribution
2002-10-25M. Ouellette 38
Step 3: Estimate the Magnitude of the Uncertainty Contributors 2) Type B, Unknown Distrib’n
Standard Uncertainty, u, = ± 0.02 mm / 3 = ± 0.012 mm
E.g., An incomplete instrument specification:
“The anvils of the micrometer are certified to be parallel within ± 0.02 mm.”
• No coverage factor or level of confidence,
• No distribution --> is spec. based on a stdev?
2002-10-25M. Ouellette 39
Step 4: Document, Document, Document!
Your uncertainty budget is a living document that is revised as your measurement process changes and as your understanding of it improves. Therefore...
Write down how you arrived at each of the uncertainty estimates that you have listed.For example of documenting, see last slide.
File this info with the uncertainty budget.
2002-10-25M. Ouellette 40
Any questions so far?
?
2002-10-25M. Ouellette 41
Recap on Steps 1 to 4
1. Identify the contributors to measurement uncertainty.
2. Decide on consistent uncertainty units.
3. Estimate magnitude of each uncertainty contributor, and express each as a standard uncertainty.
The hardest part is done!
4. Document the basis for your estimates.
2002-10-25M. Ouellette 42
Step 5: Combine the standard uncertainties into one number
Use the RSS method:
• Take the square of each std uncertainty, u
• Add up the squares
• Take the square root of the sum
This gives combined standard uncertainty, uc, of your measurement, with ~68% confidence.
2002-10-25M. Ouellette 43
Step 5: Combine the standard uncertainties into one number
E.g., std uncertainty, u, in ppm of reading:
1. Cal of ref std: 0.75 ppm
2. Long term drift of ref std: 1.16 ppm
3. Repeatability, n=5: 0.07 ppm
Combined std uncertainty, uc, =
sqrt ( 0.752 + 1.162 + 0.072 ) = 1.4 ppm
i.e., ~68% confident that true value is within± 1.4 ppm of the reported result.
2002-10-25M. Ouellette 44
Step 6: Expand the combined the standard uncertaintyWhat if we want to report uncertainty with a different level of confidence rather than ~68% ?
No problem! Simply multiply combined std uncertainty, uc, by an appropriate coverage factor, k. The product is called Expanded Combined Uncertainty, Uc.
Coveragefactor, k
Level ofConfidence
0.676 50%
1 68.27%
1.645 90%
1.960 95%
2 95.45%
2.576 99%
3 99.73%
Step 6: Expand the combined the standard uncertainty
Coveragefactor, k
Level ofConfidence
0.676 50%
1 68.27%
1.645 90%
1.960 95%
2 95.45%
2.576 99%
3 99.73%
Exercise: Combined std unc,uc, is 1.4 ppm of rdg;
We want to report uncertainty with a level of confidence of “approximately 95%.”
Expanded Combined Uncertainty,Uc, = 1.4 ppm x 2 = 2.8 ppm
2002-10-25M. Ouellette 46
Step 7: Reality check
The preceding simplified approach is based upon several assumptions, including:
• Dominant contributor(s) to uncertainty are known with reasonable certainty;
- For Type A: Have enough data ( 10 points)- For Type B: No wild guessing
2002-10-25M. Ouellette 47
Step 7: Reality check
So identify the dominant contributor(s).
1. Cal of ref std: 0.75 ppm
2. Long term drift of ref std: 1.16 ppm
3. Repeatability, n=5: 0.071 ppm
Dominant contributor is drift. If wild guess or only few data points, then get a more reliable (otherwise a more conservative) estimate if possible. Otherwise follow detailed steps in the GUM on degrees of freedom.
2002-10-25M. Ouellette 48
Step 7: Reality check
1. Cal of ref std: 0.75 ppm
2. Long term drift of ref std: 1.12 ppm
3. Repeatability, n=5: 0.071 ppm
Small n (of 5) for repeatability is good enough because this contributor isn’t dominant.
Focus on the biggies: Long term drift & ref std cal.
2002-10-25M. Ouellette 49
Step 8: Reporting the result
“The measured result is 10,000.051 ± 2.8 ppm for a level of confidence of approximately 95%, k=2.”
“The measured result is 10,000.051 ± 0.028 for a level of confidence of approximately 95%, k=2.”
Better...
Better yet...
“The measured result is 10,000.051 ± 0.028 . The reported uncertainty is expanded using a coverage factor k=2 for a level of confidence of approximately 95%, assuming a normal distribution.”
Not Recommended...
2002-10-25M. Ouellette 50
Recap on all Steps
1. Identify the contributors to measurement uncertainty.
2. Decide on consistent uncertainty units.
3. Estimate magnitude of each uncertainty contributor, and express each as a standard uncertainty.
4. Document the basis for your estimates.
5. Combine the standard uncertainties.
6. Expand combined std uncertainty to represent desired confidence.
7. Reality check.
8. Report the result.
Source of Uncertainty Value ppm
ProbabilityDistribution
DivisorStandard
Uncertainty ppm
Repeatability 0.071 Normal 1 0.071
Calibration uncertainty ofstandard resistor
1.5 Normal 2 0.75
Uncorrected drift of standardresistor since last calibration
2.0 Rectangular 3 1.155
Effect of temperature of oilbath
0.5 Rectangular 3 0.289
Effect of voltmeter onmeasurement of standard
resistor0.2 Rectangular 3 0.115
Effect of voltmeter onmeasurement of unknown
resistor0.2 Rectangular 3 0.115
Combined standarduncertainty, uc
Assumed normal(k=1)
1.418
Expanded combineduncertainty, Uc
Assumed normal(k=2)
2.9
A more Complete Example Uncertainty BudgetFor cal of a 10 k resistor by voltage intercomparison in oil
A more Complete Example Uncertainty BudgetWhere the numbers came from
• Repeatability from 5 random repeat measurements in ppm from nominal:+10.4, +10.7, +10.6, +10.3, +10.5 --> stdm of 0.071 ppm.
• Cal of std resistor from cal certificate saying uncertainty in reported result is 1.5 ppm at a level of confidence of approximately 95% (k=2);1.5 ppm / 2 = 0.75 ppm.
• Drift of ref std is based upon several years of calibration history and comparisions with a check standard. A trend line was fitted to the historical data and was used to correct the ref std’s value at the date that it was used. The estimated uncertainty in the correction is 2 ppm based upon an analysis of the regression residuals.
• Temperature effects were estimated from the control limits of the stirred oil bath ( 0.2°C) and the manufacturer’s specs for the resistors’ temperature coefficient (worst case 2.5 ppm/°C). Bath homogeneity was measured and found to be negligible (< 0.05°C).
• Voltmeter effects are limited to linearity and resolution because it is used as a transfer device only. According to the mfr’s specs, the combined effects of linearity & resolution are 0.2 ppm in the range of measurement. The uncertainty is considered twice: once for each resistor. Go back to Step 4
Top Related