Measurement Uncertainties and Inconsistencies Dr. Richard Young Optronic Laboratories, Inc.
-
Upload
monica-warren -
Category
Documents
-
view
216 -
download
0
Transcript of Measurement Uncertainties and Inconsistencies Dr. Richard Young Optronic Laboratories, Inc.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Measurement Measurement Uncertainties and Uncertainties and InconsistenciesInconsistencies
Measurement Measurement Uncertainties and Uncertainties and InconsistenciesInconsistencies
Dr. Richard YoungDr. Richard Young
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Dr. Richard YoungDr. Richard Young
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
IntroductionIntroductionIntroductionIntroduction
The concept of accuracy is generally The concept of accuracy is generally understood.understood.
“…“…an accuracy of 1%.”an accuracy of 1%.” What does this mean?What does this mean?
•99% inaccurate?99% inaccurate?
The concept of accuracy is generally The concept of accuracy is generally understood.understood.
“…“…an accuracy of 1%.”an accuracy of 1%.” What does this mean?What does this mean?
•99% inaccurate?99% inaccurate?
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
IntroductionIntroductionIntroductionIntroduction
The confusion between the concept The confusion between the concept and the numbers has lead national and the numbers has lead national laboratories to abandon the term laboratories to abandon the term accuracy.accuracy. Except in qualitative terms e.g. Except in qualitative terms e.g.
high accuracy.high accuracy.The term now used is uncertainty.The term now used is uncertainty.
“…“…an uncertainty of 1%.”an uncertainty of 1%.”
The confusion between the concept The confusion between the concept and the numbers has lead national and the numbers has lead national laboratories to abandon the term laboratories to abandon the term accuracy.accuracy. Except in qualitative terms e.g. Except in qualitative terms e.g.
high accuracy.high accuracy.The term now used is uncertainty.The term now used is uncertainty.
“…“…an uncertainty of 1%.”an uncertainty of 1%.”
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
IntroductionIntroductionIntroductionIntroduction
Sometimes…Sometimes… Users do not know the Users do not know the
uncertainty of their results.uncertainty of their results. They interpret any variations as They interpret any variations as
inconsistencies.inconsistencies.
Sometimes…Sometimes… Users do not know the Users do not know the
uncertainty of their results.uncertainty of their results. They interpret any variations as They interpret any variations as
inconsistencies.inconsistencies.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Uncertainty vs. Uncertainty vs. InconsistencyInconsistency
Uncertainty vs. Uncertainty vs. InconsistencyInconsistency
Laboratories give different values, Laboratories give different values, but the difference is within their but the difference is within their combined uncertainties…combined uncertainties… Pure chance.Pure chance.
Laboratories give different values, Laboratories give different values, and the difference is outside their and the difference is outside their combined uncertainties…combined uncertainties… Inconsistency.Inconsistency.
Laboratories give different values, Laboratories give different values, but the difference is within their but the difference is within their combined uncertainties…combined uncertainties… Pure chance.Pure chance.
Laboratories give different values, Laboratories give different values, and the difference is outside their and the difference is outside their combined uncertainties…combined uncertainties… Inconsistency.Inconsistency.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
What is uncertainty?What is uncertainty?What is uncertainty?What is uncertainty?
“…“…an uncertainty of 1%.”an uncertainty of 1%.” But is 1% the maximum, average But is 1% the maximum, average
or typical variation users can or typical variation users can expect?expect?
Uncertainty is a statistical quantity Uncertainty is a statistical quantity based on the average and standard based on the average and standard deviation of data.deviation of data.
“…“…an uncertainty of 1%.”an uncertainty of 1%.” But is 1% the maximum, average But is 1% the maximum, average
or typical variation users can or typical variation users can expect?expect?
Uncertainty is a statistical quantity Uncertainty is a statistical quantity based on the average and standard based on the average and standard deviation of data.deviation of data.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
StatisticsStatisticsStatisticsStatistics
““There are three types of lies: There are three types of lies: lies, damned lies and statistics.lies, damned lies and statistics.””
--attributed to Benjamin Disraeliattributed to Benjamin Disraeli
““There are three types of lies: There are three types of lies: lies, damned lies and statistics.lies, damned lies and statistics.””
--attributed to Benjamin Disraeliattributed to Benjamin Disraeli
““The difference between statistics and The difference between statistics and experience is time.”experience is time.”
--Richard YoungRichard Young
““The difference between statistics and The difference between statistics and experience is time.”experience is time.”
--Richard YoungRichard Young
Statistics uses past experience to predict Statistics uses past experience to predict likely future events.likely future events.
Statistics uses past experience to predict Statistics uses past experience to predict likely future events.likely future events.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
StatisticsStatisticsStatisticsStatistics
We toss a coin:We toss a coin: It is equally likely to be heads or It is equally likely to be heads or
tails.tails.We toss two coins at the same time:We toss two coins at the same time:
There are 4 possible outcomes:There are 4 possible outcomes:• Head + Head• Head + Tail• Tail + Head• Tail + Tail
We toss a coin:We toss a coin: It is equally likely to be heads or It is equally likely to be heads or
tails.tails.We toss two coins at the same time:We toss two coins at the same time:
There are 4 possible outcomes:There are 4 possible outcomes:• Head + Head• Head + Tail• Tail + Head• Tail + Tail
These 2 are the same These 2 are the same and hence twice as and hence twice as
likely to happen as the likely to happen as the others.others.
These 2 are the same These 2 are the same and hence twice as and hence twice as
likely to happen as the likely to happen as the others.others.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
StatisticsStatisticsStatisticsStatistics Now let us throw Now let us throw
10 coins.10 coins. There are 1024 There are 1024
possibilities (2possibilities (21010).). What if we threw What if we threw
them 1024 times, them 1024 times, and counted each and counted each time a certain time a certain number of heads number of heads resulted…resulted…
Now let us throw Now let us throw 10 coins.10 coins.
There are 1024 There are 1024 possibilities (2possibilities (21010).).
What if we threw What if we threw them 1024 times, them 1024 times, and counted each and counted each time a certain time a certain number of heads number of heads resulted…resulted…
0
50
100
150
200
250
300
0 1 2 3 4 5 6 7 8 9 10
Number of Heads
Nu
mb
er o
f O
ccu
rren
ces
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
StatisticsStatisticsStatisticsStatisticsAlthough the Although the
outcome of each outcome of each toss is random…toss is random…
...not every result ...not every result is equally likely.is equally likely.
If we divide the If we divide the number of number of occurrences by the occurrences by the total number of total number of throws…throws… We get We get probability.probability.
Although the Although the outcome of each outcome of each toss is random…toss is random…
...not every result ...not every result is equally likely.is equally likely.
If we divide the If we divide the number of number of occurrences by the occurrences by the total number of total number of throws…throws… We get We get probability.probability.
0
50
100
150
200
250
300
0 1 2 3 4 5 6 7 8 9 10
Number of Heads
Nu
mb
er o
f O
ccu
rren
ces
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
StatisticsStatisticsStatisticsStatisticsHere is the same Here is the same
plot, but shown as plot, but shown as probability.probability.
Probability is just a Probability is just a number that number that describes the describes the likelihood between:likelihood between: 0 = never 0 = never happenshappens
1 = always 1 = always happenshappens
Here is the same Here is the same plot, but shown as plot, but shown as probability.probability.
Probability is just a Probability is just a number that number that describes the describes the likelihood between:likelihood between: 0 = never 0 = never happenshappens
1 = always 1 = always happenshappens
0
0.05
0.1
0.15
0.2
0.25
0.3
0 1 2 3 4 5 6 7 8 9 10
Number of Heads
Pro
bab
ility
of
Occ
urr
ence
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
StatisticsStatisticsStatisticsStatisticsGauss described a Gauss described a
formula that formula that predicted the predicted the shape of any shape of any distribution of distribution of random events.random events. Shown in redShown in red
It uses just 2 It uses just 2 values:values: The averageThe average The standard The standard deviationdeviation
Gauss described a Gauss described a formula that formula that predicted the predicted the shape of any shape of any distribution of distribution of random events.random events. Shown in redShown in red
It uses just 2 It uses just 2 values:values: The averageThe average The standard The standard deviationdeviation
0
0.05
0.1
0.15
0.2
0.25
0.3
0 1 2 3 4 5 6 7 8 9 10
Number of Heads
Pro
bab
ility
of
Occ
urr
ence
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
StatisticsStatisticsStatisticsStatistics Now throw 100 coins…Now throw 100 coins… Now throw 100 coins…Now throw 100 coins…
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0 10 20 30 40 50 60 70 80 90 100
Number of Heads
Pro
bab
ility
of
Occ
ure
nce
We have an average We have an average = 50= 50
We have an average We have an average = 50= 50
And a standard And a standard deviation = 5deviation = 5
And a standard And a standard deviation = 5deviation = 5
And the familiar And the familiar bell-shaped bell-shaped distribution.distribution.
And the familiar And the familiar bell-shaped bell-shaped distribution.distribution.
The The Gaussian Gaussian curve fits curve fits exactly.exactly.
The The Gaussian Gaussian curve fits curve fits exactly.exactly.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
ConfidenceConfidenceConfidenceConfidence Now throw 100 coins…Now throw 100 coins… Now throw 100 coins…Now throw 100 coins…
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0 10 20 30 40 50 60 70 80 90 100
Number of Heads
Pro
bab
ility
of
Occ
ure
nce
Since the total Since the total probability must =1, probability must =1,
the standard the standard deviation marks off deviation marks off certain probabilities.certain probabilities.
Since the total Since the total probability must =1, probability must =1,
the standard the standard deviation marks off deviation marks off certain probabilities.certain probabilities.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0 10 20 30 40 50 60 70 80 90 100
Number of Heads
Pro
bab
ility
of
Occ
ure
nce
ConfidenceConfidenceConfidenceConfidence Now throw 100 coins…Now throw 100 coins… Now throw 100 coins…Now throw 100 coins…
Since the total Since the total probability must =1, probability must =1,
the standard the standard deviation marks off deviation marks off certain probabilities.certain probabilities.
Since the total Since the total probability must =1, probability must =1,
the standard the standard deviation marks off deviation marks off certain probabilities.certain probabilities.
About 67% of About 67% of all results lie all results lie
within within 1 1 standard standard deviation.deviation.
About 67% of About 67% of all results lie all results lie
within within 1 1 standard standard deviation.deviation.
““I am 67% confident that a I am 67% confident that a new throw will give new throw will give
between 45 and 55 heads.”between 45 and 55 heads.”
““I am 67% confident that a I am 67% confident that a new throw will give new throw will give
between 45 and 55 heads.”between 45 and 55 heads.”
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
0
0.01
0.02
0.03
0.04
0.05
0.06
0.07
0.08
0.09
0 10 20 30 40 50 60 70 80 90 100
Number of Heads
Pro
bab
ilit
y o
f O
ccu
ren
ce
ConfidenceConfidenceConfidenceConfidence Now throw 100 coins…Now throw 100 coins… Now throw 100 coins…Now throw 100 coins…
Since the total Since the total probability must =1, probability must =1,
the standard the standard deviation marks off deviation marks off certain probabilities.certain probabilities.
Since the total Since the total probability must =1, probability must =1,
the standard the standard deviation marks off deviation marks off certain probabilities.certain probabilities.
About 95% of About 95% of all results lie all results lie
within within 2 2 standard standard
deviations.deviations.
About 95% of About 95% of all results lie all results lie
within within 2 2 standard standard
deviations.deviations.
““I am 95% confident that a I am 95% confident that a new throw will give new throw will give
between 40 and 60 heads.”between 40 and 60 heads.”
““I am 95% confident that a I am 95% confident that a new throw will give new throw will give
between 40 and 60 heads.”between 40 and 60 heads.”
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data
Real data, such as the result of a Real data, such as the result of a measurement, is also characterized measurement, is also characterized by an average and standard by an average and standard deviation.deviation.
To determine these values, we must To determine these values, we must make measurements.make measurements.
Real data, such as the result of a Real data, such as the result of a measurement, is also characterized measurement, is also characterized by an average and standard by an average and standard deviation.deviation.
To determine these values, we must To determine these values, we must make measurements.make measurements.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data NVIS radiance measurements are unusual.NVIS radiance measurements are unusual.
The signal levels at longer wavelengths The signal levels at longer wavelengths can be very low – close to the dark can be very low – close to the dark level of the system.level of the system.
The signal levels at longer wavelengths The signal levels at longer wavelengths dominate the NVIS radiance result.dominate the NVIS radiance result.
The uncertainty in results close to the The uncertainty in results close to the dark level can be dominated by PMT dark level can be dominated by PMT noise.noise.
Therefore: Variations in NVIS results can Therefore: Variations in NVIS results can be dominated by PMT noise.be dominated by PMT noise.
NVIS radiance measurements are unusual.NVIS radiance measurements are unusual. The signal levels at longer wavelengths The signal levels at longer wavelengths
can be very low – close to the dark can be very low – close to the dark level of the system.level of the system.
The signal levels at longer wavelengths The signal levels at longer wavelengths dominate the NVIS radiance result.dominate the NVIS radiance result.
The uncertainty in results close to the The uncertainty in results close to the dark level can be dominated by PMT dark level can be dominated by PMT noise.noise.
Therefore: Variations in NVIS results can Therefore: Variations in NVIS results can be dominated by PMT noise.be dominated by PMT noise.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data
The net signal from the PMT is used The net signal from the PMT is used to calculate the spectral radiance.to calculate the spectral radiance.
Dark current, which is subtracted Dark current, which is subtracted from each current reading during a from each current reading during a scan, contains PMT noise.scan, contains PMT noise.
Scans at low signals contain PMT Scans at low signals contain PMT noise.noise.
The net signal from the PMT is used The net signal from the PMT is used to calculate the spectral radiance.to calculate the spectral radiance.
Dark current, which is subtracted Dark current, which is subtracted from each current reading during a from each current reading during a scan, contains PMT noise.scan, contains PMT noise.
Scans at low signals contain PMT Scans at low signals contain PMT noise.noise.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data
PMT noise present in each of these PMT noise present in each of these current readings does not have the current readings does not have the same effect on results:same effect on results: A high or low dark reading will A high or low dark reading will
raise or lower ALL points.raise or lower ALL points. Current readings during scans Current readings during scans
contain highs and lows that contain highs and lows that cancel out to some degree.cancel out to some degree.
PMT noise present in each of these PMT noise present in each of these current readings does not have the current readings does not have the same effect on results:same effect on results: A high or low dark reading will A high or low dark reading will
raise or lower ALL points.raise or lower ALL points. Current readings during scans Current readings during scans
contain highs and lows that contain highs and lows that cancel out to some degree.cancel out to some degree.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data
1.7E-12
1.8E-12
1.9E-12
2E-12
2.1E-12
2.2E-12
2.3E-12
0 20 40 60 80 100 120 140 160 180 200
Measurement #
Dar
k C
urr
ent
[A]
Excel: “= average()” Excel: “= average()” 2E-12 2E-12Excel: “= average()” Excel: “= average()” 2E-12 2E-12Excel: “= stdev()” Excel: “= stdev()” 1E-13 1E-13Excel: “= stdev()” Excel: “= stdev()” 1E-13 1E-13
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data
-6E-13
-4E-13
-2E-13
0
2E-13
4E-13
6E-13
0 20 40 60 80 100 120 140 160 180 200
Measurement #
Ne
t s
ign
al
[A]
Dark = min
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data
-6E-13
-4E-13
-2E-13
0
2E-13
4E-13
6E-13
0 20 40 60 80 100 120 140 160 180 200
Measurement #
Ne
t s
ign
al
[A]
Dark = min
Dark = max
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
Real DataReal DataReal DataReal Data
-6E-13
-4E-13
-2E-13
0
2E-13
4E-13
6E-13
0 20 40 60 80 100 120 140 160 180 200
Measurement #
Ne
t s
ign
al
[A]
Dark = minDark = maxDark = average
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
dGC
dGAs
A
NVISa
930
450930
450
11
CalculationsCalculationsCalculationsCalculations We can describe the effects of noise on We can describe the effects of noise on
class A NVIS radiance mathematically:class A NVIS radiance mathematically: s s is the standard deviation of the noiseis the standard deviation of the noise C(C() is the calibration factors) is the calibration factors GGAA(() is the relative response of class A ) is the relative response of class A
NVISNVIS
We can describe the effects of noise on We can describe the effects of noise on class A NVIS radiance mathematically:class A NVIS radiance mathematically: s s is the standard deviation of the noiseis the standard deviation of the noise C(C() is the calibration factors) is the calibration factors GGAA(() is the relative response of class A ) is the relative response of class A
NVISNVIS
Dark subtractionDark subtractionDark subtractionDark subtraction
Signal averagingSignal averagingSignal averagingSignal averaging
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
CalculationsCalculationsCalculationsCalculations
A similar equation, but using NVIS A similar equation, but using NVIS class B response instead of class A, class B response instead of class A, can give the standard deviation in can give the standard deviation in NVISb radiance.NVISb radiance.
The standard deviations should be The standard deviations should be scaled to the luminance to give the scaled to the luminance to give the expected variations in scaled NVIS expected variations in scaled NVIS radiance.radiance.
A similar equation, but using NVIS A similar equation, but using NVIS class B response instead of class A, class B response instead of class A, can give the standard deviation in can give the standard deviation in NVISb radiance.NVISb radiance.
The standard deviations should be The standard deviations should be scaled to the luminance to give the scaled to the luminance to give the expected variations in scaled NVIS expected variations in scaled NVIS radiance.radiance.
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
CalculationsCalculationsCalculationsCalculations
Noise can be reduced by multiple Noise can be reduced by multiple measurements.measurements.
If we generalize the equation to If we generalize the equation to include multiple dark readings (Ninclude multiple dark readings (NDD) )
and scans (S):and scans (S):
Noise can be reduced by multiple Noise can be reduced by multiple measurements.measurements.
If we generalize the equation to If we generalize the equation to include multiple dark readings (Ninclude multiple dark readings (NDD) )
and scans (S):and scans (S):
930
450930
450
930
450
dGC
dGNS
dGN
As
AD
AD
NVISa Brain overloadBrain overloadBrain overloadBrain overload
Optronic Laboratories, Inc.Optronic Laboratories, Inc.
SpreadsheetSpreadsheetSpreadsheetSpreadsheet
Moving on to the benefits…Moving on to the benefits…Moving on to the benefits…Moving on to the benefits…
IntroducingIntroducingIntroducingIntroducing
The SpreadsheetThe SpreadsheetThe SpreadsheetThe Spreadsheet