Introduction to Error Analysis - unibas.chpcpraktikum/Experimentdesign/Intro error... · 2 1. Why...
Transcript of Introduction to Error Analysis - unibas.chpcpraktikum/Experimentdesign/Intro error... · 2 1. Why...
UNIVERSITÄT BASEL – DEPARTEMENT CHEMIE
Introduction to Error Analysis
Physikalisch-Chemisches Praktikum
Jens Gaitzsch, Nico Bruns, Katarzyna Kita, Corinne Vebert
2016
2
1. Why is error analysis important?
First of all, every measurement of any value (in chemistry, biology, physics, ...) always has
some error. Therefore, every time a result of an experiment should be presented in the following way:
(X ± ΔX) [units]; for example (20 ± 2)0C.
In our laboratory work, we want to have possibly small errors, and for analysis of our results
we always need to estimate those errors. Depending on the error, we can discuss the results. For example,
if we obtain a value and its error is 120 %, that means either we have made some serious mistake with
the calculations, or something was very wrong with the experiment, but in general such a result does not
make much sense, and is useless in the laboratory practice.
2. What types of errors can we have?
There are a few types of errors, some of them are easier to eliminate than others. Firstly, we
have large errors, which in general are dependent on the person performing the experiment and / or
documenting the data, and result from human mistakes. This is called the human error and can be
detected and avoided quite easily For example, we measured 19 0C, but we wrote down 9 0C, or we write
1.9 0C instead of 0.19 0C, etc. It could also happen that we make a mistake and instead of measuring the
size of one object we measure a different object, or we confuse radius with diameter, and so on. As a
rule, such 'strange' results, need to be always eliminated from data analysis, and they are not included in
error calculations either. In order to eliminate those errors, we need to work carefully, pay attention to
small details, perform necessary calibrations and control experiments. It is also good to work in small
groups, where one person can check on what their partner does and control data documentation.
Additionally, we always want to repeat the experiment, to make sure that the result is reproducible.
Next, we have systematic errors, which in general cannot be avoided. They result from the fact
that we use some equipment (machines), which by definition are never 100 % accurate. These errors
cause a trend away from the true value in the result, the result is thus less accurate since systematic
errors affect the accuracy of a result. This means that the measured values are constantly off in a certain
direction.
Finally, we have statistical errors, which are difficult to avoid as well and affect the precision
or reproducibility of a measurement. They appear because there are always small differences in the
measured values depending on external conditions (temperature, pressure, humidity). Therefore, it can
happen than the measurements taken in summer will slightly differ from the winter results. Since
solutions are always also not 100% homogeneous and sample preparation is also not 100% reproducible,
results can differ there as well. We can minimize such errors by careful machine calibration, careful
sample preparatioin and trying to avoid dramatic changes in external conditions.
These types of errors are independent from each other and can occur in any combination. An example
on how systematic errors (accuracy) and statistical errors (precision) affect a result, is given in the
following image:
3
3. Ways to treat errors
3.1. One can treat errors in three ways:
If a variable has been measured several times (at least 10 times) it can be treated according to the laws
of statistics (→ Gaussian error propagation) in order to calculate a mean value and a standard deviation.
However, this case is quite rare for the experiments in the PC Praktikum, because most often the number
of individual measurements is not high enough for a statistical treatment. In many cases (also in reality),
the standard deviation is already considered for 5 measurement points.
If you are able to measure a variable only a couple of times ,e.g., due to time limitations, the error has
to be estimated as a maximal error.
If the results can be theoretically described by a mathematical function, it is possible to estimate the
error by analysis of the corresponding graph. This can be done by hand or solved numerically by a
computer. Details for a linear regression will be discussed later in the script
3.2. Proper estimation of errors
If errors can’t be treated statistically, it is your responsibility as a researcher to estimate maximal errors
of a single variable. For example, the error could be the last digit of an instrument, e.g. 0.01 mg for a
4
scale with a display showing 126.56 mg. However, a lot of instruments (especially ones with a digital
display) show more digits than their actual accuracy allows. Therefore, it is wise to think twice about
the accuracy of an instrument. The scale might have an accuracy of 0.02 mg. Instruments are normally
calibrated and the accuracy is stated in the manual or in some kind of certificate. If this information is
not available, it is your job to estimate the error and it is always better to overestimate it, in the case of
the scale for example 0.05 mg!
The error might also be larger than estimated from the display due to other experimental facts. E.g. a
stop-watch might be accurate to 0.02 s. However, if you stop the time of an experiment manually, your
reaction time will be a lot longer than those 0.02 s. Therefore, your reaction time has to be taken as
maximal error.
3.3. How to state errors
A single variable 𝑓 was measured with an error ∆𝑓. The experimental result can be written as:
𝑓 ± |∆𝑓|
There are several ways to state errors:
Absolute error ∆𝑓
Relative error ∆𝑓/𝑓
Per cent error 100% ∙ ∆𝑓/𝑓
Example: 𝑓 = 39.23 cm; error: |∆𝑓|=0.28 cm
This result can be written as:
𝑓 = (39.23 ± 0.28) cm; ∆𝑓/𝑓 = ±0.007, or 100% ∙ ∆𝑓/𝑓 = ± 0.7%
𝑓 = (39.2 ± 0.3) cm
It is not exact to write 𝑓 = 39.2 cm ± 0.7%.
5
3.4. Decimal places and significant figures
The result of a measurement is x = 4.21596847 ± 0.00686521. In this case, the third decimal place is
already affected by the error. Therefore, it does not make sense to state all the following decimal places.
This is thus the last figure of the number, which can be trusted, it is the last significant figure.
Significant figures tell you, how certain a number is. It is thus important to know which digits of a
number are significant. As a rule, all non-zero digits are significant, all zeros which are between
significant figures and all zeros in decimals, which are behind the last non-zero digit are also significant
figures. A result “210”, for example, has 2 significant figures. This number tells you that the actual
number could be anywhere between 205 and 215. A result named “211” has 3 significant figures and
the actual result could be anywhere between 210.5 and 211.5. In case the “210” from above is supposed
to have 3 significant figures (the “0” is an accurate number), the annotation 21.0 * 101 can be used to
reflect this. In this annotation it is obvious that the “0” is not there to fill the space, but a significant
figure.
The result is always presented up to and including the first uncertain decimal place. (In rare cases the
second uncertain decimal place can also be stated.) The error has to be presented with a maximum of
two decimal places different from cero. The error has to be rounded up.
Applying this rule to the example results in 5 significant figures (first example) or 4 significant figures
(second example):
x = 4.2160 ± 0.0069 or x = 4.216 ± 0.007
Usually, the amount of significant figures varies for different sets of data and numbers with a different
amount of significant figures need to be used in one calculation. Here, certain rules apply. For additions
and subtraction, the number with the lowest degree of certainty determines the amount of significant
figures. For “210 + 23”, for example, the 210 is only defined until the “tens” digit, the 23 is significant
until the “singles”. The result can now not be more defined than the “210” was and the result is “210
+ 23 =230”, which is now still defined until the “tens” digit. This is done best if the numbers are written
underneath each other and are cut off at the number which is less defined. The result then needs to be
rounded up or down according to the normal rules. Some examples are shown here with the significant
figures written in bold font:
2 1 0 2 0 . 1 0 3 3 1 0 0 0 3 5 . 4
+ 2 3 - 5 . 3 0 + 5 3 2 0 0 0 - 0 . 4 6
= 2 3 0 1 4 . 8 0 8 4 2 0 0 0 4 . 9
For multiplications and divisions, the number with the least amount of significant figures determines the
amount of significant figures in the result. If “3.00” (3 significant figures) is multiplied with “1.5” (2
significant figures) (e.g. 3.00 * 1.5), the result thus has 2 significant figures and the equation should read
“3.00 * 1.5 = 4.5”. The “4.5” also has only 2 significant figures, like the “1.5”. Again, some examples
below:
10.0 * 1.90 = 19.0 (both starting numbers hat 3 significant figures)
100 * 1.9 = 200 (one starting number had only 1 significant figures)
6
4. Treating experimental values
4.1. Mean value
The mean value ā is defined as
�̅� =1
𝑛∑ 𝑎𝑖
𝑛
𝑖=1
where n is the number of experiments (repetitions), ai are the results of individual experiments, and the
sum extends from 1 to n. In general, repeating the experiment n times increases the accuracy of the mean
value, because it will be more influenced by the most often obtained results. In laboratory, every
experiment should be repeated at least three times!
4.2. Standard deviation
When we have performed an experiment several times (as a rule of thumb: more than 5 repetitions), we
can use the standard deviation to estimate the errors. Standard deviation (Sn) is a statistical measure of
the error, and can only be applied to a large series of data (many repetitions). It is generally smaller than
the maximum error and it is a measure of the width of the data distribution around the real value, i.e. a
measure for the accuracy of a single measurement.
Definition: 𝑆𝑛 = √1
𝑛−1∑ (𝑎𝑖 − �̅�)2𝑛
𝑖=1
The value with the largest deviation from the mean is called the maximum error. Despite its name, the
standard deviation can still be larger than this value! The maximum error only describes the largest
experimentally detected error.
4.3. Sample calculation
We measure the weight of a product and obtain the following results:
n weight [mg]
1 148.4
2 148.1
3 149.6
4 149.0
5 149.9
Mean value:
�̅� = (148.4 + 148.1 + 149.6 + 149.0 + 149.9) / 5 = 149.0 cm
7
With respect to significant figures, it is important to note the number of experiments, here “5” is no
experimentally obtained value with some kind of error, but a fix value with indefinite significant figures.
It is thus correct to give the final value 4 significant figures as all measured values have 4 of them.
The standard deviation is thus:
n weight [mg] Deviation 𝑎𝑖 − �̅�
1 148.4 0.6
2 148.1 0.9 (Maximum error)
3 149.6 0.6
4 149.0 0.0
5 149.9 0.9
𝑆𝑚 = √1
𝑛−1∑ (𝑎𝑖 − �̅�)2𝑛
𝑖=1 = √(0.62+0.92+0.62+0+0.92)
5−1= √
2.34
5−1= 0.765 ≈ 0.8
Please note that in this example the maximum error is smaller than the standard deviation, which is
normal and nothing to worry about if the difference is not extraordinary large (more than 3 times larger).
The relative error is: 𝑆𝑚,𝑟𝑒𝑙 =𝑆𝑚
𝑚∗ 100% =
0.8
149.0∗ 100% = 0.5%
The final result would thus look as follows: 𝑚 = (149.0 ± 0.8) mg (0.5 % error)
0.5 % is a very small error. In many experiments, errors up to 20 % are tolerated. If the error is higher
than 20 %, something is wrong (probably calculations). If the error exceeds 100 %, something is VERY
wrong: check calculations again and think what could have produced such an error. Every time, errors
need to be discussed in PC Praktikum reports!
5. Error propagation
5.1. Maximal Error
In most cases, the result of an experiment depends on more than one measured value. Each value has to
be determined individually and each of these values has its error. So how do these errors influence the
error of the final result? They can accumulate or they can cancel each other out to some degree. The
latter case will be discussed later (Gaussian error propagation). Here, we will assume the case that all
individual errors falsify the true value into the same direction. We will therefore discuss the error
propagation of the maximal error.
For example, if we make a solution, the mass of this solution (C) is a sum of the mass of solvent
(A) and what is dissolved (B), both with corresponding errors. So if C = A + B, then ΔC = ΔA + ΔB. In
the most general case, if the result of an experiment F is a function of several individual values x,y,z,…
We can write:
F = f(x,y,z,…)
A Taylor expansion gives (in a first approximation) ∆F:
8
∆𝐹 = |𝜕𝐹
𝜕𝑥| |∆𝑥| + |
𝜕𝐹
𝜕𝑦| |∆𝑦| + |
𝜕𝐹
𝜕𝑧| |∆𝑧| + ⋯
Three important cases will be highlighted:
a) 𝐹 = 𝑥 + 𝑦 + 𝑧 + ⋯ Resulting Error: |∆𝑅| = |∆𝑥| + |∆𝑦| + |∆𝑧| + ⋯
Addition of absolute errors of the individual values
b) 𝐹 = x ∙ y ∙ z ∙ … Resulting Error: |∆𝐹
𝐹| = |
∆𝑥
𝑥| + |
∆𝑦
𝑦| + |
∆𝑧
𝑧| + ⋯
Addition of relative errors of the individual values
c) 𝐹 = 𝑥𝑎𝑦𝑏𝑧𝑐 … Resulting Error: |∆𝐹
𝐹| = |𝑎| ∙ |
∆𝑥
𝑥| + |𝑏| ∙ |
∆𝑦
𝑦| + |𝑐| ∙ |
∆𝑧
𝑧| + ⋯
If F is a more complicated function of x,y,z,… (log, exp, sin, etc.), or a combination of addition and
multiplication, the error has to be calculated explicitly by the original Taylor expansion., i.e., the
function has to be differentiated with respect to each variable.
5.2. Gaussian error propagation (used in PC Praktikum)
If the errors of individually measured values x, y, z, and so on were determined by means of statistics
(i.e. standard deviation or confidence range, or error of a linear regression), the error SF of a function
F = f(x,y,z,…)
can be calculated with the Gaussian error propagation:
SF = √(𝜕𝐹
𝜕𝑥)
2
(∆x)2 + (𝜕𝐹
𝜕𝑦)
2
(∆𝐵)2 + (𝜕𝐹
𝜕𝑧)
2
(∆z)2 + ⋯
Requirements for the Gaussian error propagation are that the errors in x, y, z, are not correlated
(=independent of each other) and random.
Example: We have an ideal gas equation, pV = nRT. We perform an isothermal experiment (T = const.)
and measure the change of volume (V) with pressure (p), so we need to find the error ΔV. Our
'experimental' equation takes the form: 𝑉 =𝑛𝑅𝑇
𝑝 , where n and R are constants (and as a rule we do not
consider any errors for physical constants, therefore Δn = 0 and ΔR = 0). We have, however, errors from
measuring temperature, ΔT, and pressure, Δp (which we should know from performing multiple
measurements and estimating the standard deviations of T and p).
By applying the above formula, we can see that:
∆𝑉 = √(𝜕𝑉
𝜕𝑇)
2
(∆𝑇)2 + (𝜕𝑉
𝜕𝑝)
2
(∆𝑝)2
9
Now we need to find partial derivatives 𝜕𝑉
𝜕𝑇 and
𝜕𝑉
𝜕𝑝, which are:
𝜕𝑉
𝜕𝑇=
𝑛𝑅
𝑝, so (
𝜕𝑉
𝜕𝑇)
2=
𝑛2𝑅2
𝑝2
𝜕𝑉
𝜕𝑝= −
𝑛𝑅𝑇
𝑝2 , so (𝜕𝑉
𝜕𝑝)
2=
𝑛2𝑅2𝑇2
𝑝4
And the last step is to insert the partial derivatives into the general equation:
∆𝑉 = √𝑛2𝑅2
𝑝2(∆𝑇)2 +
𝑛2𝑅2𝑇2
𝑝4(∆𝑝)2
After including all numbers (and checking for units!), we obtain ΔV.
6. Graphical data presentation
In many cases, we need to present our results graphically, and most often there is a mathematical
function that theoretically describes what the graph should look like.
Example: The theoretical dependence of absorbance on concentration is 𝐴 = 𝜀 ∙ 𝑐 ∙ 𝑙, where l is the light
pathway through the solution, and ε is extinction coefficient, a physicochemical parameter
characterizing the ability of a compound to absorb electromagnetic radiation. If we need to find this
parameter, we can measure the absorbance of several solutions of different concentration and plot
absorbance versus concentration. From the slope of a line that is fitted to the data points, we can easily
calculate 𝜀: 𝜀 = 𝑠𝑙𝑜𝑝𝑒/𝑙.
Below a few comments about how to prepare a good graph:
0.000 0.005 0.010 0.015 0.020 0.0250.0
0.2
0.4
0.6
0.8
1.0
Absorb
ance
c, mol/dm3
10
Graphics are nowadays always prepared on a computer (see above). The axes have to be labeled,
including units, e.g. „length (mm)”. The scale on the axes has to have equal increments (0-10-20, or 0-
500-1000, whatever fits to your experiment), please do not mark the values that correspond to your
experimental data. If you repeated the experiment a few times and obtained a mean value, only the mean
goes on the graph. Every experimental point has to have error bars at least in the Y direction, but ideally
in both X and Y directions.
Please note, that from your experiments you will only have data points, so it does not make too
much sense to connect them in your graph by any lines, especially if you do not know what the line
should be. We will come back to that point later.
7. The least squares method
In many experiments, we measure the dependence of a physical value as a function of another
value, and the relationship between the two is predicted by theory. It could be a linear dependence (𝑦 =
𝑎𝑥 + 𝑏), an exponential growth / decay, or many more mathematical functions. In general, fitting of the
obtained data with a theoretical model provides fitting parameters (for example a and b in the linear fit),
which have some physico-chemical meaning and could be interesting to us. Now the question is: how
to find good fitting parameters, when we have experimental points only, like for example in the graph
above?
We will consider the simplest case of a linear dependence. For such fitting (which is often called
'linear regression'), we have to make sure that we have enough data points (remember: it is always
possible to draw a straight line through two points: the line will be a perfect fit, but may have little to no
physical meaning, if the points themselves have large errors). In general, 5 points are enough for a
reasonable fit.
The 'best' line is defined such as it lies the closest to the experimental points. Practically, we
look for the smallest sum of square distances between the points and the line (and therefore the name:
'the least squares method'). When we have a linear function 𝑦 = 𝑎𝑥 + 𝑏, this procedure allows for the
calculation of the slope (a) and the intercept (b) from the following formulas:
a =𝑛 ∑ 𝑥𝑖𝑦𝑖 − ∑ 𝑥𝑖 ∑ 𝑦𝑖
𝑛 ∑ 𝑥𝑖2 − (∑ 𝑥𝑖)2
𝑏 =∑ 𝑥𝑖
2 ∑ yi − ∑ 𝑥𝑖 ∑ 𝑥𝑖𝑦𝑖
𝑛 ∑ 𝑥𝑖2 − (∑ 𝑥𝑖)2
where n is the number of points taken for fitting, xi and yi are the coordinates of the data points,
respectively, and the sums extend from 1 to n.
11
The calculations of a and b can be done automatically by most calculators and with computer
programs (linear fit option). There are also on-line tools that calculate function parameters, see e.g.
http://www.chemie.unibas.ch/~huber/Statistik/LinReg/index.html.
For example, fitting of the points from our example graph with OriginProTM gives slope of 38.8
dm3/mol and intercept of 0.004, so the function is: A = 38.8 c + 0.004.
8. Regression errors
Since the fitted line does not go through all experimental points and only as close to them as
possible, we need to discuss fitting errors. In other words, we should ask how good the fit is, or: how
well does the fitted line follow the experimental data?
One common approach is to analyze the so-called coefficient of determination R2 (R is the
correlation coefficient). In a data set accounted for by a statistical model, R2 is the proportion of
variability in that set of data. The value of R2 lies between 0 and 1, and a good fit is described by R2
well above 0.9 (in our example we had R2 = 0.99931, which means a very good fit). The perfect fit has
R2 = 1. The formula to calculate R is the following:
𝑅 =𝑛 ∑ 𝑥𝑖𝑦𝑖 − ∑ 𝑥𝑖 ∑ 𝑦𝑖
√𝑛 ∑ 𝑥𝑖2 − (∑ 𝑥𝑖)2 √𝑛 ∑ 𝑦𝑖
2 − (∑ 𝑦𝑖)2
Again, R2 is normally computed by calculators and computer programs together with linear regression
parameters.
On the other hand, R2 does not inform us directly about individual errors of the slope and
intercept of the fitted line. We can calculate them from the following formulas. Both of them are
numerically stable, e.g. small deviations in the input value do not result in large differences in the value
computed by the formula.:
0.000 0.005 0.010 0.015 0.020 0.0250.0
0.2
0.4
0.6
0.8
1.0
Absorb
ance
c, mol/dm3
12
∆𝑎 = √𝑛
𝑛 − 2
∑ 𝑦𝑖2 − 𝑎 ∑ 𝑥𝑖𝑦𝑖 − 𝑏 ∑ 𝑦𝑖
𝑛 ∑ 𝑥𝑖2 − (∑ 𝑥𝑖)2
∆𝑏 = ∆𝑎√1
𝑛∑ 𝑥𝑖
2
In conclusion: when performing your experiments, try to do as many repetitions as possible, think if the
measured value depends on other values, and always discuss the experimental errors in your reports!
9. Further Reading
John R. Taylor: An Introduction to Error Analysis: The Study of Uncertainities in Physical
Measurements, 2nd edition, University Science Books, Sausalito, 1997.
Manfred Drosq: Der Umgang mit Unsicherheiten: Ein Leitfaden zur Fehleranalyse, Facultas
Universitätsverlag, Wien, 2006.
The Journal of Cell Biology, Error bars in experimental biology, Vol. 177, No. 1, April 9, 2007, 7–11,
http://www.jcb.org/cgi/doi/10.1083/jcb.200611141