Experimental measurement uncertainties.

navigation page

Antennas lab 1999

Occasional notes in electronics series.

What are measurement errors (sometimes called "uncertainties"), and why are they important?

Except in straightforward counting situations, there is always a difference between the measured value of a variable and the true value. This difference is called the "error". By the nature of an error, it must be an indeterminate quantity, or an "uncertainty", for if we knew what it was precisely we could adjust for it and we would then know the true value precisely. The value of an error can therefore only be estimated; it cannot be known precisely. A measurement which does not report the likely range of possible errors contains limited information, for we can be quite sure that the true value differs from that reported and we have no idea of the possible size of this difference. Uncertainties are sometimes quoted as a fixed number at a 90% confidence level, or somesuch figure. There are established procedures for estimating such figures; a useful site is the National Institute of Standards and Technology (NIST) in the USA.

An example. Consider the statement "Guildford is in the county of Sussex". Factually this is wrong, and its use to someone who suspects the statement may be wrong is minimal. Now consider the statement "Guildford lies in the county of Sussex plus or minus one county in every direction". Factually this is right; it contains more information and gives the reader an idea of where Guildford might be, and the possible range by which the statement could be wrong. Even if the statement were wrong, the reader would think it more likely that Guildford was in Middlesex than in Yorkshire, for example.

Now consider the statement "Guildford is in the county of Sussex, or some other county beginning with S". The error specified here points the reader to different possibilities; Somerset rather than Middlesex or Yorkshire. We see that the specification of the possible error guides the reader and adds information to the measurement.

Now consider the problem of determining the value of pi experimentally, using a piece of cotton thread and a ruler. The method is to wrap the thread around a circular object and measure its length (for the circumference); then to find the maximum chord across the base of the object with the thread and measure its length again (for the diameter). You then divide the circumference by the diameter to get pi, with a calculator. You repeat for a number of different sized circular objects (beer cans, glasses, car tyres, dustbins, etc.) For each of these measurements you can estimate the maximum likely error. The repeated measurements will give, hopefully, randomly distributed errors.

Now pi is 3.14159265... and there is an infinite amount of information contained in an unlimited expansion of pi. If we could report the measurement absolutely precisely and accurately, we would need an unlimited amount of computer memory.

I can ask you a very simple question. How many of the digits of pi listed above can I hope to determine using this experimental method? If you can answer this question reasonably correctly, you have learned a lot about experimental error analysis and its importance.

Much of error estimation is a matter of experience acquired from many years of experimentation. There a few algorithmic rules for combining errors, particularly if they are known to be random errors or maximum possible errors; such rules have to be tempered with experience and judgement in any given situation.

To take our example, if we use a cotton thread of diameter 0.2mm then the error in circumference due to the diameter of the thread will certainly be less than .5mm. However the thread may stretch between being wrapped and measured. We need to estimate how much this stretch could be. Say 1mm. Then again, we can only measure the length of the thread with the ruler to about 0.5mm precision.

You now have choices. If you reckon these errors are random and uncorrelated you should add up the squares of the relative error and take the square root. On the other hand, if you think they may be systematic or correlated or maximum possible errors, you may prefer to add the errors directly.

In either case you come up with a possible error in the circumference of between 1 and 2mm. There is a similar possible error in the diameter. There is no need for much precision in the estimate of this error.

Now consider an object of diameter 10cms, circumference 31.41... cms. You can determine the diameter to 1 or 2 parts in 100 (1-2 mm in 10 cm) and the circumference to 1 or 2 parts in 310 (1-2 mm in 31 cms). Adding the errors directly we can determine the error in pi as about 2 parts in 100. The empirical value of pi lies between about (notice the word "about" here) 3.08 and 3.20. You might be able to do a bit better with a larger object to start with. An object of diameter 1 metre would only improve the error to give pi to about 1 more decimal place.

Now consider how much computer memory you need to store the first million digits of pi. It's about a megabyte. Experimentally we can only determine two of these digits by this method. (2 bytes). Any measurement of a continuous variable which you quote absolutely precisely conveys an infinite amount of information. Reporting the error limits the information content of the reported value to that of the measurement. In this case if we know to start with that pi is about three, the only outcome of our experimental procedure is to tell us the next digit is probably 1.

Compared to how much information is actually contained in the true value of the quantity we are trying to measure, our experiment only reveals a very small portion of this infomation. We can expect to have to work very much harder experimentally to reduce the possible error range and acquire more of this information.

Experiments are basically procedures that "ask a question of nature". The quantity of information in the answer is important when we come to compare the outcome of the experiment with theory. In practice, errors are estimated and plotted as error bars, and the error bars should bracket the theoretical prediction. If they don't, either the theory is wrong, or the model is wrong, or very occasionally some terrible systematic error has happened in the experiment which has not been considered.

A comparatively large error bar tells us less information about where the theoretical or true value might lie than does a small error bar. If we consider errors in a measurement of a function y = f(x) as having an area dxdy (dx is the small error in the x value, and dy is the small error in the y value) then the amount of "information" contained in plotting the point x,y with its error rectangle dx,dy depends on what we consider the possible ranges Dx and Dy of x and y to have been BEFORE we carried out the experiment.

In other words, doing the experiment and measuring this one point has reduced our range of possibility from Dx,Dy to dx,dy and there are DxDy/(dxdy)=N ways of chosing where the point lies within our pre-existing ranges Dx,Dy. The amount of information produced by our experiment is therefore log2(N) bits per point.

In practice, measuring our first point reduces the uncertainties Dx,Dy about where the next (adjacent) point might lie. After we have taken a modest number of points which lie on a slowly bending smooth curve, we might expect that the return of information on the investment of experimental effort to take more points is becoming marginal. It is from these considerations that the seasoned experimentalist decides how much data to take in a given experimental situation.

The mathematics of combining error estimates.

Consider two measurements x and y. Suppose the two actual values, or true values, are x0 and y0. Suppose the estimated errors are dx and dy. Then

		    x = x0 +/-  dx

		    y = y0 +/-  dy

Then the error in the quantity xy is got from considering

		   x0(1 +/- epsilon)y0(1 +/- delta)


which is           x0y0(1 +/- epsilon +/- delta +/- epsilon*delta)

then               x^2 = x*x = x0*x0*(1+epsilon)*(1+epsilon)

                       = x0^2(1 + 2*epsilon + epsilon^2)

Similarly, the error in x^2 is just 2*epsilon, but here the errors are correlated (they are the same) and so the alternative of adding the sums of the squares of the errors and taking the square root is inappropriate.

In practice, since there was a certain amount of judgement in estimating the range to put in for epsilon and delta in the first place, there is little to choose between these two different estimates. All we are interested in is the order of the sizes of the errors, we are not interested in precise values for the errors. It is this aspect of error theory which often confuses the novice.

Of course, if you can show the errors are uniformly distributed and random it is possible to refine the error analysis quantitatively. But in a variable with a Gaussianly distributed added random error, there is no limit to the possible size of the error, although the larger the error the less probable it is.

Exercise for the student. What are reasonable estimates for the errors in the quantities (x+y), and in x/y, and in (x^2)/(y^2)?

Answers: (epsilon + delta), (epsilon + delta), (2*epsilon + 2*delta) or the square roots of the sums of the squares of the terms in each of the brackets.

Reference. Klaas B Klaasen, "Electronic measurement and Instrumentation" Cambridge University Press 1996 0-521-47729-8

Feedback from a reader

Dr. Jefferies,

I read the above note with interest. I believe it is more correct to refer to measurement "errors" as measurement uncertainties in modern usage.

You might be interested to read NIST technical note 1297 "Guidelines for Evaluating and Expressing the Uncertainty of NIST Measurement Results". This is freely available from the NIST website. The UKAS version of this probably costs tens of pounds.

Regards,

Leslie Green CEng MIEE, Senior Principal Engineer

--------------------- response

Leslie,

You have kindled my interest.
Thus, the Shorter Oxford Dictionary....

"Error" Middle English errour, Old French errour, errur, modern erreur, :-Latin error. See Err + -or. The form error dates from 1753.

1.) The action of wandering; hence a devious or winding course. Now only poetical, 1594.
2.) (obsolete). Chagrin, fury; extravagance of passion 1460
3.) The condition of erring in opinion; the holding of mistaken beliefs; a mistaken belief; false beliefs collectively. Also "personified" ME.
4.) Something incorrectly done through ignorance or inadvertance; a mistake ME.

then

d.) Mathematical. The difference between an approximate result and the true determination 1726.

plus various other technical uses.

If we look at "uncertainty" or "uncertainties"

the Shorter Oxford Dictionary has

uncertainty: Late Middle English.

1.) The quality of being uncertain in respect of duration, continuance, occurrence, etc.
2.) The state of not being definitely known or perfectly clear; vagueness, doubtfulness. Late ME.
---b. Something not definitely known or knowable; a doubtful point (late ME).
3.)The state or character of being uncertain in mind; hesitation, irresolution 1548.

I don't know how this strikes you, but I think that for what we are talking about here, the term "error" has clear historical, mathematical, and connotational advantages. The term as defined subsumes the connotation of being an indeterminate quantity. So I shall, I think, stick with it on the web page, pointing out the alternative usage of "uncertainty" and its (dis)advantages.

Thanks again.

regards

David Jefferies.