Single-Measurement Uncertainty

About this Article
By Ian Instone, it was first presented at the Institution of Electrical Engineers, London in May 1993 at their Colloquium entitled "Uncertainties in electrical measurements".

Note: Throughout this paper uncertainties are referred to by their more descriptive title, random uncertainties and systematic uncertainties. Random uncertainties (U_R) have now become known as Type A uncertainties, and systematic uncertainties (U_S)  have become known as Type B uncertainties.

Introduction

There are several different methods for calculating measurement uncertainty. All require the metrologist to produce multiple sets of identical measurements on the item being calibrated and then performing some rather tedious and complicated calculations to arrive at an estimate of the measurement uncertainty. Whilst each has its merits, any of the recognized methods of combining uncertainty contributors will result in a value within about 10% of any other method. The principal flaw is the quantity of measurements required on every item tested which will either result in very high costs being passed on to the customer, or a compromise might be reached whereby the instrument is calibrated at significantly less points. To address this issue, a method is required which does not routinely involve measurement repetition for the sole purpose of calculating the uncertainty. This paper proposes a simplified method of analysis, including safeguards which can be built into the calibration process to monitor the quality of the assessment.

Balancing Customer-Supplier Needs

In the cost-conscious calibration business environment, it is crucial to ensure that the quality and quantity of measurements provided precisely matches the customer's requirements. A low quality level will have an impact upon integrity and too many testpoints will have an adverse effect upon costs. In assessing measurement uncertainty, guidance can be sought from a variety of sources; many measurement accreditation bodies publish suitable material. Unfortunately, these documents assume the device to be certified is a simple artifact requiring few testpoints. In such conditions it is practical to follow the guidance and report the average value of several repeated measurements. However, in the case of complex, microprocessor controlled measuring instruments such as a spectrum analyzer, customers expect a calibration report to show the results for over twenty parameters with each being measured at , maybe, fifteen points. To generate this quantity of test data can take several hours using automation and several days if measurements are performed manually. It is unreasonable to expect a calibration laboratory to perform several calibrations on such an item solely for the purpose of calculating a component of the uncertainty budget.

Systematics

It is important that a similar understanding of "calibration" exists between both the supplier and consumer of the service. At the standards laboratory level it is normal to supply a certificate which can used as "correction" data for the equipment. With complex multi-parameter instruments it is not possible to supply enough calibration data to enable reliable corrections to be made, so certificates often show the equipment's calibration status along with supporting measurement data. Much of the assessment of measurement uncertainty is possible without ever having seen the instrument requiring calibration. Systematic uncertainties can be fully assessed using the paper specifications of both the measuring instruments and the unit to be calibrated. It is suggested that this process is carried-out before even agreeing to calibrate the instrument; there is no better way of being sure of your ability to provide an adequate calibration on the product. This will result in a very conservative estimate of measurement uncertainty. However, provided that the measurement method and equipment is similar to that recommended by the equipment manufacturer, the resulting uncertainties should be adequate to establish the equipment's compliance with specification or otherwise. Systematic uncertainties are calculated using any one of the recognized methods.

Figure 1 -- The effect of measurement uncertainty upon test limits

The magnitude of the systematic uncertainty will be used as a means of determining the level of random uncertainty assessment required. For an uncertainty to be considered adequate in checking for compliance, it should be less than one third of spec. of the unit-under-test (ISO10012-1:1992, Guidance to para.4-3). In Figure 1, it can be seen that at these levels the measurement uncertainty has negligible effect when determining compliance provided that both the specification and the uncertainty are at the same confidence probability. Similarly, if it can be shown that the random uncertainty is less than one third of the total systematic uncertainty, then it can be seen to be negligible. Where the random uncertainty exceeds this fraction, it will have to be assessed using the traditional methods.

Repeatability

A first assessment of random uncertainty is made by repeating all measurements twice in fairly rapid succession. If the first set of measurements were at the positive extreme of random uncertainty and the second set at the negative extreme then the difference between them would be twice the worst case random uncertainty. If we assume that the "worst case" is represented by +/-3 standard deviations then the difference between the highest value measured at any point and the lowest value will be 6 sigma. Since measurement uncertainties are usually assessed at 95% confidence probability (approximately +/-2 standard deviations), use of the difference between maximum and minimum values as the random uncertainty means there is considerable over-estimation, assuming that the (max - min) value is the true maximum and minimum. To find these values it will be necessary to perform a great many measurements but, if the measurement process is understood, it may be possible to drastically reduce this work.

Figure 2 -- HP8753B network analyzer linearity by experiment.
Random uncertainty as LINEAR (left) and LOG (right) plots

In the example shown in Figure 2, it is obvious that the random uncertainty follows a law and can be predicted. There is some noise present at the input of the measuring system which gives rise to the horizontal part of the (log) plot. There is also some noise generated at the output stages which is demonstrated by the positive going line. The 8753B values were generated using statistical methods based upon a measurement sample of 401 values at each point.

Figures 3 - 5  show the value of determining a "law" by which the random uncertainty varies with another, connected, influence. Having established this on one unit, it is feasible to reduce the random uncertainty assessment points by several orders, perhaps only assessing U_R at 20dB intervals. This is of course still a very costly exercise so for "run-of-the-mill" specification compliance type calibrations faster methods based upon type testing a sample instrument and applying the values obtained to all of the calibrations performed is proposed.

Figure 3 -- Maximum to minimum differences for 401 points

Figure 4 -- Max to min differences for 5 points

Figure 5 -- Max to min values for just 2 points

Whilst it is obvious that the range of random uncertainty values obtained is less uniform,  Figures 3 - 5 show that we are, nevertheless, able to predict a random uncertainty contribution based upon only two sets of measurements providing that some knowledge of the measuring instruments is available. However, the approach may not be suitable when only a few measurements are to be performed across the range.

The Uncertainty Of A Single Measurement

The above plots demonstrate that we can often reduce the random uncertainty assessment down to two sets of measurements. However, even with two sets of measurements performed on complex instruments there will be an unacceptable cost burden. The information above needs to be translated into an uncertainty budget based upon one measurement being performed upon the instrument being calibrated. Statistical approximation is needed when estimating uncertainties for a single measurement. If an infinite quantity of measurements were performed at every point the limits of random uncertainty would be described by the (max - min) value. The limits are often assumed to be at the (mean +/-3 sigma) points, therefore:

(max - min) = (+/-3 sigma) = (6 sigma)

For a 95% confidence probability with an infinite quantity of measurements performed at every point we are required to calculate the (2 x sigma) points. In using this method of calculating random uncertainty (max - min), we would have a very pessimistic assessment but this needs to be balanced against the limited quantity of measurements used. Normally, when reporting the mean of a quantity of measurements, the standard deviation would be divided by the square root of the quantity so when only one measurement is performed the random uncertainty (U_R) will remain the same.

i.e.
For the mean of many measurements:

U_R = (t  x  standard deviation) / (N^½)

Where "t" is Student's t for the quantity (N) of measurements performed or, for only one measurement:

U_R = (t  x  standard deviation) / 1

Where the standard deviation is calculated from a typical measurement set and where 1 is the square root of 1, only one measurement performed on unit under test,

or using methods above :

U_R = (max - min)

Confidence, despite the Short-cut

The methods described here are based upon "type testing" the first of each instrument seen. To ensure that this method remains valid it is necessary to periodically audit the process by checking the random uncertainty contribution using a different test sample. Ideally, unless the procedure identifies specific items of test equipment (by serial number) then substitute equipment of the same model and type should also be used.

It is possible when assessing uncertainties that it becomes unnecessary to perform the rather tedious repeated set of measurements to establish values for the random uncertainty contribution. In many cases the random uncertainty contribution will be known to be negligible with respect to the systematic contribution, with negligible being defined as less than one third of the systematic uncertainty. In these cases it is perfectly feasible to make the one-third U_S allowance without going through the measurement process. In other cases it is possible that the major random uncertainty contribution is a well documented type (e.g. connector repeatability). In these cases the random uncertainty contribution could be the larger of either the documented allowance, or one-third U_S.

Total Uncertainty

There are, unfortunately, no short cuts when calculating the total measurement uncertainty. Although the specifications of the various measuring instruments may be quoted at a variety of confidence probabilities it is reasonable to assume that they will be for at least 95%. To use a specification which is quoted at 99.7% probability (+/-3 sigma) and assume it to be at 95% (+/-2 sigma) will produce an very conservative measurement uncertainty. The decision to convert specifications from one confidence probability to another will depend upon the accuracy ratios involved, in most cases it will be acceptable to leave the confidence probabilities as they are and gain additional confidence from the rather pessimistic values produced.