Ian Rooney
PQ Systems Pty Ltd

Whether assessing product quality simply to meet customer specifications or as part of a quality improvement initiative, it is important to have an understanding of the measurement system used to make the assessment. It is common to accept measurements as being correct, particularly if:

• The data was generated in a logical manner
• Approved procedures or methods were used.
• An experienced operator took the measurement.
• The number looked right!
• The measurement instrument used was calibrated.

When a bad measurement is observed, such as one that is out of specification, immediate action typically focuses on the production process. Rarely is an investigation of the measurement system initiated. There tendency is to believe the data produced, but is this the right thing to do?

Ideally, measurement systems should always produce numbers that are accurate and reliable. However, this is frequently not the case. It is important to understand that product assessed as out of specification can be the result of a poor measurement system, rather than a poor process. An added dimension to this situation arises when measurements are used in analytical studies, such as those used in statistical process control or six sigma improvement projects. In these cases we rely on accurate, reliable data to make decisions.

Problems arising from poor measurement systems can be avoided by conducting a thorough assessment of the measurement system, then implementing appropriate strategies to remedy any errors found.

If a measurement system has significant variation, test results cannot be relied upon when assessing product. For this reason many organisations re-test samples that fail, to make sure they really have a problem before taking action. If a measurement system is highly variable, this is obviously a sensible approach. Taking action on a poor test result could potentially make the process worse. But doesn’t the same logic mean that the good test results should also be questioned? Where does this end? Does every test have to be performed twice?

The variation in the measurement system causes a degree of uncertainty that can be estimated and managed. This is known as measurement resolution. The method of estimating uncertainty involves calculating probable error. The probable error of measurement variation can be applied to specifications giving a guide to those test results that need to be re-tested, and determining those that do not. Pictorially, probable error applied to specification limits as shown in figure 1.

The shaded areas can be thought of as bands of uncertainty. Those results that land outside the shaded areas can be classified as definitely in or out of specification, even taking into account measurement variation. However, it is not possible to determine whether the results in the shaded areas are either in or out of specification. They must be re-tested.

The variation emanating from the measurement system effectively narrows the specifications. Calculating probable error quantifies the narrowing effect and consequently provides a better assessment of product. Samples in the shaded areas cannot be correctly assessed as in or out of specification so care needs to be taken.

The calculation for probable error assumes that the measurement system error forms a normal curve, a valid assumption for most measurement systems. If a distribution is created comprising thousands of repeat measurements a normal curve can be expected. Probable error is defined as the median uncertainty, the median being the middle point of the distribution, with approximately 50% of the data falling either side of this point. Further, the interval that exists between plus and minus 0.67 standard deviations either side of the median point of the distribution contains 50% of the data and is known as the median uncertainty. This means that approximately half of the measurements can be thought of as having an uncertainty of less than 0.67 standard deviations while approximately half has an uncertainty greater than 0.67 standard deviations.

So with these statistics in mind, probable error is calculated by multiplying standard deviation by 0.67.

The probable error figure can now be applied to the specifications. The error is built around the specifications to determine whether the product is truly in or out of specification. Adding and subtracting the probable error to each of the specifications effectively widens and tightens the specification range as shown in figure 3.

If a more cautious approach is preferred, that is 5 in 6 chances is too risky for the particular application, then the limits can be adjusted to 2 x probable error. This has the effect of increasing the band of uncertainty around each specification and increasing the chance of product conforming to 19 in 20. If preferred the limits can be adjusted to 3 x probable error, increasing the odds to 99 in 100.

When the widened and tightened specifications are applied, the rules to determine the acceptability of a test result and therefore the product are as follows:

• Result beyond the widened specification - Result is outside the specification.
• Result is between the tightened specification - Result is inside the specification.
• Result is between the widened and tightened specification (within the bands of uncertainty) - Result is borderline.
• Borderline results need to be re-tested to determine their disposition.

If a measurement system has extremely high variation, it is possible that there will be no area between the tightened specifications. This shows a system in urgent need of improvement, as the measurement system can never prove a product is in specification!

This method gives the user confidence in test results, and enables logical decisions to be made. It enables results to be viewed with confidence as the impact of the measurement variation has been defined.

As always, I look forward to your comments and questions. I’m at support@pqsystems.com

Like to know more about measurement systems analysis? Please view our seminar schedule for a location near you.