Many industries, from aerospace to medical device manufacturing, are required to show customers, auditors, or regulatory bodies that their measurement devices are acceptable for use. But how do you prove that?

One of my clients needed to show the Biomedical Advanced Research and Development Authority (BARDA) that a measurement device they were using to inspect incoming materials was acceptable as they transitioned from R&D into production. The goal was to produce a new medical device that met the “five nines” (99.999%) specification conformance goal of BARDA.

To achieve such high conformance rates, you need high-quality vendors. The client needed to move to a domestic supplier and show that they were capable of providing materials that would allow them to meet the five-nines requirement. This can be difficult to test, since the very device you are using to measure vendor specification conformance might be unreliable. Additionally, this device measured a number of different characteristics on parts with different shapes and materials.

To fully assess a measurement system, you must combine an understanding of its:
• Stability through time (control)
• Measurement error (precision)
• Measurement error compared to spec (capability)
• Ability to detect process changes (discrimination)
• Ability to get the “right” number (accuracy)

To do this, many might think the obvious step would be to do the typical gauge R&R study. However, the “measure 10 parts two to three times each” gauge R&R studies are not sufficient to prove that a gauge meets these requirements. A typical gauge R&R only estimates the measurement error, and that only in a quick snapshot. It provides no information on control or accuracy. Given that they were measuring different parts with different targets, that type of study also doesn’t provide any information on discrimination.

Instead, we did an accelerated long-term measurement systems analysis (MSA) and monitored the results of nine very different parts and characteristics over time.

This is the essential first step, but even a complete MSA is not enough to prove to you or your customers that the gauge is acceptable for your use.

For example, even though AIAG’s MSA handbook warns against it, many companies use the %R&R (the ratio of the measurement error to the specification width) as their sole acceptance criterion. It is one, but only one, aspect of acceptability.

Creating good acceptance criteria is a meeting of the technical and the business worlds. When you create your criteria, you must balance the ability of the device to perform its function along with the risk and economics of using it.

Here are some things to define in your acceptance criteria prior to starting your testing:
• The gauge must demonstrate ongoing control. This is a requirement for any measurement device.
• Require precision and accuracy such that:
a) The risk that a good specimen being misclassified as nonconforming, or a bad specimen being classified as conforming, is tolerable.
b) The device is useful for process monitoring, reaction, and improvement.
• Determine that the device is practical, including:
a) The cost of purchase, maintenance, and calibration
b) The time and personnel needed to make the measurements
c) The likely uptime and time to maintain

I helped that client save a lot of money and time by showing them how their gauge with a %R&R of 26.5% had an infinitesimally small chance of misclassifying incoming materials and was perfectly acceptable. In another case, a measurement system found to have a 7.6 %R&R was found to suddenly shift average partway through the week, failing the control criterion. This helped to find deficiencies in the measurement standard operating procedure and prevented future misclassification of incoming materials.

Measurement system analysis and determination of acceptability aren’t boxes to check and then move on from; they are the ongoing foundation on which any data-driven decision is made. If you would like to learn more about MSAs, check out the video below. To learn how to do the three types of MSA, test accuracy, create acceptance criteria, and handle destructive tests, take my self-paced and online Measurement System Analysis for Continuous Data course.

(Here’s a link to free software that will do MSA calculations for you. )