## How to Understand a Measurement System Analysis

### A true MSA measures more than just the gauge

Published: Tuesday, September 10, 2013 - 08:46

Measurement system analysis of uncertainty is one topic in lean Six Sigma training that is too often ignored or under-taught. I believe that it is under-taught because most instructors have never used or understood it. Therefore, this column will dive deep into what it is and why you should learn about it. Keep in mind that this methodology is not for destructive sampling or attribute or classification gauges. It only works on measurement systems that allow remeasuring of the same sample and reporting a continuous value for the measurement output.

### Measurement system analysis or gauge study

I first learned about this topic as a measurement system analysis (MSA). The term “gauge study” is only one element of the process, but many consider gauge study and MSA to be synonyms. MSA is probably the best term for the concept because its title implies measuring more than just the gauge. A true MSA includes estimating the impact of the gauge, the fixturing or setup of the gauge, the operator, and the variation over time.

### The best reference book

The best reference for MSA is the Automotive Industry Action Group’s (AIAG) blue book, *Measurement System Analysis, Fourth Edition*. This book has all the information on the definitions and methods that are in common use, along with very good examples. When I taught at Bechtel, we required every lean Six Sigma Black Belt to obtain a copy. Currently I teach out of Forrest Breyfogle’s book, *Integrated Enterprise Excellence, Volume 3,* which has licensed portions of the AIAG material, and it is also an excellent reference book.

### Ratio of items to appraisers to repeat measurements

The general answer to this ratio is to measure multiple items, multiple times, by multiple people. The most common ratios when all the analysis was performed by hand were 10 items, two measurements, by three people. I seem to recall that this ratio provided a good balance of the uncertainties, but it is far from being a requirement. You do need a minimum of two appraisers and a minimum of two measurements of each item by each appraiser, but the number of items to be measured can vary a good deal.

My guidance to most people is to consider where you believe the greatest sources of variation exist and increase the factor that would allow the best estimation of that factor.

• If the difference in appraisers (people) is expected to be high, use three or four appraisers of widely different experience levels

• If you expect that slight differences in the measured items may impact the reported measurement, make sure you include a full range of eight to 12 items that include at least one item outside the specification.

• If you believe that there may be a time-to-time change in the measurement system because of adjustments or setups methods, then include at least three replicated measurements from each appraiser of each item.

### Analysis method

There are two accepted MSA methods: ANOVA and X-bar R. I have always trusted the ANOVA method because it provides the ability to evaluate an interaction between the appraiser and the items. The X-bar R method is preferred when you are doing MSA by hand or using a calculator. When using a statistical analysis program, I recommend always using the ANOVA method.

### What is precision or gauge repeatability and reproducibility?

There are two components of uncertainty in a MSA: repeatability and reproducibility. This combination is labeled as the precision of a measurement system.

Repeatability is the variation found in a measurement system when the same item is measured over and over again without changing its position or who appraised it, and all at the same time. Get it: repeated measurements. This uncertainty estimate is really the smallest error you can get on a measurement system without fundamentally changing the equipment or the measurement process. It is reported as a standard deviation.

**Repeatability: variation in measuring something the same way, same person, same time, same conditions, same, same, same.**

Reproducibility is the variation found when trying to reproduce the measurement under different conditions. These different conditions will include the difference in appraisers, the difference in fixturing or positioning the item in the measurement tool, different times, and different calibrations. When this value is high, it implies that the measurement process is inadequate. It is reported as a standard deviation.

**Reproducibility: variation in measuring something in different ways, different times, different people, different, different, different.**

The precision is considered the true measurement system uncertainty. Precision is the square root of the sum of the squared repeatability value and squared reproducibility value. The precision value is the one that is compared to the specification and the process variability to determine the goodness. A large precision value may derive from a large repeatability value, a large reproducibility value, or both values being large. The component that has the largest contribution to the precision is where you address improvements.

**Precision: The total measurement system variation estimate. It includes repeatability and reproducibility.**

### What is “good” for a measurement system?

This is probably the most difficult part of the entire MSA concept to get across to students because the definition of “good” can depend on who taught you. The AIAG reference books list multiple methods to judge goodness, but I believe there are only two that matter: percent of the tolerance and percent of the process. Comparing the precision value to the requirements or specifications will tell you about the ability of the measurement system to be used for quality assurance. Comparing the precision to the process variation reported by the measurement system tells you about the ability of the measurement system to be used for quality control.

AIAG provides a standard measure of goodness as a ratio of the precision to a variation source. For quality assurance use, the ratio is (6*precision / width of the specification), which is reported as a percentage of tolerance. For quality control and SPC use, the ration is (precision / std-deviation of the process), which is reported as a percentage of the process.

For both percentages we apply the following general rules:

• >30 percent as unacceptable

• 20 percent to 30 percent as marginal

• 10 percent to 20 percent as acceptable

• < 10 percent as excellent

Some organizations consider 20 percent as a maximum allowable, but I have seen no single business sector or industry that has a standard; it has always seemed company-dependent.

### Measures of goodness that have little value

Many books recommend using the percentage of the test as a measure of goodness. This is reported as the (precision / standard deviation of the measurements in the test), which is reported as a percentage of test. This is not considered a reliable measure of goodness because the value will change each time you run the MSA and use different parts. This is not a good characteristic because it means that I can choose items that have widely varying values and make the percentage better than if I choose a series of items that have similar values.

A second common measure of goodness is the number of categories, which again is a ratio of the precision and the standard deviation of the tested items. I have the same problem with this measure as I do the percentage of the test.

## Comments

## Percentage of acceptability

I have a book from Dr. Wheeler Evaluating the Measurement Process III and I fully agree with his opinion about percentages of acceptability of measurement system. In short these percentages are deteriorating the measurement system. If the measurement system is stable, predictable and consistent validated by process behavior charts, I will not say that MSA with R&R = 35% is unacceptable. Depends on use. One of the purpose of measurement system is to be able to track process changes and for that is MSA with R&R = 50% still enough. I work in a machinery company with lots of single-purpose process measures and with this approach we should scrap half of them and then stop the lines. Unfortunately our customers request this approach because they are widely using AIAG studies as a Holy Bible which is unfortunately wrong.

I propose to read Dr. Wheeler´s article "Problems with Gauge R&R Studies" here on Quality Digest for more information and for different view. http://www.qualitydigest.com/inside/twitter-ed/problems-gauge-rr-studies.html

## MSA

MSA is probably one of the least understood and one of the most abused of investigation methods on reliability of measuring systems: I've seen it carried out on destructive measurements, by laboratory technicians instead of line operators only because Registrars' auditors request it. Even when the system is evidently stable, third party auditors request MSA be carried out once a year, at a company's great expense of time and money. And we are talking of auditors as graduated engineers, not of common laymen. It is therefore no wonder that companies' metrologists seldom care for it: once the output is within specs, it's cooked and ready for serving.