Assuming there are still questions about the suitability of a gauge for a particular application after a gauge repeatability and reproducibility (GR&R) study, Measurement Systems Analysis, 4th Edition (MSA-4), published by AIAG in 2010, recommends using an additional study in the calibration environment for more information on bias, linearity, or stability. Each of these studies can provide information about repeatability on reference standards and other variables, in addition to information on the variable for which they are named.
ADVERTISEMENT |
The most popular study for that purpose is the bias study. It requires one operator, and one reference standard (or calibrated part). Typically, it uses 25 repetitions. This article will use popular software, and published data, to demonstrate all the uses for a bias study.
Step 1: A gauge R&R study (not shown)
A routine GR&R study on a gauge showed GRR% tolerance = 1.68% (maximum 30%). This was an astonishingly good result, but it seemed too good to be true. The gauge was, for unknown reasons, suspected of having high uncertainty. Because the gauge was suspected of having a problem, and the GR&R study had failed to detect that problem, the quality department decided to do a bias study for further investigation. There would be follow-up steps as required. At each step the investigation might end, or might continue to another step.
Step 2: Collect data for bias study
The published data in figure 1 are from the same gauge mentioned above. It is an automated test device that uses a dial gauge.
Step 3: Analyze bias
Figure 2 shows the bias analysis as presented in the stability module of GAGEtrak software. The bias, meaning the average difference between the measured value and the reference value, is “significantly large.” The software decides this based on the 95% confidence bounds, which are a type of t test. If the lower limit is positive, or the upper limit is negative, the software marks it “unacceptable,” meaning the bias is too big to ignore. The report shows EV % of tolerance = repeatability % of tolerance = 5.5%, meaning repeatability on the reference standard is good enough that we can trust the 95% confidence bounds. (“EV,” which stands for equipment variation, is the abbreviation for repeatability in gauge studies, because a regular abbreviation would be confused with reproducibility.)
Since they should not ignore the bias, their choices are:
1. Adjust the gauge to remove the systematic error (they are already trying unsuccessfully to do that)
2. Compensate for the systematic error, e.g., subtract 0.0005 from each reading (rarely practical in production)
3. Treat bias as a random variable, which would increase the uncertainty of calibration
The team decided on option No. 3. They also decided to add a step to check the overall uncertainty of calibration, to be sure the gauge is still OK to use with the added uncertainty. Before proceeding they would look at the short-term stability analysis that the software was ready to display.
Step 4: Analyze short-term stability
Figure 3 shows the short-term stability analysis in the form of a control chart. There is some minor instability but no apparent change in bias (see the individual chart in figure 3). There is also no apparent change in repeatability (see the moving-range chart in figure 3). The team concluded also that the “out of control” point was not caused by a typo or a bad reading. Rather, the repeatability estimate just happens to be smaller than the resolution. That had probably enabled the control charts to detect a routine round-off error.
Step 5: Create preliminary uncertainty budget for calibration
The conclusion of the bias analysis was that there was a bias problem. But the team wanted to treat bias as a random variable rather than compensate for it. They also decided to find the overall calibration uncertainty and make sure the gauge would still be OK to use without compensating for systematic error from bias. They knew the bias analysis, done by the software, had already calculated all or most of the statistics that would be needed.
The next step would be to prepare a form called an “uncertainty budget.” This form is used to summarize and combine all of the appropriate uncertainty contributors. GAGEtrak software locates the uncertainty budget in the linearity module, and the linearity module is able to do bias studies. For convenience, we re-entered the same data into the linearity module, which duplicated the bias analysis and automatically generated a preliminary uncertainty budget. (See figure 4.)
Discussion of uncertainty budgets
Uncertainty budgets have been around a long time and are used for calculating the expanded uncertainties we are accustomed to seeing on calibration certificates from outside calibration sources. Uncertainty budgets are not covered in MSA-4. Uncertainty budgets are, of course, described in the help files of software that can use them, and uncertainty budgets now have their own reference manuals similar to MSA-4. They are listed at the end of this article.
The uncertainty budget in figure 4 was proposed by GAGEtrak software, using traditional estimators. We call it “preliminary” because it is common to make a couple of changes and additions before the final calculations.
What the uncertainty budget columns mean
The first column, “Uncertainty Contributor,” is self-explanatory. The second column, “Type,” designates each uncertainty as type A if the plus or minus statistic was calculated from a current study. If the statistic was not calculated, or was from a study not current, it is designated type B.
The “Plus or Minus” column is there because we can’t always directly estimate variation by standard deviations. A more versatile way is to ask for a plus or minus value and imagine what the shape of the probability distribution would look like if you could see a histogram. To illustrate what is meant by shape, figure 5 below shows a histogram made by the software using the bias data from this example. With experience, we would expect or imagine bias data to be a normal (i.e., bell) shape. Some might say this particular sample looks more like a triangle, but that is untypical for bias. The choice of probability distribution should always be based on the “imaginary” typical shape. The software we are using will usually recommend a shape. (If the software doesn’t recommend a shape, and we can’t decide what shape would be appropriate, we would consult one of the manuals listed below, or follow the suggestion in the next paragraph.)
The “Probability Distribution” column has a drop-down list with shapes listed on it such as normal (bell-shaped), triangular, and rectangular. Rectangular probability distributions are popular because they occur fairly often, and they are also considered the safe choice when the shape is unknown. Based on the chosen shape, the software will make an appropriate choice for the “Divisor” column, and use it to calculate the “Uncertainty Contribution” column. The Uncertainty Contribution column is summed by a special method called RSS (root sum square).
Those six are all the columns required for a basic uncertainty budget. This software has two additional columns. “Sensitivity Coefficient” is a place to insert a multiplier, when that would be convenient, and “df” (degrees of freedom) is used to calculate the t statistic for 95% confidence. To calculate expanded uncertainty, we customarily use a multiplier of 2, called a “coverage factor.” Because 2 is actually an approximation of t for 95% confidence, we can compare the approximation to the actual. If t were large compared to 2, it would mean the team should do something to compensate for small sample size.
Step 6: Edit the preliminary uncertainty budget
See figure 6 for the edited version of the uncertainty budget.
1. Linearity is changed to type B. It was not calculated by the bias study. The study team had assumed linearity was negligible, based on calibration experience with this gauge.
2. They didn’t want to use the traditional estimator for uncertainty from bias. They wanted to treat bias as a random variable. They consulted a reference manual we will call VDA-5 (listed below). VDA-5 recommended using Avg. Bias from figure 3 as the plus or minus value and changing the probability distribution to rectangular.
3. Resolution is type B because it was not calculated. It came from a database or from observation. VDA-5 also recommended, if the divisor for resolution was 1.732, to double it or do something equivalent. They entered a multiplier of 0.5 in the sensitivity coefficient column as a convenient equivalent.
4. They entered the uncertainty of reference standard from the certificate.
5. They entered the uncertainty of the maximum permissible error (MPE). An MPE had been specified in the calibration procedure.
6. VDA-5 offers a list of capability index and ratio formulas that can be used to decide whether the expanded uncertainty is too much. They selected “% tolerance” because it is easy to calculate and had also been used in the GR&R study and the bias study.
7. Resolution % tolerance = 100 (resolution / tolerance) = 100 (0.0005 / 0.0400) = 1.25% (5% maximum)
8. Calibration-uncertainty % tolerance = 100 (2 × expanded uncertainty / tolerance) = 100(2 × 0.002326 / 0.0400) = 11.63% (15% maximum)
For calibration-uncertainty % tolerance, 11.63% is within the acceptable range of 15% but is nothing to brag about. The team still could not explain why the gauge was suspected of high uncertainty. So far, the biggest uncertainty contributor was the reference standard. They decided to take a final step and extend the uncertainty budget to include production issues. Then the study would be called “measurement uncertainty” rather than calibration uncertainty.
Step 7: Extend the uncertainty budget to include measurement uncertainty
In this case, that means including calibration uncertainty, plus uncertainty contributions from GR&R and from uncontrolled temperatures. (The VDA-5 manual shows how to calculate temperature uncertainties.) See figure 7.
Measurement-uncertainty % tolerance = 100 (2 × expanded uncertainty / tolerance) = 100 (2 × 0.0000435 / 0.0400) = 21.75% (30% maximum).
The study team had found the problem they were looking for. Technically, 21.75% is within the 30% maximum. However, this company considers over 20% as “needs improvement.” Looking at both uncertainty budgets to find the major uncertainty contributors, it turns out the dominant uncertainty contributor is temperature uncertainty from setup. Specifically, the setup procedure does not compensate for difference of expansion between the working standard and the test part. The second largest contributor is the uncertainty of the reference standard.
Conclusions
The software makes it easy to do “what-if?” analysis by temporarily placing a zero or other multiplier in the sensitivity coefficient column for an item of interest. In this case, repeatability and reproducibility are already excellent. However:
• If temperature uncertainty from setup could be made negligible with a more advanced setup procedure, measurement-uncertainty % tolerance would drop to 12.78% instead of 21.75%.
• If uncertainty of the reference standard could be reduced 50%, it wouldn’t help much.
Manuals for uncertainty budgets
These manuals are available in multiple languages.
VDA-5, the manual we referred to earlier, is published by the Verband der Automobilindustrie (VDA), the German association of the automotive industry, as Capability of Measurement Processes, Volume 5 (VDA, 2011). Examples are included in the manual. The example in this article is based on VDA-5, Annex F.4, on pages 121–125.
More detail on basic uncertainty budgets and gauge capability formulas is available in the ISO standard ISO 22514-7:2012—“Statistical methods in process management—Capability and performance— Part 7: Capability of measurement processes.”
For information about the studies used to generate the numbers, see MSA-4.
For those interested in a theoretical approach that covers both simple and complex uncertainty budgets, see Evaluation of Measurement Data—Guide to the Expression of Uncertainty in Measurement (GUM), published in 2008 by the Bureau International des Poids et Mesures and available on the BIPM website. It can be read online, or downloaded free. It is also available as an ISO document. GUM generally does not address application-type questions such as which estimator to use, or how much is too much.
Credits
Software used for this article is GAGEtrak calibration management software furnished by CyberMetrics, Phoenix, Arizona.
Gary Phillips is a measurement systems consultant for CyberMetrics, a Quality Digest content partner.
Add new comment