Few things breed more misunderstanding than the term "significance" in statistical process control. Quality and statistics professionals often use the word in one sense while their audience understands it in a completely different way.

For example, I once helped a team of process engineers experiment on a new vapor-phase soldering process. Being new, the process produced much variation from many unknown sources. This statistical noise overwhelmed any signals caused by the controlled changes we made to the variables we were studying. I commented that, based on the experiments' results, none of the effects were significant. An engineer disagreed, saying that, according to well-  established scientific principles, all the variables tested were important.

He was right. So was I. Our miscommunication stemmed from different interpretations of "significant."

When discussing statistical significance, statisticians and quality engineers usually are referring to significance tests such as Student's t-test for equality of means. In simple terms, significance testing answers the question, "Are these things really different, or could the difference be due to pure chance?"

However, problems arise when these test results are presented to nonstatisticians. In common parlance, "significance" and "importance" are synonymous. When statisticians speak of "statistical  significance" (often dropping the "statistical" and adding to the confusion), then "importance" and "significance" bear no relationship to one another. In fact, the following can be said:

 A difference or effect can be important but insignificant.

 A difference or effect can be both significant and important.

 A difference or effect can be unimportant and insignificant.

 A difference or effect can be significant but of no importance to the situation at hand.

In short, "important" and (statistically) "significant" are not synonymous. Consider the following examples:

 Important but not significant -- A hospital's fatality rate for coronary artery bypass surgery doubled from one week to the next. The loss of life clearly is important. However, if the average death rate is low -- say, 0.5 percent -- and the sample size is small -- 100 surgeries per month -- then one month's fatality rate can double the previous month's without being (statistically) significant.

 Significant and important -- A part's average hole size falls below the lower control limit, and the part won't work in the field. This agreement in meaning between significant and important is, however, purely coincidental. The fact that it often occurs only adds to the confusion in other circumstances.

 Unimportant and not significant --  The part's hole size changes by a statis-tically meaningless amount, and the part still functions perfectly. Again, agreement in meaning here can cause confusion at other times.

 Significant but not important -- Carpenter A saws wall studs to within ±1/64" of the nominal length, while Carpenter B saws them with twice the variation at ±1/32". The contractor can tolerate ±1/8". Because each carpenter cuts many studs, even small differences are statistically significant. However, a difference of only 1/64" in a sawing process's total spread is of no economic or architectural importance.

Significance is a technical term when used in statistical science. It has a precise meaning as determined by the statistical test being applied. While not advisable, it's possible to determine statistical significance without knowing what the numbers mean or how the results will be used.

Importance, on the other hand, is a nontechnical term. The questions being answered are implicit: Important to whom? Important for what purpose? Unlike analyzing statistical significance, pat answers can't be given in advance regarding a particular number's importance.

It's important to think before reacting to an observed difference, whether it's significant or not. Figure 1 provides some useful guidelines for the thinking process.

About the author

Thomas Pyzdek is president and CEO of Pyzdek Management Inc. Comments can be e-mailed to him at Tom Pyzdek .