In the article, “Four Control Chart Myths from Foolish Experts,” by Davis Balestracci (Quality Digest Daily, March 30, 2011) the following comments were made regarding what Balestracci considers statistical process control (SPC) myths:

“Myth No. 4: Three standard deviation limits are too conservative.
Reality: Walter A. Shewhart, the originator of the control chart, deliberately chose three standard deviation limits. He wanted limits wide enough so that people wouldn’t waste time interpreting noise as signals (a Type I error). He also wanted limits narrow enough to detect an important signal that people shouldn't miss (avoiding a Type II error). In years of practice he found, empirically, that three standard deviation limits provided a satisfactory balance between these two mistakes. My experience has borne this out as well. I’ve seen two standard deviation limits commonly used because people, especially in medicine, are obsessed that they might ‘miss something.’”

The real reality

The people in medicine were right, and the statistician should have listened to them because they clearly understood Shewhart’s thinking better than the author.

As Shewhart wrote on page 276 of his book, Economic Control of Quality of Manufactured Product, (D. Van Nostrand Co., 1931):

“How then shall we establish allowable limits on the variability of samples? Obviously, the basis for such limits must be, in the last analysis, empirical. Under such conditions it seems reasonable to choose limits q1 and q2 on some statistic such that the associated probability P is economic in the sense now to be explained. If more than one statistic is used, then the limits of all statistics should be chosen so that the probability of looking for trouble when any one of the chosen statistics falls outside its own limits is economic.”

Now while Shewhart was concerned with Type 1 and Type 2 error rates, he was smart enough, being trained as a physicist, to recognize that the error rate was only half of the issue. The other half was the cost of sampling, testing, and product failure. So even if the error rate is small if the cost per failure is large, then we must try to prevent the failure from happening. We do this by selecting a sample size, a sampling frequency, and control limits that reflect the economic risk we are exposing ourselves to. Three sigma limits are a reasonable choice for the average economic risk, but the average may not address the economic extremes very well.

Therefore, beginning with the seminal work of A. J. Duncan, Ph.D., in “The Economic Design of X-Bar Charts Used to Maintain Current Control of a Process ” (Journal of American Statistical Association, Vol. 51), there has been extensive research on the economic design of control charts. D.C. Montgomery, Ph.D., showed that under the assumptions of the Duncan model for the mean control chart:
• The magnitude of the process shift that we wish to detect are largely determined by the sample size.
• The sampling frequency is mainly caused by the penalty cost of production in the out-of-control state and the rate of occurrence of the search for assignable cause signals.
• The width of the control limits is chiefly affected by the cost of looking for assignable causes.
• The mean control chart should not be arbitrarily designed. [Montgomery, 1980]

So what are the implications of the economic design of control charts, and do they support the “one size fits all” concept of control chart design? The answer is no, as Montgomery stated, for in literally hundreds of peer-reviewed research papers, we see optimal economic designed charts with control limits that range from 1 to 4 sigma limits. However, like any model you have to apply good judgment when generating an economic model because getting reasonable cost estimates can be a daunting task.

For example, consider the heart pacemaker manufacturing case. Suppose we are sampling the process and performing an electrical test at a rate of five units per hour, computing the average, and plotting the result on a trend chart, when an observation arises that appears to be significantly different from our historical experience. The question is, should we search for assignable causes? To answer this question, we need to know how unusual the observation is and the costs of searching for an assignable cause vs. the cost we might face if we don’t search.

The cost of searching for an assignable cause in this case would probably mean sending the suspect unit to the lab, where electronic tests would be performed to validate the unusual observation, and microscopic inspection of the unit or other tests might be performed to try and to identify the causes of failure.

Suppose the unit has low output strength due to a cold solder joint, and the cost of searching and finding the cause is about $1,000. Now, assume that the unusual unit was not captured in a sample of the process production, and thus there was no indication of a problem on our trend chart, and hence no search was performed. The suspect unit would be encapsulated and the pacemaker placed in a patient, where it might fail catastrophically and the patient die. The cost of not searching is then perhaps $10 million in the civil litigation and significant negative publicity.

The cost ratio of inspection cost to field-failure cost is $1,000/$10,000,000 = 1 × 10–4. Now the test cost is small, and the search cost of $1,000 is still a small percentage of the field-failure cost. So what should we do? We have a production line making product that is cheap to test, but we are only checking five units per hour. So we apply W. Edwards Deming’s acceptance sampling rule: If the process fraction defective is greater than the break-even point (BEP), then do 100-percent inspection; otherwise, do no inspection. The BEP is the ratio of inspection cost to field-failure cost, which in this case is tiny. If the defect rate is p = 0.001, then clearly we are above the break-even point, so 100 percent is justified.

Let’s see what the Duncan model says about designing an optimal economic control chart for the pacemaker production process. Using Chiu’s model [Chiu, 1974] we use the cost data from the first example given by Duncan p. 236, where d = 2, l = 0.01, e = 0.05, D = 2, T = 50, W = 25, b = 0.50, c = 0.10, and M = 100 are the increase in the cost-of-escape failures per hour.

The optimal economic design for the mean control for this process is:
n = 5, h = 1.4, k = 3.2, where n is the subgroup sample size, h is the time between samples, and k is the sigma multiplier for the control limit.

Now, suppose we change M from 100 to 10,000, then h drops to 0.14, an order of magnitude decrease. So as M goes to infinity, h goes to zero. This tells us we need to do 100-percent sampling. However, our original process had a subgroup size n = 5, so we change this to n = 1, and this changes the economically optimal control limit value to from k = 3.2 to k = 0.7. So it looks like you should select a control limit multiplier that is less than 3 in this case if you want to save money, not kill anybody, and stay out of court.

In “Weakness of the Economic Design of Control Charts” (Technometrics, 1986, Vol. 28), author W. H. Woodall, Ph.D., points out some of the potential problems with economic-designed control charts, and his concerns need to be addressed. For example, some plans may have high false-alarm rates (like the pacemaker example). This can certainly be a problem for humans, but if the search for assignable causes can be automated, then it should not undermine operator confidence in the control tool. At the other extreme, if field-failure cost is low, the control limit may be high enough to allow many defects to escape, which can destroy the customer’s confidence in the product. This is why it is critically important to try and capture the intangible cost of customer dissatisfaction (i.e., negative advertising, and civil liability) in the field-failure cost. Further, some economic plans can have poor statistical performance (i.e., poor ARLo and ARLx). The application of some simple common sense guidelines (see notes below) can usually reduce these risks to tolerable levels and make economically designed control charts significantly less risky than adopting the one-size-fits-all approach.

This view fits perfectly with Shewhart’s pragmatic economic view of a control chart as a statistical heuristic to distinguish the two types of variation. The control limits were developed as a guide to help the practitioner decide when it was economically reasonable to expend resources in the search for assignable causes.

Note: If the economic model recommends large subgroups sizes n > 12, just default to n = 12 and reduce h, the time between samples. If the program recommends k < 2, then just default to k = 2 (unless you have solved the automated search problem), and if it recommends k > 4, just default to 4.

References:
1. Shewhart, W. A. (1931). Economic Control of Quality of Manufactured Product ISBN 73890760

2. Duncan, A. J. (1956). “The Economic Design of Mean-Charts used to Maintain Current Control of a Process,” Journal of American Statistical Association, Vol. 51, No. 274, pp. 228-242.

3. Chiu, W. K., and Wetherill, G. B. (1974). Journal of Quality Technology, Vol. 6, No. 2.

4. Woodall, W. H. (1986). “Weakness of the Economic Design of Control Charts,” Technometrics, Vol. 28, pp. 408-409.