Since “Sick Sigma” was published in April 2006, the public has become much more aware of Six Sigma’s failings. There is little risk today of being burnt at the stake for pointing out Six Sigma’s many faults. Last fall, even Dilbert discredited Six Sigma, pointing out that Six Sigma companies’ performance trail the Standard & Poor 500 index. It’s instructive to look at Six Sigma’s history in a little detail.

Motorola engineer Bill Smith suggested that where processes weren’t controlled, “batch-to-batch variation could be as much as +/–1.5 sigma off target.” Smith pulled the 1.5 value out of the air. In reality, where processes are uncontrolled, there is no limit to how far they can go wrong. Batch-to-batch variation might be 3, 5, 7 sigma or whatever, off target. Nevertheless, Smith’s figure of 1.5 was to become the core of today’s Six Sigma.

Why did Smith pick 1.5? At that time Motorola made extensive use of Shainin precontrol charts, as well as Shewhart charts. Precontrol charts define three bands—green, yellow and red. The green band is defined as 50 percent of the specification limits around the target, the yellow band is between the green band and the specification limits, and the red band is outside these limits. When a process moves into red or yellow bands, the methodology prescribes a set of corrective actions. If the process mean drifts within the green band, no action is necessary. It’s been shown that such procedures actually increase process variation. Because Motorola used a Cp=1 at the time, the green band was equivalent to +/–1.5 sigma off target. The mean was allowed to drift in this band. It’s been suggested that Smith confused precontrol charts’ +/–1.5 sigma drifting mean with control charts.

From 1987 on, Motorola changed its design tolerances to +/–6 sigma. That is, it changed the previous tolerance of Cp=1 to Cp=2, to reduce defects. It appears that this change was at Smith’s suggestion. “Another way to improve yield is to increase the design specification width. This influences the quality of product as much as the control of process variation does,” Smith said. In other words, to reduce defects, simply change the specification. This is perhaps the most fundamental flaw in Six Sigma, because its quality doesn’t relate to how well a process is performing. Six Sigma quality simply reflects where specification limits have been set.

Bill Smith made another major error. He introduced numbers relating to normal distributions and—without any justification—intimated that they apply to all processes. If he had read Shewhart’s work from 60 years previously, he would have realized that we can never know the exact form of any data distribution, nor do we need to. Donald Wheeler has shown that even with 3,200 data points, we can’t be sure what form a distribution has beyond +/–2.95 sigma. There’s no justification for assuming that any particular data set is normally distributed. Many data sets such as the time-based processes common in service industries, will be very skewed. Six sigma’s assumptions of normality don’t apply.

Sigma is a measure of data dispersion—the spread of data. Bill Smith used sigma values to give defect levels for normal distributions. The common practice—using defect counts to estimate sigma levels—is nonsensical, like trying to estimate the shape of a dog by looking at the extreme tip of its tail. For example, it’s impractical to collect sufficient data to distinguish whether any data set is a better fit to a Burr or normal distribution, and at six sigma an incorrect choice gives a 15,000-percent error.

After Smith pulled 1.5 sigma out of his hat, Mikel Harry attempted to justify it theoretically, as discussed in “Sick Sigma.” Harry based his justification on work by Bender and Evans, who looked at tolerances in the thickness of stacks of disks. Of course, stacks of disks bear no relation whatsoever to typical processes. Harry later suggested, “We employed the value of 1.5 since no other empirical information was available at the time of reporting” and later “The 1.5 constant would not be needed as an approximation.”

Harry also introduced his “Z shift” equation. This equation is derived from the sums of squares equality, where for a group of data points :

Overall variation = Variation between groups + variation within groups

or

SSoverall = SSbetween + SSwithin , where SS represents sums of squares.

Unlike control charts, to which Harry refers, this equation dispenses with time. That is, the sequence of data sets is irrelevant to the above equality. Despite this, Harry renamed the SSwithin figure “short term” and the SSoverall figure “long term.” To add further to his error, these figures use exactly the same data. In this context, there is neither “short term” nor “long term.” If that weren’t enough, Harry bases his equation on six sets of five data points. For 30-minute sampling, this represents just 2.5 hours. Hence, his short-term and long-term figures are 2.5 hours.

Harry’s erroneous derivation of the 1.5 made another change to Smith’s claims. Instead of the +/–1.5 applying to uncontrolled processes, as Smith had suggested, Harry doesn’t make this distinction. The nonsensical 1.5 became universal for all processes.

By the turn of the 21st century, Six Sigma was in full swing. Its appeal to management was its ability to reduce defects to a claimed level of 3.4 defects per million opportunities (DPMO). This 3.4 DPMO is based on the assumption that all processes have normally distributed data and that all processes experience a long-term drift or shift of +/–1.5 sigma. Some take this even further and make the outlandish claim that “all processes are out of control 13–14 percent” of the time, because of this shift. However with the old version of 1.5 starting to be challenged, a new theory was needed to support Six Sigma. Around 2003, Mikel Harry obliged.

Harry’s new theory is based on the fact that the standard deviation for any set of variable data has an error in its ability to estimate sigma for a population. With a very large data set, the standard deviation will be a good estimate of sigma for the population. Conversely, a small data set won’t give an accurate estimate. We can estimate the error in the estimate of sigma using a chi-square test. Depending on the number of samples and the probability that the standard deviation (SD) is an accurate estimate of sigma, we find various values.

For example, if we take a sample of 100 points, we can be 99-percent sure that the estimate of sigma is less than +1.2 SD or 90-percent sure it is less than +1.1 SD. Harry takes a special case of 30 points and a 99-percent confidence interval to give:

0.744 SD < sigma < 1.487 SD.

Harry then ignores the left-hand side of the equation, multiplies it by 3 (for 3 sigma control limits) ; subtracts 3 and gets a +/–1.5 correction to control limits. The left-hand side of the equation is ignored, because it would have given a value of –0.77 instead of the required –1.5.

By taking different values, the correction can be anything from 0.0 to over +/–50 (2 points, 99-percent confidence). Fortunately, we don’t need to make such corrections to control limits. As Shewhart pointed out, control limits don’t depend on probability. Control limits are economic limits and are always exactly 3.0.

Reigle Stewart, Harry’s partner, has added yet another calculation he calls a "dynamic mean off-set" as:

3/sqrt(n), where 3 is the value for control limits and n is the subgroup size. For n=4 he gets “1.5.”

“This means that the classic Xbar chart can only detect a 1.5 sigma shift (or larger) in the process mean when subgroup size is 4,” Stewart says. He is quite incorrect. Such data is readily available from average run length (ARL) plots.

In summary, the 1.5 doesn’t exist, despite the many attempts to prop it up. Calculations involving 1.5 are hence meaningless. That is, 3.4 DPMO is meaningless. Six sigma tables are meaningless. Sigma levels are meaningless. These measures should never be used.

Finally, it should be observed that Six Sigma’s 3.4 defects per million theory is similar to Phil Crosby’s “zero defects.” Both are based on a suggestion that product is good if it’s inside specification limits and bad if it’s outside. Costs related to processes are more important than just defects. Costs aren’t a step function, where costs are low when product is in specification and high when product is out of specification. Taguchi proposed that costs vary in some continuous way in relation to product characteristic, as described by his “loss function.” The result of Taguchi’s analysis can be shown to be that good quality means “on target with minimum variance.”

Forget about 3.4 DPMO. Manage processes well, and defects will take care of themselves.