“Jim, how do we know that your project made any improvement?” asked the Six Sigma champion.
The Black Belt candidate looked confused. “Um, the sponsor said he was happy with the outcome,” he offered. The members of the certification board looked skeptically at one another:
“But the only metrics you’ve shown us are from after the project was completed.” Another board member asked hopefully, “Didn’t you measure the key process output variable before you started the project?” The Black Belt candidate shook his head. “OK. thanks, Jim. We’ll get back to you,” said the certification board chairwoman. Jim wasn’t certified.
The certification criteria required that the Black Belt’s project produce a measurable improvement in a metric important to customers, shareholders or employees. Measurable improvement requires a basis for comparison, for example, a change for the better compared to a baseline measurement. Jim could only show that things were pretty good after the project was completed, but he couldn’t answer the question, “Were things better than they were before you did the project?” Without this, he obviously couldn’t establish that the project actually caused any improvement. Without a baseline measurement, the board couldn’t determine if things got better, worse or stayed the same. Jim had to either find historical data to help prove his point or use another project for certification.
Key process output variables (or CTQs) are important, of course. But so are key process input variables. Six Sigma clarifies the desired outcomes for all stakeholders, identifies the drivers and root causes necessary to achieve them, uses data wisely to help guide organizations as they try to address root causes and uses a systematic approach to continuously improve. Statistical thinking is a necessary part of Six Sigma because it does two things:
Without statistical thinking there’s no way to avoid superstitious learning. Following is an excerpt from BF Skinner’s classic paper “’Superstition’ in the Pigeon,” which describes this phenomenon in a laboratory experiment:
“Whenever we present a state of affairs which is known to be reinforcing… we must suppose that conditioning takes place, even though we have paid no attention to the behavior of the organism in making the presentation. A simple experiment demonstrates this to be the case.
A pigeon is brought to a stable state of hunger by reducing it to 75 percent of its weight when well fed. It’s put into an experimental cage for a few minutes each day. A food hopper attached to the cage may be swung into place so that the pigeon can eat from it. A solenoid and a timing relay hold the hopper in place for five seconds at each reinforcement.
If a clock is now arranged to present the food hopper at regular intervals with no reference whatsoever to the bird’s behavior, operant conditioning usually takes place. In six out of eight cases, the resulting responses were so clearly defined that two observers could agree perfectly in counting instances. One bird was conditioned to turn counter-clockwise about the cage, making two or three turns between reinforcements. Another repeatedly thrust its head into one of the upper corners of the cage. A third developed a ’tossing’ response, as if placing its head beneath an invisible bar and lifting it repeatedly.
Two birds developed a pendulum motion of the head and body, in which the head was extended forward and swung from right to left with a sharp movement followed by a somewhat slower return. The body generally followed the movement and a few steps might be taken when it was extensive. Another bird was conditioned to make incomplete pecking or brushing movements directed toward but not touching the floor. None of these responses appeared in any noticeable strength during adaptation to the cage or until the food hopper was periodically presented. In the remaining two cases, conditioned responses weren’t clearly marked.
The conditioning process is usually obvious. The bird happens to be executing some response as the hopper appears; as a result it tends to repeat this response. If the interval before the next presentation isn’t so great that extinction takes place, a second ’contingency’ is probable. This strengthens the response still further and subsequent reinforcement becomes more probable. It’s true that some responses go unreinforced and some reinforcements appear when the response has not just been made, but the net result is the development of a considerable state of strength.”
Of course, people aren’t pigeons, but they too learn from watching perceived signals from their environment. For example, there’s a statistical phenomenon known as regression to the mean. Simply put, regression to the mean states that a measurement relatively far from the mean is likely to be followed by a result that’s nearer to the mean. Managers typically take action when things “get out of hand” and when results perceived to be atypical are observed. Because it’s likely that these atypical results will be followed by results closer to normal, even if nothing is done, the managers’ behavior will be rewarded with the expected result. Managers conclude that their action caused the favorable result and, therefore, the behavior will be reinforced. Without good statistical guidelines, managers tend to believe that their actions caused the observed effect. Like the pigeon, managers repeat their reinforced behavior.
Statistical thinking minimizes superstitious learning in two ways. First, it provides unambiguous, statistically derived guidelines to tell project managers when an observed result is due to an important cause that can be readily determined, and not due to a chance combination of many small causes. It does this by providing managers with a set of analytical tools (usually graphical) that help them understand the behavior of dynamic processes over time. Second, statistical thinking is aided by powerful experimental design tools that help project managers investigate models of cause and effect by using feed forward instead of feedback.
Feed forward means that managers develop predictive models, then test them by carefully examining their ability to predict the future. The differences between the specialists’ predictions and the actual results (i.e., error) are studied to help them determine if their models are systematically wrong. Unlike superstitious learning from favorable results, feed forward models can be manipulated to produce results that are farther from the mean as well as results nearer to the mean. A well planned design of experiments (DOE) project will include such confirmatory runs.
Statistical thinking also focuses managers’ attention on variation as well as averages. Their statistical guidelines and experiments incorporate variability into the calculations. This is critical because customers feel variation. Without the statistical sophistication of Six Sigma, managers tend to only look at average performance, thus missing this critical component of performance.
