Control Charts: Simple Elegance or Legalized Torture?

Once again, I’m beginning to understand Deming’s hatred of statistical ‘hacks’

Published: Monday, January 6, 2014 - 10:26

For all the talk about the power of control charts, I can empathize when audiences taking mandated courses on quality tools are left puzzled. When I look at training materials or books, their tendency is to bog down heavily in the mechanics of construction without offering a clue about interpretation.

Some seminars even teach all seven control charts! And then there is the inevitable torturous discussion of "special cause tests" (usually the famous eight Western Electric rules). People are then left even more confused. Does each test signal need to be individually investigated, i.e., treated as a special cause? Not to worry—most people usually investigate only the points outside the control limits. The focus tends to be on individual observations. But what if there is one underlying explanation generating many of these signals that has nothing to do with individual outliers, e.g., a step change?

Someone once presented me with the graph shown in figure 1. (Yes, the y-scale started at 0.) It almost convinces you that there is a trend, eh?

Figure 1: Trend analysis of  percentage goal compliance

I can almost picture Six Sigma Black Belt No. 1 scolding them: "Now, now, now. Test the data for normality, and if it passes, you need to plot that as a control chart." Note: It does indeed pass (p-value = 0.507), but the test is totally inappropriate and irrelevant.

Using standard control chart software that also performs the eight Western Electric tests, the individuals' chart for percentage conformance is shown in figure 2:

Figure 2: Individuals Chart of percentage conformance to goal (click here for larger image)

Look at all those special causes in figure 2 shown on the top line of the graph. Sixteen of the 52 data points generate 30 signals. Where should you start? Many people would investigate the four points outside the three standard deviation limits (observations No. 9 and Nos. 50–52).

Then Black Belt No. 2 says, "The control chart needs to be adjusted for the trend." There's plenty of customer-friendly software that will do just that (figure 3):

Figure 3: Trend-adjusted percentage conformance to goal with seven special cause signals (click here for larger image)

That was obviously the solution. Figure 3 shows we're down from 30 special cause signals to seven. Better still, there are no data points outside the limits. Now what? Perhaps investigate each signal, but which one do you start with? And what about observations with more than one signal?

As I've said many times: The computer will do anything you want.

Trend lines and bar graphs seem to be the two most ubiquitous tools that people use for (alleged) analysis. Regression was my favorite course in grad school, and I'm very good at it. But I rarely use it. In my 30 years as a statistical practitioner, I have never ever seen an appropriate use of a trend line on data from a service industry (e.g., healthcare) plotted over time. Never. I have the utmost respect for my distinguished colleague Tom Nolan, and he agrees.

Over the years, I have developed an increasing affection for the much-neglected run chart: a time plot of your process data with the median drawn in as a reference (yes, the median—not the average). It is "filter No. 1" for any process data and answers the question: "Did this process have at least one shift during this time period?" (This is generally signaled by a clump of eight consecutive data points either all above or below the median.)

If it did, then it makes no sense to do a control chart at this time because the overall average of all these data doesn't exist. (Sort of like: If I put my right foot in a bucket of boiling water and my left foot in a bucket of ice water, on average, I'm pretty comfortable.)

All the time, I hear people saying, "We use run charts," but I see very few of them. It's generally taught as a boring prelude to the (allegedly) more important and powerful control chart. Usually that is left to the end of the training because it is the most difficult, which further complicates matters—peoples' brains are already full.

So, they stumble by rote through the hand calculations of each of the seven chart types and are then assured that they can now use their company's designated computer software to generate future charts—aided by the smart sofware add-in of guiding one to the allegedly correct chart. Many computer packages don't generate run charts. Well, who needs them? Why not jump right to the more advanced control chart analysis of all the data? One can then look at the special cause signals and try to find reasons for each individual signal.

The poor run chart: It does not find individual special-cause observations, but that is not its purpose. The control chart is "filter No. 2"—plotting the data after the shifts have been determined, which then usually reduces the number of special cause signals and results in a lot less confusion.

What does the run chart of these data in figure 4 tell us?

Figure 4: Run chart of percentage conformance to goal

With the y-axis scale a lot healthier and no control limits as a distraction, doesn't it look like the process "needle" shifted twice—around August 17 (observation  No. 21) and February 17 (observation  No. 47)? In fact, when I asked the clients about those two dates, they looked at me like I was a magician and asked, "How did you know?" Those dates coincided with two major interventions to improve this process. As the chart in figure 4 shows, they worked—two distinct needle bumps (step-change special cause)—not a continuously increasing improvement trend.

In other words, a process goes from what it is "perfectly designed" to get with its original inputs and transitions to what it is "perfectly designed" to get based on the new inputs. It eventually settles in to the new average based on these inputs. This also puts the trend special-cause test (six successive increases or decreases) into perspective: A step change can manifest as a trend during the transition. It won't continue.

So, the correct resulting control chart is shown in figure 5...

Figure 5: Percentage conformance to goal sans special cause (click here for larger image)

...and there's not a special cause to be found (other than the programmed step changes).

Interpretation: Their original performance was 78.5 percent. Their first intervention improved the process to 83.8 percent, and their second intervention improved that further to 91 percent.

They had recently started yet another intervention, and based on the last four data points, it's looking relatively promising. What would indicate success?

1. There are three immediate increases. Two or three more would be good evidence (trend transition).
2. Maybe a weekly performance will go outside the upper limit (97.8%).
3. Maybe the next four to six weeks will all be above the average (91%).
4. Two out of three consecutive weeks' performances will be between 95.5 percent (two standard deviations above the average) and 97.8 percent (three standard deviations' upper limit). This is a very useful test known as the two-out-of-three rule.

And, not to overreact, if performance goes down from one week to the next, be advised that it could differ as much as 8.4 percent from the previous week simply due to common cause (upper limit of the moving range chart). What could be simpler—if taught correctly?

Once again, I'm beginning to understand Deming's curmudgeonliness and his hatred of what he termed statistical "hacks."

How many hours are you spending in meetings looking at trends?

About The Author

Davis Balestracci

Davis Balestracci is a past chair of ASQ’s statistics division. He has synthesized W. Edwards Deming’s philosophy as Deming intended—as an approach to leadership—in the second edition of Data Sanity (Medical Group Management Association, 2015), with a foreword by Donald Berwick, M.D. Shipped free or as an ebook, Data Sanity offers a new way of thinking using a common organizational language based in process and understanding variation (data sanity), applied to everyday data and management. It also integrates Balestracci’s 20 years of studying organizational psychology into an “improvement as built in” approach as opposed to most current “quality as bolt-on” programs. Balestracci would love to wake up your conferences with his dynamic style and entertaining insights into the places where process, statistics, organizational culture, and quality meet.