*That’s*fake news.

*Real*news COSTS. Please turn off your ad blocker for our web site.

Our PROMISE: Our ads will never cover up content.

Six Sigma

Published: Wednesday, May 28, 2014 - 10:52

My recent columns have emphasized the need for critical thinking to understand the process that produces any data. By just plotting data in their naturally occurring time order, many important questions arise about the data’s “pedigree” (a term coined by my respected colleagues Roger Hoerl and Ron Snee), including question one: “How were these data defined and collected?”

That simple time plot can easily be converted to a run chart—a time-ordered plot of data with the *median *drawn in as a reference line. This helps to answer question two: “What was the state of the process that produced these data?”

Two intuitively simple rules can be applied to the chart to determine whether a process has exhibited significant shift during the plotted period:

**Rule 1:** A trend of *six consecutive* increases or decreases, indicating a transition to a new process average. If the total number of plotted points is *20 or fewer*, you can use *five* increases to indicate a special cause.

Actually, the question of when to use five vs. when to use six is pretty much moot: This test is rarely triggered. I give it to tell you what a trend is *not*.

*Special case of Rule 1:* Some recent articles are declaring four rather than six successive increases or decreases as a special cause. This is true if (and only if) you’ve made a specific intervention and are applying it to the immediate post-intervention data points. If the *first four subsequent* data points trend in your desired direction, then the intervention was effective. Even then, most effective changes manifest so quickly that you won’t even have time to observe this signal!

**Rule 2:** A cluster of *eight consecutive* points either all above or all below the median. This is by far the most common test triggered in plots of historical data.

*Special case of Rule 2:* As above, the same articles have relaxed the eight consecutive points above to six. Once again, this is true if (and only if) you’ve made a specific intervention and are applying it to the immediate post-intervention data points. If the *first six subsequent *data points are all above or below the median in your desired direction, then the intervention was effective.

We now come to question 3: Given the answers to 1 and 2, are the current analysis and interpretation appropriate?

What would you do if you were presented with the run charts in figures 1–3 below? (Spoiler: Actually, these are three plots of the exact same data.)

**Figure 1:**

**Figure 2:**

**Figure 3:**

The same data? How can that be, you ask? Well, I’m going to tell you how these data were defined and collected (question 1 above).

I generated 61 random, normally-distributed observations (average = 50, standard deviation = 5).

Figure 1 (Process 1) is a run chart of observations 12 through 61, i.e., the last 50 observations.

Plotting a set of data in its naturally occurring time order via a run chart assesses the state of the process that produced it (question 2 above). If you apply the two rules above, there are, appropriately, no special causes.

Figure 2 (Process 2) is a run chart of the same data, except I’ve “rolled” three previous observations into each data point and taken the resulting average. This is similar to a common technique used in financial data analysis, the four-quarter rolling average.

So, given the original 61 observations, the first data point is the average of observations nine through 12, the second data point averages observations 10 through 13, all the way to the final data point, which averages observations 58 through 61. Each data point has three observations’ worth of “memory” from its immediate predecessor. So it is indeed, once again, a plot of observations 12 through 61.

Figure 3 (Process 3) is a run chart whose data were obtained similarly to process 2, except that 11 previous observations (rather than three) were rolled into each data point and averaged.

This is similar to the common 12-month rolling average analysis (from my experience, a very common basis for determining days outstanding for accounts receivable—a number that really seems to make executives perspire).

So, given my original data (i.e., 61 observations), the first data point is the average of observations one through 12; the second data point averages observations two through 13, all the way to the final data point, which averages observations 50 through 61. Each data point has 11 observations’ worth of “memory” from its immediate predecessor. So, it too is a plot of observations 12 to 61.

As you see in figures 2 and 3, the two statistical run chart analysis rules have multiple violations; however, because of the memory inherent in each data point, the usual runs analysis for taking action is *inappropriate for the way these data were defined and collected *(question 3 above) because the observations are not independent. Needless to say, control charts would be equally invalid.

Next, in figure 4 I proceeded to generate 50 more observations from the same process (average = 50, standard deviation = 5). These data were then appended to the Process 1 graph above, resulting in the run chart for these combined 100 observations.

**Figure 4:**

As expected, due to the non-occurrence of the two run chart special-cause rules, this plot confirms this process stability of the two combined sets of data.

In figure 5 I proceeded as before to generate and plot the rolling averages of four by appending the second set of data to the first and continuing the previous “rolling” calculations.

**Figure 5:**

In figure 6, I did the same for the rolling averages of 12, appending the second set of data to the first and continuing the previous “rolling” calculations.

**Figure 6:**

Note that the initial impression in the second half of the rolling average graphs in both figure 5 and figure 6 is totally different from first half—and equally incorrect. Worse yet, they create the appearance of special causes that* aren't even there.*

The point is, even though the underlying process is stable, “rolling” the numbers creates the appearance of rampant instability—all in the name of innocently trying to reduce variation through (allegedly) “smoothing” them. As W. Edwards Deming was so fond of saying, “For every problem there is a solution: Simple... obvious... and wrong!”

This also creates extra, nonvalue-added work in an organization when explanations for these artificially created special causes (no doubt called “trends”) are demanded... and found... and acted upon.

And Wall Street does love that 52-week rolling average.

As you can see, mathematical manipulations don’t make a process’s inherent variation disappear. “It is what it is” and must be understood so that appropriate action can be taken. Ultimately, it isn’t as difficult as one would think but initially, it is most certainly counterintuitive.

How much longer can we as a profession sit on our hands and wait for execs to “get” it?