Featured Product
This Week in Quality Digest Live
Six Sigma Features
OpusWorks
Eight unique best-practice sessions featuring 11 process improvement and thought leaders
Harish Jose
Learning how to better ask “Why?”
Richard Harpster
Good news? You are probably already doing it.
Donald J. Wheeler
Does your approach do what you need?
James J. Kline
Quality professional organizations need to adjust their body of knowledge to include an understanding of big data

More Features

Six Sigma News
Sept. 28–29, 2022, at the MassMutual Center in Springfield, MA
Elsmar Cove is a leading forum for quality and standards compliance
Is the future of quality management actually business management?
Too often process enhancements occur in silos where there is little positive impact on the big picture
Collect measurements, visual defect information, simple Go/No-Go situations from any online device
Good quality is adding an average of 11 percent to organizations’ revenue growth
Floor symbols and decals create a SMART floor environment, adding visual organization to any environment
A guide for practitioners and managers
Making lean Six Sigma easier and adaptable to current workplaces

More News

Davis Balestracci

Six Sigma

Are Your Processes ‘Too Variable’ to Apply Statistical Thinking?

I doubt it

Published: Wednesday, May 28, 2014 - 10:52

My recent columns have emphasized the need for critical thinking to understand the process that produces any data. By just plotting data in their naturally occurring time order, many important questions arise about the data’s “pedigree” (a term coined by my respected colleagues Roger Hoerl and Ron Snee), including question one: “How were these data defined and collected?”

That simple time plot can easily be converted to a run chart—a time-ordered plot of data with the median drawn in as a reference line. This helps to answer question two: “What was the state of the process that produced these data?”

Two intuitively simple rules can be applied to the chart to determine whether a process has exhibited significant shift during the plotted period:

Rule 1: A trend of six consecutive increases or decreases, indicating a transition to a new process average. If the total number of plotted points is 20 or fewer, you can use five increases to indicate a special cause.

Actually, the question of when to use five vs. when to use six is pretty much moot: This test is rarely triggered. I give it to tell you what a trend is not.

Special case of Rule 1: Some recent articles are declaring four rather than six successive increases or decreases as a special cause. This is true if (and only if) you’ve made a specific intervention and are applying it to the immediate post-intervention data points. If the first four subsequent data points trend in your desired direction, then the intervention was effective. Even then, most effective changes manifest so quickly that you won’t even have time to observe this signal!

Rule 2: A cluster of eight consecutive points either all above or all below the median. This is by far the most common test triggered in plots of historical data.

Special case of Rule 2: As above, the same articles have relaxed the eight consecutive points above to six. Once again, this is true if (and only if) you’ve made a specific intervention and are applying it to the immediate post-intervention data points. If the first six subsequent data points are all above or below the median in your desired direction, then the intervention was effective.

Beware of a common smoothing technique

We now come to question 3: Given the answers to 1 and 2, are the current analysis and interpretation appropriate?

What would you do if you were presented with the run charts in figures 1–3 below? (Spoiler: Actually, these are three plots of the exact same data.)

Figure 1: Run chart of Process 1

Figure 2: Run chart of Process 2

Figure 3: Run chart of Process 3

The same data? How can that be, you ask? Well, I’m going to tell you how these data were defined and collected (question 1 above).

I generated 61 random, normally-distributed observations (average = 50, standard deviation = 5).

Figure 1 (Process 1) is a run chart of observations 12 through 61, i.e., the last 50 observations.

Plotting a set of data in its naturally occurring time order via a run chart assesses the state of the process that produced it (question 2 above). If you apply the two rules above, there are, appropriately, no special causes.

Figure 2 (Process 2) is a run chart of the same data, except I’ve “rolled” three previous observations into each data point and taken the resulting average. This is similar to a common technique used in financial data analysis, the four-quarter rolling average.

So, given the original 61 observations, the first data point is the average of observations nine through 12, the second data point averages observations 10 through 13, all the way to the final data point, which averages observations 58 through 61. Each data point has three observations’ worth of “memory” from its immediate predecessor. So it is indeed, once again, a plot of observations 12 through 61.

Figure 3 (Process 3) is a run chart whose data were obtained similarly to process 2, except that 11 previous observations (rather than three) were rolled into each data point and averaged.

This is similar to the common 12-month rolling average analysis (from my experience, a very common basis for determining days outstanding for accounts receivable—a number that really seems to make executives perspire).

So, given my original data (i.e., 61 observations), the first data point is the average of observations one through 12; the second data point averages observations two through 13, all the way to the final data point, which averages observations 50 through 61. Each data point has 11 observations’ worth of “memory” from its immediate predecessor. So, it too is a plot of observations 12 to 61.

As you see in figures 2 and 3, the two statistical run chart analysis rules have multiple violations; however, because of the memory inherent in each data point, the usual runs analysis for taking action is inappropriate for the way these data were defined and collected (question 3 above) because the observations are not independent. Needless to say, control charts would be equally invalid.

Next, in figure 4 I proceeded to generate 50 more observations from the same process (average = 50, standard deviation = 5). These data were then appended to the Process 1 graph above, resulting in the run chart for these combined 100 observations.

Figure 4: Run chart of process data combined

As expected, due to the non-occurrence of the two run chart special-cause rules, this plot confirms this process stability of the two combined sets of data.

In figure 5 I proceeded as before to generate and plot the rolling averages of four by appending the second set of data to the first and continuing the previous “rolling” calculations.

Figure 5: Run chart of combined data, rolling averages of 4

In figure 6, I did the same for the rolling averages of 12, appending the second set of data to the first and continuing the previous “rolling” calculations.

Figure 6: Run chart of combined data, rolling averages of 12

Note that the initial impression in the second half of the rolling average graphs in both figure 5 and figure 6 is totally different from first half—and equally incorrect. Worse yet, they create the appearance of special causes that aren't even there.

‘Smoothing’ variation to make it go away doesn’t make it go away

The point is, even though the underlying process is stable, “rolling” the numbers creates the appearance of rampant instability—all in the name of innocently trying to reduce variation through (allegedly) “smoothing” them. As W. Edwards Deming was so fond of saying, “For every problem there is a solution: Simple... obvious... and wrong!”

This also creates extra, nonvalue-added work in an organization when explanations for these artificially created special causes (no doubt called “trends”) are demanded... and found... and acted upon.

And Wall Street does love that 52-week rolling average.

As you can see, mathematical manipulations don’t make a process’s inherent variation disappear. “It is what it is” and must be understood so that appropriate action can be taken. Ultimately, it isn’t as difficult as one would think but initially, it is most certainly counterintuitive.

How much longer can we as a profession sit on our hands and wait for execs to “get” it?

Discuss

About The Author

Davis Balestracci’s picture

Davis Balestracci

Davis Balestracci is a past chair of ASQ’s statistics division. He has synthesized W. Edwards Deming’s philosophy as Deming intended—as an approach to leadership—in the second edition of Data Sanity (Medical Group Management Association, 2015), with a foreword by Donald Berwick, M.D. Shipped free or as an ebook, Data Sanity offers a new way of thinking using a common organizational language based in process and understanding variation (data sanity), applied to everyday data and management. It also integrates Balestracci’s 20 years of studying organizational psychology into an “improvement as built in” approach as opposed to most current “quality as bolt-on” programs. Balestracci would love to wake up your conferences with his dynamic style and entertaining insights into the places where process, statistics, organizational culture, and quality meet.