Two Questions That Get You Nowhere

‘The purpose is not to have charts. The purpose is to use the charts.’

Published: Tuesday, April 29, 2014 - 15:34

In my column, “The Universal Process Flowchart × Four,” I challenged you to look at the everyday use of data as a huge source of hidden waste. I suggested looking at a sample of any routine meeting’s raw data and asking eight questions, the last of which could be the most important of all: Does a plot of these data in their naturally occurring time order exist? How could you construct one?

Here is some wisdom from Dr. Donald Berwick—offered almost 20 years ago during his 1995 Institute for Healthcare Improvement Forum plenary speech, “Run to Space”:

“Plotting measurements over time turns out, in my view, to be one of the most powerful devices we have for systemic learning.... Several important things happen when you plot data over time. First, you have to ask what data to plot. In the exploration of the answer you begin to clarify aims, and also to see the system from a wider viewpoint. Where are the data? What do they mean? To whom? Who should see them? Why? These are questions that integrate and clarify aims and systems all at once. Second, you get a leg up on improvement. When important indicators are continuously monitored, it becomes easier and easier to study the effects of innovation in real time, without deadening delays for setting up measurement systems or obsessive collections during baseline periods of inaction. Tests of change get simpler to interpret when we use time as a teacher.... So convinced am I of the power of this principle of tracking over time that I would suggest this: If you follow only one piece of advice from this lecture when you get home, pick a measurement you care about and begin to plot it regularly over time. You won't be sorry.”

Note: He doesn’t say to construct a control chart, but just plot the data in their naturally occurring time order. From the questions he and I suggest, can you see how this will start a whole new—and more productive—conversation?

Quality improvement in healthcare was still relatively new in 1995, but interest was swelling. A tsunami of training has since ensued that ignores this sound basic message in favor of teaching a gamut of quality tools, many of which are statistical. These include all seven types of control charts and even more advanced (and worthless) statistics to get “belts.”

Rather than asking deep questions to gain further insight into critical thinking skills inherent in the questions above, two questions that seem to be of greatest concern to practitioners are:

1. “Which chart do I use for which situation?”
2. “When and how often should I recalculate my limits?”

These are the wrong questions!

Let me emphasize a key point with a comment by my respected colleague Donald Wheeler: “The purpose is not to have charts. The purpose is to use the charts.... You get no credit for computing the right number—only for taking the right action. Without the follow-through of taking the right action, the computation of the right number is meaningless.”

How many of you remember the torture of doing control chart calculations by hand—all seven? The time spent on these minutiae is sheer agony and wastes time with the wrong emphasis. I’ll deal with the different charts in a future column, in which I will also explain my overwhelming preference for the control chart for individuals (I-chart) as a first choice.

It never fails: I’ll be barely done describing the technique of I-charts—never mind teaching it—when, as if on cue, someone will ask, “When and how often should I recalculate my limits?” I’m at the point where it triggers my “fingernails on the blackboard” internal reaction.

If you feel the instinct to ask that question, pause and think of how you would answer these critical-thinking questions from me instead:

1. Could you please show me any existing data (or describe an actual situation) that are making you ask me this question?
2. Please tell me why this situation is important.
3. Please show me a run chart of these data plotted over time.
4. What ultimate actions would you like to take with these data?
5. What “big dot” in the board room are these data and chart going to affect? Or less tactfully:
5a. Who cares whether the limits are correct or not?

Once you supply me with the answers to one and two, we can begin a dialogue, during the course of which I would be happy to answer your question about limits. I will talk about that in my next column. Knowing when to recalculate limits is relatively useful information, but for the moment, you've got far more important things to do.

Answering Berwick’s questions and mine is important preliminary work in addressing any significant issue. In the process leading up to actually getting a chart, they address the issue alluded to in statistician John Tukey’s comment (paraphrased), “The more you know what is wrong with your data, the more useful it becomes.”

Now, I can hear some of you asking, “Could you please give me some examples?” to which I reply, “Show me some existing data from a meeting you regularly attend about a problem that just won’t seem to go away.” And: “Please tell me why this situation is important.”

Can you see where this is going?

About The Author

Davis Balestracci

Davis Balestracci is a past chair of ASQ’s statistics division. He has synthesized W. Edwards Deming’s philosophy as Deming intended—as an approach to leadership—in the second edition of Data Sanity (Medical Group Management Association, 2015), with a foreword by Donald Berwick, M.D. Shipped free or as an ebook, Data Sanity offers a new way of thinking using a common organizational language based in process and understanding variation (data sanity), applied to everyday data and management. It also integrates Balestracci’s 20 years of studying organizational psychology into an “improvement as built in” approach as opposed to most current “quality as bolt-on” programs. Balestracci would love to wake up your conferences with his dynamic style and entertaining insights into the places where process, statistics, organizational culture, and quality meet.