‘What’s the Trend?’

Wrong question!

Published: Monday, August 11, 2014 - 11:13

In my last column, I showed how hidden special causes can sometimes create the appearance of common cause, but the purpose of common-cause strategies is to deal with this and smoke them out. When there's an underlying structure to how these data were collected, or when one can somehow code each individual data point with a trace to a process input, stratifying the data accordingly can many times expose these special causes.

Even the simple coding of individual points on a graph can be every bit as effective as the more formal tool of a stratified histogram. I’m going to take the issue of understanding variation in count data further in the next couple of columns. I’ll begin here by looking at two scenarios.

‘What's the trend?’

I once consulted with a medical center and attended its monthly meeting on medication errors. Right before the meeting, I was handed the 24 monthly reports from the previous two years. Not being sure what to do, I noticed something in the top right corner of each: a small table citing the number of errors for this month, last month, and same month last year.

The meeting began, and I tuned out the people asking, “What’s the trend?’ I used these 24 data tables to sketch a run chart of the past three years of performance. It looked like this:

Applying the usual run chart rules, it is common cause, but do you see a pattern to the months with high values? I stopped the meeting dead in its tracks by asking, “What happens in July?’ Turns out, besides a lot of vacations, that’s when the new medical residents start. Do you think that might be the reason? In fact, because this seems to be a predictable event, why not try to prevent it?

Let’s look at a similar meeting. A manufacturing plant had 45 accidents one year and set an aggressive goal for the next year of reducing them by at least 25 percent. The subsequent total was 32—a 28.9-percent decrease. However, a trend analysis (below) showed the decrease was more on the order of 46.2 percent (4.173 vs. 2.243).

Kudos to the safety committee and its hard work at the monthly safety meetings, during which each month’s accidents are individually dissected to find the “root cause”—i.e., each undesirable variation (or accident) is treated as a special cause. Also note there are three months with zero events. The reasons for these were also discussed and used to evaluate the effects of previous recommended solutions. Based on these data, approximately 80 actions (reactions to 77 accidents plus three months of zero) have been implemented during the past two years, including new policies, new reporting forms, new awareness posters, additional visual safety alerts in “dangerous” places, and plant safety meetings. All this hard work must have surely paid off.

25 percent is a ‘big’ number, but it’s just a number

Key question: Is the process that produced this year's 32 accidents truly different from the process that produced the 45 accidents from the year before? If the answer is yes, one should be able to create a run chart of individual monthly results and see at least one of three things:
1. A trend of six successive decreases and/or
2. A run of eight consecutive points above the median during the first year and/or
3. A run of eight consecutive points below the median during the second year

Take a look at the run chart below. What do you think?

What part of ‘no trend lines’ don’t people understand?

Common cause—and an expected monthly range of zero to eight! What has been the effect of the special-cause strategy of looking at each accident individually (and scraping it like a piece of burnt toast)? The chart strongly suggests that this strategy doesn't seem to be working. There is no evidence of improvement. Even though accidents “shouldn't” happen and the company met an aggressive reduction goal, this facility remains "perfectly designed" to produce them—at the same rate.

Suppose that no coding of the individual observations helped? What now, a common occurrence with incident data or nonconformance data? One does not have to accept this level of performance, but to paraphrase a favorite saying of W. Edwards Deming's, “Statistics on the number of accidents by themselves don't help to improve the number of accidents.” Nor is the answer a total process redesign with the vague objective of “reducing accidents.”

Do I see the Ishikawa cause-and-effect diagram from hell in this manufacturer’s future as it attempts to determine “what causes accidents?” I hope not.

The fact that the plot demonstrates common cause means that neither individual monthly points nor individual accidents can be treated as special cause—even if they can be explained after the fact. Where does one go from here? To common-cause strategies. There are additional tools, neither commonly taught nor taught with this in mind, that are most enlightening. We’ll talk about these more in my next column.

About The Author

Davis Balestracci

Davis Balestracci is a past chair of ASQ’s statistics division. He has synthesized W. Edwards Deming’s philosophy as Deming intended—as an approach to leadership—in the second edition of Data Sanity (Medical Group Management Association, 2015), with a foreword by Donald Berwick, M.D. Shipped free or as an ebook, Data Sanity offers a new way of thinking using a common organizational language based in process and understanding variation (data sanity), applied to everyday data and management. It also integrates Balestracci’s 20 years of studying organizational psychology into an “improvement as built in” approach as opposed to most current “quality as bolt-on” programs. Balestracci would love to wake up your conferences with his dynamic style and entertaining insights into the places where process, statistics, organizational culture, and quality meet.