Our PROMISE: Our ads will never cover up content.

Our children thank you.

I hope this little diversion into design of experiments (DOE) that I’ve explored in my last few columns has helped clarify some things that may have been confusing. Even if you don’t use DOE, there are still some good lessons about understanding the ever-present, insidious, lurking cloud of variation.

Building on my June column, consider another of C. M. Hendrix’s “ways to mess up an experiment”: Insufficient data to average out random errors (aka, a failure to appreciate the toxic effect of variation).

This is where the issue of sample size comes in, and it’s by no means trivial.

The ability to detect effects depends on your process’s standard deviation, which in the tar scenario from my May column simulation was +/– 4 (the real process was actually +/– 8).

Here’s a surprising reality for many: The number of variables doesn’t necessarily directly determine the number of experiments. But let’s continue the tar scenario:

Today I want to concentrate on the foundation of what is most commonly taught as design of experiments (DOE)—factorial designs.

Elsewhere I’ve mentioned three of C.M. Hendrix’s “ways to mess up an experiment.” After 35 years of teaching DOE, I’ve concluded that he pretty much captures the universal initial class experience I described in these additional ways to mess up:

• No comprehensive strategy, i.e., attacking one dependent variable at a time

• Too many experiments concentrated in one region of experimental space, or too few variables

• Attempting to optimize by manipulating one variable at a time

• Failure to appreciate the role of interactions

In my last column I explained how many situations have an inherent response surface, which is the “truth.” However, any experimental result represents this true response, which is unfortunately obscured by the process’s common-cause variation. Regardless of whether you are at a low state of knowledge (factorial) or a high state of knowledge, the same sound design principles apply.

**The contour plot: a quadratic ‘French curve’**

Response surface methodology’s objective is to model a situation by a power series *truncated after the quadratic terms*. In the case of three independent variables (x1, x2, x3), as in the tar scenario from my column, “90 Percent of DOE Is Half Planning,” in May 2016:

Y = B0 + (B1 x1) + (B2 x2) + (B3 x3) + (B12 x1 x2) + (B13 x1 x3) + (B23 x2 x3) + [B11 (x1**2)] + [B22 (x2**2)] + [B33 (x3** 2)]

Which designs give the best estimates of these B coefficients?

I’ve mentioned that design of experiments (DOE) is one of the few things worth salvaging from typical statistical training, and I thought I’d talk a bit more about DOE in the next couple of columns. The needed discipline for a good design is similar when using rapid-cycle plan-do-study-act (PDSA).

Doing a search on the current state of DOE in improvement education, I observed that curricula haven’t changed much in the last 10 years and still seem to favor factorial designs or orthogonal arrays as a panacea.

The main topics for many basic courses remain:

• Full and fractional factorial designs

• Screening designs

• Residual analysis and normal probability plots

• Hypothesis testing and analysis of variance (ANOVA)

The main topics for advanced DOE courses usually include:

• Taguchi signal-to-noise ratio

• Taguchi approach to experimental design

• Response-surface designs

• Hill climbing

• Mixture designs

In part one yesterday, we looked at stats of the Boston Red Sox bullpen, a typical example of baseball’s tendency to find special cause in just about anything. *The Boston Globe* article on which these two columns are based has been a gold mine for teaching many useful, basic concepts about variation. Today we’ll continue the analysis with a closer focus on common cause.

Once again, for those of you who aren’t interested in the statistical mechanics but want to be aware of how this type of analysis can drastically change one’s thinking, just skip to the bottom-line conclusions where indicated. For my non-U.S. readers, I hope you’ll be able to follow the analysis philosophy and see parallels to your favorite sports and news articles.

Any italics in direct quotes are mine, and if I make comments within a quote, I show that by inserting [**DB:**...].

In honor of baseball season, I’m going to apply some *simple* statistical thinking to my favorite sport in a two-part series today and tomorrow. I want anyone to be able to enjoy this, so I’ll mark any technical statistics as optional reading. For those of you interested only in the interpretations, I'll offer the “bottom line” conclusions, many of which I think will surprise you.

Maybe you can’t do the math and don’t even want to, but you should at least realize the importance of understanding these types of analyses and have access to someone who can do them. In many similar daily situations you encounter, anything else would be data insanity.

April Fool’s Day (today) and the opening of baseball season (this Sunday) are upon us. To mark the first event, I’ll let my distinguished colleague Donald Wheeler make some eloquent and crucial statistical points that turn out to be, well, *laughably simple.* (No fooling!) Regarding the baseball connection, please bear with me.

Wheeler is one of the world’s leading W. Edwards Deming statistical experts, and he’s been an influential stealth mentor of mine for 30 years. He’s an excellent theoretician who can certainly get appropriately technical with statistics if he needs to; however, when he does, it’s usually to soundly deflate what Deming called “such nonsense” being taught today.

This article is based on some ideas from my respected colleague Mark Hamel. Despite the lean framework, these ideas apply to any improvement approach—all of which come from the same theory, lean included.

During the past 35 years, quality has evolved from the necessary evil of quality control to what can easily be considered a self-sustaining organizational improvement sub-industry. Leaders still try to inspire people with passionate lip service in scheduled, cliché-ridden speeches (or videos) about committing to excellence, becoming world class, and dazzling customers. But in reality the leader’s commitment yields—predictably—vague results that stem from the naive hope for improvement in general (and perhaps, increased cultural cynicism).

Marketers are relentless in their efforts to seduce you with fancy tools, acronyms, Japanese terminology—and promises—about *their versions* of formal improvement structures such as Six Sigma, lean, lean Six Sigma, or the Toyota Production System, each with its own unique toolbox.

In my last column, I discussed the need of becoming more effective by* using far fewer tools* in conjunction with critical thinking to understand variation.

In the midst of all this, I’ve seen W. Edwards Deming’s and Walter A. Shewhart’s brilliant, fundamental plan-do-study-act (PDSA) morph to an oft-spouted platitude. I laugh when I hear people casually comment that PDSA and plan-do-check-act (PDCA) are the same thing. They’re not.

As I was preparing this column, one of my resources referred to chapter 48 of the 2,500-year-old *Tao te Ching* (quoted below), which, as some of you know, is one of my favorite sources of wisdom. It really tied today’s message together, and I hope you can apply its wisdom to your improvement efforts.

“To obtain a diploma requires the storage of trivia.

To obtain the Great Integrity [Tao] requires their abandonment.

“The more we are released from vested fragments of knowledge,

the less we are compelled to take vested actions,

until all is done without doing.

“When the ego interferes in the rhythms of process,

there is so much doing!

But nothing is done.”

Matthew E. May has published a thought-provoking article whose points I’d like to share as you settle into your post-holiday work rhythm.