Featured Product
This Week in Quality Digest Live
Quality Insider Features
Gleb Tsipursky
Believing things will normalize soon, many companies are unprepared for new waves of restrictions
Harry Hertz
Communities of Excellence 2026 can help communities work together using a Baldrige-based, community-centric framework
Michael Weinold
GE’s exit from the lighting business is a warning to other players in the sector
Ken Voytek
Productivity will be even more critical as we recover from the current health and economic crisis
Jeffrey Phillips
Putting data to work

More Features

Quality Insider News
3D scans help Chicago Jet Group retrofit old Dassault Falcon avionics system
Real-time data collection and custom solutions for any size shop, machine type, or brand
Lloyd Instruments launches the LS5 high-speed universal testing machine
Measure diameter, ovality of wire samples, optical fibers and magnet wire, including transparent products
Training, tips and tricks, unboxing, and product videos provide additional information for users
How to develop an effective strategic plan and make the best major decisions in the context of uncertainty and ambiguity
Collect measurements, visual defect information, simple Go/No-Go situations from any online device
Laser scanning also used to help create safety covers for credit card readers
A complimentary webinar for novices to experts on May 27-28, 2020

More News

Steven Ouellette

Quality Insider

Blocking Out the Nuisance, Part Two

On the practical benefits of blocking

Published: Wednesday, September 14, 2011 - 11:37

Yesterday in “Blocking Out the Nuisance, Part One,” we saw the results of an experiment done by That Guy Over There that didn’t control for the environmental variable of humidity. There was a lot of variability in that experiment, so we didn’t see a whole lot of improvement to be made. I also gave you the data from a blocked design that you created which controlled for humidity. Let’s take a look at the analysis and see if we can find out anything more.

You perform the analysis at α = 0.05, test and pass the assumptions for analysis of variance (ANOVA), and find the following:

Figure 1: Analysis of variance (ANOVA) with humidity blocked

 

In the immortal words of the philosopher Neo, “Whoa.”

You see an entirely different result than That Guy Over There, don’t you? First off, with an adjusted R2 of 0.996 and an overall ω2 of 0.9979, you properly conclude that you have nailed it. You see a significant three-way interaction between all of the factors:

Figure 2: Temperature and speed interaction at feed = 1



Figure 3: Temperature and speed interaction at feed = 2

 


Figure 4: Temperature and speed interaction at feed = 3


This is exactly the same process as was tested in the first experiment. The reason we can detect more effects is that in this design, we are controlling for the different levels of humidity as a blocked factor. To get an idea of what is going on, here is an analysis of the same data but without including the blocked factor:

Figure 5: Analysis of variance (ANOVA) of blocked data without humidity
These are the exact same data as before, but the variability due to the humidity is unassigned, and so it goes into the error term.

 

To see what I mean, compare just these parts:


Figure 6: Blocked term and error


Do you see how the total variability is exactly the same, but how the error sum of the squares in the second one is the sum of the variability of the humidity and the error?


7949.362 + 25.765 = 7975.127


All we did by assigning the variability to the humidity is subtract it out of the error. (Note that we do also lose a degree of freedom from that error, but that is far outweighed by the huge decrease.) Since the error term is what we divide each effect’s mean square by in order to get our F-statistic, we are dividing by a lot smaller number, meaning that we get a lot bigger F-statistics.

For example, the F-statistic of the speed*feed interaction is 1.218 without the block (which is probably going to be tough to differentiate from 1.000 without a huge sample size), as opposed to 131.697 for the blocked analysis, which if there is any fairness in the cosmos will always be a significant effect. So, all else being equal (except for that drop in the degrees of freedom), it is a whole lot easier to see that the different effects are significant: There is a lot less noise over which to hear the process screaming out to you.

Practical benefits of blocking

What are the practical benefits of the blocked analysis?
• You have learned a lot more about the process, so you can choose better settings to maximize the expansion in the face of changing humidity.
• On the other hand, with the unblocked design, That Guy Over There missed significant factors, due to the humidity, and chose the wrong setting, due to not randomizing.

Let’s take a look at our optimum settings. Because we blocked humidity, we are assuming that there is no interaction, so the optimal settings are going to be the same for different levels of humidity, but they will result in different expansions depending on where the humidity is that day.


 Figure 7: Prediction of optimum for blocked experiment


Now compare this table in figure 7 to the table in figure 5. You can see that during times of low humidity, we predict a much higher expansion, and for times of high humidity, a lower expansion. In both cases, we have a lot less variability.

Let’s make a picture to compare our predictions:


Figure 8: Comparison of predictions for humidity blocked and unblocked experiments


The experiment without blocking has a lot higher variability since all that variability due to humidity is unexplained and going into the error term. This unexplained variability washed out effects that really were significant, misleading That Guy Over There into thinking that none of the settings except temperature had an effect. Therefore, the additional variation that is really caused by these process settings gets thrown into the expected variation, too. Even worse, since That Guy Over There didn’t randomize, and since the low temp setting just happened to mostly correspond with a low humidity time frame, he concludes that the best setting for temperatures was the low one, when in fact it was the high one.

This means that when That Guy Over There implements his settings, he will not get 79.013 on average across all the other settings and humidity like he expects, but something closer to 77.65. That expansion is about 10 points lower than the average you would expect with our optimum settings across the humidity range (about 88.7%).

By the way, if you are wondering why our lower prediction of 73.773 percent is below what he thinks he will get, remember that his prediction is averaged across all humidities and settings. Ours is just for the high humidity. In reality, using his settings when the humidity was high, we would expect to see 63.34 percent.

You also now have a powerful argument to go back to management and see if they will be willing to start air conditioning the shop floor. Your customers are not going to see those nice little distributions but will be getting expansions all across the range between the two of them as the humidity varies, so it is doubtful they will notice much of an improvement. Your process is capable of high expansion with very low variability if you could just control humidity. (Or perhaps find a way of making your process robust to the humidity variation, which is exactly what happened in the real experiment.)

Summing up variable identification

Blocking properly accounts for variables that might affect the output but that we don’t care to include in the experimental design, such as nuisance variables. Blocking allows us to more easily see significant effects without increasing the sample size. By identifying all potential factors, including ones we don’t plan on incorporating into the experiment, we can avoid making major errors that will doom us to failure even before we set up the first experimental run.

So don’t be That Guy Over There, running all sorts of experiments but not getting replicable results, mocked by those who read this article. Use blocking and succeed.

More on the next few steps coming up in my next column.

Discuss

About The Author

Steven Ouellette’s picture

Steven Ouellette

Steven Ouellette is the Lead Projects Consultant in the Office for Performance Improvement at the University of Colorada, Boulder. He has extensive experience implementing the systems that allow companies and organizations to achieve performance excellence, as well as teaching Master's-level students the tools used in BPE. He is the co-editor of Business Performance Excellence with Dr. Jeffrey Luftig. Ouellette earned his undergraduate degree in metallurgical and materials science engineering at the Colorado School of Mines and his Masters of Engineering from the Lockheed-Martin Engineering Management Program at the University of Colorado, Boulder.