Planning the Research Study: Part 2

How to identify, track, and measure things related to what we are working on

Published: Tuesday, August 16, 2011 - 11:19

Let's face it—many industrial researchers, including Six Sigma Black Belts, do a terrible job of planning the research they need to do to perform their jobs efficiently. See that guy over there? Yeah, he is the one I am talking about, so you should read this article so you can help the poor bloke. In my last couple of articles I have been covering a process to plan a research study—in this one we continue with the planning phase and confront an often neglected step for good experimental design—nuisance variables.

In my previous article, I covered the first part of how to plan a study. Remember, this can be any type of study—anything from the plan for a mega project spanning an entire company to a sub-sub-project where we are investigating a small component of that process.

Again, here is the cycle I'll be using.

Figure 1: The Research Design Process from Design of Experiments in Quality Engineering by Jeffrey T. Luftig and Victoria S. Jordan (McGraw-Hill, 1998)
Click here for larger image.

Now at this point we should have a good idea of what we are working on and why. The next few steps are going to define how we are going to identify, track, and measure things related to what we are working on.

Define/select the dependent variables and criterion measures

OK, first some terminology. A variable is just something that can take on different values. Anything from a temperature to a count or rate can be a variable. The dependent variable is the value of that thing in which we are really interested in learning about. It might be "surface quality" or "customer loyalty" or "pain level."  However, while it might be easy to put words on what we are interested in, we still have to figure out a measure of that dependent variable. Often we cannot measure the dependent variable directly, we can only measure one or more proxies for that variable, and that is the criterion measure. So surface quality might have a criterion measure of "surface roughness over a 1 mm line." And in fact, there might be a number of measures that add up to our best description of this thing called "surface quality."  The criterion measure of customer loyalty might be "percent of customer product supplied by us," or perhaps "customer rating of likelihood of buying from us next year." A criterion measure for "pain level" could be "patient-reported scale of 1 to 10, with 10 being the worst pain imaginable."

If you are doing experimental research, as you monkey around with process settings, you are hoping to change the dependent variable, which would hopefully be reflected in your data generated from the criterion measure.

Now as you can see, the criterion measure is not the same thing as the dependent variable. The relationship between the criterion measure and the dependent variable is what gives data its measurement level, which in turn dictates what you are allowed to do with it statistically. (If you want to learn more about this, you might start with my article, "How Do You Measure Up?") Your decisions right here will affect what happens when you finally get data for analysis.

Identify and classify treatment, independent, and nuisance variables

Now that we know what we want to measure (and it's relationship back to the dependent variable) we need to start classifying all the variables that are going to be involved in the study, and by that I mean all the variables that are present, whether you want to study them or not.

If we are doing an experiment, we will have independent variables, and the subsets of them called treatment and nuisance variables, that we will have to understand. (If we are doing a nonexperimental study, we will have just independent and dependent variables.)

Independent variables. Independent variables are factors that might influence your dependent variable, whether you know about them or not. If you know about them, you are going to have to handle them one way or the other in your study. Independent variables that you include in an experiment are called "treatment variables."

Treatment variables. Treatment variables are those factors that you are going to manipulate directly during the study. These are the "inputs" that you hope will affect your "outputs" (the dependent variables). Now treatment variables can be incorporated into the study as fully crossed factors (I can run any and all combinations of the factors) or as nested factors.

Nested factors. Nested factors have been incorporated, but it doesn't make any sense to cross them, or it doesn't help us answer our research question to do so. Let's say I make the same polymer in two production lines. Because these are different machines, the minimum and maximum temperatures, speeds, and feed rates that we use for them are completely different for the two lines.  I need Line in my experiment, since I need to find the optimal settings for each line. It wouldn't make sense to run an experiment with Temperature at four levels: 150° (low for Line 1), 175° (low for Line 2), 210° (high for Line 1) and 250° (high for Line 2). If I did that, I would have combinations that don't make sense on both lines. So I say I will run Low and High Temperature on each, but since the actual low temperature on one is not the low on the other, I run the Low and High for each line on each line. Temperature is therefore nested within Line.

Nuisance variables. Independent variables that you know about and you are not interested in studying or can't control, still have to be accounted for and are called nuisance variables. In the real world of business or industrial research we get a lot of these. For example, if I know that humidity affects polymer expansion, but I can't climate control my factory that makes the polymer, then I have to somehow account for humidity, which will change throughout the day and through the year.

What do you think—couldn't we just ignore humidity? I mean how do we account for it?

Next month, I'll show you what happens if you ignore these types of nuisance variables and the amazing, often neglected power of blocking that can save you lots of money and help you find solutions that would have otherwise been hidden.

About The Author

Steven Ouellette

Steve Ouellette, ME, CMC started his career as a metallurgical engineer. It was during this time that his eyes were opened to find that business was not a random series of happenings, but that it could be a knowable enterprise. This continues to fascinate him to this day.

He started consulting in 1996 and is a Certified Management Consultant through the Institute for Management Consulting. He has worked in heavy and light industry, service, aerospace, higher education, government, and non-profits. He is the President of The ROI Alliance, LLC. His website can be found at steveouellette.com.

Steve has a black belt in aikido, a non-violent martial art, and spent a year in Europe on a Thomas J. Watson Fellowship studying the “Evolution, Fabrication, and Social Impact of the European Sword."