Our PROMISE: Our ads will never cover up content.
Our children thank you.
Steven Ouellette
Published: Tuesday, June 14, 2011 - 05:30 Although we may use the define, measure, analyze, improve, control (DMAIC) mnemonic to help guide us through our problem solving, that doesn’t really give us a lot of specific direction (as I bemoan in my Top 10 Stupid Six Sigma Tricks No. 4). Good experimental design technique is critical to being able to turn problems into solutions, and in my experience Black Belts have not been introduced to a good process to do this. If you know someone whose first thought is, “Let’s go collect some data to see what is going on,” then read on to avoid losing millions of dollars in experimental mistakes. OK, so we all know that Six Sigma is mostly repackaging things that have been around for awhile. The stated intention was to make using statistics and experimental design easy enough so that process experts could use them to solve problems in their processes. A laudable goal, but as you probably can guess by now, I think we have gone too far at times and ended up with “Black Box Black Belts” who have only been trained to enter data, click on buttons, and go with the answer that comes out of the software. This will work occasionally (giving the poor Black Belts a sense of false confidence), but will eventually turn around and bite them in the… back. One of those areas that got dropped was the process of performing research—industrial or otherwise. If someone doesn’t know about this process (or even that there is one), he is at grave risk of doing a lot of work only to have nothing in the end to show for it, or a “solution” that is worse than the problem. When do we use an experimental design process? Well, as you will see below, since it is compatible with DMAIC (or whatever problem-solving process you prefer), it is applicable to the entire project. But let's say that during the course of solving the big problem (scrap rate, for instance), you also find that you need to study a smaller component of the process with data (maybe the concordance of those scrapping the product). So here is the rule: Any time you set out to collect data, you should be more than halfway through your research design process. Yes, I said “any time,” and I mean it. It doesn’t take long to do, and it will save you tons of time and money. So let’s go over a process for designing research. There are others, but this is the one I think is best. In later articles, I’ll explore some of the specific (and overlooked) tools used in the steps. This time, I’ll just do an overview of the process. To start with, here is the research design process I’ll be using: Figure 1: The research design process (from Design of Experiments in Quality Engineering, by Jeffrey T. Luftig and Victoria S. Jordan, McGraw-Hill, 1998) As Douglas Adams wrote, “Don’t panic!” There is nothing here that is conceptually difficult. First off, take a look at the size of each of the slices, which are more or less proportional to the time that should be spent on it during research. In Black Belt training, we spend a lot of time on the statistical tools because it takes time for people to learn everything they need to know to be effective and not dangerous. But in terms of actually working on a project, a far higher proportion of time should be spent on planning the experiment. If you do your planning correctly, the analysis is a very small part of the study. Note also that the data are not collected until after the planning phase. I know it seems obvious, but I have talked with more than one Black Belt who told me stories of starting off collecting data or designing an experiment before they really understood what the problem was. Another thing that should be obvious now that we are looking at a picture of the process, is that you don’t design the experiment until you have figured out the variables you want to vary (treatments) as well as the variables for which you need to control (all other independent variables). Many an experiment has not been confirmed because of a failure to do the latter. Finally, everything in the research process follows the plan-do-study-act process popularized by W. Edwards Deming. Of course this is no surprise because PDSA and DMAIC are nothing more than different interpretations of the scientific method. Well, that gets us started, anyway. Next month I’ll delve more into the individual steps and highlight some underutilized tools that will save your sanity, as well as a lot of money. If you can’t wait, well, buy yourself a copy of the book I referenced. It doesn’t teach you how to do the stats, just how to actually run an experiment in the real world once you do know the stats. Quality Digest does not charge readers for its content. We believe that industry news is important for you to do your job, and Quality Digest supports businesses of all types. However, someone has to pay for this content. And that’s where advertising comes in. Most people consider ads a nuisance, but they do serve a useful function besides allowing media companies to stay afloat. They keep you aware of new products and services relevant to your industry. All ads in Quality Digest apply directly to products and services that most of our readers need. You won’t see automobile or health supplement ads. So please consider turning off your ad blocker for our site. Thanks, Steve Ouellette, ME, CMC started his career as a metallurgical engineer. It was during this time that his eyes were opened to find that business was not a random series of happenings, but that it could be a knowable enterprise. This continues to fascinate him to this day. He started consulting in 1996 and is a Certified Management Consultant through the Institute for Management Consulting. He has worked in heavy and light industry, service, aerospace, higher education, government, and non-profits. He is the President of The ROI Alliance, LLC. His website can be found at steveouellette.com. Steve has a black belt in aikido, a non-violent martial art, and spent a year in Europe on a Thomas J. Watson Fellowship studying the “Evolution, Fabrication, and Social Impact of the European Sword." Don’t Design the Experiment Until You Research the Process
Applied research the smart way
Our PROMISE: Quality Digest only displays static ads that never overlay or cover up content. They never get in your way. They are there for you to read, or not.
Quality Digest Discuss
About The Author
Steven Ouellette
© 2022 Quality Digest. Copyright on content held by Quality Digest or by individual authors. Contact Quality Digest for reprint information.
“Quality Digest" is a trademark owned by Quality Circle Institute, Inc.
Comments
What is you defintion of "data?"
What is you definition of "data?" Does the process of observation result in data?
Understanding the Process...
I'm sure that as in everything, the quality of the Six Sigma BB training varies dramatically. I have been to several DOE training classes and many other statistical related classes. But the Six Sigma training I received through the Whirlpool Corporate Training Center was a phenomenal, mind-blowing experience.
The basic premise that the entire 5 weeks of training was centered around the following saying, "Practical, Graphical and Analytical".
It is my belief that if the project planning is done properly and at a level of detail befitting a "Black Belt", that many of the problems could be solved by using the "Practical" aspect of the Black Belt approach without spending wasted time with ineffective DOE's. But that depends on the level of the detail done as part of investigation into the process. If done properly, the BB will have identified, not just the inputs, transformations and outputs, but the interactions, interdependencies, controllable variables and the noise variables. If a process mapping approach is used, then the known’s and unknown’s will also be identified. This also includes a detailed investigation into what the perceived problem or project really is. I believe that if this phase is done correctly, the scope of the project or even what problem is being address might change some from the initial starting point.
I also believe that another common mistake is that organizations are convinced that a single DOE will provide a magical answer or the absolute perfect process. One of the first things I learned in the Whirlpool training program was that doing a DOE was a process. An iterative process that might only give you a hint of the direction you need to head, and rarely the optimal answer on the first DOE.
Too many times I have seen people quick to run a DOE to solve a problem they didn't even fully understand. Then when the analytical aspect of the DOE did not yield and magical solution, they would become skeptical of DOE's in general.
To me, the Black Belt training I received would still be well worth the effort if we had spend almost no time on the Analytical aspect of the training program. The planning, the understanding of the process, the critical thinking involved in a well done thought map has benefited me almost on a daily basis. The statistical and analytical knowledge I gained has been extremely helpful, but the initial aspects of approaching a problem and the planning and process investigation has forever change the way I approach things on a daily basis.
I'm not trying to negate the value of the analytical aspect but so many times, the use of the analytical tools only will not of much value without the initial part of the process.
thanks,
Terry A. Henson
Necessary, but Sometimes not Sufficient
A very good question!
(Bullets used to separate paragraphs...)
<edited to correct ambiguities>