Improvement can be achieved in one or more of the many characteristics of any given product or
process. In most situations, however, improvement primarily implies that performance is enhanced. Experimental design is one technique that can be learned and applied to determine product or
process design for improved performance.
For a high-volume manufactured part, the two statistical performance characteristics that manufacturers typically aim to achieve are improving the mean (average) and reducing the
variability around the mean. For improvement, our goal is to move the performance of a population of parts to the target and minimize the variability around it (see figures 1 and 2).
No matter the application, performance consistency is a desirable characteristic to achieve. Performance consistency is achieved when the performance is on-target most of the time.
Figure 1: Performance Before Experimental Study
Figure 1: Performance Before Experimental Study
An effective way to improve performance is to optimize the engineering designs of products or processes by experimental means. A structured and economical way to study
projects whose performance depends on many factors is to apply the experimental method known as "design of experiments," a statistical technique introduced in the 1920s by Ronald A. Fisher in
England. In the 1950s, Genichi Taguchi of Japan proposed a much-standardized version of the technique for engineering applications. His prescription for experiment
designs, a new strategy to incorporate the effects of uncontrollable factors and ability to quantify the performance improvement in terms of dollars by use of a loss function,
made the DOE technique much more attractive to the practicing engineers and scientists in all kinds of industries.
In January, John Wiley & Sons Inc. published my book on DOE/Taguchi technique. Intended primarily for the self-learner, the book takes the reader through the entire
application and analysis process in 16 different steps. One who learns the topics covered in these 16 steps well will be able to handle more than 99 percent of the
situations common to manufacturing and production activities. Following are the 16 steps you will need to master DOE using the Taguchi approach for your own product and process design improvement.
Design of experiments and the Taguchi approach
A quick review and understanding of the Taguchi version of DOE is essential before
diving into the subject. The purpose here is to gather a clear understanding of what DOE is and understand how Taguchi standardized the experiment design process to
make the technique easier to apply.
Step 2: Definition and measurement of improvement
No experiment that lacks the means to measure its results is complete or useful. A
clear definition of objectives and measurement methods allows us to compare two individual performances, but a separate yardstick is needed to compare performances
of one population (multiple products or processes) with another. In general, individual performance measures are different for different projects, but consistency is the means
by which we measure population performance. Consistent performance produces reduced variations around the target (when present) and results in reduction of scrap,
rejection and warranty. In this step, you learn how population performances are measured and compared.
Common experiments and analyses methods
A common practice for studying single or multiple factors is to experiment with one factor at a time while holding all others fixed. This practice is attractive, as it's simple
and supported by common sense. However, the results are often misleading and fail to reproduce conclusions drawn from such an exercise. A more effective method for
these situations is to study their effect simultaneously by setting up experiments following the DOE technique. This step should lead to some understanding of basic DOE principles.
Designing experiments using orthogonal arrays
The word "design" in "design of experiments" implies a formal layout of the
experiments that contains information about how many tests are to be carried out and the combination of factors included in the study. Once the project is identified, the
objectives and factors and their levels are determined by following a recommended sequence of discussion in a planning meeting. There are many possible ways to lay out
the experiment; the best method depends on the project. A number of standard orthogonal arrays (number tables) have been constructed to facilitate designs of
experiments. Each of these arrays can be used to design experiments to suit several experimental situations. This step should be devoted to learning about the different
orthogonal arrays and understanding how easy it is to design experiments by using them.
Designing experiments with two-level factors
Experiments that involve studies of factors with two levels are both simple and common. There are a set of orthogonal arrays (designated as L-4, L-8, L-12, L-16,
L-32, L-64, etc.) created specifically for two-level factors. Experiments of all sizes can be easily designed using these arrays, as long as all factors involved are tested at two
levels. By completing this step, you will learn how quickly experiments involving two-level factors can be designed and analyzed using the standard orthogonal arrays.
Designing experiments with three-level and four-level factors
When only two levels of factors are studied, the factors' behavior is necessarily
assumed to be linear. When nonlinear effects are suspected, more than two levels of the factors are desirable. Although many larger two-level orthogonal arrays can be
modified to accommodate three-level and four-level factors, a set of standard arrays such as L-9, L-18, L-27, modified L-16 and modified L-32 are also available for this
purpose. This step should help you learn the design and analysis of these more complex experiments.
Step 7: Analysis of variance (ANOVA)
Calculations of result averages and averages for factor-level effects, which only involve simple arithmetic operations, produce answers to major questions that were
unconfirmed in the earlier steps about the project. However, questions concerning the influence of factors on the variation of results --in terms of discrete proportion --can
only be obtained by performing analysis of variance. In this step, you'll learn how all analysis of variance terms are calculated. Utilize this step to review a number of
example analyses to build your confidence in interpreting the experimental results.
Designing experiments to study interactions between factors
Interaction among factors, which is one factor's effect on another, is quite common in industrial experiments. When experiments with factors don't produce satisfactory
results, or when interactions among factors are suspected, the experiment must accommodate interaction studies. In this step, your objective will be learning how to
design experiments to include interaction and how to analyze the results to determine if interaction is present. You will also learn how to determine the most desirable condition
in cases in which interaction is found to be significant. Although interactions among several factors, and between factors at three or four levels, are also present, studies
and corrections for interaction between two two-level factors will suffice for most situations.
The materials in steps 1-8 prepare you for many applications in the production floor. As long as the factors you want to study are all at the same level, you're able to design
experiments using one of the available orthogonal arrays. You're also able to analyze the results of such experiments following the standard method of analysis, which uses the
averages (means) of the multiple sample test results of individual experiments, and determine the optimum design conditions. With the knowledge you should gather in
these steps, you can indeed apply the DOE to solve most production problems whose solutions lie in finding the proper combination of the controllable factors, instead of some special causes.
The reality, however, is that you will often have factors at mixed levels; some will be at three-level, some at four-level, and many at two-level. You also need to learn how to
analyze the results for variability. Recall that it's the reduction of variability, which instills performance consistency, that we're after. The following additional steps
address these items and prepare you to handle most every type of experimental situation.
If your applications always involve production problem solving, you may find that
your knowledge up to this point is quite adequate for the job. Nevertheless, you may want to sharpen your application skills before proceeding to learn about the advanced
concepts in the technique described in the eight steps that follow.
Step 9: Experiments with mixed-level factors
Experiment designs with all of the factors at one level are easily handled using one of
the available standard arrays. But these standard arrays can't always accommodate many mixed-factor situations that you might find in industrial settings. Most
mixed-level designs, however, can be accomplished by altering the standard orthogonal arrays. Your goal will be to learn the procedure by which columns of an array are
modified to upgrade and downgrade the number of levels in creating a new column. This way, a two-level array can be modified to have three-level and four-level columns.
Conversely, to accommodate a factor with a lesser number of levels, a four-level column can be reduced to a three-level, and a three-level column to two-level, by a
method known as "dummy treatment."
Step 10: Combination designs
For some applications, the factors and levels are such that standard use of the
orthogonal array doesn't produce an economical experimental strategy. In such situations, a special experiment design technique such as a combination design might
offer a significant savings in number of samples. This step will familiarize you with the necessary assumptions that must be made in order to lay out experiments using
combination design. With this technique, generally, two two-level factors are studied by assigning them to a three-level column.
Step 11: Robust design strategy
Variations among parts manufactured to the same specifications are common even when attempts are made to keep all factors at their desired levels. Remember, variation
reduction is our ultimate goal. When performance is consistently on-target (the desired value), the customer perceived quality of the product is favorably affected. Variation is
most often due to factors that are not controllable or are too expensive to control. These are called the "noise factors." In robust design methodology, the approach is not
to control the noise factors, but to minimize their influence by adjusting the controllable factors that are included in the study. This new strategy, promoted by Taguchi,
reduces variability without actually removing the cause of variation.
Step 12: Analysis using signal-to-noise (S/N) ratios
The traditional method of calculating average factor effects and thereby determining the desirable factor levels (optimum condition) is to look at the simple averages of the
results. Although average calculation is relatively simple, it doesn't capture the variability of results within a trial condition. A better way to compare the population
behavior is to use the mean-squared deviation, which combines effects of both average and standard deviation of the results. For convenience of linearity and to accommodate
wide-ranging data, a logarithmic transformation of MSD (called the signal-to-noise ratio) is recommended for analysis of results. This step will teach you how MSD is
calculated for different quality characteristics and how analysis using S/N ratios differs from the standard practice. When the S/N ratio is used for results analysis, the
optimum condition identified from such analysis is more likely to produce consistent performance.
Step 13: Results analysis using multiple evaluation criteria
Often, a product (or process) is expected to satisfy multiple objectives. The result in this case comprises multiple evaluation criteria, which represents performance in each
of the objectives. It's common practice, however, to analyze only one criteria at a time because different objectives are likely to be evaluated by different criteria, each of
which has different units of measurement and relative weighting. When the results are analyzed separately for different criteria and the desirable design conditions are
determined, there is no guarantee that the factor combination will all be alike. An objective way to analyze the results is to combine the multiple evaluations into a single
criterion, which incorporates the units of measurements and the relative weights of the individual criterion of evaluation. You should devote your time during this step to
learning the principles involved in formulation of an overall evaluation criterion for analysis of multiple objectives, when present.
Step 14: Quantification of variation reduction and performance improvement
Most of your DOE applications allow you to determine optimum design that is expected to produce an overall better performance. The improvement of performance
often means that either the average or the variations (or both) have improved. When the new design is put into practice (i.e., the recommended design is incorporated), it's
expected to reduce scrap and warranty costs. In turn, this reduction more than offsets the cost of the new design. The expected monetary savings from the improved design
can be calculated by using Taguchi's loss function. In this step, you'll learn how to estimate the expected savings from the improvement predicted by the experimental
results. Further, you'll also learn how the expected improvement in performance from the new design is expressed in terms of capability improvement indexes such as Cp and Cpk.
Step 15: Effective experiment planning
As far as the benefits from the technique are concerned, experiment planning is the most important among the different application activities. Therefore, it's a required first
and necessary step in the application process. Planning for DOE/Taguchi requires structured brainstorming with project team members. The nature of discussions in the
planning session is likely to vary from project to project and is best facilitated by one who is well-versed in the technique. Your effort in this step will be to learn the
structure of proven planning sessions documented by experienced application specialists.
Review of example case studies
The application knowledge gained in steps 1-15 could be overwhelming if you didn't have immediate projects on which to practice. One way to build more confidence and
extend your application expertise is by familiarizing yourself with numerous types of case studies with complete experiment design and results analysis. In this final step,
you should seek out and thoroughly review complete project application reports. Complete case studies should contain discussions under most of the following topics:
Project title or problem definition
Evaluation criteria and quality characteristic
Identified factors and levels and those that are included in the study
Suspected interactions and those that are selected for the initial study
Uncontrollable factors (noise factors) and how they were treated
Sequence of running of the experimental conditions
Measured results, which represent evaluation of different objectives
Main effects indicating the trend of factors' influence
Analysis of variance for relative influence of the factor to the variation of results
Optimum condition and the expected performance
Improvement and expected monetary savings
Graphical representation of variation reduction expected from the improved design
Now that you have an idea about the topics and the study sequence, one question remains: How do you actually go about learning them?
To get yourself comfortable with DOE application knowledge, you will need to understand four phases in the application process: (1) experiment planning, (2)
experiment design, (3) results analysis and (4) interpretation of results. Of these, you need not --and may not be able to afford the time --to be too good with experiment
design and number crunching. These are mundane tasks, so feel comfortable letting a computer program do the work for you; your focus should be to learn the practiced
and proven discipline of how to plan an experiment following a structured sequence of discussion. The experiment planning process requires more the art of teamwork than
experimental science. Only the experienced can describe and share methods that have worked. Look for references that describe and teach the technique through application examples.
Both experiment planning and interpretation analysis are areas you'll want to gain control over. The nature of discussions and findings in these areas are always
project-specific. As the experimenter, you'll know far more about these two areas than anyone else. Good knowledge of the project objectives, how objectives are evaluated,
how the factors included in the study were selected, and so on will help you confidently interpret results from the routine analysis. You will benefit most when your
reference book stresses application rather than theory.
1. Taguchi, Genichi. System of Experimental Design. New York: UNIPUB, Kraus International Publications, 1987.
2. Roy, Ranjit K. A Primer on the Taguchi Method. Dearborn, Michigan: Society of Manufacturing Engineers, 1990.
3. Roy, Ranjit K. Design of Experiments Using the Taguchi Approach: 16 Steps to Product and Process Improvement. New York: John Wiley & Sons, 2001.
About the author
Ranjit K. Roy, Ph.D., P.E. (M.E.) is an engineering consultant specializing in
Taguchi approach of quality improvement. Roy has achieved international recognition as a consultant and trainer for his down-to-earth teaching style of the Taguchi
experimental design technique. He is the author of the texts Design of Experiments Using Taguchi Approach: 16 Steps to Product, Process Improvement and A Primer on the Taguchi Method
, and of Qualitek-4 software for design and analysis of Taguchi experiments.