Wave Dynamics Articles 

coffee cup on table

Design of Experiments: A Powerful Tool for Process Improvement

        by Randy Ashbaugh

September 16, 2011

Design of Experiments, also called DOE, is one of the most powerful tools in the Six Sigma Black Belt’s arsenal for process improvement. It is a method of experimentation that reveals the hidden secrets surrounding why a process behaves in a certain way. In this article we will explore some of the uses of DOE, the steps involved in doing a DOE, describe some key information obtained from the analysis results and conclude with some advice on how to cash in on the knowledge obtained from the experiments. So where can we apply this method of improvement?

Well, any process that has controllable inputs and identifiable measurable outputs can qualify for DOE experimentation. While traditionally it has been applied very successfully to manufacturing processes, it can be applied to other processes as well. Customer service processes like the admissions and treatment process at a hospital or a loan origination process at a bank could qualify for this kind of experiment. Of course in order to run the experiments you will need to be able to control the inputs. Some inputs like the number of physicians on duty at a hospital ER could be easy to control. Other factors like number of patients at any one time would be more difficult to deal with in an experiment. But there are ways to minimize the effects of these difficult inputs in an experiment and still obtain useful results. So the answer is yes, any process can be considered for DOE. So how does the process work?

We start the process by identifying the goal or objective of the experiment. We may be trying to reduce scrap, reduce surgery infection rates, increase throughput in an ER or optimize the quality of a weld. The list of objectives is endless. Next we need to identify the inputs that affect the chosen output. Usually this is a team activity with subject matter experts participating in a brainstorming session. The inputs can be captured effectively on a fishbone diagram showing all of the potential factors. Once the team identifies all of these they need to narrow the list to the most likely factors that will affect the outcome. Since two level DOE designs are common, the team must also select two setting levels for each main factor. For instance in a molding process where mold temperature was identified as a main factor we would need to select a high and low temperature for the experiment. Once this is done for all inputs the Black Belt will typically enter these into a statistical software program like Minitab® to create an experiment and a run sheet. The run sheet shows each experiment run and the settings for each factor for each run. All other variables are kept at the same setting to minimize their influence on the experiment. A fractional factorial Taguchi DOE design is one of the most popular designs because it requires fewer experimental runs for the information it ultimately provides. Fewer experimental runs equate to less cost, but it is wise to keep in mind that several DOE experiments may be required to ultimately get the results we need. Each situation has to be evaluated independently to determine if the objective has been met or not.

All of this work leads to knowledge, the part we’ve been waiting for. To illustrate an analysis we pulled data from an experiment illustration in “The McGraw-Hill 36-Hour Course Six Sigma” by Greg Brue and Rod Howes (Brue ,2006). We entered the experimental raw data into Minitab® and printed out some of the results to illustrate what we learned. The experiment was conducted to learn which input factors of an electro magnet contributed most to its ability to lift a group of BB’s.

The main results of a DOE are summarized in an Analysis of Variance table (Figure 1). This table has several columns, but the one of most interest is the “p” column. This column describes the significance of each input factor, the lower the number the more significant the input. A P-value of .05 or lower for an input indicates statistical significance to a 95% confidence level. If it is statistically significant, consider it a key variable worthy of further study and optimization. In this case the Number of Turns is the most significant factor.

Also of interest is the pareto of the effects (Figure 2). This chart shows the relative contribution of each variable to the end result. It is calculated as the sum of squares (SS) of the input factor divided by the total sum of squares of all the input factors. In this case you can see the Number of Turns is number one with the Nail Size second and the interaction of Nail Size and Number of Turns also contributing to pick-up power.

                           Figure 2

Also examine the main effects plot of each factor (Figure 3). This plots each input level against the measured output. Steep slopes indicate a significant input. You see this steep slope on Number of Turns and Nail Size plots.

                         Figure 3

Lastly look at the interaction plots of the factors (Figure 4). If the lines tend to cross one another it indicates an interaction. If the interactions are strong or they are interactions involving one of the significant factors, further experimentation including the interaction may be required to optimize the process. In this case Nail Size and Number of turns show the greatest interaction. Other smaller interactions are also present. If these electromagnets were an actual product at ElectromagnetsRus, the experiment would have revealed the best process settings to get the maximum pick up power. You will see from the plot that N_Turns at a level “1” combined with Nail_Siz at level "1" produced the most pick up power. Therefore, it would have lead directly to product optimization and happier customers.

                          Figure 4

It is important to watch the magnitude of the error variable. A high error variable could indicate that an important variable was not included in the experiment. Low values of error increase the confidence level of the overall experiment. In the Analysis table (Figure 1) we see a sum of squares value for error equal to 3981. This is about 25% of the total sum of squares indicating a moderate error was present in the experiment, but probably not enough to create great concern. Now that we have learned all of this, what is next?

Once we are armed with the knowledge, now we have to apply it. It is important to first optimize the significant input factors and then control them. Run additional optimization experiments if necessary. Then run to the new settings and verify the results. If the results are good, the next step is to establish a new permanent standard and communicate it to the organization. This is sometimes the most difficult part of the improvement process for those who like to tinker with the process. It is important to help the “tinkerer” understand that the new standard comes from proven statistical analysis science. Also making things difficult is the fact that usually the experiments reveal something totally unexpected. If you think about it, this unexpected result makes perfect sense. It is usually the factor no one suspected that caused the problem to go unresolved. And lastly, you will have to put controls on things that may not have been controlled before. This may present a challenge to the organization, but it is far from a major problem.

DOE is a great tool that every organization should have the ability to utilize. Companies like Wave Dynamics in the Greenville area of South Carolina are experts at performing this type of analysis. Knowledge gives us the power to change the face of things. Statistical analysis gives us that edge we need to make sense of things that just don’t seem to make sense on the surface. It truly reveals the hidden secrets in any process.

Reference :

Brue G., Howes R. (2006). The McGraw-Hill 36-Hour Course Six Sigma. Mcgraw-Hill. New York, NY.