INTRODUCTION The term experiment is defined as the systematic procedure carried out under controlled conditions in order to discover an unknown effect, to test or establish a hypothesis, or to illustrate a known effect. When analyzing a process, experiments are often used to evaluate which process inputs have a significant impact on the process output, and what the target level of those inputs should be to achieve a desired result (output). Experiments can be designed in many different ways to collect this information. Design of Experiments (DOE) is also referred to as Designed Experiments or Experimental Design - all of the terms have the same meaning. Experimental design can be used at the point of greatest leverage to reduce design costs by speeding up the design process, reducing late engineering design changes, and reducing product material and labor complexity. Designed Experiments are also powerful tools to achieve manufacturing cost savings by minimizing process variation and reducing rework, scrap, and the need for inspection. PREPARATION Some of the basic requirements to properly understand Design of experiment some statistical tools like:- a. Histogram b. Statistical process control c. Regression and correlation analysis
4
COMPONENTS OF EXPERIMENTAL DESIGN Consider the following diagram of a cake-baking process (Figure 1). There are three aspects of the process that are analyzed by a designed experiment:
a. Factors, or inputs to the process. Factors can be classified as either controllable or uncontrollable variables. In this case, the controllable factors are the ingredients for the cake and the oven that the cake is baked in. The controllable variables will be referred to throughout the material as factors. Potential factors can be categorized using the Fishbone Chart (Cause & Effect Diagram) b. Levels, or settings of each factor in the study. Examples include the oven temperature setting and the particular amounts of sugar, flour, and eggs chosen for evaluation c. Response, or output of the experiment. In the case of cake baking, the taste, consistency, and appearance of the cake are measurable outcomes potentially influenced by the factors and their respective levels.
5
PURPOSE OF EXPERIMENTATION a. Comparing Alternatives. In the case of our cake-baking example, we might want to compare the results from two different types of flour. If it turned out that the flour from different vendors was not significant, we could select the lowest-cost vendor. If flour were significant, then we would select the best flour. The experiment(s) should allow us to make an informed decision that evaluates both quality and cost. b. Identifying the Significant Inputs (Factors) Affecting an Output (Response) - separating the vital few from the trivial many. Wemight ask a question: "What are the significant factors beyond flour, eggs, sugar and baking?" c. Achieving an Optimal Process Output (Response). "What are the necessary factors, and what are the levels of those factors, to achieve the exact taste and consistency of Mom's chocolate cake? d. Reducing Variability. "Can the recipe be changed so it is more likely to always come out the same?" e. Minimizing, Maximizing, or Targeting an Output (Response). "How can the cake be made as moist as possible without disintegrating?" f. Improving process or product "Robustness" - fitness for use under varying conditions. "Can the factors and their levels (recipe) be modified so the cake will come out nearly the same no matter what type of oven is used?" g. Balancing Trade-offs when there are multiple Critical to Quality Characteristics (CTQC's) that require optimization. "How do you produce the best tasting cake with the simplest recipe (least number of ingredients) and shortest baking time?
6
EXPERIMENT DESIGN GUIDELINES The Design of an experiment addresses the questions outlined above by stipulating the following: a. The factors to be tested. b. The levels of those factors. c. The structure and layout of experimental runs, or conditions. A well-designed experiment is as simple as possible - obtaining the required information in a cost effective and reproducible manner. When designing an experiment, pay particular heed to four potential traps that can create experimental difficulties: a. Unexplained variation: - In addition to measurement error (explained above), other sources of error. Error refers to all unexplained variation that is either within an experiment run or between experiment runs and associated with level settings changing b. Noise Factors: - They are the uncontrollable factors that induce variation under normal operating conditions. These factors, such as multiple machines, multiple shifts, raw materials, humidity, etc., can be built into the experiment so that their variation doesn't get lumped into the unexplained, or experiment error. A key strength of Designed Experiments is the ability to determine factors and settings that minimize the effects of the uncontrollable factors. c. Correlation: - Can often be confused with causation. Two factors that vary together may be highly correlated without one causing the other -they may both be caused by a third factor. Brainstorming exercises and Fishbone Cause & Effect Diagrams are both excellent techniques to deal with this situation. d. Combined effects or interactions:- between factors demand careful thought prior to conducting the experiment.
7
EXPERIMENT DESIGN PROCESS
TEST OF DOE ONE FACTOR EXPERIMENT One of the most common types of experiments is the comparison of two process methods, or two methods of treatment. There are several ways to analyze such an experiment depending upon the information available from the population as well as the sample. One of the most straight-forward methods to evaluate a new process method is to plot the results on an SPC chart that also includes historical data from the baseline process, with established control limits. Then apply the standard rules to evaluate out-of-control conditions to see if the process has been shifted. An alternative to the control chart approach is to use the F-test (F-ratio) to compare the means of alternate treatments. This is done automatically by the ANOVA (Analysis of Variance) function.
8
FULL FACTORIALS As their name implies, full factorial experiments look completely at all factors included in the experimentation. In full factorials, we study all of the possible treatment combinations that are associated with the factors and their levels. They look at the effects that the main factors and all the interactions between factors have on the measured responses. If we use more than two levels for each factor, we can also study whether the effect on the response is linear or if there is curvature in the experimental region for each factor and for the interactions. Full factorial experiments can require many experimental runs if many factors at many levels are investigated. FRACTIONAL FACTORIALS Fractional factorials look at more factors with fewer runs. Using a fractional factorial involves making a major assumption - that higher order interactions (those between three or more factors) are not significant. Fractional factorial designs are derived from full factorial matrices by substituting higher order interactions with new factors. To increase the efficiency of experimentation, fractional factorials give up some power in analyzing the effects on the response. Fractional factorials will still look at the main factor effects, but they lead to compromises when looking into interaction effects. o This compromise is called confounding. o Just because we have confounded main factor and interaction effects doesnt mean fractional factorials are a poor choice. The risks we are taking are well worth it. o Three-way and higher interactions are rare and even two-way interactions are not that commonplace. The efficiency in experimentation more than makes up for the confounding of results that we get.
9
SCREENING EXPERIMENTS Screening experiments are the ultimate fractional factorial experiments. These experiments assume that all interactions, even two-way interactions, are not significant. They literally screen the factors, or variables, in the process and determine which are the critical variables that affect the process output. There are two major families of screening experiments: Drs. Plackett and Burman developed the original family of screening experiments matrices in the 1940s. Dr. Taguchi adapted the PlackettBurman screening designs. He modified the PlackettBurman design approach so that the experimenter could assume that interactions are not significant, yet could test for some two-way interactions at the same time. RESPONSE SURFACE ANALYSIS Response surface analysis is an off-line optimization technique. Usually, 2 factors are studied; but 3 or more can be studied. With response surface analysis, we run a series of full factorial experiments and map the response to generate mathematical equations that describe how factors affect the response. EVOP EVOP (evolutionary operations) is an online optimization technique. Usually, two factors are studied using small, step changes in factor levels to incrementally explore the operating bounds of the process. MIXTURE EXPERIMENTS The designs we have looked at so far work fine for variables like temperature, pressure, or time and even for material substitutions. But they will not work in situations where we need to study how changes in the formulation affect the final properties of a material. When dealing with formulations, there are added constraints on the experimenter. When dealing with composition, the sum of all of the weight fractions of all the components must add up to 1.0 and each of the individual components must have a weight fraction between 0 and 1.0. Mixture experiments provide techniques to operate within these constraints.
10
EXPERIMENTAL STRATEGY When setting up an experimental strategy, it is usually best to start with screening experiments to separate out the important (significant) factors from the many factors in a process. From there we can experiment further on the significant factors and study their interactions with fractional factorial or full factorial experiments. In some cases, once we have identified the power factors, we may want to optimize the response using the power factors in one of the two major DOE techniques for optimizing processes, Response Surface Analysis or EVOP.
TAGUCHI METHODS Dr. Genichi Taguchi is a Japanese statistician and Deming prize winner who pioneered techniques to improve quality through Robust Design of products and production processes. Dr. Taguchi developed fractional factorial experimental designs that use a very limited number of experimental runs. Traditional thinking is that any part or product within specification is equally fit for use. In that case, loss (cost) from poor quality occurs only outside the specification. However, Taguchi makes the point that a part marginally within the specification is really little better than a part marginally outside the specification. As such, Taguchi describes a continuous Loss Function that increases as a part deviates from the target, or nominal value .The Loss Function stipulates that society's loss due to poorly performing products is proportional to the square of the deviation of the performance characteristic from its target value.
11
Taguchi adds this cost to society (consumers) of poor quality to the production cost of the product to arrive at the total loss (cost). Taguchi uses designed experiments to produce product and process designs that are more robust - less sensitive to part/process variation
SUMMARY Designed experiments are an advanced and powerful analysis tool during projects. An effective experimenter can filter out noise and discover significant process factors. The factors can then be used to control response properties in a process and teams can then engineer a process to the exact specification their product or service requires. A well-built experiment can save not only reduces project time but also solve critical problems which have remained unseen in processes. Specifically, interactions of factors can be observed and evaluated. Ultimately, teams will learn what factors matter and what factors do not.
12
APPENDIX 1. https://www.moresteam.com/toolbox/design-of-experiments.cfm 2. Webster's Ninth New Collegiate Dictionary 3. Mark J. Anderson and Patrick J. Whitcomb, DOE Simplified (Productivity, Inc. 2000). 4. http://www.qualitytrainingportal.com/resources/doe/doe_types.htm