Technical Analysis:

Design of Experiments

Dr. Ir. M. Sabri, MT. IPM

Design of Experiments
 Design of experiments is also called statistically
designed experiments
 The purpose of the experiment and data analysis is to
find the cause and effect relationship between the output
and experimental factors in a process

2
 Design of Experiments
 First develop on Agricultural Experiment Design of
experiments
 Used to improved farm yields in the early1930s.
 The output, or response, variable y is the yield of a certain
farm crop.
 Controllable factors, such as x (x1, x2,…, xn), “farm
variables,” such as the amount of various fertilizers applied,
watering pattern, and selection of seeds.
 Uncontrollable factors, such as z (z1, z2,…, zn), is soil types,
weather patterns, and so on.
 In early agricultural experiments, the experimenter would
want to find the cause-and-effect relationship between the
yield and controllable factors, specifically.
 How different types of fertilizers, application quantities,
watering patterns, and types of seeds would influence crop
yield.
3
Design of Experiments
 In any DOE project, we change the experimental factors and
observe the effects on the output.
 The data obtained in the experiment will be used to fit empirical
models relating output y corresponding to experimental factors.
 Mathematically, we are trying to find the following functional
relationship:
y  (x1, x2, .., xn)  
 where ε is experimental error, or experimental variation.
 The existence of ε means that there may not be an exact functional
relationship between y and (x1,x2,...,xn).
 Due to
◦ Uncontrollable factors (z1,z2,…,zn) will influence the response y
but are not accounted for in the equation
◦ There are experimental and measurement errors on both y and
(x1,x2,…,xn) in the experiment.

4
Steps of DOE
Step 1: Project definition
 Identify the objective of the project and find the scope of
the problem.
 For example,
 In a product design, we need to identify what we want to
accomplish.
◦ Such as
 Do we want to reduce defect?
 Do we want to improve the current product’s performance?
 What is the performance?
 What is the project scope?
 Do we work on a subsystem or a component?

5
Steps of DOE
Step 2: Selection of response variable (output)
 Next is to select the response variable y. In selecting this
variable, the experimenter should determine if it could
provide useful information about the process under
study.
 Usually, theresponse variable is a key performance
measure of the process.
 What y should be
◦ Acontinuous variable, which would make data analysis much
easier and meaningful
◦ A variable that can be easily and accurately measured

6
Steps of DOE
Step 3: Choice of factors, levels, and ranges
 Steps 2 and 3 can be done simultaneously.
 Identify all the important factors which may significantly
influence the response variable.
 There are two kinds of factors
◦ A continuous factor can be expressed by continuous
real numbers.
◦ For example, weight, speed, and price are continuous
factors.
◦ Adiscrete factor is also called as category variable, or
attributes.
◦ Forexample, type of machines, type of seed, and type
of operating system are discrete factors.
 In a DOE project, each experimental factor will be
changed at least once

7
Steps of DOE
Step 3: Choice of factors, levels, and ranges
 The numbers of settings of a factor in the experiment are
called levels.
 For a continuous factor, these levels often correspond to
different numerical values.
 For example, two levels of temperature could be given as 200
and 300°C.
 For continuous factors, the range of the variable. If the range
of variable is too small, then we may miss lots of useful
information. If the range is too large, then the extreme values
might give infeasible experimental runs.
 For a discrete variable, the number of levels is often equal to
“the number of useful choices.”
 For example, if the “type of machine” is the factor, then the
number of levels depends on
 “How many types are there?” and “Which types do we want to
test in this experiment?

8
Steps of DOE
Step 4: Select an experimental design
 The type of experiment design to be selected will depend on
the number of factors, the number of levels in each factor,
and the total number of experimental runs that we can afford
to complete.
 In this example we consider a primarily full factorial designs
and fractional factorial designs.
 If the numbers of factors and levels are given, then a full
factorial experiment will need more experimental runs, thus
becoming more costly, but it will also provide more
informationabout the process under study.
 The fractional factorial will need a smaller number of runs,
thus costing less, but it will also provide less information
about the process.
 We will discuss how to choose a good experimental design in
subsequent sections.
9
Steps of DOE
Step 5: Perform the experiment
 When running the experiment, we must pay attention to
the following:
◦ Check performance of gauges and/or measurement devices first
◦ Check that all planned runs are feasible.
◦ Watch out for process drifts and shifts during the run.
◦ Avoid unplanned changes (e.g., swap operators at halfway point).
◦ Allow some time (and backup material) for unexpected events.
◦ Obtain buy-in from all parties involved.
◦ Preserve all the raw data
◦ Record everything that happens.
◦ Reset equipment to its original state after the experiment.

10
Steps of DOE
 Step 6: Analysis of DOE data
 Used statistical methods.
 From the analysis of experimental data, we are able to obtain the
following results:
 Identification of significant and insignificant effects and interactions.
◦ Not all the factors are the same in terms of their effects on the output.
◦ If the impact of response is relatively small then this factor might be
insignificant.
◦ Sometimes, two or more factors may interact, in which case their effects on
the output will be complex.
◦ If none of the experimental factors are found to be significant, in which case
the experiment is inconclusive.
◦ Analysis of variance can identify significant and insignificant factors for DOE.
 Ranking of relative importance of factor effects and interactions.
◦ Analysis of variance (ANOVA) can identify the relative importance of each
factor by giving a numerical score

11
Steps of DOE
 Step 6: Analysis of DOE data
 Empirical mathematical model of response versus
experimental factors.
◦ DOE can provide an empirical mathematical model relating the
output y to experimental factors.
◦ The form of the mathematical model could be linear or
polynomial, plus interactions.
◦ Can provide graphical presentations of the mathematical
relationship between experimental factors and output, in the form
of main-effects charts and interaction charts.
 Identification of best factor level settings and optimal
output performance level.
◦ If there is an ideal goal for the output, for example, if yis the yield
in an agricultural experiment, then the ideal goal for ywould be
“the larger, the better.”
12
Steps of DOE
 Step 7: Conclusions and recommendations
 conclusion about the project.
 If there is enough information, we might be able to recommend
some changes to the process to improve its performance.
Sometimes, the data analysis cannot provide enough information, in
which case we may have to do more experiments.
 When the analysis of the experiment is complete, we must verify
whether the conclusions are good. These are called confirmation
runs.
 The interpretation and conclusions from an experiment may include
a “best” setting to use to meet the goals of the experiment. Even if
this “best” setting were included in the design, you should run it
again as part of the confirmation runs to make sure that nothing has
changed and that the response values are close to their predicted
values.
 In an industrial setting, it is very desirable to have a stable process.
Therefore, one should run more than one test at the “best” settings.
13
Steps of DOE
 Step 7: Conclusions and recommendations
 A minimum of three runs should be conducted. If the time between
actually running the experiments and conducting the confirmation runs
is more than a few hours, the experimenter must be careful to ensure
that nothing else has changed since the original data collection.
 If the confirmation runs don’t produce the results you expected, then
you need to check to see that nothing has changed since the original
data collection.
 Verify that you have the correct settings for the confirmation runs.
 Revisit the model to verify the “best” settings from the analysis.
 Verify that you had the correct predicted value for the confirmation runs.
 If you don’t find the answer after checking these four items, the model
may not predict very well in the region that you decided was “best.”
 However, you still will have learned something from the experiment,
and you should use the information gained from this experiment to
design another follow-up experiment.

14
Factorial Experiment
 Most industrial experiments involve two or more experimental
factors
 factorial designs are the most frequently used designs
 all combinations of factor levels will be tested in the experiment
 For example,
 we have two factors in the experiment, say, factors A and B. If A has
a levels, B has b levels, then in a factorial experiment, we are going
to test all ab combinations.
 In each combination, we may duplicate the experiment several
times, say, n times. Then, there are n replicates in the experiment.
If n=1, then we call it a single replicate. Therefore, for two factors,
the total number of experimental observations is equal to abn.

15
Factorial Experiment
 An article in Industrial Quality Control (1956, vol. 13, no. 1, pp.
5–8) describes an experiment to investigate the effect of the type of
glass and the type of phosphor on the brightness of a television
tube.
 The response variable is the current necessary (in microamps) to
obtain a specific brightness level.
 The data are listed in Table 12.1.In this example, we can call the
glass type factor A, so the number of A levels is a=2;
 the phosphor type, factor B, with b=3 as the number of B levels. The
number of
 replicates in this example is n =3. Total experimental observations
2 x3x3=18 runs.
 Both glass and phosphor types are discrete factors.
 We would like to determine the following:

16
Factorial Experiment
 We would like to determine the following:
 How the glass and phosphor types will affect the brightness of the
TV, and whether these effects, if any, are significant, and whether
there are any interactions.
 Y is defined to be the current needed to achieve a certain brightness
level, so the smaller current means higher efficiency. We would like
to find a glass phosphor combination that gives the best efficiency.

17
Factorial Experiment
 The layout of the two factorial experiment

 Mathematical model
 If we denote A as x1 and B as x2, then one possible mathematical
model is
 Equation 1
 Here f1(x1) and f2(x2) are the main effects of A and B, respectively,
and f12(x1, x2) is the interaction of A and B.

 Equation 2

18
Factorial Experiment
 What is interaction means
 If no interaction
 Equation 3

 where f1(x1) is a function of x1 alone and f2(x2) is a function of x2

alone; we call Equation 3 the additive model. However, if the
interaction effect is not equal to zero, then we do not have an
additive model.

19
Factorial Experiment

20
Factorial Experiment

21