EXPERIMENTAL AND RESEARCH

METHODS
TEGT3762

THE EXPERIMENTAL DESIGN

 DESIGN OF EXPERIMENTS (DOE)
What is experimental design?
The (statistical) design of experiments (DOE) is

an efficient procedure for planning

experiments so that the data obtained can be
analyzed to yield valid and objective
conclusions
 a systematic, rigorous approach to engineering
problem-solving that applies principles and
techniques at the data collection stage so as to
ensure the generation of valid, defensible, and
supportable conclusions.

In addition, all of this is carried out under the

constraint of a minimal expenditure of runs, time,
and money.
3
 An Experimental Design is the laying out of a
detailed experimental plan in advance of
doing the experiment.

Well chosen experimental designs maximize

the amount of "information" that can be
obtained for a given amount of experimental
effort.
 Applications of design of experiments

Comparative

Screening/Characterizing

Modeling

Optimizing

5
 Comparative
assessing whether a change in a single factor
has in fact resulted in a change/improvement
to the process as a whole.
choosing between alternatives, with narrow
scope
suitable for an initial comparison

6
 Screening/characterisation
understanding" the process as a whole in the
sense that he/she wishes (after design and
analysis) to have in hand a ranked list of
important to unimportant factors (most
important to least important) that affect the
process (considering multiple factors).

to identify which factors/effects are important

when you have 2 - 4 factors and can perform a
full factorial design
7
 Modeling
interested in functionally modeling the
process with the output being a good-fitting
mathematical function, and in having good
estimates of the coefficients in that function.

Good fitting (=high predictive power)

Good estimates (=maximal accuracy)

Y=mx + c

8
 Optimization/Response surface
interested in determining optimal settings of
the process factors;

i.e, to determine for each factor, the level of

the factor that optimizes the process
response.

9
 Optimization
hit a target
maximize or minimize a response
reduce variation by locating a region where
the process is easier to manage
 RESEARCH DESIGN
After identifying the objective listed above
that corresponds most closely to your specific
goal, you can:
Proceed to select the experimental factors
Select the appropriate design
Process variables include both inputs and
outputs - i.e., factors and responses
 Include all important factors (based on
engineering judgment).

Check the factor settings for impractical or

impossible combinations - i.e., very low
pressure and very high gas flows.

Include all relevant responses.

 Introduction: What is meant by DOE?
Experiment -
a test or a series of tests in which
purposeful changes are made to the
input variables or factors of a system
so that we may observe and identify
the reasons for changes in the
output response(s).

13
 A designed experiment
-is an investigation in which a specified framework
is provided in order to observe, measure and
evaluate groups/treatments with respect to a
designated response

-the researcher controls the elements of the

framework in order to obtain data which are
statistically valid and the results can be inferred to
the general population from which the sample was
taken from
 There are two types of variables in an
experimental study:

factors-:- controlled variables selected by the

researcher for making comparisons, depending
on the objective of the research (independent
variable)

measurements:- these are the response

variables/observations that are recorded after
the factors have (dependent variables)
Example
Researchers want to evaluate the quality of plastic
irrigation pipes.

The study involved a total of 24 pipes, with 12 pipes

randomly selected from each of the two manufacturing
plants.

The pipes were manufactured using one of two water

temperatures (200 oC and 175 oC) and one of the three
types of hardeners.

The compressive strength of each pipe was determined

for analysis.
 For this experiment the factors of interest
are:-
Manufacturing company (2 levels)
Temperature (2 levels)
Hardeners (3 levels)
 The response variable is the compressive
strength of the pipe (units of measurement)
 Treatment
Treatments are conditions constructed from
the factors in the study.

In a single factor experiment, the factor is the

same as the treatment.

Once you have two or more factors in the

experiment, then the treatments will be a
combination of these factors.
 Treatment combinations
Manufacturing company (2 levels)- A, B
The Temperature (2 levels)- 175 oC and 200 oC
The hardeners (3 levels)- BKF, BPA and Aziridine
BFK-Plastic Acrylic Casting Resin
BPA-Bisphenol A

Treatment combinations =2x2x3 =12

Unique combinations of the factor levels
 FACTORIAL TREATMENT DESIGN
Treatment Manufacturer Temperature Hardener
1 A 175 BKF
2 A 175 BPA
3 A 175 AZARINIDE
4 A 200 BKF
5 A 200 BPA
6 A 200 AZARINIDE
7 B 175 BKF
8 B 175 BPA
9 B 175 AZARINIDE
10 B 200 BKF
11 B 200 BPA
12 B 200 AZARINIDE
 Experimental unit
An experimental unit is the physical entity to
which the treatment is randomly assigned e.g.
the pipe, a petri dish, an animal, a person, a
plot etc.

Measurements are done on the experimental

unit.
 Replication
it is the application of each treatment to
more than one experimental unit.

Minimum replication is two, but this is not to

be encouraged since any accident in the
experiment will result in no results at all, at
least three replications are allowed, but the
more the replication the better the
experimental results.
 Why replication is important
Replication
i. enables the experimental error to be
estimated,
ii. can improve on the precision of the
estimated parameters
iii. helps to reduce the level of noise or
unexplained variation/experimental error
in the data
Treatment Manufacturer Temperature Hardener Compressive
strength
1 A 175 BKF
2 A 175 BPA
3 A 175 AZARINIDE
4 A 200 BKF
5 A 200 BPA
6 A 200 AZARINIDE
7 B 175 BKF
8 B 175 BPA
9 B 175 AZARINIDE
10 B 200 BKF
11 B 200 BPA
12 B 200 AZARINIDE
13 A 175 BKF
14 A 175 BPA
15 A 175 AZARINIDE
16 A 200 BKF
17 A 200 BPA
18 A 200 AZARINIDE
19 B 175 BKF
FACTORS
Manufacturing company (2 levels)- A, B
The Temperature (2 levels)- 175 oC and 200 oC
The hardeners (3 levels)- BKF, BPA and Aziridine
BFK-Plastic Acrylic Casting Resin
BPA-Bisphenol A

Treatment combinations =2x2x3 =12

If the treatments are replicated 2 times, then the
experiment has 24 experimental units
Full factorial design/ fractional factorial
designs
 Randomisation
it is a procedure that is undertaken to ensure
that each experimental unit has an equal
chance of being assigned to any of the
experimental treatments.
 Why is randomization important
Randomisation
i. removes bias, since it has to be done by a mechanical
process where no subjective human influence can affect
the outcome.
ii. removes the correlation between the errors (time series
data, there is autocorrelation and the analysis is specific
for this type of data, repeated measures data e.g.
growth) and
iii. ensures independence of results from each of the
experimental units.
Experimental error
Experimental error is the variation in the
responses among experimental units that are
assigned the same treatment and are
observed under the same experimental
conditions

Experimental error can never be zero in

statistical experiments
 What causes experimental error
The natural differences in the experimental units
prior to receiving the treatment

The variation in the devices that record the

measurements (calibration of equipment)

The variation in setting the treatment conditions

The effect on the response variable of all

extraneous factors other than the treatment (e.g.
environmental conditions, genetic variation)
Control of experimental error

The are many potential sources of experimental error

in an experiment

When the variance is large (inflated), the precision of

our inferences will be greatly compromised.

Any techniques that can be implemented to reduce

experimental error will lead to improved experiment
and more precise estimates

31
Blocking
Blocking is one way of controlling extraneous
conditions
Blocking can be defined as the grouping of
experimental units into comparable homogeneous
groups with each group representing a level of the
blocking variable, units in each block are as similar
as is possible but differ from units in the next block
significantly.
Blocking factors can be pH, material type, moisture
level, gradient, age, time of experimentation, weight,
position, etc.
32
 The experimental units are placed into groups based on
their similarity with respect to a characteristic that may
affect the response
Units in a block are considered homogenous
Treatments are randomly assigned separately within each
block
In a randomised complete block design (RCBD), the
experimental units in a block are equal to the treatments
being compared
Alternative is a randomised incomplete block design
(RIBD)
 Strategy of Experimentation
Strategy of experimentation
Best guess approach (trial and error)
can continue indefinitely
cannot guarantee best solution has been found
One-factor-at-a-time (OFAT) approach
inefficient (requires many test runs)
fails to consider any possible interaction between factors
Factorial approach (invented in the 1920s)
Factors varied together
Correct, modern, and most efficient approach
Can determine how factors interact
Used extensively in industrial R and D, and for process
improvement.
34
 Statistical Design of Experiments
All experiments should be designed experiments
Unfortunately, some experiments are poorly
designed - valuable resources are used
ineffectively and results inconclusive
Statistically designed experiments permit
efficiency and economy, and the use of statistical
methods in examining the data result in scientific
objectivity when drawing conclusions.

35
 DOE is a methodology for systematically
applying statistics to experimentation.
DOE lets experimenters develop a mathematical
model that predicts how input variables interact
to create output variables or responses in a
process or system.
DOE can be used for a wide range of
experiments for various purposes including
nearly all fields of engineering.
Use of statistics is very important in DOE and
the basics are covered in a first course in an
engineering program.
36
 In general, by using DOE, we can:
Learn about the process we are investigating
Screen important variables
Build a mathematical model
Obtain prediction equations
Optimize the response (if required)

Statistical significance is tested using ANOVA, and

the prediction model is obtained using regression
analysis.

37
 Applications of DOE in Engineering Design
Experiments are conducted in the field of
engineering to:
evaluate and compare basic design configurations
evaluate different materials
select design parameters so that the design will work
well under a wide variety of field conditions (robust
design)
determine key design parameters that impact
performance

38
 INPUTS OUTPUTS
(Factors) (Responses)
X variables Y variables

People

Materials

PROCESS: responses related

Equipment to performing a
service

responses related
Policies to producing a
A Ble nding of produce
Inputs which
Ge ne rates responses related
Procedures
Corresponding to completing a task
Outputs

Methods

Env ironment Illustration of a Proce ss

39
 INPUTS OUTPUTS
(Factors) (Responses)
X variables Y variables

Type of
cement

compressive
Percent water
strength

PROCESS:
Type of
modulus of elasticity
Additiv es

Percent
Discov e ring modulus of rupture
Additiv es
Optimal
Concre te
Mixing Time M ixture Poisson's ratio

Curing
Conditions

% Plasticizer Optimum Concre te M ixture

40
 INPUTS OUTPUTS
(Factors) (Responses)
X variables Y variables

Type of Raw
Material

Mold
Temperature

Holding PROCESS: thickness of molded

Pressure part

% shrinkage f rom
Holding Time
mold size
M anufacturing
Inje ction number of defective
Gate Size
M olde d Parts parts

Screw Speed

Moisture M anufacturing Inje ction M olde d

Content Parts

41
 INPUTS OUTPUTS
(Factors) (Responses)
X variables Y variables

Impermeable layer
(mm)

Initial storage PROCESS:

(mm)

Coef f icient of R-square:

Inf iltration Predicted vs
Observed Fits
Rainfall-Runoff
M odel
Coef f icient of
Recession
Calibration

Soil Moisture
Capacity
(mm)

M odel Calibration
Initial Soil Moisture
(mm)

42
 Basic Principles

Statistical design of experiments (DOE)

the process of planning experiments so that
appropriate data can be analyzed by statistical
methods that results in valid, objective, and
meaningful conclusions from the data
involves two aspects: design and statistical
analysis

43
 Every experiment involves a sequence of
activities:
Conjecture - hypothesis that motivates the
experiment
Experiment - the test performed to investigate
the conjecture
Analysis - the statistical analysis of the data
from the experiment
Conclusion - what has been learned about the
original conjecture from the experiment.

44
 Three basic principles of Statistical DOE
Replication
allows an estimate of experimental error
allows for a more precise estimate of the sample mean
value
Randomization
cornerstone of all statistical methods
average out effects of extraneous factors
reduce bias and systematic errors
Blocking
increases precision of experiment
factor out variable not studied
45
 Guidelines for Designing Experiments
Recognition of and statement of the problem
need to develop all ideas about the objectives of the
experiment - get input from everybody - use team
approach.
Choice of factors, levels, ranges, and response
variables.
Need to use engineering judgment or prior test results.
Choice of experimental design
sample size, replicates, run order, randomization,
software to use, design of data collection forms.

46
 Performing the experiment
vital to monitor the process carefully. Easy to
underestimate logistical and planning aspects in a
complex R and D environment.
Statistical analysis of data
provides objective conclusions - use simple graphics
whenever possible.
Conclusion and recommendations
follow-up test runs and confirmation testing to validate
the conclusions from the experiment.
Do we need to add or drop factors, change
ranges, levels, new responses, etc.. ???

47
 Using Statistical Techniques in
Experimentation - things to keep in mind
Use non-statistical knowledge of the problem
physical laws, background knowledge
Keep the design and analysis as simple as possible
Dont use complex, sophisticated statistical techniques
If design is good, analysis is relatively straightforward
If design is bad - even the most complex and elegant
statistics cannot save the situation
Recognize the difference between practical and
statistical significance
statistical significance practically significance

48
 Experiments are usually iterative
unwise to design a comprehensive experiment at the
start of the study
may need modification of factor levels, factors,
responses, etc.. - too early to know whether
experiment would work
use a sequential or iterative approach
should not invest more than 25% of resources in the
initial design.
Use initial design as learning experiences to accomplish
the final objectives of the experiment.

49
 Factorial v.s. OFAT
Factorial design - experimental trials or runs are
performed at all possible combinations of factor
levels in contrast to OFAT experiments.

Factorial and fractional factorial experiments are

among the most useful multi-factor experiments
for engineering and scientific investigations.

50
 The ability to gain competitive advantage requires
extreme care in the design and conduct of
experiments. Special attention must be paid to joint
effects and estimates of variability that are provided
by factorial experiments.

Full and fractional experiments can be conducted

using a variety of statistical designs. The design
selected can be chosen according to specific
requirements and restrictions of the investigation.

51
 Factorial Designs
In a factorial experiment, all
possible combinations of
factor levels are tested
The golf experiment:
Type of driver (over or regular)
Type of ball (balata or 3-piece)
Walking vs. riding a cart
Type of beverage (Beer vs water)
Time of round (am or pm)
Weather
Type of golf spike
Etc, etc, etc

52
 One-factor-at-a-time experiments (OFAT)
OFAT experiments are regarded as easier to implement,
more easily understood, and more economical than
factorial experiments. Better than trial and error.
OFAT experiments are believed to provide the optimum
combinations of the factor levels.
Unfortunately, each of these presumptions can generally
be shown to be false except under very special
circumstances.
The key reasons why OFAT should not be conducted
except under very special circumstances are:
Do not provide adequate information on interactions
Do not provide efficient estimates of the effects
53
 Approaches to Experimentation: One-
Factor-at-a-Time
One-factor-at-a-time
procedure (2 level example)
run all factors at one condition
repeat, changing condition of one factor
continuing to hold that factor at that condition, rerun with another
factor at its second condition
repeat until all factors at their optimum conditions
slow, expensive: many tests
can miss interactions!

54
 One-Factor-At-A-Time
Process: Yield = f(temperature, pressure)

50% yield
30% yield

Max yield: 50% at 78C, 130 psi?

55
 One-Factor-At-A-Time

A better view of the maximum yield!

Optimized yield is over 85%

Process: Yield = f(temperature, pressure)

56
 Factorial vs OFAT ( 2-levels only)
Factorial OFAT
2 factors: 4 runs 2 factors: 6 runs
3 effects 2 effects
3 factors: 8 runs 3 factors: 16 runs
7 effects 3 effects
5 factors: 32 or 16 runs 5 factors: 96 runs
31 or 15 effects 5 effects
7 factors: 128 or 64 7 factors: 512 runs
runs 7 effects
127 or 63 effects
57
 Example: Factorial vs OFAT

Factorial OFAT

high high
Factor B B
low
low

low high low high

Factor A A

E.g. Factor A: Reynolds number, Factor B: k/D

58
 FACTORIAL (2k) DESIGNS (k = 2):
GRAPHICAL OUTPUT
Neither factor A nor Factor B have an effect on
the response variable.

59
 FACTORIAL (2k) DESIGNS (k = 2):
GRAPHICAL OUTPUT
Factor A has an effect on the response
variable, but Factor B does not.

60
 FACTORIAL (2k) DESIGNS (k = 2):
GRAPHICAL OUTPUT
Factor A and Factor B have an effect on the
response variable.

61
 FACTORIAL (2k) DESIGNS (k = 2): GRAPHICAL
OUTPUT
Factor B has an effect on the response variable, but only if factor A is
set at the High level. This is called interaction and it basically
means that the effect one factor has on a response is dependent on
the level you set other factors at. Interactions can be major problems
in a DOE if you fail to account for the interaction when designing your
experiment.

62
 Design of Engineering Experiments
Basic Statistical Concepts
Simple comparative experiments
The hypothesis testing framework
The two-sample t-test
Checking assumptions, validity
Comparing more than two factor levelsthe
analysis of variance
ANOVA decomposition of total variability
Statistical testing & analysis
Checking assumptions, model validity
Post-ANOVA testing of means

63
 Example
Formulation of a cement mortar
Original formulation and modified formulation
10 samples for each formulation
One factor formulation
Two formulations:
two treatments
two levels of the factor formulation

64
 Portland Cement Formulation
Observation Modified Mortar Unmodified Mortar
(sample), j (Formulation 1) y1 j (Formulation 2) y2 j

1 16.85 17.50
2 16.40 17.63
3 17.21 18.25
4 16.35 18.00
5 16.52 17.86
6 17.04 17.75
7 16.96 18.22
8 17.15 17.90
9 16.59 17.96
10 16.57 18.15

65
 Basic Statistical Concepts

Experiences from above example

Run each of above observations
Noise, experimental error, error the individual runs
difference
Statistical error arises from variation that is
uncontrolled and generally unavoidable
The presence of error means that the response
variable is a random variable
Random variable could be discrete or continuous

66
 Basic Statistical Concepts

Describing sample data

Graphical descriptions
Dot diagramcentral tendency,
spread
Box plot
Histogram

67
Graphical View of the Data
Dot Diagram
Dotplots of Form 1 and Form 2
(means are indicated by lines)

18.3

17.3

16.3

Form 1 Form 2

68
 Box Plots
Boxplots of Form 1 and Form 2
(means are indicated by solid circles)

18.5

17.5

16.5

Form 1 Form 2

69
 Histogram

70
 Two level design experiments
The most popular experimental designs are two-
level designs.
Why only two levels?
There are a number of good reasons why two is the
most common choice amongst engineers:
it is ideal for screening designs,
simple and economical;
it also gives most of the information (required to go
to a multilevel response surface experiment if one is
needed. )
 Example
Two factors: results in distinct 4 treatments
(see table)
Bottle type:
glass
plastic
Chemical for disinfecting
- Chlorine
- metabisulphate
 Bottle type

CHEMICAL Glass Plastic

Chlorine Glass, Chlorine Plastic ,Chlorine
Metabisulphate Glass, Metabisulphate Plastic, metabisulphate
 Design selection guide

Number Comparative Screening Response Surface

of Factors Objective Objective Objective

1 1-factor completely _ _
randomized design

2-4 Randomized block Full or fractional Central composite

design factorial or Box-Behnken

5 or more Randomized block Fractional factorial Screen first to

design or Plackett-Burman reduce number of
factors