Factorial designs

Independent variable
One-way (between subjects) ANOVA has one
independent variable that can have several levels or
conditions

Example: what is the effect of sleep deprivation on

exam performance?

• 2 levels: 0 hrs, 10 hrs

• 3 levels: 0 hrs, 5 hrs, 10 hrs

• 4 levels: 0 hrs, 4 hrs, 8hrs, 9 hrs

 Independent variable
In many cases, desirable to have multiple levels of the
independent variable, and they are not necessarily ordered
2 levels: presence vs. absence

3 levels: control + 2 types of instruction

• no instructions, study, test

4 Levels: control + 3 levels

• placebo, 100 mg, 250 mg, 500 mg
• 3 levels of dosage lets us test for curvilinear effects

In a non-experimental design, this can be a participant variable

“No experience with mental health problems” vs. “Had family members with
mental health problems,” vs. “Had mental health problems oneself”
 Multiple Levels
120
Intelligence Judgment

114

108

102

96

90
Less Attractive Attractive More Attractive
 Multiple Levels
50

40
Performance

30

20

10

0
Control 100 mg 250 mg 500 mg
Condition
 One-factor design
Despite having multiple levels, all of these
examples have only 1 independent variable
and are analyzed with one-way ANOVA
 Factorial designs
An experiment with 2 or more independent
variables (or factors), with each factor having
multiple levels or conditions

2 × 2 design:
• two factors (there are 2 numbers)
• each has two levels (the number 2)

2 × 3 design:
• two factors (there are 2 numbers)
• one has 2 levels, one has 3 levels
 Example: sleep and memory
Studied using a 2 x 4 between-groups design

Factor 1 – sleep state

• sleep (experimental) or awake (control)

Factor 2 – delay
• assess after 1 hour, 2 hours, 4 hours, or 8
hours

Dependent variable – number of items

remembered from a list Jenkins & Dallenbach (1924)
 Example 1: sleep and memory
A 2 × 4 design means 8 groups or conditions
• if 10 people tested per group in a between-subjects
design, then 80 people tested

Factor 2: delay

1 hour 2 hours 4 hours 8 hours

Asleep
Factor 1:
sleep state
Awake

Jenkins & Dallenbach (1924)

Example 1: sleep and memory
Awake Asleep

12

10
Number Recalled

0
0 2 4 6 8 10
Delay (Hours)

Jenkins & Dallenbach (1924)

 Example 1: sleep and memory
Each level of each factor appears in combination
with each level of every other factor
• test sleep subjects after 1, 2, 4, or 8 hours

• test awake subjects after 1, 2, 4, or 8 hours

Ø different effects of sleep at different delays

Jenkins & Dallenbach (1924)

 Example 1: sleep and memory
Awake Asleep
If tested only one 12
delay, results would be 10

Number Recalled
incomplete or 8

misleading: 6
4
2
1 hour: small effect
0
0 2 4 6 8 10
8 hours: large effect Delay (Hours)

Jenkins & Dallenbach (1924)

 Example 1: Sleep and memory
State and delay interact: Awake Asleep

• 1 hour: small effect 12

10
• 8 hours: huge effect

Number Recalled
8
6
Different effects of delay 4
depending on state 2
0
0 2 4 6 8 10
Different effects of state Delay (Hours)

depending on delay

Jenkins & Dallenbach (1924)

 Some other factorial designs
2×2 2×3

Factor 2 Factor 2

Level 1 Level 2 Level 1 Level 2 Level 3

Level 1 Level 1
Factor 1 Factor 1
Level 2 Level 2

Number of numbers = number of factors

Individual numbers = number of levels
Order matters: the first number represents factor 1
 A 2 × 2 × 2 factorial design
Number of numbers = 3 so 3 factors
Each factor has 2 levels

Factor 3 Level 1 Factor 3 Level 2

Factor 2 Factor 2

Level 1 Level 2 Level 1 Level 2

Level 1
Factor 1
Level 2
 A 2 × 2 × 2 factorial design
Factor 1: modality (vision vs. audition)
Factor 2: type of information (factual vs. conceptual)
Factor 3: delay (immediate vs. delayed test)

Factor 3 Level 1: Immediate Factor 3 Level 2: Delayed

Factor 2 Factor 2

Factual Conceptual Factual Conceptual

Vision
Factor 1
Audition
 A 2 × 2 × 3 factorial design
Number of numbers = 3 so 3 factors
Two factors have 2 levels and one factor has 3 levels

Factor 3 Level 1 Factor 3 Level 2 Factor 3 Level 3

Factor 2 Factor 2 Factor 2

Level 1 Level 2 Level 1 Level 2 Level 1 Level 2

Level 1
Factor 1
Level 2
 2 × 2 factorial design
A very common design
• not too complex
• not too simple
• allows examination of more than 1 factor at a time

Analysis
• ANOVA
• main effects
• interactions
Example 2: perception of managers’ skill

Subjects received either positive or negative

feedback from either a female or male
manager
• 2 × 2 between-subjects factorial

Subjects then asked to rate the manager’s skill

• higher scores mean more skilled
• lower scores mean less skilled
Example 2: perception of managers’ skill
2 x 2 between-subjects design

female, positive

female, negative
Population Sample
male, positive

male, negative

Random Random
sampling assignment
Example 2: perception of managers’ skill

Dependent variable: rating of manager’s skill

Factor 1: type of feedback

• positive
• negative

Factor 2: gender of manager

• female
• male
Sinclair & Kunda (2000)
Example 2: perception of managers’ skill

Main effects: basically the same as for a one-

way design, but we collapse over (ignore)
levels of the other factors
• ignoring type of feedback, what is the effect of gender
on skill rating of female and male managers?

• ignoring gender of manager, what is the effect of type

of feedback on skill rating of managers?
Example 2: perception of managers’ skill
Rated skill of manager
Manager’s Gender

Female Male Row Means

Positive 9.2 8.8 9.0

Feedback
Negative 6.8 8.0 7.4

Col. Means 8.0 8.4

No main effect of gender

Main effect of feedback
 Two main effects
Collapsed over Feedback Collapsed over Gender

10 10

Rating of Manager
Rating of Manger

9 9

8 8

7 7

6 6
Female Male Positive Negative

No main effect of gender

Significant main effect of feedback
 Interaction
An interaction between two factors occurs
when the effect of one independent variable
on the dependent variable is different across
the levels of the other independent variable

• graphically, interactions are present when results

are not parallel lines

• if the lines are parallel, there is no interaction

 Interaction
Gender of Manager
Female Male

10
Rating of Manager
9

No effect of 8
gender with
positive Significant
feedback 7 effect of
gender with
6 negative
Positive Negative feedback
Type of Feedback

Gender has different effects on ratings of skill

depending on whether feedback is positive or negative
 Interaction
Type of Feedback
Positive Negative

10
Rating of Manager
9

7
Significant
No effect of
effect of
6 feedback
feedback
Female Male for males
for females
Gender of Manager

Type of feedback has different effects on ratings of skill

depending on whether manager is female or male
 Same data in each graph
Gender of Manager Type of Feedback
Female Male Positive Negative

10 10
Rating of Manager

Rating of Manager
9 9

8 8

7 7

6 6
Positive Negative Female Male
Type of Feedback Gender of Manager
 Interaction
If we ran study with only positive vs negative feedback:
• managers' skill rated higher following positive compared to
negative feedback
• does not tell you anything about effect of gender of manager

If we ran study with only female vs male managers:

• null results: skills of female and male managers did not differ
• does not tell you anything about effect of feedback valence

Need a factorial design to assess how two factors influence

each other
Possible outcomes of 2 × 2 factorial
Factors A and B
• 2 main effects
Ø A is significant and B is significant

• 1 main effect
Ø A is significant and B is not, or B is significant and A is not

• no main effects
Ø neither A nor B is significant

• each of the above can occur with or without a

significant interaction
 Rule of thumb
In the absence of a statistical test:
• determine main effects from table of means

• determine interaction from plot

 Numbers same
Outcome 1 as in Figure
11.4, page 216

Row
1 2
Means

1 5 5 5.0
A
2 5 5 5.0

Col.
5.0 5.0
Means

Main effect of A: No
Main effect of B: No
Interaction (A × B): No
 Outcome 8

Row
1 2
Means

1 1 9 5.0
A
2 9 1 5.0

Col.
5.0 5.0
Means

Main effect of A: No
Main effect of B: No
Interaction (A × B): Yes
 Outcome 2

Row
1 2
Means

1 1 1 1.0
A
2 9 9 9.0

Col.
5.0 5.0
Means

Main effect of A: Yes

Main effect of B: No
Interaction (A × B): No
 Outcome 6

Row
1 2
Means

1 1 5 3.0
A
2 9 5 7.0

Col.
5.0 5.0
Means

Main effect of A: Yes

Main effect of B: No
Interaction (A × B): Yes
 Outcome 3

Row
1 2
Means

1 9 1 5.0
A
2 9 1 5.0

Col.
9.0 1.0
Means

Main effect of A: No
Main effect of B: Yes
Interaction (A × B): No
 Outcome 7

Row
1 2
Means

1 5 5 5.0
A
2 9 1 5.0

Col.
7.0 3.0
Means

Main effect of A: No
Main effect of B: Yes
Interaction (A × B): Yes
 Outcome 4

Row
1 2
Means

1 5 1 3.0
A
2 9 5 7.0

Col.
7.0 3.0
Means

Main effect of A: Yes

Main effect of B: Yes
Interaction (A × B): No
 Outcome 5

Row
1 2
Means

1 1 1 1.0
A
2 9 1 5.0

Col.
5.0 1.0
Means

Main effect of A: Yes

Main effect of B: Yes
Interaction (A × B): Yes
Parallel lines: no interactions
Lines not parallel: interactions
Example 3: flower growth (2 × 2)

Factor 1: amount of sunlight

• low vs normal

Factor 2: amount of water

• low vs normal

Dependent variable: number of flowers

Example 3: flower growth (2 × 2)

Two potential main effects

• sunlight
• water

One potential interaction

• sunlight × water
Example 3: flower growth (2 × 2)
Number of Flowers
Sunlight

Low Normal Row Means

Low 22 26 24.0
Water
Normal 11 37 24.0

Col. Means 16.5 31.5

Main effect of sunlight

No main effect of water
 Example 3: flower growth (2 × 2)
Collapsed over Water Collapsed over Sunlight

40 40
Number of Flowers

Number of Flowers
30 30

20 20

10 10

0 0
Low Sunlight Normal Sunlight Low Water Normal Water

Main effect of sunlight

No main effect of water
 Example 3: flower growth (2 × 2)
Same data/info in each graph
Low Sunlight
Low Water Normal Water
Normal Sunlight
40 40
Number of Flowers

Number of Flowers
30 30

20 20

10 10

0 0
Low Sunlight Normal Sunlight Low Water Normal Water

Interaction: Water has different effects at different levels of sunlight,

and sunlight has different effects at different levels of water
Example 4: subliminal self-help tapes (2 × 2)

Factor 1: tape label

• memory vs self-esteem

Factor 2: tape content

• memory vs self-esteem

Dependent variable: perceived memory

improvement

Greenwald et al. (1991)

Example 4: subliminal self-help tapes (2 × 2)
Perceived Memory Improvement
Tape Label

Memory Self Esteem Row Means

Memory 0.45 0.13 0.29

Tape
Content Self Esteem
0.47 0.20 0.34

Col. Means 0.46 0.17

Main effect of tape label

No main effect of tape content
 Example 4: subliminal self-help tapes (2 × 2)

Collapsed over Tape Label Collapsed over Tape Content

0.6 0.6
0.5 0.5
Perceived Memory

Perceived Memory
Improvement

Improvement
0.4 0.4
0.3 0.3
0.2 0.2
0.1 0.1
0 0
Memory Content Self Esteem Content Memory Label Self Esteem Label

No main effect of tape content

Main effect of tape label
 Example 4: subliminal self-help tapes (2 × 2)
Same data/info in each graph
Memory Content Memory Label
Self Esteem Content Self Esteem Label
0.6 0.6
0.5 0.5
Perceived Memory

Perceived Memory
Improvement

Improvement
0.4 0.4
0.3 0.3
0.2 0.2
0.1 0.1
0 0
Memory Label Self Esteem Label Memory Content Self Esteem Content

No interaction: label has same effect

at different levels of content
Example 5: social snacking (2 × 2)
Cover story: experiences viewing movies

Factor 1: confederate size

• thin (105 lbs/48 kg, BMI = 20) vs obese (180 lbs/82 kg,
BMI = 33)

Factor 2: confederate consumption

• 2 vs. 30 M&Ms

Dependent variable: # of M&Ms subject ate

McFerran et al. (2010)
Example 5: social snacking (2 × 2)
Example 5: social snacking (2 × 2)
Number of M&Ms eaten by subject
Size

Thin Obese Row Means

2 9.82 6.25 8.12

Amount
30 3.20 4.26 3.72

Col. Means 6.51 5.26

Main effect of amount

Main effect of size
 Example 5: social snacking (2 × 2)
Collapsed over Size Collapsed over Amount

10 10

8 8
# Eaten by Subject

# Eaten by Subject
6 6

4 4

2 2

0 0
2 30 Thin Obese

Main effect of amount confederate ate

Main effect of weight of confederate
 Example 5: social snacking (2 × 2)
Same data/info in each graph
2 30 Thin Obese

10 10

8 8
# Eaten by Subject

# Eaten by Subject
6 6

4 4

2 2

0 0
Thin Obese 2 30

Interaction: amount consumed by confederate has different effect when

confederate is thin vs obese, and confederate’s size has different effects
depending on amount of food consumed by confederate
Example 6: factors affecting eating (2 × 2)

Cover story: subjects participate in a taste test

study in which the server is a confederate

Factor 1: confederate size (within-subjects)

• thin vs obese

Factor 2: subject status (between-subjects)

• dieting vs not dieting

Dependent variable: amount of food eaten

McFerran et al. (2010b)
Example 6: social snacking (2 × 2)
Example 6: factors affecting eating (2 × 2)
Amount of food eaten
Size of server

Thin Obese Row Means

Subject’s Not 7.4 5 6.2

dieting
Status Dieting 5 7.4 6.2

Col. Means 6.2 6.2

No main effect of dieting status

No main effect of size
 Example 6: factors affecting eating (2 × 2)

Collapsed over Size Collapsed over Status

10 10

8 8
Amount Eaten

Amount Eaten
6 6

4 4

2 2

0 0
Not Dieting Dieting Thin Obese

No main effect of dieting status

No main effect of size
 Example 6: factors affecting eating (2 × 2)
Same data/info in each graph
Thin Obese
Not Dieting Dieting
10 10

8 8
Amount Eaten

Amount Eaten
6 6

4 4

2 2

0 0
Thin Obese Not Dieting Dieting

Interaction: dieters ate more snacks when the server was

obese, but non-dieters ate more when the server was thin
 Factorial designs can be:
• completely between-subjects, with each subject in
only one condition

• completely within-subjects (i.e., repeated measures),

with each subject experiencing all conditions

• mixed, with one factor manipulated between subjects

(e.g., gender) and another factor manipulated within
subjects (e.g., dieting status)
 Advantages of factorial designs

1) more efficient because they consider

multiple main effects at the same time

2) more comprehensive because they

consider how one variable affects another

3) more external validity

 Practical issues
Although it is possible to have many factors, each
with a large number of levels, it is impractical

Might see:
• 2×2×2
• 2×3

Gets difficult to think about

• 2×2×2×2
• 3×3
 3 × 3 factorial design
2 factors with 3 levels in each
Factor 1: sunlight
• low, medium, high

Factor 2: water
• low, medium, high

9 total conditions
• Low S, Low W • Med S, Low W • High S, Low W
• Low S, Med W • Med S, Med W • High S, Med W
• Low S, High W • Med S, High W • High S, High W
 3 × 3 factorial design
Two potential main effects
• sunlight
• water

One potential interaction

• sunlight × water
 3 × 3 factorial design
Number of Flowers

Sunlight

Row
Low Med High
Means

Low 22 26 21 23.0

Water Med 11 37 24 24.0

High 15 30 21 22.0

Col. Means 16.0 31.0 22.0

Main effect of sunlight

No main effect of water
 3 × 3 factorial design
Collapsed over Water Collapsed over Sunlight

40 40
Number of Flowers

Number of Flowers
30 30

20 20

10 10

0 0
Low Sun Med Sun High Sun Low Water Med Water High Water

Main effect of sunlight

No main effect of water
 3 × 3 factorial design
Low Water Low Sun
Med Water Med Sun
High Water High Sun
40 40

Number of Flowers
Number of Flowers

30 30

20 20

10 10

0 0
Low Sun Med Sun High Sun Low Med High
Water Water Water

Interaction: sunlight has different effects at different levels of water

 2 × 2 × 2 factorial design
Factor 1: gender of manager
• female vs male

Factor 2: gender of subjects

• female vs male

Factor 3: type of feedback

• positive vs negative
 2 × 2 × 2 factorial design
Three potential main effects:
• gender of manager
• gender of subject
• type of feedback

Four possible interactions:

• gender of manager × gender of subject
• gender of manager × feedback
• gender of subject × feedback
• gender of manager × gender of subject × feedback
 Problem
As the factorial design gets more complex,
the results also get more complex

It may not be possible to understand the

outcome
• consider an interaction among 4 factors
• question whether such a design is appropriate