Scribd
Upload
45 views26 pages

Check Your Calendar : If You Are Qualified, Fill Out Make-Up Exam Form by 5pm This Friday, Attach Necessary Documents

This lecture discusses interpreting the results of ANOVA and performing multiple comparisons to determine which group means are statistically different. It reviews checking assumptions, conducting visual checks using boxplots and scatterplots of means, and using the Tukey and Dunnett's methods to make pairwise comparisons between groups while controlling for Type I error. Examples using hypothetical data and the cereal design dataset in SAS are provided.

Uploaded by

AZ Ndingwan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
45 views26 pages

Check Your Calendar : If You Are Qualified, Fill Out Make-Up Exam Form by 5pm This Friday, Attach Necessary Documents

This lecture discusses interpreting the results of ANOVA and performing multiple comparisons to determine which group means are statistically different. It reviews checking assumptions, conducting visual checks using boxplots and scatterplots of means, and using the Tukey and Dunnett's methods to make pairwise comparisons between groups while controlling for Type I error. Examples using hypothetical data and the cereal design dataset in SAS are provided.

Uploaded by

AZ Ndingwan
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 26

Check your calendar…

If you are qualified, Fill out make-up


exam form by 5pm this Friday, attach
necessary documents.

3/23/12 Lecture 20 1
Interpreting the ANOVA
results
Chapter 9

3/23/12 Lecture 20 2
Assumptions (prior to ANOVA)
•  Two important assumptions for ANOVA
1.  Constant variance: The variances of the k
populations are the same.
–  Check this with the ratio of the largest and smallest
standard deviations, the ratio must be < 2

2.  Each of the k populations follows a normal


distribution.
–  Check this by looking at QQplots for each group

3/23/12 Lecture 19 3
Quick sidenote

•  ANOVA is a good example of a situation where often a


nonsignificant test is actually useful.
–  Suppose we are comparing a new drug to several standard
drugs already used
–  Suppose also that the new drug is less expensive to produce
–  In this case, mostly what we’d like to show is that the new
drug is at least effective as the other standard drugs used
–  So in this situation, a non-significant ANOVA is a great result!
•  Remember:
statistical significance ≠ practical significance
3/23/12 Lecture 19 4
9.3
Interpreting ANOVA results
•  If the results are significant in ANOVA, we’d
like to know explicitly which means are
different

•  Remark: If insignificant in ANOVA, we don’t


have to try further steps…

•  Two benefits of ANOVA


1.  Single test with single chance (α) of type I error
2.  Better estimation of error among all groups
•  By comparing all the groups simultaneously, we get a
better picture of the overall error among groups.
3/23/12 Lecture 20 5
How do we know which means are
different?
•  We need some "Supplementary Analysis" to tell that.
–  One way is to do a check visually, using the "effect plots"
•  Scatter plot of means
•  Side-By-Side Boxplots

–  Another way is to perform multiple comparison of means


•  Tukey's method
•  Dunnett's method
•  many more: LSD, Scheffe, Bonferroni, FDR, etc.

Actually, We can compare the means pairwise and keep the two
benefits of ANOVA, we do this by adjusting the T value we use to
compare the two means.

3/23/12 Lecture 20 6
Visual Check - Boxplots

3/23/12 Lecture 20 7
Visual Check – Scatterplot of
Means

3/23/12 Lecture 20 8
Multiple comparisons (General Concepts)

•  How far apart do two means need to be to


be statistically significant?
–  This value can be calculated directly similar to
what we did with confidence intervals
•  It functions like a t from a t-test
–  Generally, the critical value (t for example) is
modified to “correct” the inflated type I error
rate to keep it at the desired α level (like 0.05)
•  So instead of an α error rate for each test, we get an
“family” α error rate—one rate for the entire
comparison
3/23/12 Lecture 20 9
Tukey’s Method
•  Controls the type I error rate directly by modifying
the T value
MSE 1 1 MSE if n = n
T = qα ( + ) or T = qα i j
2 ni n j ni
•  qα comes from the studentized range tables
–  Table IX on pages 577-578
–  Df of qα is (k, n – k) for single-factor ANOVA

•  If the difference in two means is greater than this


critical value, we say those two means are
statistically significantly different

3/23/12 Lecture 20 10
More simply

•  Calculate T, calculate the differences xi − x j

•  If xi − x j < T, means are not significantly different

•  If xi − x j > T, means ARE significantly different

•  Remark: Tukey’s method is conservative,


sometimes its conclusion will be inconsistent with
that by using ANOVA test results.
If we took a less stringent alpha level we might see
some of the significant differences.
3/23/12 Lecture 20 11
Example—generic
•  Let’s say we are comparing 4 means with equal sample
sizes of ni = 5 for all i. With an MSE of 10.

•  Looking at Table IX, we have k = 4, and Error df = n – k


= 16
–  qα = 4.05

MSE 10
•  So, T = qα = 4.05 = 5.73
ni 5

•  Any difference of means more than 5.73 apart would be


significant different
3/23/12 Lecture 20 12
Example—generic

•  Suppose that you have four treatment groups


and the treatment means are:
• TRT 1: 52
• TRT 2: 63
• TRT 3: 58
• TRT 4: 54
•  Which pairs are significantly different?

3/23/12 Lecture 20 13
Example (from Monday’s Class)
•  For the cereal example, let’s use Tukey’s method using
α = 0.01

•  The means are (arranged in descending order):

n4 = 5, x4 = 27.2
n3 = 4, x3 = 19.5
n1 = 5, x1 = 14.6
n2 = 5, x2 = 13.4
•  Note, group 3’s sample size, what effect will that have on
the comparisons?

3/23/12 Lecture 20 14
Another Method: Using SAS code

proc glm data=cereal alpha=0.01;


class design;
model cases = design;
means design / tukey cldiff;
run;

3/23/12 Lecture 20 15
Example (cont) using SAS
•  Notice it stars the pairs
that are significantly
different.

•  So the only pairs that are


significantly different are:
1 and 4
2 and 4

3/23/12 Lecture 20 16
Alternate SAS code

proc glm data=cereal alpha=0.01;


class design;
model cases = design;
means design / tukey lines;
run;

3/23/12 Lecture 20 17
Example (cont) using SAS
•  Same information as before,
differences are:
1 and 4
2 and 4
•  Notice the nice “groupings”
though

3/23/12 Lecture 20 18
Dunnett’s Multiple
Comparison

3/23/12 Lecture 20 19
3/23/12 Lecture 20 20
Multiple Comparison – Dunnett’s
Method

3/23/12 Lecture 20 21
Dunnett’s Method
•  Functions like a Tukey, just uses a different T

1 1
T = tα (k − 1, n − k ) MSE ( + )
ni nC
•  tα comes from the Dunnett’s t table
–  Table X on page 579
–  Only use when one of the groups is a control group
–  Only interested in comparing the “other” groups to
the control group
•  Again we take pairwise differences, xi − xC
3/23/12 Lecture 20 22
Example (Cereal Design) using SAS
•  Dunnett’s SAS code, pretend design 1 was the
regular design already used

proc glm data=cereal alpha=0.01;


class design;
model cases = design;
means design / dunnett(“1”) cldiff;
run;

Note: lines doesn’t work with Dunnett’s


3/23/12 Lecture 20 23
Example (cont) using SAS
Dunnett's t Tests for cases
•  So if group1 was the
NOTE: This test controls the Type I experimentwise error for
comparisons of all treatments against a control. control, only group
4 is significantly
Alpha 0.01
Error Degrees of Freedom 15 different
Error Mean Square 10.54667
Critical Value of Dunnett's t 3.43026

Comparisons significant at the 0.01 level are indicated by ***.

Difference
design Between Simultaneous 99%
Comparison Means Confidence Limits

4-1 12.600 5.554 19.646 ***


3-1 4.900 -2.573 12.373
2-1 -1.200 -8.246 5.846

3/23/12 Lecture 20 24
Summary about ANOVA and
Multiple Comparison
•  ANOVA (Analysis of Variances)
–  Check the assumptions (constant variance/normality)
–  Be able to do most of ANOVA by hand or by SAS both
•  Lots of hand calculations
•  Be able to read and interpret SAS output
–  For Hw, do it either way you like, but for the exam be prepared to
do both!

•  Multiple Comparison methods (ONLY when ANOVA result is


significant)
–  are useful in other situations, but they all involve calculating a T
value and using it to compare pairs of means
–  Tukey’s is approprirate if there’s no control group; try Dunnett’s
if there is any control(s)

3/23/12 Lecture 20 25
After Class
•  Hw#8, due by 5pm next Monday

•  Start review of Exam 2 (Ch.7, 8 and


Monday’s complete notes)
–  Practice Test 2
–  Hw5-8, Lab 3 and 4

3/23/12 Lecture 20 26

You might also like