Scribd
Upload
4 views

Module03 Anova

The document discusses ANOVA (Analysis of Variance) and its application in statistical analysis, particularly in determining if different settings significantly affect outcomes, such as distance traveled in various settings. It covers key concepts including the Central Limit Theorem, important sampling distributions, assumptions for ANOVA, and methods for hypothesis testing like Tukey’s test and Least Significant Difference. Additionally, it emphasizes the importance of experimental design and the distinction between fixed and random effects models.

Uploaded by

A216Kasish Agarwal
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
4 views

Module03 Anova

The document discusses ANOVA (Analysis of Variance) and its application in statistical analysis, particularly in determining if different settings significantly affect outcomes, such as distance traveled in various settings. It covers key concepts including the Central Limit Theorem, important sampling distributions, assumptions for ANOVA, and methods for hypothesis testing like Tukey’s test and Least Significant Difference. Additionally, it emphasizes the importance of experimental design and the distinction between fixed and random effects models.

Uploaded by

A216Kasish Agarwal
Copyright
© © All Rights Reserved
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 28

ANOVA

Prof. S. Roychowdhury
The Statapult
Example: More than 2 Levels

• More than 2 samples


• Distance travelled in setting 1 (inches)
11,13,12,10,11
• Distance travelled in setting 2 (inches)
17,14,13,15,15
• Distance travelled in setting 3 (inches)
19,17,21,23,18
Question: Does setting significantly
affect travelling distance?
Plasma Etching
Plasma Etching

Etch Rate
Power (W) 1 2 3 4 5
160 575 542 530 539 570
180 565 593 590 579 610
200 600 651 610 637 629
220 725 700 715 685 710
1 Factor: More than 2 Levels

• Concept of ANOVA
• ANOVA Table
• Formulas
• Conclusion
Central Limit Theorem (CLT)
• Definition: If 𝑥1 , … , 𝑥𝑛 are independent random variables with mean 𝜇𝑖 and variance 𝜎𝑖2 ,
𝑦−σ𝑛𝑖=1 𝜇𝑖
and if 𝑦 = 𝑥1 + ⋯ + 𝑥𝑛 , then the distribution of approaches the 𝑁 0,1
σ𝑛 2
𝑖=1 𝜎𝑖
distribution as 𝑛 approaches infinity.
(Montgomery D.C., Introduction to Statistical Quality Control)

• It implies that the sum of 𝑛 independently distributed random variables is approximately


normal, regardless of the distribution of the individual variables.

• If 𝑥𝑖 are independent and identically distributed (IID), and distribution of each 𝑥𝑖 does not
depart radically from normal distribution, then CLT works quite well for 𝑛 ≥ 3 𝑜𝑟 4.
(common in SQC problems)
Important Sampling Distributions Derived
from Normal Distribution
1. 𝜒 2 distribution: If 𝑥1 , . . 𝑥𝑛 are standard normally and
independently distributed then 𝑦 = 𝑥12 + 𝑥22 … + 𝑥𝑛2
follow chi-squared distribution with 𝑛 degrees of
freedom.
2. 𝑡-distribution: If 𝑥 is standard normal variable and 𝑦 is chi-
squared random variable with 𝑘 degrees of freedom, and𝑥 if
𝑥 and 𝑦 are independent then the random variable 𝑡 = 𝑦 is
𝑘
distributed as 𝑡 with 𝑘 degrees of freedom.
3. If 𝑤 and 𝑦 are two independent random chi-sq distributed
variables with 𝑢 and 𝑣 degrees of freedom, then the ratio
𝑤
𝑢
𝐹= 𝑦 follows F distribution with (𝑢, 𝑣) degrees of freedom
𝑣
ANOVA
Minitab: Stat-> ANOVA-> One way ANOVA,
Graphs: Select Boxplot, Normal Probability Plot of Residuals

Example 3: Null hypothesis is rejected at 𝛼 = 0.05. The average distance


travelled differs significantly with the settings.
ANOVA How do I
explain
(model) the
variation ?
ANOVA
Means Model
𝒚𝒊𝒋 = 𝝁𝒊 + 𝝐𝒊𝒋
𝑖 = 1, . . 𝑎 levels
𝑗 = 1, . . 𝑛 observations in each level
Null hypothesis 𝐻0 : 𝜇1 = 𝜇2 … . = 𝜇𝑎
𝐻1 : 𝜇𝑖 ≠ 𝜇𝑗 for at least one pair of 𝑖, 𝑗

Effects Model
𝒚𝒊𝒋 = 𝝁 + 𝝉𝒊 + 𝝐𝒊𝒋
𝑖 = 1, . . 𝑎 levels
𝑗 = 1, . . 𝑛 observations in each level
Null hypothesis 𝐻0 : 𝜏1 = 𝜏2 … . = 𝜏𝑎 = 0
𝐻1 : 𝜏𝑖 ≠ 0 for at least one 𝑖
ANOVA
Grand
Mean Random Error
Component 𝑁(0, 𝜎 2 )
Linear Statistical Model
𝒚𝒊𝒋 = 𝝁 + 𝝉𝒊 + 𝝐𝒊𝒋
𝑖 = 1, . . 𝑎 levels
Treatment Effect
𝑗 = 1, . . 𝑛 observations in each level
(Effect due to level i)
ANOVA Calculations

𝑎 𝑛 𝑎 𝑎 𝑛
2 2 2
෍ ෍ 𝑦𝑖𝑗 − 𝑦.
ത . = 𝑛 ෍ 𝑦ത𝑖 . −𝑦ത.. + ෍ ෍ 𝑦𝑖𝑗 − 𝑦ത𝑖 .
𝑖=1 𝑗=1 𝑖=1 𝑖=1 𝑗=1

𝑆𝑆𝑇 = 𝑆𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡 + 𝑆𝑆𝐸


Assumption: Each of the 𝑎 populations is coming from Normal distribution.
If 𝐻0 is true then 𝐹0 (see ANOVA Table) follows 𝐹 distribution with 𝑎 − 1
and 𝑎 𝑛 − 1 degrees of freedom.
Procedure: Reject 𝐻0 if 𝐹0 > 𝐹𝛼,𝑎−1,𝑎(𝑛−1) or P-value <0.05 , meaning differences
among the average responses at different levels are significant.
Assumptions for ANOVA
• The model error is assumed to be independent and
normally distributed random variables, with mean 0 and
variance 𝜎 2
• 𝑦𝑖𝑗 ~𝑁(𝜇 + 𝜏𝑖 , 𝜎 2 )
• 𝜖𝑖𝑗 ~𝑁(0, 𝜎 2 )
• Observations are mutually independent
• The variance 𝜎 2 is assumed to be same for all levels of the
factor.
Statistical Analysis
• 𝑆𝑆𝑇 is the sum of square of normally distributed random
variables.
𝑆𝑆𝑇
• It can be shown 2 ~𝜒 2 (𝑁 − 1)
𝜎
𝑆𝑆𝐸
• It can be shown 2 ~𝜒 2 (𝑁 − 𝑎) and
𝜎
𝑆𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑠 2
• ~𝜒 (𝑎 − 1) if 𝐻0 : 𝜏𝑖 = 0 ∀𝑖 is true
𝜎2
• But all three sum of squares are not independent.
Cochran’s Theorem
• Let 𝑍𝑖 be 𝑁𝐼𝐷(0,1) for 𝑖 = 1,2, … 𝜈 and
σ𝜈𝑖=1 𝑍𝑖2 = 𝑄1 + 𝑄2 + ⋯ + 𝑄𝑠
Where 𝑠 ≤ 𝜈, 𝑄𝑖 has 𝜈𝑖 degrees of freedom.
Then 𝑄1 , . . 𝑄𝑠 are independent chi-square random variables
with 𝜈1 , … , 𝜈𝑠 degrees of freedom respectively, if and only if

𝜈 = 𝜈1 + ⋯ + 𝜈𝑠
Cochran’s Theorem
𝑆𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑠 𝑆𝑆𝐸
Cochran’s theorem implies that and are
𝜎2 𝜎2
independently distributed chi-square random variables.

So if 𝐻0 of no difference in treatment is true the ratio


𝑆𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑠 /(𝑎−1)
𝐹0 = = 𝑀𝑆𝑇𝑟𝑒𝑎𝑡𝑚𝑒𝑛𝑡𝑠 /𝑀𝑆𝐸 is distributed
𝑆𝑆𝐸 /(𝑁−𝑎)
as 𝐹 with (𝑎 − 1) and (𝑁 − 𝑎) dofs.
ANOVA Calculations
ANOVA F-crit

𝑭𝟎.𝟎𝟓,𝟐,𝟏𝟐
Key Considerations for ANOVA
• Experiments have to be performed in random order so
that the environment in which the treatments are applied
is as uniform as possible. The experimental design should
be completely randomized design.
• When “𝑎”no. treatment levels are specifically chosen by
the experimenter, the conclusions cannot be extended for
similar treatments that were not considered. This is called
Fixed Effects Model
• When "𝑎“ no. of treatment levels are chosen randomly
out of a larger population of treatments, 𝜏𝑖 (treatment
effects) are random variables and we try to estimate the
variability in 𝜏𝑖 . This is called Random Effects Model.
Key Considerations for ANOVA
• In Fixed Effects Model we test hypotheses about the
treatment means.
• The conclusions for FEM are only applied to the factor
levels considered.
• The model parameters (𝜇, 𝜏𝑖 , 𝜎 2 ) can be estimated from
FEM.
• In Random Effects Model, conclusions can be exteneded
to all treatments.
• In REM, 𝜏𝑖 s are random variables, and the variability of
the 𝜏𝑖 s are estimated.
Which Factors are Significant

no
F-test finds
Terminate
Significance?

Perform Pair-wise
“LSD” Comparisons Terminate
Or Tukey’s test
Least Significant Difference
1 1
▪ LSD = 𝑡1−𝛼 , 𝑛𝑜. 𝑜𝑓 𝑑𝑜𝑓𝑠 𝑜𝑓 𝑟𝑒𝑠𝑖𝑑𝑢𝑎𝑙𝑠 * 𝑀𝑆𝐸 ( + )
2 𝑛1 𝑛2

▪ Check if |𝑦ത𝑖 − 𝑦ത𝑗 | > 𝐿𝑆𝐷 for all 𝑖, 𝑗 ∈ 1 … 𝑎, 𝑖 ≠ 𝑗


▪ You can make plots to show which
1

2
3
levels are different

Final conclusion: Setting 3 provides significantly more


travel, than setting 1 and 2. There is no significant difference
among the average distances travelled as a result of settings 1
and 2.
Tukey’s Test
• When null hypothesis is rejected, all pairwise means to be
compared 𝐻0 : 𝜇𝑖 = 𝜇𝑗 ; 𝐻1 : 𝜇𝑖 ≠ 𝜇𝑗
• Tukey’s procedure has significance level 𝛼 when sample sizes
are equal and atmost 𝛼 when sample sizes are not equal. It
keeps family error rate at selected level 𝛼.
• Tukey’s procedure uses Studentized Range Distribution
ഥ
y𝑚𝑎𝑥 −ഥ
ymin
• Studentized Range Statistic q =
𝑀𝑆𝐸
𝑛
𝑀𝑆𝐸
• If |𝑦ത𝑖 . −𝑦ത𝑗. | > 𝑇𝛼 = 𝑞 1−𝛼 (𝑎, 𝑓)
𝑛
• where 𝑎 = number of treatment levels and 𝑓 is dof associated
with MSE, then two means are significantly different
Confidence Intervals
• 100 1 − 𝛼 confidence intervals for all pairs of means may be
constructed
𝑀𝑆𝐸
𝑦ത𝑖 . −𝑦ത𝑗 . −𝑞1−𝛼 𝑎, 𝑓 ≤ 𝜇𝑖 − 𝜇𝑗
𝑛
𝑀𝑆𝐸
≤ 𝑦ത𝑖 . −𝑦ത𝑗 . + 𝑞1−𝛼 𝑎, 𝑓
𝑛
For unequal sample sizes:
Check if the absolute differences of sample means is greater
than
𝑞1−𝛼 𝑎,𝑓 1 1
𝑇𝛼 = 𝑀𝑆𝐸 +
2 𝑛𝑖 𝑛𝑗
Estimating Model Parameters
• 𝑦𝑖𝑗 = 𝜇 + 𝜏𝑖 + 𝜖𝑖𝑗 For 𝑖 = 1, . . 𝑎 ; 𝑗 = 1, … 𝑛
• The model parameters can be estimated as below:
• 𝜇ො = 𝑦.ത.
• 𝜏Ƹ 𝑖 = 𝑦ത𝑖 . −𝑦.
ത . For 𝑖 = 1, . . 𝑎
• 𝑖 𝑡ℎ treatment mean: 𝜇ො𝑖 = 𝑦ത𝑖 .
Confidence interal of 𝑖 𝑡ℎ treatment mean:
𝑀𝑆𝐸 𝑀𝑆𝐸
𝑦ത𝑖 . − 𝑡1−𝛼,𝑁−𝑎 ≤ 𝜇𝑖 ≤ 𝑦ത𝑖 . + 𝑡1−𝛼,𝑁−𝑎
2 𝑛 2 𝑛
Model Adequacy Checking
• To check the assumption of normality of the error term
find the residuals: 𝑒𝑖𝑗 = 𝑦𝑖𝑗 − 𝑦ො𝑖𝑗 = 𝑦𝑖𝑗 − 𝑦ത𝑖 .
• The residuals should be structureless.
• Normal probability plot of residuals is an effective way to
check the assumption of normality
• Check outliers with standardized residuals
𝑒𝑖𝑗
𝑑𝑖𝑗 =
𝑀𝑆𝐸
68% of 𝑑𝑖𝑗 should be within ±1, 95% within ±2 and almost
all (99.73%) should be within ±3
Variance Stabilizing Transformation
Observations Transformation
′
Poisson Distribution Square root: 𝑦𝑖𝑗 = 𝑦𝑖𝑗 or
′
𝑦𝑖𝑗 = 1 + 𝑦𝑖𝑗
′
Lognormal Distribution Logarithmic: 𝑦𝑖𝑗 = log 𝑦𝑖𝑗
Binomial Distribution ′
Arcsin: 𝑦𝑖𝑗 = arcsin√(𝑦𝑖𝑗 )

You might also like