Scribd
Upload
11 views16 pages

Hypothesis Test Errors

The document discusses hypothesis testing, including Type I and Type II errors, and methods to calculate them using R programming. It also explains the Bonferroni correction for multiple tests, and provides an overview of ANOVA and its types, including one-way and two-way ANOVA, as well as the Kruskal-Wallis test for non-parametric data. The document includes examples of statistical tests and their interpretations in the context of agricultural data analysis.

Uploaded by

Chaya Anu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
11 views16 pages

Hypothesis Test Errors

The document discusses hypothesis testing, including Type I and Type II errors, and methods to calculate them using R programming. It also explains the Bonferroni correction for multiple tests, and provides an overview of ANOVA and its types, including one-way and two-way ANOVA, as well as the Kruskal-Wallis test for non-parametric data. The document includes examples of statistical tests and their interpretations in the context of agricultural data analysis.

Uploaded by

Chaya Anu
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PPTX, PDF, TXT or read online on Scribd
You are on page 1/ 16

Hypothesis Test Errors

Hypothesis Test Errors


• Hypothesis testing is performed with the objective of obtaining a p-value in
order to quantify evidence against the null statement H0.
• This is rejected in order to quantify evidence against the null statement H0.
• This is rejected in favor of the alternative, HA, if the p-value is itself less
than a predefined significance level α, which is conventionally 0.05 or 0.01.
• As touched upon, this approach is justifiably criticized since the choice of α
is essentially arbitrary;
• A decision to reject or retain H0 can change depending solely upon the α
value
Type I Error
• A Type I error, also known as a false positive or alpha error, occurs when a
null hypothesis is rejected when it is actually true.
• In statistical hypothesis testing, the null hypothesis (H0) often represents a
baseline assumption, such as no effect or no difference between groups.
• A Type I error is the probability of incorrectly rejecting H0 when it is true,
and this probability is represented by the significance level (alpha).
• In R programming, you can calculate the Type I error rate (also known as
the alpha level or false positive rate) by simulating data and comparing the
proportion of false positives to the total number of tests conducted.
# Set the parameters alpha <- 0.05
sample_size <- 30
num_simulations <- 10000
# Set the seed for reproducibility
set.seed(123)
# Initialize the counter for false positives
false_positives <- 0
# Perform the simulations
for (i in 1:num_simulations)
{
# Generate two samples from the same normal
# distribution (null hypothesis is true)
sample1 <- rnorm(sample_size, mean = 0, sd = 1)
sample2 <- rnorm(sample_size, mean = 0, sd = 1)
# Conduct a t-test test_result <- t.test(sample1, sample2)
# Check if the p-value is less than the alpha level
if (test_result$p.value < alpha) { false_positives <- false_positives + 1 } }
# Calculate the Type I error rate
type1_error_rate <- false_positives / num_simulations
# Print the Type I error rate cat("Type I Error Rate:", type1_error_rate)
output
# Print the Type I error
rate > cat("Type I Error Rate:",
type1_error_rate)
Type I Error Rate: 0.0481
• In this example, we run 10,000 simulations
where we draw two samples from the same
normal distribution,
and conduct a t-test for each pair of samples.

• We count the number of times we reject the null


hypothesis when it is true (false positives) and
divide it by the total number of simulations to
Bonferroni Correction
• When several hypothesis tests are conducted, you can curb the
multiple testing problem with respect to committing a Type I error by
using the Bonferroni correction.
• The Bonferroni correction suggests that when performing a total of N
independent hypothesis tests, each under a significance level of α,
you should instead use αB = α/N for any interpretation of statistical
significance.
• The Bonferroni and other corrective measures were developed in an
attempt to formalize remedies to making a Type I error in multiple
tests.
Type II error
• Type II error, also known as a false negative, occurs when you fail to
reject the null hypothesis when it’s actually false.
• In hypothesis testing, this error is denoted as β (beta).
• To calculate Type II error in R, you need to know the effect size
(difference between the null and alternative hypotheses), sample size,
standard deviation, and the desired significance level (alpha).
effect_size <- 0.5
# The difference between null and alternative
hypotheses
sample_size <- 100
# The number of observations in each group
sd <- 15
# The standard deviation
alpha <- 0.05
# The significance level
# Calculate Type II Error
pwr_result <- pwr.t.test( n = sample_size, d =
effect_size / sd, sig.level = alpha, type =
"two.sample", alternative = "two.sided" )
type_II_error <- 1 - pwr_result$power
# Print Type II Error
print(type_II_error)
In this example, we are using the pwr package to calculate the power
of the test,
output
and then subtracting it from 1 to obtain the Type II error (β).
Remember to adapt the parameters according to your specific problem.
Output:
# Print Type II Error
> print(type_II_error)
[1] 0.9436737
Analysis of Variance
• ANOVA is a statistical Test for estimating how a quantitative dependent
variable changes according to the levels of one or more
categorical independent variables
• ANOVA tests whether there is a difference in means of the groups at
each level of the independent variable.
• The null hypothesis (H0) of the ANOVA is no difference in means, and
the alternative hypothesis (Ha) is that the means are different from one
another.
Types of ANOVA test
• One-way ANOVA: One-way When there is a single categorical independent
variable (also known as a factor) and a single continuous dependent
variable, an ANOVA is employed.
• It seeks to ascertain whether there are any notable variations in the
dependent variable’s means across the levels of the independent variable.

• Two-way ANOVA: When there are two categorical independent variables


(factors) and one continuous dependent variable, two-way ANOVA is used
as an extension of one-way ANOVA.
• You can evaluate both the direct impacts of each independent variable and
how they interact with one another on the dependent variable.
The Dataset (crop.data)

• Here the ‘density’, ‘block’, and ‘fertilizer’ listed as categorical variables with the
number of observations at each level (i.e. 48 observations at density 1 and 48
observations at density 2).
• ‘Yield’ should be a quantitative variable with a numeric summary (minimum,
median, mean, maximum).
Performing ANOVA
One.way<-aov(yield ~ fertilizer,data=crop.data
Summary(one.way)
The model summary first lists the independent variables being tested in the model (in this case we have only
one, ‘fertilizer’) and the model residuals (‘Residual’).
All of the variation that is not explained by the independent variables is called residual variance.
• The rest of the values in the output table describe the independent variable and the residuals:
• The Df column displays the degrees of freedom for the independent variable (the number of
levels in the variable minus 1), and the degrees of freedom for the residuals (the total number
of observations minus one and minus the number of levels in the independent variables).
• The Sum Sq column displays the sum of squares (also known as the total variation between
the group means and the overall mean).
• The Mean Sq column is the mean of the sum of squares, calculated by dividing the sum of
squares by the degrees of freedom for each parameter.
• The F value column is the test statistic from the F test. This is the mean square of each
independent variable divided by the mean square of the residuals. The larger the F value, the
more likely it is that the variation caused by the independent variable is real and not due to
chance.
• The Pr(>F) column is the p value of the F statistic. This shows how likely it is that the F value
calculated from the test would have occurred if the null hypothesis of no difference among
group means were true.
• The p value of the fertilizer variable is low (p < 0.001), so it appears that the type of fertilizer
used has a real impact on the final crop yield.
Two-way ANOVA
• In the two-way ANOVA example, we are modeling crop yield as a function of
type of fertilizer and planting density.
• First we use aov() to run the model, then we use summary() to print the
summary of the model.
• two.way<-aov(yield ~ fertilizer+density,data=crop.data
• Summary(two.way)
• Adding planting density to the model seems to have made the model better:
• It reduced the residual variance (the residual sum of squares went from 35.89 to 30.765), and both
planting density and fertilizer are statistically significant (p-values < 0.001).
Kruskal-wallis test
• A Kruskal-Wallis test is used to determine whether or not there is a statistically significant
difference between the medians of three or more independent groups.
• This test is the nonparametric equivalent of the one-way ANOVA and is typically used when the
normality assumption is violated.
• The Kruskal-Wallis test does not assume normality in the data and is much less sensitive to
outliers than the one-way ANOVA.
• Here are a couple examples of when you might conduct a Kruskal-Wallis test:
• Example 1: Comparing Study Techinques
• You randomly split up a class of 90 students into three groups of 30. Each group uses a different
studying technique for one month to prepare for an exam.
• At the end of the month, all of the students take the same exam. You want to know whether or
not the studying technique has an impact on exam scores.
• From previous studies you know that the distributions of exam scores for these three studying
techniques are not normally distributed so you conduct a Kruskal-Wallis test to determine if there
is a statistically significant difference between the median scores of the three groups.

You might also like