Hypothesis Testing

Purpose of Hypothesis Testing

Test of hypothesis is a process of testing of the significance regarding the parameters of the
population on the basis of sample drawn from it.

 The main purpose of hypothesis testing is to help the researcher in reaching a conclusion
regarding the population by examining a sample taken from that population.
 The test only indicates whether the hypothesis is supported or not supported by the
available data.
 Through the hypothesis testing researcher or investigator can determine whether or not
such statements are compatible with the available data.

What is Hypothesis Testing?

Hypothesis is usually considered as the principal instrument in research. The main goal in many
research studies is to check whether the data collected support certain statements or predictions. A
statistical hypothesis is an assertion or conjecture concerning one or more populations. Test of
hypothesis is a process of testing of the significance regarding the parameters of the population on
the basis of sample drawn from it. Thus, it is also termed as “Test of Significance’. In short,
hypothesis testing enables us to make probability statements about population parameter. The
hypothesis may not be proved absolutely, but in practice it is accepted if it has withstood a critical
testing.

Points to be considered while formulating Hypothesis

Hypothesis should be clear and precise.

Hypothesis should be capable of being tested.

Hypothesis should state relationship between variables.

Hypothesis should be limited in scope and must be specific.

Hypothesis should be stated as far as possible in most simple terms so that the same is easily
understandable by all concerned.

Hypothesis should be amenable to testing within a reasonable time.

Hypothesis must explain empirical reference

Types of Statistical Hypotheses

There are two types of statistical hypotheses:

1. Null Hypothesis (H0) – A statistical hypothesis that states that there is no difference between a
parameter and a specific value, or that there is no difference between two parameters.

2. Alternative Hypothesis (H1 or Ha) – A statistical hypothesis that states the existence of a
difference between a parameter and a specific value, or states that there is a difference between
two parameters. Alternative hypothesis is created in a negative meaning of the null hypothesis.
Suppose, we want to test the hypothesis that the population mean (µ) is equal to the hypothesized
mean (µ H0) = 100. Then we would say that the null hypothesis is that the population mean is equal
to the hypothesized mean 100 and symbolically we can express as:

H0: µ = µ H0 = 100

If our sample results do not support this null hypothesis, we should conclude that something else
is true. What we conclude rejecting the null hypothesis is known as alternative hypothesis. In other
words, the set of alternatives to the null hypothesis is referred to as the alternative hypothesis. If
we accept H0, then we are rejecting H1 and if we reject H0, then we are accepting H1. For H0: µ =
µHo =100, we may consider three possible alternative hypotheses as follows:
Type I and Type II Error:

In the process of accepting or rejecting a null hypothesis, we may encounter two types of errors. We
may wrongly reject a true null hypothesis. This leads to an error, which we call type I error. The
second kind of error, called type II error, occurs when we accept a null hypothesis, when it is false,
that is when an alternative is true.
When no error is committed, we arrive at a correct decision. The correct decision may be achieved
in two ways: accepting a true null hypothesis or rejecting a false null hypothesis. Four possible
outcomes with associated types of error that we commit in our decision are shown in the
accompanying table:

Decision H0 is true / H1 is false Ho is false / H1 is true

Reject H0 Type I error Correct decision
P(Type I error) = α P(Correct decision)= 1-β
Accept H0 Correct decision Type II error
P(Correct decision)= 1-α P(Type II error) = β

The Probability of committing a type I error is usually denoted by α and is commonly referred to as
the level of significance of a test:
α = P (type I error) = P (rejecting H0 when H0 is true)
The Probability of committing type II error is usually denoted by β:
β = P (type II error) = P (accepting H0 when H1 is true)
The complement of β, i.e. 1- β is commonly known as the power of a test.
1- β=1-P (type II error) = P (rejecting H0 when H1 is true )

Alpha Value (α)

The number alpha is the threshold value that we measure p-values against. It tells us how extreme
observed results must be in order to reject the null hypothesis of a significance test.

The value of alpha is associated with the confidence level of our test. The following lists some
levels of confidence with their related values of alpha:

 For results with a 90 percent level of confidence, the value of alpha is 1 — 0.90 = 0.10.
 For results with a 95 percent level of confidence, the value of alpha is 1 — 0.95 = 0.05.
 For results with a 99 percent level of confidence, the value of alpha is 1 — 0.99 = 0.01.

And in general, for results with a C percent level of confidence, the value of alpha is 1 — C/100.
P-Values

The other number that is part of a test of significance is a p-value. A p-value is also a probability,
but it comes from a different source than alpha. Every test statistic has a corresponding probability
or p-value. This value is the probability that the observed statistic occurred by chance alone,
assuming that the null hypothesis is true.

 If p-value is less than .01, the results are regarded as highly significant.
 If p-value is between .01 and .05, the results are regarded as statistically significant.
 If p-value is between .05 and .10, the results are regarded as only tending towards
statistical significance.
 If p-value is greater than .10, the results are considered as not significant.

Difference Between P-Value and Alpha

To determine if an observed outcome is statistically significant, we compare the values of alpha

and the p-value. There are two possibilities that emerge:

 The p-value is less than or equal to alpha. In this case, we reject the null hypothesis. When
this happens, we say that the result is statistically significant. In other words, we are
reasonably sure that there is something besides chance alone that gave us an observed
sample.
 The p-value is greater than alpha. In this case, we fail to reject the null hypothesis. When
this happens, we say that the result is not statistically significant. In other words, we are
reasonably sure that our observed data can be explained by chance alone.

Confidence Level

A confidence level is how much uncertainty there is with any particular statistic. Confidence
intervals are often used with a margin of error. It tells you how confident you can be that the results
from a poll or survey reflect what you would expect to find if it were possible to survey the entire
population. Confidence intervals are intrinsically connected to confidence levels.
Confidence Interval & Confidence Level

Confidence levels are expressed as a percentage (for example, a 95% confidence level). It means
that should you repeat an experiment or survey over and over again, 95 percent of the time your
results will match the results you get from a population (in other words, your statistics would be
sound!). Confidence intervals are your results and they are usually numbers. For example, you
survey a group of pet owners to see how many cans of dog food they purchase a year. You test
your statistic at the 99 percent confidence level and get a confidence interval of (200,300). That
means you think they buy between 200 and 300 cans a year. You’re super confident (99% is a very
high level!) that your results are sound, statistically.

Procedure of Hypothesis Testing

When you are evaluating a hypothesis, you need to account for both the variability in your sample
and how large your sample is. Based on this information, you'd like to make an assessment of
whether any differences you see are meaningful, or if they are likely just due to chance. This is
formally done through a process called hypothesis testing.

Five Steps in Hypothesis Testing:

1. Specify the Null Hypothesis

2. Specify the Alternative Hypothesis
3. Set the Significance Level (a)
4. Calculate the Test Statistic and Corresponding P-Value
5. Drawing a Conclusion

Step 1: Specify the Null Hypothesis

The null hypothesis (H0) is a statement of no effect, relationship, or difference between two or
more groups or factors. In research studies, a researcher is usually interested in disproving the null
hypothesis.

Examples:

 There is no difference in intubation rates across ages 0 to 5 years.

 The intervention and control groups have the same survival rate (or, the
 intervention does not improve survival rate).
 There is no association between injury type and whether or not the patient
received an IV in the prehospital setting.

Step 2: Specify the Alternative Hypothesis

The alternative hypothesis (H1) is the statement that there is an effect or difference. This is usually
the hypothesis the researcher is interested in proving. The alternative hypothesis can be one-
sided (only provides one direction, e.g., lower) or two-sided. We often use two-sided tests even
when our true hypothesis is one-sided because it requires more evidence against the null hypothesis
to accept the alternative hypothesis.

Examples:

 The i n t u b a t i o n success rate differs with the age of the patient

being treated (two-sided).
 The time to resuscitation from cardiac arrest is lower for the intervention
group than for the control (one-sided).
  There is an association between injury type and whether or not the patient
received an IV in the prehospital setting (two sided).

Step 3: Set the Significance Level (α)

The significance level (denoted by the Greek letter alpha— α) is generally set at 0.05. This means
that there is a 5% chance that you will accept your alternative hypothesis when your null hypothesis
is actually true. The smaller the significance level, the greater the burden of proof needed to reject
the null hypothesis, or in other words, to support the alternative hypothesis.

Step 4: Calculate the Test Statistic and Corresponding P-Value

Hypothesis testing generally uses a test statistic that compares groups or examines associations
between variables. When describing a single sample without establishing relationships between
variables, a confidence level is commonly used.

The p-value describes the probability of obtaining a sample statistic as or more extreme by chance
alone if your null hypothesis is true. This p-value is determined based on the result of your test
statistic. Your conclusions about the hypothesis are based on your p-value and your significance
level (α).

Example:

 P-value = 0.01 This will happen 1 in 100 times by pure

chance if your null hypothesis is true. Not likely to happen
strictly by chance.

Example:

 P-value = 0.75 This will happen 75 in 100 times by pure

chance if your null hypothesis is true. Very likely to occur
strictly by chance.

Step 5: Drawing a Conclusion

1. P-value <= significance level (α): Reject your null hypothesis in favor of your alternative
hypothesis. Your result is statistically significant.
2. P-value > significance level (α): Fail to reject your null hypothesis. Your result is not
statistically significant.
Hypothesis testing is not set up so that you can absolutely prove a null hypothesis. Therefore,
when you do not find evidence against the null hypothesis, you fail to reject the null hypothesis.
When you do find strong enough evidence against the null hypothesis, you reject the null
hypothesis. Your conclusions also translate into a statement about your alternative hypothesis.
When presenting the results of a hypothesis test, include the descriptive statistics in your
conclusions as well. Report exact p-values rather than a certain range.

For example, "The intubation rate differed significantly by patient age with younger patients have
a lower rate of successful intubation (p=0.02)." Here are two more examples with the conclusion
stated in several different ways.

Example:

 H0: There is no difference in survival between the intervention and control group.
 H1: There is a difference in survival between the intervention and control group.
 a = 0.05; 20% increase in survival for the intervention group; p-value = 0.002

Conclusion:

 Reject the null hypothesis in favor of the alternative hypothesis.

 The difference in survival between the intervention and control group was statistically
significant.
 There was a 20% increase in survival for the intervention group compared to control
(p=0.001).

Example:

 H0: There is no difference in survival between the intervention and control group.
 H1: There is a difference in survival between the intervention and control group.
 a = 0.05; 5% increase in survival between the intervention and control group; p-value =
0.20

Conclusion:

 Fail to reject the null hypothesis.

 The difference in survival between the intervention and control group was not statistically
significant.
 There was no significant increase in survival for the intervention group compared to
control (p=0.20).
 What is a test statistic?
A test statistic is a random variable that is calculated from sample data and used in a hypothesis test.
You can use test statistics to determine whether to reject the null hypothesis. The test statistic compares
your data with what is expected under the null hypothesis. The test statistic is used to calculate the p-
value.

A test statistic measures the degree of agreement between a sample of data and the null hypothesis. Its
observed value changes randomly from one random sample to a different sample. A test statistic
contains information about the data that is relevant for deciding whether to reject the null hypothesis.
The sampling distribution of the test statistic under the null hypothesis is called the null distribution.
When the data show strong evidence against the assumptions in the null hypothesis, the magnitude of
the test statistic becomes too large or too small depending on the alternative hypothesis. This causes
the test's p-value to become small enough to reject the null hypothesis.

For example, the test statistic for a Z-test is the Z-statistic, which has the standard normal distribution
under the null hypothesis. Suppose you perform a two-tailed Z-test with an α of 0.05, and obtain a Z-
statistic (also called a Z-value) based on your data of 2.5. This Z-value corresponds to a p-value of
0.0124. Because this p-value is less than α, you declare statistical significance and reject the null
hypothesis.

Different hypothesis tests use different test statistics based on the probability model assumed in the
null hypothesis. Common tests and their test statistics include:
Hypothesis test Test statistic

Z-test Z-statistic

t-tests t-statistic

ANOVA F-statistic

Chi-square tests Chi-square statistic

 Deciding between Z Test and T-Test

What is the Z Test?

z tests are a statistical way of testing a hypothesis when either:

 We know the population variance, or

 We do not know the population variance but our sample size is large n ≥ 30

If we have a sample size of less than 30 and do not know the population variance, then we must
use a t-test.

We perform the One-Sample Z test when we want to compare a sample mean with the population
mean.
Performing a t-test
The t-test estimates the true difference between two group means using the ratio of the difference in
group means over the pooled standard error of both groups. You can calculate it manually using a
formula, or use statistical analysis software.

T-test formula
The formula for the two-sample t-test (the Student’s t-test) is shown below.

In this formula, t is the t-value, x1 and x2 are the means of the two groups being compared, s2 is the
pooled standard error of the two groups, and n1 and n2 are the number of observations in each of the
groups.

Key Differences Between T-test and Z-test

The difference between t-test and z-test can be drawn clearly on the following grounds:
1. The t-test can be understood as a statistical test which is used to compare and analyze whether
the means of the two population is different from one another or not when the standard deviation
is not known. As against, Z-test is a parametric test, which is applied when the standard deviation
is known, to determine, if the means of the two datasets differ from each other.

2. The t-test is based on Student’s t-distribution. On the contrary, z-test relies on the assumption that
the distribution of sample means is normal. Both student’s t-distribution and normal distribution
appear alike, as both are symmetrical and bell-shaped. However, they differ in the sense that in a
t-distribution, there is less space in the center and more in the tails.

3. One of the important conditions for adopting t-test is that population variance is unknown.
Conversely, population variance should be known or assumed to be known in case of a z-test.

4. Z-test is used to when the sample size is large, i.e. n > 30, and t-test is appropriate when the size
of the sample is small, in the sense that n < 30.
Chi-square test
A Chi-square test is performed to determine if there is a difference between the theoretical
population parameter and the observed data.
 Chi-square test is a non-parametric test where the data is not assumed to be normally
distributed but is distributed in a chi-square fashion.
 It allows the researcher to test factors like a number of factors like the goodness of fit,
the significance of population variance, and the homogeneity or difference in
population variance.
 This test is commonly used to determine if a random sample is drawn from a population
with mean µ and the variance σ2.

Formula
The chi-squared test is done to check if there is any difference between the observed value and
expected value. The formula for chi-square can be written as;

or
χ2 = ∑(Oi – Ei)2/Ei
where Oi is the observed value and Ei is the expected value.

Example of Categorical Data

Let us take an example of a categorical data where there is a society of 1000 residents with
four neighbourhoods, P, Q, R and S. A random sample of 650 residents of the society is taken
whose occupations are doctors, engineers and teachers. The null hypothesis is that each
person’s neighbourhood of residency is independent of the person’s professional division. The
data are categorised as:

Categories P Q R S Total

Doctors 90 60 104 95 349

Engineers 30 50 51 20 151

Teachers 30 40 45 35 150

Total 150 150 200 150 650

Assume the sample living in neighbourhood P, 150, to estimate what proportion of the whole
1,000 people live in neighbourhood P. In the same way, we take 349/650 to calculate what
ratio of the 1,000 are doctors. By the supposition of independence under the hypothesis, we
should “expect” the number of doctors in neighbourhood P is;
150 x 349/650 ≈ 80.54
So by the chi-square test formula for that particular cell in the table, we get;
(Observed – Expected)2/Expected Value = (90-80.54)2/80.54 ≈ 1.11
Some of the exciting facts about the Chi-square test are given below:
The Chi-square statistic can only be used on numbers. We cannot use them for data in terms of
percentages, proportions, means, or similar statistical contents. Suppose, if we have 20% of 400
people, we need to convert it to a number, i.e. 80, before running a test statistic.
A chi-square test will give us a p-value. The p-value will tell us whether our test results are
significant or not.
However, to perform a chi-square test and get the p-value, we require two pieces of information:
(1) Degrees of freedom. That’s just the number of categories minus 1.
(2) The alpha level (α). You or the researcher choose this. The usual alpha level is 0.05 (5%),
but you could also have other levels like 0.01 or 0.10.