HYPOTHESIS TESTING
Frances Chumney, PhD
CONTENT OUTLINE
Logic of Hypothesis Testing
Error & Alpha
Hypothesis Tests
Effect Size
Statistical Power
HYPOTHESIS TESTING 2
how we conceptualize hypotheses
LOGIC OF HYPOTHESIS TESTING
HYPOTHESIS TESTING 3
HYPOTHESIS TESTING LOGIC
Hypothesis Test
statistical method that uses sample data to
evaluate a hypothesis about a population
The Logic
State a hypothesis about a population, usually concerning a population
parameter
Predict characteristics of a sample
Obtain a random sample from the population
Compare obtained data to prediction to see if they are consistent
HYPOTHESIS TESTING 4
STEPS IN HYPOTHESIS TESTING
Step 1: State the Hypotheses
Null Hypothesis (H0)
in the general population there is no change, no difference, or no relationship;
the independent variable will have no effect on the dependent variable
o Example
• All dogs have four legs.
• There is no difference in the number of legs dogs have.
Alternative Hypothesis (H1)
in the general population there is a change, a difference, or a relationship;
the independent variable will have an effect on the dependent variable
o Example
• 20% of dogs have only three legs.
HYPOTHESIS TESTING 5
STEP 1: STATE THE HYPOTHESES (EXAMPLE)
Example
How to Ace a
Statistics Exam
little known facts about
the positive impact of
alcohol on memory
during “cram” sessions
HYPOTHESIS TESTING 6
STEP 1: STATE THE HYPOTHESES (EXAMPLE)
Dependent Variable
Amount of alcohol consumed the night before a statistics exam
Independent/Treatment Variable
Intervention: Pamphlet (treatment group) or No Pamphlet (control group)
Null Hypothesis (H0)
No difference in alcohol consumption between the two groups the night before a
statistics exam.
Alternative Hypothesis (H1)
The treatment group will consume more alcohol than the control group.
HYPOTHESIS TESTING 7
STEP 2: SET CRITERIA FOR DECISION
Example
Exam 1 (Previous Semester): μ = 85
Null Hypothesis (H0): treatment group
will have mean exam score of M = 85
(σ = 8)
Alternative Hypothesis (H1): treatment
group mean exam score will differ
from M = 85
HYPOTHESIS TESTING 8
STEP 2: SET CRITERIA FOR DECISION
Alpha Level/Level of Significance
probability value used to define the (unlikely) sample outcomes
if the null hypothesis is true; e.g., α = .05, α = .01, α = .001
Critical Region
extreme sample values that are very unlikely to be
obtained if the null hypothesis is true
Boundaries determined by alpha level
If sample data falls within this region (the shaded tails), reject the null
hypothesis
HYPOTHESIS TESTING 9
STEP 2: SET CRITERIA FOR DECISION
Critical Region Boundaries
Assume normal distribution
Alpha Level + Unit Normal Table
Example: if α = .05, boundaries of critical region divide middle 95% from
extreme 5%
o 2.5% in each tail (2-tailed)
HYPOTHESIS TESTING 10
STEP 2: SET CRITERIA FOR DECISION
Boundaries for Critical Region
α = .05 z = ±1.96
α = .01 z = ±2.58
α = .001 z = ±3.30
HYPOTHESIS TESTING 11
STEP 3: COLLECT, COMPUTE
Collect data
Compute sample mean
Transform sample mean M to z-score
M
z
M
Example #2
95 85 10
z 8.85
1.13 1.13
HYPOTHESIS TESTING 12
STEP 4: MAKE A DECISION
Compare z-score with boundary of critical region for selected level of
significance
If…
z-score falls in the tails, our mean is significantly different from H0
o Reject H0
z-score falls between the tails, our mean is not significantly different from H0
o Fail to reject H0
HYPOTHESIS TESTING 13
HYPOTHESIS TESTING: AN EXAMPLE (2-TAIL)
How to Ace a Statistics Exam…
Population: μ = 85, σ = 8
Hypotheses
o H0: Sample mean will not differ from M = 85
o H1: Sample mean will differ from M = 85
Set Criteria (Significance Level/Alpha Level)
o α = .05
HYPOTHESIS TESTING 14
HYPOTHESIS TESTING: EXAMPLE (2-TAIL)
How to Ace a Statistics Exam…
Collect Data & Compute Statistics
o Intervention to 9 students
o Mean exam score, M = 90
8
M 2.67
9 3
90 85 5
z 1.87
HYPOTHESIS TESTING
2.67 2.67 15
HYPOTHESIS TESTING: EXAMPLE (2-TAIL)
How to Ace a Statistics Exam…
Decision: Fail to reject H0
Reject Reject
H0 H0
σM = 2.67
-1.96 z=0 +1.96
μ = 85 M = 90
HYPOTHESIS TESTING 16
REVISITING Z-SCORE STATISTICS
A Test Statistic
Single, specific statistic
Calculated from the sample data
Used to test H0
Rule of Thumb…
Large values of z
o Sample data pry DID NOT occur by chance – result of IV
Small values of z
o Sample data pry DID occur by chance – not result of IV
HYPOTHESIS TESTING 17
uncertainty leads to error
ERROR & ALPHA
HYPOTHESIS TESTING 18
UNCERTAINTY & ERROR
Hypothesis Testing = Inferential Process
LOTS of room for error
Types of Error
Type I Error
Type II Error
HYPOTHESIS TESTING 19
TYPE 1 ERRORS
error that occurs when the null hypothesis is rejected even though
it is really true; the researcher identifies a treatment effect
that does not really exist (a false positive)
Common Cause & Biggest Problem
Sample data are misleading due to sampling error
Significant difference reported in literature even though it isn’t real
Type I Errors & Alpha Level
Alpha level = probability of committing a Type I Error
Lower alphas = less chances of Type I Error
HYPOTHESIS TESTING 20
TYPE II ERRORS
error that occurs when the null hypothesis is not rejected
even it is really false; the researcher does not identify
a treatment effect that really exists (a false negative)
Common Cause & Biggest Problem
Sample mean in not in critical region even though there is a treatment effect
Overlook effectiveness of interventions
Type II Errors & Probability
β = probability of committing a Type II Error
HYPOTHESIS TESTING 21
TYPE I & TYPE II ERRORS
Experimenter’s Decision
Actual Situation
No Effect, Effect Exists,
H0 True H0 False
Reject H0 Type I Error
Retain H0
Type II Error
HYPOTHESIS TESTING 22
SELECTING AN ALPHA LEVEL
Functions of Alpha Level
Critical region boundaries
Probability of a Type I error
Primary Concern in Alpha Selection
Minimize risk of Type I Error without maximizing risk of Type II Error
Common Alpha Levels
α = .05, α = .01, α = .001
HYPOTHESIS TESTING 23
testing null hypotheses
HYPOTHESIS TESTS
HYPOTHESIS TESTING 24
HYPOTHESIS TESTS: INFLUENTIAL FACTORS
Magnitude of difference between sample mean and population mean (in z-
score formula, larger difference larger numerator)
M
z M
M n
Variability of scores (influences σM; more variability larger σM)
Sample size (influences σM; larger sample size smaller σM)
HYPOTHESIS TESTING 25
HYPOTHESIS TESTS: ASSUMPTIONS
Random Sampling
Independent Observations
Value of σ is Constant
Despite treatment
Normal sampling distribution
HYPOTHESIS TESTING 26
NON-DIRECTIONAL HYPOTHESIS TESTS
Critical regions for 2-tailed tests
α = .05 z = ±1.96
α = .01 z = ±2.58
α = .001 z = ±3.30
HYPOTHESIS TESTING 27
DIRECTIONAL HYPOTHESIS TESTS
Critical regions for 1-tailed tests
Blue or Green tail of distribution – NOT BOTH
α = .05
α = .05
α = .01
α = .01
α = .001
α = .001
z = -3.10 z = -1.65 z = +1.65 z = +3.10
z = -2.33 z = +2.33
HYPOTHESIS TESTING 28
ALTERNATIVE HYPOTHESES
Alternative Hypotheses for 2-tailed tests
Do not specify direction of difference
Do not hypothesize whether sample mean should be lower or higher than
population mean
Alternative Hypotheses for 1-tailed tests
Specify a difference
Hypothesis specifies whether sample mean should be lower or higher than
population mean
HYPOTHESIS TESTING 29
NULL HYPOTHESES
Null Hypotheses for 2-tailed tests
Specify no difference between sample & population
Null Hypotheses for 1-tailed tests
Specify the opposite of the alternative hypothesis
Example #2
o H0: μ ≤ 85 (There is no increase in test scores.)
o H1: μ > 85 (There is an increase in test scores.)
HYPOTHESIS TESTING 30
HYPOTHESIS TESTS: AN EXAMPLE (1-TAIL)
How to Ace a Statistics Exam…
Population: μ = 85, σ = 8
Hypotheses
o H0: Sample mean will be less than or equal to M = 85
o H1: Sample mean be greater than M = 85
Set Criteria (Significance Level/Alpha Level)
o α = .05
HYPOTHESIS TESTING 31
HYPOTHESIS TESTS: AN EXAMPLE (1-TAIL)
How to Ace a Statistics Exam…
Collect Data & Compute Statistics
o Intervention to 9 students
o Mean exam score, M = 90
8
M 2.67
9 3
M 90 85 5
z 1.87
M 2.67 2.67
HYPOTHESIS TESTING 32
HYPOTHESIS TESTS: AN EXAMPLE (1-TAIL)
How to Ace a Statistics Exam…
Decision: Reject H0
Reject
H0
σM = 2.67
z=0 +1.65
μ = 85 M = 90
HYPOTHESIS TESTING 33
estimating the magnitude of an effect
EFFECT SIZE
HYPOTHESIS TESTING 34
EFFECT SIZE
Problem with hypothesis testing
Significance ≠ Meaningful/Important/Big Effect
o Significance is relative comparison: treatment effect compared to standard error
Effect Size
statistic that describes the magnitude of an effect
Measures size of treatment effect in terms of (population) standard
deviation
HYPOTHESIS TESTING 35
EFFECT SIZE: COHEN’S D
Not influenced by sample size
mean difference
Cohen’s d =
standard deviation
Evaluating Cohen’s d
d = 0.2 – Small Effect (mean difference ≈ 0.2 standard deviation)
d = 0.5 – Medium Effect (mean difference ≈ 0.5 standard deviation)
d = 0.8 – Large Effect (mean difference ≈ 0.8 standard deviation)
Calculated the same for 1-tailed and 2-tailed tests
HYPOTHESIS TESTING 36
probability of correctly rejecting a false null hypothesis
STATISTICAL POWER
HYPOTHESIS TESTING 37
STATISTICAL POWER
the probability of correctly rejecting a null hypothesis when it
is not true; the probability that a hypothesis test will identify
a treatment effect when if one really exists
A priori
Calculate power before collecting data
Determine probability of finding treatment effect
Power is influenced by…
Sample size
Expected effect size
Significance level for hypothesis test (α)
HYPOTHESIS TESTING 38
