Key terms / Power of a Test

• Hypothesis Testing: a formal procedure for using observed data to decide between two The power of a test is the probability that the test will find convincing
competing claims (called hypotheses). The claims are usually statements about a
parameter, like a population proportion p or the population mean μ. evidence for Ha when the specific alternative value of the parameter is true.
• Null Hypothesis (Ho): the claim that we weigh evidence against in a significance test.
Power = 1 - P(Type II Error)
• Alternative Hypothesis (Ha or H1): the claim that we are trying to find evidence to support.
• Test Statistic: measures how far a sample statistic is from what we would expect if the null Four ways to increase Power of a Hypothesis Test
hypothesis is true, in standard deviation units. 1. Increase the sample size, n.
• P-Value: the probability of getting evidence for the alternative hypothesis Ha as strong or 2. Increase the significance level, α.
stronger than the observed evidence when the null hypothesis Ho is true.
3. Choose alternative parameter value that’s further away from null.
• How to calculate: either using z/t table or technology (invNorm() or invT())
4. For means, decrease variability of population, σ.
• Statistical Significance: when the p-value is smaller than alpha (α).
• One-sided Test: when the alternative hypothesis is either > or < the null value. Relationship: As P(Type I Error) = α increases, P(Type II Error) decreases,
• Two-sided Test: when the alternative hypothesis is different from (≠) the null value. and Power increases.
• Significance Level (alpha value): the value that we use as a boundary for deciding whether
an observed result is unlikely to happen by chance alone when the null hypothesis is true. If For AP Stats, we are not required to know how to calculate Power.
not given, assume α = 0.05.
Interpreting p-value *Be sure to replace [brackets] with context!
Assuming that the true [parameter in context*] is true, there is a [p-value] probability of getting a sample statistic of [sample mean/proportion] or more extreme just by chance in a random sample of [n].

Hypothesis testing steps (PCCc)

Parameter: State the parameter of interest (be sure to include the word “all”, “true”, or “population” in your description).
• Ex. We are estimating the true proportion of students who watch soccer in Florida OR We are estimating the population mean travel time to work for Florida residents.
Conditions: Be sure to check the appropriate conditions (it would be a “SIN” to forget them). Please see the conditions that need to be verified below on the right.
• To earn full credit, students must state and verify each of the conditions by substituting the correct values in the appropriate formulas.
Calculations: Students must either substitute the correct values into the appropriate formula OR name the test (see names below) and report the test statistic AND p-value.
One Population Proportion Two Population Proportions
By hand using formula: Using Calculator: Hypothesis Test Confidence Interval
1-prop z-test
Specify:
x (successes) n (sample size)
po (null parameter value) Note: we assume Ho is true and use the pooled proportion, Note: we DO NOT assume Ho is true so we DO NOT use the
Note: we assume Ho is true and use p, not p-hat.
p-hat c, not p-hat 1 and 2, respectively. pooled proportion, p-hat c, and use p-hat’s 1 and 2.
AP Tip: When naming a test, do not use calculator terms (calculator speak).
Instead, refer to the test by its full name. Using Calculator: Proper Name: Using Calculator: Proper Name:
1-prop z-test (on calculator) is a one proportion z-test 2-prop z-test two proportion z-test 2-prop z-interval two proportion z-interval
Conclusion: when making a decision, we must do three things: 1) compare p-value to alpha level, 2) reject/fail to reject the null accordingly, and 3) state that we have suffi-
cient/insufficient evidence to support the alternative hypothesis in the context of the problem.
• For example, “Since 0.0236 < 0.05, we reject the null. We have sufficient evidence to support the alternative hypothesis that the true proportion of [in context].”

Type I/II Errors Checking conditions

Type I Error: rejecting the null and Proportions (Categorical): when working Means (Quantitative): when working with
finding convincing evidence that the
with sample proportions we are collecting sample means we are collecting a value with
alternative is true when the null is true.
binary responses (yes/no) from each member. units from each member.
Also known as a false positive.
When there are two populations, each condition must be verified for each sample, respectively. Therefore, there should be two random
P(Type I Error) = alpha/significance samples, independence for each, and normality for each. Additionally, there is a fourth condition (please see last line below)*.
level
The following three conditions must be verified for either setting and two of them are
Type II Error: failing to reject the null and not finding convincing evidence that the alternative
is true when the alternative is true. Also known as a false negative. the same (the third, Normality, is where they differ):
P(Type II Error) = 1 - Power 1) Simple Random Sample or Randomized Experiment (representative of population)
Note: we always interpret errors in the context of the problem and NOT using vague/generic terms. 2) Independence* - Sample must be less than 10% of population (since we are
sampling without replacement). To verify: N > 10n - Also, see below in red*
Sampling distributions 3) Normality: Large Counts Condition 3) Normality: Central Limit Theorem
From AP Stats Formula Sheet: Normality Condition for Proportions: Normality Condition for Means (module 7):
At least 10 “successes” and 10 “failures”. To verify, we must show that only one of the
following criteria has been met:
When n is large, we can use a Normal
probability model to approximate 1. We are told the shape of population distribution
is normal/approximately normal OR
binomial probabilities if the Large Counts
2. n ≥ 30 and state that the CLT applies OR
Condition is met (be sure to substitute
3. n < 30 and we verify the shape of the
values to verify): distribution of the sample is not strongly skewed
and there are no outliers.

Calculator Tip: When estimating a population mean and the sample size is less than
30, use the calculator to create a boxplot to check for strong skewness and outliers.
Note: Don’t forget that we must pool (p-hat c as seen above) when conducting a two-proportion *Additional Condition: When there are two populations, we must verify that the samples are independent of
z-test and NOT pool (using both p-hat 1 and 2) when building a two-proportion z-interval. one another. This is either told in the scenario or must be reasonably inferred from scenario.
• Many students lose credit on the AP Statistics exam when defining parameters because their description refers to the sample
instead of the population or because the description isn't clear about which group of individuals the parameter is describing. When
defining a parameter, we suggest including the word “all”, “true”, or “population” in your description to make it clear that you
aren't referring to a sample statistic.
• Terminology matters. Never just say “the distribution.” Always say the “distribution of [blank]”, be careful to distinguish the
distribution of the population, the distribution of sample data, and the distribution of a statistic. Likewise don't use ambiguous
terms like “sample distribution” which could refer to the distribution of the sample data or to the sampling distribution of a
statistic. You will lose credit on the free-response questions for misusing statistical terms.
• Notation matters. The symbols all have specific and different meanings. Either use notation correctly–or don’t use it at all. You can
expect to lose credit if you use incorrect notation.
• The free response section almost always has a question that asks students to calculate a probability of some sort. Students should
always check the necessary conditions before calculating a probability even if the question doesn't specifically ask for the
conditions. Students will not be asked to perform a probability calculation in a context where the conditions have not been met.
There may, however, be a question that focuses on just the conditions. In this case, the conditions may not be met.
• The Random and Independence conditions are the same for sampling distributions that involve proportions and means. The only
condition that changes is the Normality condition. When working with proportions we must check the Large Counts Condition and
when working with means we must check the criteria for the Central Limit Theorem.
• If a free-response question asks you to complete a hypothesis test, you are expected to do the entire four-step process. That in-
cludes clearly defining the parameter, checking conditions, reporting calculations, and stating the conclusion. Don’t forget for one
population there are three conditions and when there are two populations there is a fourth condition (the independent samples
condition).
• When your sample size is fewer than 30 observations AND the population shape is not given to be approximately normal, it is not
enough just to make a graph of the data on the calculator when assessing Normality. You must sketch the graph on your paper to
receive credit. You don't have to draw multiple graphs, any appropriate graph will do.
• There is almost always one free-response question that asks students to perform a significance test. Students will most likely be
asked if the data provide convincing evidence for the alternative hypothesis, rather than if the data provide convincing evidence
against the null hypothesis.
• When the P-value is greater than the alpha level we fail to reject the null. Instead of failing to reject the null hypothesis, many stu-
dents use language that sounds like they accept the null hypothesis. Accepting that all hypothesis will always lose credit on the
AP Statistics exam. Instead students are expected to say that “there is insufficient evidence to support the alternative in context”.
• On the AP Statistics exam, it is acceptable for students to use a confidence interval rather than the test statistic and P-value to
address a two-sided alternative hypothesis. However, if the alternative hypothesis is one-sided, students will lose credit for using
confidence interval approach unless they explicitly addressed the imperfect link between the one-sided test and the confidence
level. For instance, by adjusting the confidence level appropriately. Our recommendation for the AP Statistics exam is to always
stick with a significance test.
• Many students lose credit when defining parameters in an experiment by describing the sample proportion rather than the true
proportion. For example, “the true proportion of the men who had surgery and survived 20 years” describes the sample statistic
and not the population parameter.
• The formula for the two-sample z interval for p1 - p2 often leads to calculation errors by students. As a result, it is recommended to
use the calculator’s 2-PropZInt feature to compute the confidence interval on the AP Statistics exam. Be sure to name the
procedure (2-sample z-interval for p1 - p2) in the Calculations step and give the interval computed by the calculator.
• When identifying the parameter of interest, it is essential to state which proportion is p 1 and which is p2. Because hypothesis
testing looks at whether there is a statistically significant difference between proportions, your alternative hypothesis is not
relevant without knowing which proportion is represented by which statistic.
• For any two-sample hypothesis test or interval, you must check and state all four conditions for both samples. If you do not include
the work for both, you will not get credit for checking the conditions.
• “Significant” in the statistical sense does not mean “important”. It means simply “not likely to happen by chance alone”.
Please note the last line contains the various confidence levels. When in need of a t-critical value (t*), see where confidence level
column intersects the appropriate degrees of freedom. For z*, use df = infinity row (last row) and confidence level.