6 Hypothesis Testing
Learning Goals
- Understand the mathematical foundation of and know how to correctly interpret the results of a hypothesis test
- Derive and implement a hypothesis test that can be used to distinguish between two conflicting hypotheses
- Understand the differences and relationships between type I error, type II error, and power, as well as the factors that influence each of them
- Calculate the power or type II error probability for a given hypothesis test
6.1 Hypothesis Testing Framework
Textbook Reading Guide
Read: Sections 6.1–6.3 (pages 343–359)
Definitions:
- null hypothesis
- alternative hypothesis
- test statistic
- critical region
- critical value
- significance level
- statistically significant
- p-value
Questions:
- What is the goal of hypothesis testing?
- What are the typical steps to deriving a hypothesis test?
- If we decide to reject \(H_0\), does that mean we’ve proved that \(H_0\) is false?
- How should we choose \(\alpha\)?
- What is the difference between a one-sided and a two-sided alternative hypothesis? How does this impact our hypothesis testing procedure? How does this impact our p-value?
- What is the difference between saying that we “fail to reject \(H_0\)” and saying that we “accept \(H_0\)”? Which of these can we conclude, and which of these can’t we conclude?
- How are test statistics and p-values related?
- How can we conduct a hypothesis test for the Bernoulli/Binomial parameter \(p\) if we have a small sample size?
Corresponding Videos
- HT Intro
- HT Example
6.2 Likelihood Ratio Tests
Textbook Reading Guide
Read: Section 6.5 (pages 375–378)
Definitions:
- generalized likelihood ratio
- generalized likelihood ratio test
Questions:
- What is the test statistic for a generalized likelihood ratio test?
- Why will the generalized likelihood ratio \(\lambda\) be between 0 and 1?
- How do we typically find the critical value/region \(\lambda^*\) for a generalized likelihood ratio test?
Corresponding Videos
- LRT Intro
- LRT Example
6.3 Errors and Power
Textbook Reading Guide
Read: Section 6.4 (pages 359–373)
Definitions:
- type I error
- type II error
- power
Questions:
- How is type I error related to the choice of significance level?
- What are the typical steps to calculating the probability of a type II error?
- How is type II error related to the power of a hypothesis test?
- What factors influence the power of a test? In practice, which of these factors can we control?
Corresponding Videos
- Errors and Power Intro
- Errors and Power Example