You've mastered the basics of statistical thinking. Now, let's build on that foundation to tackle some of the more nuanced and powerful ideas you'll encounter.
This is where we move from just understanding data to actively questioning it, evaluating claims, and appreciating the deep ethical questions at the heart of data science—the core of our "panoramic" approach in this course.
Think of it like a courtroom. You start with a "null hypothesis," which is like the "innocent until proven guilty" assumption. It's the boring, default state of the world (e.g., "this new soda tastes the same as the old one").
You then collect data (your evidence) to see if you have enough proof to reject that default assumption and conclude something more interesting is happening (e.g., "people actually prefer the new soda!").
We directly test the null hypothesis by assuming it's true, collecting data, and determining how likely our observed data would be if the null hypothesis were actually true. We never directly "prove" the alternative hypothesis.
A statement that assumes there is no effect, no difference, or no relationship between variables in your study.
It represents the status quo or what you would expect by chance alone. This is what you're testing against.
A statement that contradicts the null hypothesis.
It suggests there is an effect, difference, or relationship that you're investigating. This is typically what the researcher believes or wants to prove.
Key principle: The burden of proof lies with the alternative hypothesis - you need convincing evidence to overturn the assumed null state.
The logic of hypothesis testing follows these steps:
Criminal Trial Analogy
We never prove innocence directly; we either have enough evidence for guilt or we don't.
Note: We never directly "prove" the alternative hypothesis. Instead, we either:
- Reject H₀ → This provides support for H₁ (the alternative)
- Fail to reject H₀ → This doesn't prove H₀ is true, just that we don't have enough evidence against it
Medical Trial
Null: "The new drug has no effect on blood pressure compared to a placebo"
Alternative: "The new drug reduces blood pressure compared to a placebo"
Education
Null: "There is no difference in test scores between students using the new teaching method and traditional method"
Alternative: "Students using the new teaching method score higher than those using the traditional method"
Manufacturing
Null: "The defect rate is 5% or less"
Alternative: "The defect rate is greater than 5%"
In hypothesis testing, you never "prove" the null hypothesis true - you either reject it (if evidence is strong enough) or fail to reject it (if evidence is insufficient).
New Pain Relief Medication: A pharmaceutical company develops a new pain medication and conducts a small trial with 15 patients. 10 patients report improved pain relief compared to their previous medication. The company claims the new drug is more effective.
To properly test this, you'd need more data and a proper control group! A sample of just 15 patients without a control group isn't enough evidence to reject the null hypothesis.
Rejecting a true null hypothesis (false positive)
You conclude there's an effect when there actually isn't one.
The probability of making a Type I error is called alpha (α), typically set at 0.05 (5%).
Example: Concluding a treatment is effective when the improvements were actually just due to chance or placebo effect
Also known as a "false alarm"
Failing to reject a false null hypothesis (false negative)
You conclude there's no effect when there actually is one.
The probability of making a Type II error is called beta (β).
Example: Missing the fact that a medication actually does work
Also known as a "miss"
In practice, researchers typically control Type I error by setting α = 0.05, meaning they're willing to accept a 5% chance of falsely rejecting a true null hypothesis.
When we read articles in the Harvard Data Science Review, authors will make claims based on data. Understanding hypothesis testing helps us ask the right questions: Was their sample size large enough? Could their results just be a random fluke? It's a key tool for critically evaluating evidence.
These intermediate concepts are the bridge between basic statistical understanding and the complex world of data science and AI. They help us move beyond simply interpreting data to critically evaluating the systems built on that data.
By understanding these principles, you're now equipped to see beyond the "magic" of AI and appreciate the statistical foundations that make these technologies both powerful and fallible.