Confidence Intervals — Meaning, Intuition, and a Common Misconception
Goals of this post
- Understand the precise meaning of a confidence interval
- How to interpret and approach a confidence interval
- Confidence intervals from a hypothesis-testing view
- A common misconception and the truth ("probability of being inside the interval?")
Intro — Estimation in Statistics
A major goal of statistics is estimation. The statistics we usually use is parametric statistics (a subset of frequentism), which assumes a population distribution and treats parameters like the mean and variance as fixed specific values.
Parameter estimation splits into point estimation (pinning a single value) and interval estimation (assuming it lies within some interval). Interval estimation reflects uncertainty better than point estimation (expressing uncertainty as the interval's length).
Body
Q. How do we estimate the average height of adult men in our country?
- Draw a sample of 30+ and compute the mean → point estimate
- Reflect uncertainty as the estimated mean ± error (range) → interval estimate
2.1) How do we set the error range?
Start from the parameter to estimate (the population mean $\mu$). Assume the distribution of the population mean is normal (passing a normality test) and draw it.
Figure 1. Population distribution with the sample mean on the x-axis
The estimation target is $\mu$ (the population mean). The key idea:
Let the 95% of values that could be found in this distribution include the population mean. That is, the sample mean just needs to be found within Interval 1 in Figure 1.
Figure 2. A range from a sample that contains the population
Notation — $\bar{X}_n$ is the $n$-th sample mean (e.g., $\bar{X}_1$, $\bar{X}_2$), and $d_{\bar{X}_n}$ (written $d_n$) is its distance from the mean out to the boundary (e.g., $d_{\bar{X}_1} = d_1$).
2.2) Setting a range that contains the population
In Figure 1, take the boundary at distance $2d$ (95% probability of discovery) from the population mean, with the left as $\bar{X}_1$ and the right as $\bar{X}_2$. To contain the population mean, we need $d = d_1 = d_2$. Then any value between $\bar{X}_1$ and $\bar{X}_2$ contains the population mean.
Figure 3. The meaning of samples found inside vs outside the boundary
In Figure 3, $\bar{X}_3$ found inside the boundary (within 95%) contains the population mean, but $\bar{X}_4$ found in the 5% region does not. (This is on a z-distribution basis; with a sample, you do interval estimation with the t-distribution.)
Definitions
The interval of estimated mean ± $d$ is the confidence interval, $d$ is the margin of error, and the 95% probability here is the confidence level.
Summary
We set $d=d_1=d_2$ so that the 95% of values where the sample mean is found in the population distribution contain the population mean. This leads to the conclusion that if you draw 100 samples and build a confidence interval for each, about 95 of them contain the true population mean.
"Compute the confidence interval for the population mean at the 95% confidence level" or "compute the 95% confidence interval."
Confidence Intervals from a Hypothesis-Testing View
Setting significance level $\alpha=0.05$ is the same idea as setting a 95% confidence interval. The core of hypothesis testing:
If the sample's test statistic falls within the ordinary 95%, accept; if it's found in the extreme 5% region, reject.
Per Figure 3, if the sample result is in the top 5% extreme region, that confidence interval doesn't contain the population value, so we reject the null hypothesis. (Samples outside the 95% CI are sometimes treated as outliers.)
The Misconception — "95% probability it's inside the interval?" (important)
When a 95% confidence interval comes out as 172–176 cm, many interpret it as "there's a 95% probability the average height is between 172 and 176." In parametric statistics, this is wrong. The population mean is already one fixed value, and the sample has already been drawn → probability no longer exists.
Why the misconception arises
- Ambiguous phrasing — "compute the confidence interval at the 95% confidence level" feels as if the interval itself has 95% reliability.
- Because we don't know the parameter — there's a fixed truth (the parameter) we can't know, so without knowing whether the already-drawn sample's interval contains the population mean, we feel it like a probability.
Conclusion
- Before sampling: the probability that a sample-derived confidence interval contains the population mean is 95% — correct phrasing.
- After sampling: the parameter and interval are fixed → the "95% probability" phrasing is wrong.
We can't know whether the already-drawn sample falls in the 95%/5%, but we recognize it as one interval from a distribution where 95 out of 100 contain the population, and hope the parameter is inside it.
(Aside) Bayesian Statistics
In Bayesian statistics the parameter is not fixed but treated as a random variable, so even after sampling, "95% probability of being inside the interval" holds. People may accept the belief-based Bayesian view more easily. Since we all hold beliefs without knowing the truth and revise them with experience (samples), I think Bayesian statistics reflects heuristic human thinking well.
References
- Basic Statistics (Introduction) — Jayu Academy (textbook)
- Intuition gained from extensive reading of textbooks
📦 Migrated from my own Korean blog (my own writing). Original: taehyuklee.tistory.com/19
Comments