Confidence Intervals
A range that usually contains the truth, built from a mean, a standard error and a multiplier you can justify.
- Build a 95% confidence interval for a mean by hand
- Interpret a confidence interval correctly, including what it does not say
- Explain what the 95% refers to, using repeated sampling
- Choose a wider or narrower interval and say what each buys
What it means
A confidence interval is a range of plausible values for a population parameter, built from a sample estimate plus and minus a margin:
For large samples the multiplier is close to 2, which is where the familiar “±2 standard errors” comes from. Its interpretation is the subtle part:
If you repeated the study many times, about 95% of the intervals you build this way would contain the true parameter.
The 95% describes the procedure, not your particular interval. Any single interval either contains the truth or it does not, and you cannot know which. What you can say is that the method is right 95% of the time.
Why it adds something
A point estimate pretends to a precision it does not have. “Average delivery time is 18 minutes” sounds exact and is almost certainly wrong in the third digit. “Between 11.7 and 24.3 minutes, with 95% confidence” is a statement you can act on: it tells a manager they cannot yet promise a 15-minute service, because 15 is inside the plausible range.
Intervals also make comparisons honest. Two groups whose intervals overlap substantially are not clearly different, whatever their means suggest.
Worked by hand
Sample of deliveries: mean 18 minutes, standard deviation 15.3 minutes, standard error 3.06 minutes.
Step 1 — choose the confidence level. 95% is conventional. For the correct multiplier is the value with 24 degrees of freedom, which is about 2.064.
Step 2 — the margin.
Step 3 — the interval.
Step 4 — say what it means. “We estimate the mean delivery time at 18 minutes, with a 95% confidence interval from 11.7 to 24.3.” A service promise of under 12 minutes is at the very edge of plausibility; a promise of 30 is comfortably above it.
🧩 Challenge, think first, then reveal
The same study with deliveries gives the same mean and standard deviation. What is the new interval, and how much narrower is it?
💡 Answer: The standard error becomes $15.3/10 = 1.53$, the $t$ multiplier shrinks slightly to about 1.984, and the margin is about 3.0 minutes. The interval is $(15.0, 21.0)$, roughly half the width. Four times the data halves the uncertainty — the same square-root law as the standard error.
What it does not say
Four common misreadings, all worth recognising in other people’s reports:
- “There is a 95% chance the true mean is between 11.7 and 24.3.” In the frequentist reading, the true mean is a fixed number; it is the interval that is uncertain.
- “95% of deliveries take between 11.7 and 24.3 minutes.” No — that would be a prediction interval, which is much wider.
- “The interval contains 95% of the data.” It contains the mean, not the data.
- “Overlapping intervals mean no difference.” Overlap is a rough hint, not a test; intervals can overlap while the difference is still clear.
The same thing in Python
The arithmetic by hand, then the library version:
import statistics as st
times = [8, 10, 12, 15, 45]
n = len(times)
m = st.mean(times)
se = st.stdev(times) / n ** 0.5
margin = 2.064 * se # t for 24 df, 95%; for n=5 use your own df
print(round(m - margin, 1), round(m + margin, 1))scipy computes the multiplier for any sample size and confidence level:
from scipy import stats
lo, hi = stats.t.interval(0.95, df=n - 1, loc=m, scale=se)
print(round(lo, 1), round(hi, 1))And the experiment behind the interpretation: build many intervals from many samples and count how often they contain the truth.
import random, statistics as st
from scipy import stats
population = [random.gauss(50, 10) for _ in range(10_000)]
truth = st.mean(population)
hits = 0
for _ in range(500):
sample = random.sample(population, 25)
lo, hi = stats.t.interval(0.95, df=24, loc=st.mean(sample), scale=st.stdev(sample) / 5)
hits += lo <= truth <= hi
print(hits / 500) # ≈ 0.95That output — around 95% — is the entire meaning of the phrase.
Seeing it geometrically
Picture the sampling distribution from the last lesson: the interval is the central 95% of it, translated back to the data scale. In the GeoGebra confidence-interval demo you can change the confidence level and see the interval widen as you demand more certainty, and narrow it again by raising .
When to use it
- Reporting any estimate whose precision matters: survey results, benchmark timings, quality measurements.
- Comparing two groups where you want to show the size of a plausible difference.
- Deciding whether a decision is safe under the worst plausible value, not just the best estimate.
Limitations
- Valid only if the sampling is unbiased. A confidence interval around a biased estimate is a precise statement about the wrong thing.
- Assumes independence. Clustered or repeated measurements need a different formula.
- Narrow intervals are not important intervals. With a huge , a trivial difference can have a tight interval.
- It is not a prediction interval. Individual values are far more variable than the mean.
Common pitfalls
- Using 1.96 for small samples. That is the large-sample value; with the correct multiplier is above 2.7.
- Reporting the interval without saying what it is for. The mean, a difference, or a proportion — each has its own interval.
- Treating non-overlap as proof of difference. It is suggestive; a proper comparison is the right tool.
🧩 Challenges
🧩 Challenge, think first, then reveal
A report says “mean score 70, 95% CI [69.5, 70.5]” from a sample of 10,000 students, and concludes “the teaching method is clearly effective”. What is missing from that argument?
💡 Answer: A comparison. The interval says the mean is precisely known, not that it is good. Without a control group, a baseline, or a comparison interval, "70 ± 0.5" establishes only that the estimate is tight — a large $n$ makes almost any difference statistically visible, which is exactly why confidence intervals must be read alongside effect size.
🤔 Socratic Questions
- Why does a 99% interval have to be wider than a 95% one on the same data?
- If two studies give very different intervals for the same quantity, what could explain it besides a mistake?
- Why can a confidence interval never tell you the probability that your specific interval is correct?
- What would you need to build a confidence interval for a median instead of a mean?
✅ Quick check
1. What does "95% confidence" refer to?
2. Which change makes a confidence interval narrower?