Confidence interval

A confidence interval estimates an unknown number, here the population mean $\mu$, by a range computed from one sample. The rule that turns a sample into an interval is fixed in advance, and its confidence level, such as 95%, is the share of samples whose interval covers the true value. On this page every sample consists of $n$ independent draws from a normal population with $\mu = 50$ and $\sigma = 10$; each row of the stack is one sample’s interval, and the blue line is $\mu$.

Why can’t one interval claim a 95% chance of containing $\mu$? Here $\mu$ is a fixed number and the interval is what changes from sample to sample. Before the sample is drawn, the chance that its interval will cover $\mu$ is 95%; once it is drawn, that interval either contains 50 or it does not. The band shows why the rule works when $\sigma$ is known. The interval $\bar x \pm \mathrm{ME}$ covers $\mu$ exactly when $|\bar x - \mu| \le \mathrm{ME}$, that is, when $\bar x$ falls in the band $\mu \pm \mathrm{ME}$ of the same width. Since $\bar X$ is normal with standard deviation $\sigma/\sqrt{n}$, it lands in that band 95% of the time when $\mathrm{ME} = z^*\sigma/\sqrt{n}$. A probability for $\mu$ itself needs a prior distribution for $\mu$; that is a Bayesian credible interval, a different construction.

What sets the width? With $\sigma$ known, the half-width is the margin of error $z^*\sigma/\sqrt{n}$, where $z^*$ leaves half of the remaining probability in each tail of the standard normal. Each step towards 100% costs more width than the last: $z^*$ is about 1.645 at 90%, 2.576 at 99% and 3.29 at 99.9%, and no finite interval is 100% sure. The standard error of the mean is $\sigma/\sqrt{n}$ (the sampling distribution), so at a fixed level the width shrinks in proportion to $1/\sqrt{n}$. The coverage does not change with $n$: the margin is always measured in units of the spread of $\bar x$.

Why does an unknown $\sigma$ need $t$? Putting the sample’s $s$ in place of $\sigma$ makes both the centre and the width random. A sample whose $s$ comes out small gets a short interval, so misses gather among the short intervals, and with $z^*$ the coverage is $2F_{n-1}(z^*) - 1$, below the level, where $F_{n-1}$ is the distribution function of Student’s $t$ with $n - 1$ degrees of freedom. That is the distribution of $(\bar X - \mu)/(S/\sqrt{n})$ for normal data. Its tails are heavier than the normal’s, so its point $t^*$ is larger than $z^*$, and $\bar x \pm t^* s/\sqrt{n}$ covers exactly at the level. Correcting the average of $s$ alone is not enough: $s/c_4$ estimates $\sigma$ without bias, yet used with $z^*$ at a nominal 95% it still covers only about 89.5% at $n = 5$. As $n$ grows, $s$ settles near $\sigma$ and $t^*$ approaches $z^*$.