Law of large numbers

Let $X_1, X_2, \dots$ be independent draws from one distribution with a finite mean $\mu = E[X]$ (it is enough that $E|X| < \infty$), and let $\bar X_n$ be the average of the first n. The law of large numbers says that $\bar X_n \to \mu$: in probability (the weak law) and with probability one (the strong law). For a coin, $X_i$ is 1 for heads, so $\bar X_n = \hat p_n = H_n/n$ is the proportion of heads and $\mu = p$. The page follows one or more paths of 100 000 tosses and asks what makes the proportion settle.

The upper layer shows $\hat p_n$, the lower one the surplus of heads over the expected count, $H_n - np$; for a fair coin it is half of “heads minus tails”. The same deviation shrinks above and grows below. The dashed band at each n is the exact 95% range of the binomial distribution: the narrowest whole interval $[a_n, b_n]$ with at most 2.5% of $H_n$ below it and at most 2.5% above, so it holds at least 95% (the readout gives the exact coverage). In proportion it is $[a_n/n,\ b_n/n]$, in count $[a_n - np,\ b_n - np]$: the same event. With a head start of ten heads, a coupled coin that began with five shares every later toss: the two always differ by exactly 5 heads, so their proportions differ by exactly $5/n$. The lead is diluted, not repaid.