Probability & Statistics · Chi-squared test

Chi-squared test

A chi-squared test asks whether counts from independent trials are too far from the counts a hypothesis expects. It adds up (O − E)²/E over the k categories and compares the total X² with a chi-squared distribution with k − 1 degrees of freedom.

Chi-squared test in this visualization

Roll a die with kk faces NN times, each roll independent of the others, and count how often each face comes up: the observed counts O1,…,OkO_1, \dots, O_k. If the die is fair, each face is expected E=N/kE = N/k times. The chi-squared statistic X2=∑(O−E)2/EX^2 = \sum (O - E)^2/E adds up each face’s squared difference from EE, measured against EE. For a fair die and enough rolls, X2X^2 approximately follows the chi-squared distribution with k−1k - 1 degrees of freedom, and the p-value is the area under that curve to the right of the observed X2X^2. Here the counts are yellow columns with EE as an ink line, and the curve sits below them.

Why each term is divided by E

An observed count that exceeds the expected count by 9 is a big surprise when E=10E = 10 and hardly noticeable when E=1000E = 1000. Chance variation in a count grows with the count: for a fair die each count has variance E(1−1/k)E(1 - 1/k), close to EE. Dividing the squared difference by EE puts every face on that common scale, so a term near 1 is an ordinary amount of chance. Squaring lets one far-off face dominate: in the six-sided example, nineteen sixes contribute 8.1 of X2=11.2X^2 = 11.2.

Why k − 1 degrees of freedom

The counts must add up to NN, so the differences O−EO - E always add up to 0: once k−1k - 1 of them are known, the last one is fixed. For a fair die the average of X2X^2 is exactly k−1k - 1, so the same X2X^2 means less when there are more faces: 11.2 is past the 5% point for a six-sided die (p ≈ 0.048) and ordinary for a twelve-sided one (p ≈ 0.43). What a p-value does and does not tell you is the subject of hypothesis testing and p-values.

When the curve can be trusted

With a fair die, X2X^2 can take only certain values, so its true distribution is a set of separate spikes; the chi-squared curve is the smooth shape they approach as NN grows. Rerunning the rolls with a fair die estimates the true tail probability directly. With an expected count of 10 per face the estimate and the curve agree closely; with only 6 rolls they do not. A common rule of thumb asks for expected counts of at least about 5. More rolls work the other way too: multiplying every count by ten multiplies X2X^2 by ten, so with enough independent rolls even a slight lean becomes significant. X2X^2 measures the evidence against a fair die, not how unfair the die is, and a large p only means these counts are not enough to show it is unfair.

Things to notice

  • When the hypothesis gives the categories unequal probabilities pip_i, the expected counts are Ei=NpiE_i = N p_i and everything else stays the same.

Controls

  • Press or drag above a face to set its count; pressing its number under the axis only selects it.

Related