Probability & Statistics · Covariance

Covariance

The sample covariance of paired data adds the products of each pair’s deviations from the means and divides by n − 1, cov = Σ(x − x̄)(y − ȳ)/(n − 1): positive when x and y tend to move together, negative when one rises as the other falls. Each product is a signed rectangle; drag points and watch the covariance square change.

Covariance in this visualization

The sample covariance of paired data measures how two quantities vary together: the products of each pair’s deviations from the means, added and divided by n−1n - 1, cov⁡(x,y)=1n−1∑(xi−xˉ)(yi−yˉ)\operatorname{cov}(x, y) = \frac{1}{n-1}\sum (x_i - \bar{x})(y_i - \bar{y}). Each term is the product of a point’s two deviations from the means, drawn here as the rectangle between the point and the centre (xˉ,yˉ)(\bar{x}, \bar{y}). Points up and to the right of the centre, or down and to the left, give positive rectangles; the other two quarters give negative ones. Both axes use the same unit and the same scale, so the areas are true areas, and the square at the centre has area equal to the absolute value of the covariance. It extends to the right for positive covariance and to the left for negative covariance.

What its size means

Covariance carries units: those of xx times those of yy. Adding a constant to all the xx values changes nothing, but measuring xx in a unit ten times smaller multiplies the covariance by ten. Its sign says whether the points lean along a rising or a falling line; its size alone does not say how strong the tendency is. Dividing by the two standard deviations removes the units and gives the correlation coefficient r=cov⁡(x,y)/(sxsy)r = \operatorname{cov}(x, y)/(s_x s_y). Paired with itself, xx makes squares instead of rectangles, and its covariance is its variance. The divisor n−1n - 1 is the one used for the sample variance; dividing by nn instead gives the covariance of the data themselves.

Why it decides the spread of a sum

For each point, (dx+dy)2=dx2+dy2+2dxdy(d_x + d_y)^2 = d_x^2 + d_y^2 + 2d_x d_y, where dxd_x and dyd_y are its deviations. Averaged the same way, this gives var⁡(x+y)=var⁡x+var⁡y+2cov⁡(x,y)\operatorname{var}(x + y) = \operatorname{var} x + \operatorname{var} y + 2\operatorname{cov}(x, y) exactly. When xx and yy move together, their deviations add up and the sum spreads more than the two spreads added; when they move apart, the deviations partly cancel and the sum can vary less than either part. This is why combining quantities that move in opposite directions steadies a total. With “Sum x + y” on, each point is carried along a line of constant x+yx + y onto the diagonal through the centre, so the spread of those marks is the spread of the sums.

Things to notice

  • A covariance of 0 does not mean the quantities are unrelated: points on a parabola can balance exactly. It only measures a straight-line tendency.
  • One point far from the centre on both axes can supply most of the covariance by itself.
  • Overlapping rectangles each count in full; the shading does not darken where they overlap.

Controls

  • Click an empty spot to add a point; select one and press Delete to remove it, or the arrow keys to move it.

Related