Least squares

Three points rarely lie on one line, so any line misses some of them. On the left are the points, the best line and its residuals, the vertical misses whose squares add up to SSE. On the right the same three heights form one vector in space, and finding the best line means projecting that vector onto a plane. You can move the points, try a line of your own and compare two simpler models.

Why is the data a single point? Write the three heights as one vector $\vec b = (y_1, y_2, y_3)$. That is the blue arrow on the right, and its axes are named $y_1, y_2, y_3$ because each one measures a height. A line takes the values $c + m x_i$ at the three $x_i$, which is the vector $c\,\mathbf 1 + m\,\vec x$ with $\mathbf 1 = (1, 1, 1)$. In the general model $y = c + mx$, with the $x_i$ not all equal, these vectors fill the glass plane through $\mathbf 1$ and $\vec x$ as $c$ and $m$ vary. So every line you could draw on the left is one point of that plane.

Pick any line. Its residuals are the coordinates of $\vec b$ minus those of its point in the plane, so its SSE is the squared distance between the two. The best line therefore sits at the point $\vec p$ of the plane nearest to $\vec b$, the orthogonal projection. The residual vector $\vec r = \vec b - \vec p$ meets the plane at a right angle, so it is perpendicular to both $\mathbf 1$ and $\vec x$. Written out, the two right angles are the normal equations:

With Right angles and balance on, the residuals become equally stiff vertical springs. Their net force is proportional to $\sum r_i$, and their torque about the origin to $\sum x_i r_i$. The general fit makes both zero. The constant model needs only the net force to vanish, and the line through the origin only the torque.

Your line, drawn in red, is always a line of the current model, so its point $\vec q$ lies in the same plane or line as $\vec p$. The step from $\vec p$ to $\vec q$ lies there too, while $\vec r$ is perpendicular to it, so Pythagoras splits your error exactly. Your SSE reaches the minimum exactly when $\vec q = \vec p$.

The two smaller models keep one column each. Fitting $y = c$ projects $\vec b$ onto the line through $\mathbf 1$, and $c$ comes out as the mean of the heights. Fitting $y = mx$ projects onto the line through $\vec x$, or just onto the origin when every $x_i$ is 0. When the $x_i$ are not all equal, these are two different lines in the plane. Neither model contains the other, and the general line does at least as well as both. The same projection is at the heart of linear regression.