Lagrange multiplier

The problem is to find the largest or smallest value of a function $f(x, y)$ when the point is not free but must stay on a curve, the constraint $g(x, y) = c$. The method of Lagrange multipliers says where to look. Suppose $f$ and $g$ have continuous first partial derivatives near a point P of the curve, $f$ restricted to the curve has a local maximum or minimum at P, and $\nabla g(P) \neq 0$. Then there is a number $\lambda$, the Lagrange multiplier, with

These are three equations for x, y and $\lambda$. Their solutions are candidates, not answers: each one still has to be checked, and points where $\nabla g = 0$ have to be looked at separately.

The map shows the contours of $f$, the constraint curve in heavy ink, and a point P on it. At P the blue arrow is $\nabla f$, the hollow arrow shows the direction of $\nabla g$, and $\nabla f$ is split into a yellow part along the curve and a dashed part across it. Under the map, the profile plots $f$ against the distance walked along the curve.

Why does it work? Along the curve $g$ does not change, so the curve’s unit tangent $T$ is perpendicular to $\nabla g$. Walking along the curve at unit speed, $f$ changes at the rate $\nabla f\cdot T$, which is the slope of the profile at P, and the yellow arrow is $(\nabla f\cdot T)\,T$. At a highest or lowest point of the profile where $\nabla g \neq 0$, this slope is zero, so $\nabla f$ has no part along the curve. In the plane, a vector with no part along the curve is a multiple of $\nabla g$, so $\nabla f = \lambda\nabla g$, with $\lambda = 0$ if $\nabla f = 0$. Wherever the yellow arrow is not zero, walking the way it points still raises $f$, so P cannot be a maximum or minimum there.

If also $\nabla f(P) \neq 0$, the contour of $f$ through P is perpendicular to $\nabla f$, so it has the same tangent as the constraint. At a constrained maximum or minimum the constraint touches a contour instead of crossing it.

$\lambda$ has a meaning of its own. Write $f^*(c)$ for the best value on the curve $g = c$. If a point attaining this value moves smoothly as $c$ changes, then $\lambda$ is the rate at which the best value changes:

On the circle $x^2 + y^2 = c$ the largest value of $xy$ is $c/2$, and $\lambda = 1/2$ at both maxima. The λ window plots $f^*$ against $c$ with this slope. The rate depends on how the constraint is written. Written as $2g = C$ with $C = 2c$, the same curve has half the multiplier, because it now measures change per unit of C. Write it as 2g shows this.

Two curves show the limits of the method. On $y - x^3 = c$, the function $f = y$ has parallel gradients at $(0, c)$, yet along the curve $f = x^3 + c$ only levels off there and keeps rising. Parallel gradients are not enough. On $y^2 - x^3 = 0$, the smallest value of $f = x$ is at the cusp $(0, 0)$, where $\nabla g = (0, 0)$ and $\nabla f = \lambda\nabla g$ has no solution. The method misses it, so such points have to be checked by hand.

Whether a candidate is a maximum, a minimum or neither shows in the profile. It rises then falls at a local maximum on the curve, falls then rises at a local minimum, and at the cubic’s candidate it keeps rising.