Gradient: Interactive 3D Visualization

01

Gradient in this visualization

The gradient of a function $f(x, y)$ , written $\nabla f$ or $\operatorname{grad} f$ , is the vector of its partial derivatives, $\nabla f = (f_x, f_y)$ . Where $f$ is differentiable and $\nabla f \ne 0$ , it points in the direction in which $f$ increases fastest, and its length is that fastest rate. This page shows it on six surfaces together with the directional derivative, the rate of change of $f$ in a chosen direction.

The glass surface is the graph of $z = f(x, y)$ ; its contour map lies on the floor. The blue point P sits on the surface above $(a, b)$ . The red arrow u is a horizontal unit vector at the angle $\theta$ , measured counterclockwise from the positive $x$ -axis. Over a horizontal step of one unit along u, the tangent plane rises by the yellow segment: the directional derivative

Directional derivative

D_u f(a, b) = \nabla f \cdot u = f_x \cos\theta + f_y \sin\theta.

The directional derivative is also written $\dfrac{\partial f}{\partial l}$ , with the direction as the unit vector $e_l = (\cos\alpha, \cos\beta)$ ; here $\theta$ is measured counterclockwise from the positive $x$ -axis, so $\cos\alpha = \cos\theta$ and $\cos\beta = \sin\theta$ . The blue ring is the unit circle around P lifted onto the tangent plane. Its highest point lies in the direction of the gradient, $|\nabla f|$ above P, because

Steepest slope

D_u f = |\nabla f| \cos\varphi \le |\nabla f|,

where $\varphi$ is the angle between u and $\nabla f$ . The slope is largest along the gradient, zero along the contour through P, and most negative against the gradient.

02

Things to notice

The gradient is perpendicular to the contour through P. Contours drawn at equal steps of $f$ crowd together where $|\nabla f|$ is large.
As u turns once around, the map point $(a, b) + (D_u f)\,u$ traces a circle whose diameter runs from $(a, b)$ to $(a, b) + \nabla f$ . Where $\nabla f = 0$ , the circle shrinks to the point itself.
The steepest way up need not point at the summit: on a stretched hill the two directions differ.
The partial derivatives are directional derivatives: $u = (1, 0)$ gives $f_x$ , and $u = (0, 1)$ gives $f_y$ .
Where $\nabla f = 0$ (a critical point), every direction has slope 0 and there is no steepest direction. A saddle is such a point that is neither a local maximum nor a local minimum.
The yellow rise belongs to the tangent plane: it is a linear approximation. Over a short horizontal step $h$ along u, the surface itself rises by $f((a, b) + h u) - f(a, b) \approx h\,D_u f$ , and the smaller $h$ , the better the match.

03

Where it is used

Gradient descent, the basic method for training machine-learning models, lowers a function by repeated small steps against the gradient, $x \leftarrow x - \eta\,\nabla f(x)$ . Where $\nabla f \ne 0$ , a small enough step goes downhill, but the method can settle in a local minimum, or crawl near a saddle, instead of reaching the lowest point.
Because the gradient is perpendicular to level curves, it gives their normal direction. Where a smooth $f$ has a maximum or minimum on a curve $g = c$ (with $\nabla g \ne 0$ ), the two gradients are parallel, $\nabla f = \lambda \nabla g$ : the idea behind Lagrange multipliers.
In physics, a conservative force is minus the gradient of potential energy, $F = -\nabla U$ : it pushes toward lower energy, in the direction in which the energy drops fastest.

04

Controls

Drag P on the surface or its foot on the map, or drag the tip of the red arrow. Key points and key directions ( $+x$ , $+y$ , $\nabla f$ , $\perp\nabla f$ ) jump to exact values; at a key critical point $\nabla f$ is exactly 0.
Sweep turns u once around and stops on the steepest direction. Top view looks straight down, where angles on the map are true. Presentation mode opens a full-screen view for projectors (Space sweeps, ← → turn u, H hides the answers).

05

Gradient in this visualization

Things to notice

Where it is used

Controls

Related