The gradient tells you which way is downhill. The Hessian tells you the shape of the ground — bowl, dome, or saddle — and that shape is what lets Newton's method leap straight to the bottom.
You're standing somewhere on a hilly landscape in thick fog. You crouch down and feel the ground tilt under your boots. That tilt — the gradient — tells you which way is downhill and how steep it is. Good. You step that way.
But the gradient hides something crucial. Imagine you walk until the ground feels perfectly flat — zero tilt in every direction. The gradient is now the zero vector. Have you reached the bottom of a valley? Maybe. Or maybe you're balanced on the very top of a dome, about to roll off. Or maybe you're at a mountain pass — flat along the trail but dropping away on both sides into different valleys.
All three of these places have exactly the same gradient: zero. The gradient cannot tell them apart. To know whether flat ground is a bottom, a top, or a pass, you need to know how the slope is changing around you — you need the curvature.
In one dimension you already know this story. Take f(x). The first derivative f′(x) is the slope. The second derivative f″(x) tells you how the slope is changing — whether the curve is bending up like a smile (f″ > 0, a bowl) or bending down like a frown (f″ < 0, a dome).
At a flat point where f′(x) = 0, that second derivative settles the question. f″ > 0 means a minimum, f″ < 0 means a maximum. This is the second-derivative test from single-variable calculus, and you've probably used it without thinking of it as "curvature."
The whole job of this lesson is to carry that idea up into two (and then many) dimensions. In 2D a surface can curve differently in different directions — up along one axis, down along another (that's the saddle). To capture all of that curvature information we need not a single number but a small matrix of second derivatives. That matrix is the Hessian, named after the mathematician Otto Hesse.
Drag the slider to roll a marble down each surface from its flat spot. All three have zero gradient at the center. Watch what the curvature does to the marble.
Notice: in the bowl the marble returns to center (stable minimum). On the dome it rolls off and never comes back (unstable maximum). On the saddle it depends which way you nudge it — flat along one ridge, falling away the other. Same zero gradient; three completely different behaviors. The behavior is set by the curvature, and the curvature is what the Hessian records.
Before we can build the Hessian we need its raw ingredients: second partial derivatives. Let's build them from the ground up, with no hand-waving.
A function of two variables, f(x, y), is a landscape: feed it a position (x, y) and it returns a height. A partial derivative measures the slope in one direction while you hold the other variable frozen.
The notation ∂f/∂x (read "partial f partial x", also written fx) means: treat y as a constant number, and differentiate with respect to x as if it were ordinary single-variable calculus. Likewise fy = ∂f/∂y freezes x and differentiates in y.
A tiny worked example. Let g(x, y) = x²y. To get gx, freeze y (call it a constant c): g = c·x², so gx = c·2x = 2xy. To get gy, freeze x: g = (x²)·y, a constant times y, so gy = x².
A second partial derivative is just a partial derivative of a partial derivative. Each first partial fx and fy is itself a function of (x, y), so we can differentiate it again. That gives four possible second partials:
| Symbol | Built by | What it measures |
|---|---|---|
| fxx | ∂/∂x of fx | How the x-slope changes as you move in x — curvature along the x-axis |
| fyy | ∂/∂y of fy | How the y-slope changes as you move in y — curvature along the y-axis |
| fxy | ∂/∂y of fx | How the x-slope changes as you move in y — the twist |
| fyx | ∂/∂x of fy | How the y-slope changes as you move in x — the other twist |
The first two, fxx and fyy, are the pure second partials. They are the direct analog of the single-variable f″: fxx > 0 means the surface bends upward (smiles) as you walk along x.
The last two are the mixed second partials, and they carry the genuinely new information. fxy asks: as I take one step in the y-direction, how does my x-slope change? If walking north makes the east-west ground tilt differently, the surface is twisted — and that twist is exactly what distinguishes a saddle from a simple bowl.
Take f(x, y) = x³ + 4x²y. We'll compute all four second partials with every step shown.
Step 1 — first partials.
Step 2 — differentiate fx again.
Step 3 — differentiate fy again.
Look at fxy = 8x and fyx = 8x. They came out identical, even though we computed them by differentiating in opposite orders. That is not a coincidence — it is a theorem, and it's the subject of the next chapter.
The curve is a slice of a surface. The orange line is the tangent (first derivative). Drag to move the point and watch how the tangent rotates — the rate it rotates is the second derivative, the curvature.
We have four second partials. The Hessian simply packages them into a 2×2 grid — and that packaging is what turns a pile of numbers into a single object that describes curvature in every direction at once.
The layout is logical, not arbitrary. Row 1 is "how do my x-derivatives change": fxx (as I move in x) and fxy (as I move in y). Row 2 is the same story for the y-derivatives: fyx and fyy. The diagonal holds the pure curvatures; the off-diagonal holds the twist.
In the last chapter fxy and fyx came out equal. This is guaranteed by Clairaut's theorem (also called Schwarz's theorem):
Why should the order not matter? Here's the napkin intuition. fxy measures "change in the x-slope as you step in y." fyx measures "change in the y-slope as you step in x." Both are secretly measuring the same thing: the single number that says how much the surface twists as you move diagonally. A smooth surface has one twist at each point, and you can measure it by walking around the little square in either order — you end up at the same corner, so you read the same twist.
The practical payoff: the Hessian is a symmetric matrix. Its top-right entry equals its bottom-left entry, so we usually write it as:
Symmetry is not just tidy bookkeeping. A real symmetric matrix is guaranteed to have real eigenvalues and perpendicular eigenvectors — and those eigenvalues and eigenvectors are precisely the curvatures and the principal directions of the surface. Symmetry is the mathematical reason the curvature story works out cleanly. (If you've met eigenvalues, this will click in Chapter 4.)
Reusing f(x, y) = x³ + 4x²y from Chapter 1, we found fxx = 6x + 8y, fxy = 8x, fyy = 0. So the Hessian as a function of position is:
The Hessian is a matrix that depends on where you stand. Plug in a point to get a concrete matrix of numbers. At (x, y) = (1, 1):
Symmetric (8 = 8 ✓). This is exactly the kind of assembly we'll do for the full assessment function next chapter.
Time to earn the title. We compute the Hessian of a genuinely messy function — the one from the AA174A assessment — with every single step shown. If you can follow this, you can do any 2D Hessian.
Three terms. Notice each term contains both x and y, so neither partial will ever vanish completely — there's real work in every step. Let's go.
Compute fx: freeze y, differentiate in x. Go term by term.
| Term | y frozen, ∂/∂x | Result |
|---|---|---|
| 2xy | 2y is the constant on x; derivative of x is 1 | 2y |
| 6x²y | 6y is the constant; ∂/∂x of x² is 2x | 12xy |
| −3xy² | −3y² is the constant; ∂/∂x of x is 1 | −3y² |
Compute fy: freeze x, differentiate in y.
| Term | x frozen, ∂/∂y | Result |
|---|---|---|
| 2xy | 2x is the constant; derivative of y is 1 | 2x |
| 6x²y | 6x² is the constant; derivative of y is 1 | 6x² |
| −3xy² | −3x is the constant; ∂/∂y of y² is 2y | −6xy |
We have fx = 2y + 12xy − 3y². Differentiate it twice — once in x (giving fxx), once in y (giving fxy).
fxx = ∂/∂x (2y + 12xy − 3y²): freeze y again.
fxy = ∂/∂y (2y + 12xy − 3y²): now freeze x.
We have fy = 2x + 6x² − 6xy. Differentiate in x (gives fyx, the symmetry check) and in y (gives fyy).
fyx = ∂/∂x (2x + 6x² − 6xy): freeze y.
fyy = ∂/∂y (2x + 6x² − 6xy): freeze x.
Drop the four pieces into the 2×2 grid:
The Hessian above is a function of position. To get a concrete matrix, evaluate at a point. Take (x, y) = (1, 1):
This exact matrix [[12, 8], [8, −6]] is what the Code Lab will reproduce numerically by finite differences. (Spoiler from Chapter 4: with a positive 12 and a negative −6 on the diagonal and a determinant of 12·(−6) − 8·8 = −72 − 64 = −136 < 0, this point is a saddle.)
Drag x and y. The Hessian entries update live from H = [[12y, 2+12x−6y], [2+12x−6y, −6x]]. Watch the determinant flip sign as you cross into saddle territory.
We promised the Hessian could tell apart the three flavors of flat ground. Now we cash that in. The tool is the second-derivative test, and underneath it sits the deeper idea of eigenvalues.
A symmetric 2×2 matrix has two real eigenvalues, λ1 and λ2, with perpendicular eigenvectors. For the Hessian, those eigenvectors point along the surface's principal directions — the directions of pure, untwisted curving — and each eigenvalue is the curvature in that direction.
So at a critical point (gradient zero), the two eigenvalue signs decide everything:
| λ1, λ2 | Shape | Critical point |
|---|---|---|
| both > 0 | Bowl — curves up every way | Minimum |
| both < 0 | Dome — curves down every way | Maximum |
| opposite signs | Saddle — up one way, down another | Saddle point |
You rarely need to compute eigenvalues explicitly. Two summary numbers of the 2×2 Hessian H = [[a, b], [b, c]] reveal the signs for free:
The determinant equals the product of the eigenvalues; the trace equals their sum. From a product and a sum you can read both signs:
| Condition | Means | Verdict |
|---|---|---|
| det H > 0 and a > 0 | product positive, both same sign, and one is positive | Minimum (both +) |
| det H > 0 and a < 0 | product positive, both same sign, one is negative | Maximum (both −) |
| det H < 0 | product negative → eigenvalues have opposite signs | Saddle |
| det H = 0 | an eigenvalue is zero — flat direction | Inconclusive |
Recall H(1,1) = [[12, 8], [8, −6]] from Chapter 3. (At this point the gradient is ∇f(1,1) = (2+12−3, 2+6−6) = (11, 2), which is not zero — so strictly this isn't a critical point, but the Hessian still describes the local curvature, and the same arithmetic applies to any point.)
Determinant:
det H = −136 < 0. Negative determinant → eigenvalues of opposite sign → saddle-shaped curvature. The surface bends up along one direction and down along the perpendicular one. No need to find the eigenvalues themselves — the sign of the determinant settled it.
If you did want the eigenvalues, solve λ² − (tr)λ + det = 0, i.e. λ² − 6λ − 136 = 0, giving λ = (6 ± √(36 + 544))/2 = 3 ± √145 ≈ 15.0 and −9.0. One up-curvature of about +15, one down-curvature of about −9 — opposite signs, confirming the saddle.
Take g(x, y) = x² + 3y². Then gx = 2x, gy = 6y, so the only critical point is (0, 0). The Hessian is constant: H = [[2, 0], [0, 6]]. Here det H = 2·6 − 0 = 12 > 0 and fxx = 2 > 0, so (0,0) is a genuine minimum. The eigenvalues are just 2 and 6 (diagonal matrix) — both positive, both up-curving, a clean bowl. This is the picture in your head whenever you hear "convex."
Set the three entries of a symmetric Hessian. The widget computes det and trace, infers the eigenvalue signs, and names the critical point. Try to build a saddle (det < 0).
Here's the payoff. A live contour map of a quadratic surface with the Hessian's eigen-directions drawn on top. You set the three Hessian entries; the lab redraws the landscape, overlays the two principal axes (the eigenvectors), and labels the curvature along each (the eigenvalues). Switch between presets to flip instantly between a bowl, a dome, and a saddle.
The surface is the quadratic form q(x,y) = ½(a·x² + 2b·xy + c·y²), whose Hessian is exactly [[a,b],[b,c]]. Set the entries or hit a preset. The teal/warm lines are the eigenvectors; the badge names the critical point.
Play with it. Start at the default bowl (a = c = 2, b = 0): two positive eigenvalues, concentric rings, the badge reads Minimum. Now drag c down past zero — the moment c goes negative the determinant ac − b² flips sign, the rings tear open into hyperbolas, and the badge snaps to Saddle. You are watching the second-derivative test happen in real time.
Now crank up b, the mixed partial, with a and c fixed and positive. At b = 0 the eigenvectors lie along the axes. As b grows, the eigenvectors rotate off the axes — that's the twist — and once b² exceeds ac the determinant goes negative and even this "two-positive-diagonal" matrix becomes a saddle. This is the concrete reason the mixed partial cannot be ignored: enough twist turns a would-be bowl into a saddle.
This single widget contains the entire classification chapter as a feeling in your fingertips. There's no quiz here — the lab is the test. If you can drive it to each of the three shapes on demand and explain why, move on.
Gradient descent walks downhill using only the slope. It's blind to curvature, so it has no idea how big a step is safe — too small and it crawls; too big and it overshoots and bounces. The Hessian fixes this by telling the optimizer the shape of the bowl, so it can step straight to the bottom. That algorithm is Newton's method.
Near any point, a smooth function looks like a quadratic — a bowl (or dome, or saddle). The gradient and the Hessian are exactly the ingredients of that local quadratic. The second-order Taylor expansion around a point x0 is:
That last term — the one with the Hessian H — is the curvature term. It's what turns a flat tangent-plane approximation (gradient only) into a curved bowl approximation. And the bottom of a bowl is somewhere we can jump to in one shot: set the derivative of the quadratic to zero and solve.
Doing that gives the Newton step:
Compare it to a gradient-descent step, xnew = x0 − η∇f. Gradient descent multiplies the gradient by a single scalar guess η. Newton's method multiplies by H−1, the inverse Hessian — a matrix that rescales each direction by its own curvature. Steep directions (big curvature) get a small step; shallow directions (small curvature) get a big step. It's a per-direction, curvature-aware learning rate, derived rather than guessed.
Minimize g(x, y) = x² + 3y² starting from (4, 2). The gradient is ∇g = (2x, 6y), and the Hessian is constant: H = [[2, 0], [0, 6]], with inverse H−1 = [[½, 0], [0, ⅜]].
At the start point: ∇g(4, 2) = (8, 12). The Newton step:
One step. Straight to the exact minimum at the origin, no matter where we started, because g is exactly quadratic and Newton fits it perfectly. Gradient descent on the same problem would take dozens of zig-zagging steps because the bowl is stretched (6× steeper in y than x) — and a fixed η can't be right for both axes at once. The Hessian's job is precisely to undo that stretch.
Same start point, same function g = x² + k·y². Watch gradient descent (warm, timid η) zig-zag while Newton (teal) lands in one curvature-aware step. Crank the stretch k to make gradient descent suffer.
Everything in one place. If you internalize this table, you can compute and classify any 2D Hessian, and you know what it's for.
| Step | What to do |
|---|---|
| 1. First partials | fx (freeze y), fy (freeze x). Each frozen variable is a constant — keep its sign and exponent. |
| 2. Second partials | Differentiate fx in x → fxx, in y → fxy. Differentiate fy in y → fyy. |
| 3. Assemble | H = [[fxx, fxy], [fxy, fyy]]. Check the off-diagonals match (Clairaut). |
| 4. Evaluate | Plug in the point to get a matrix of numbers. |
| 5. Classify | det = ac − b². det < 0 → saddle. det > 0 & a > 0 → min. det > 0 & a < 0 → max. det = 0 → inconclusive. |
| Object | Meaning |
|---|---|
| H = [[fxx, fxy], [fxy, fyy]] | Symmetric matrix of second partials — curvature in all directions. |
| Clairaut: fxy = fyx | Mixed partials are equal → H is symmetric. Free bug-check. |
| Eigenvalues λ1, λ2 | The two principal curvatures. Signs decide bowl/dome/saddle. |
| det H = λ1λ2, tr H = λ1+λ2 | Product and sum of curvatures — read signs without solving. |
| Newton step: −H−1∇f | Curvature-aware step; one move to the min of a quadratic. |
For a function of n variables, the Hessian is an n×n symmetric matrix Hij = ∂²f / ∂xi∂xj. Everything generalizes: it's still symmetric (Clairaut), its n eigenvalues are still the principal curvatures, all-positive eigenvalues still means a minimum (the matrix is "positive definite"), and the Newton step is still −H−1∇f. In machine learning the Hessian of the loss with respect to millions of weights is too big to store, so methods like L-BFGS and Adam approximate its curvature information — but the idea you learned here is exactly what they're chasing.
"What I cannot create, I do not understand." — Richard Feynman.
You computed the Hessian by hand, classified it, and watched Newton use it. You
can now create it.
Compute the Hessian of the assessment function numerically and confirm it matches H=[[12y, 2+12x−6y],[2+12x−6y, −6x]]. Then draw the gradient field. Press Run (Python boots once, ~5s).
The lesson showed you the Hessian. This workbook makes you produce it — every exercise asks you to derive a partial, trace a classification, build a function, design a saddle, or debug a sign slip. Work them with a pencil; the answers check your arithmetic and every reveal walks the full path. Nothing is a placeholder.
Everything starts with "freeze the other variable." These build the four second partials before you ever touch the matrix.
For f = 3x²y, compute fxx and evaluate it at (x,y) = (1, 2). Enter the number.
fxx = 6y is independent of x, so only y matters.
For f = 3x²y, compute the mixed partial fxy and evaluate at (x,y) = (1, 2). Enter the number.
fxy = 6x depends on x, not y — opposite of fxx.
For f = 2xy + 6x²y − 3xy², compute fyy as a formula, then evaluate at (x,y) = (2, 5). Enter the number.
For f = 2xy + 6x²y − 3xy², fxx = 12y. Evaluate at (x,y) = (3, −1). Enter the number.
For the assessment f, fxy = 2 + 12x − 6y. Evaluate at (x,y) = (1, 1). Enter the number.
This is the off-diagonal of H(1,1) = [[12,8],[8,−6]].
Write fxx(x,y) for f = x²y + y³. (fx = 2xy, so fxx = 2y.)
function fxx(x, y) { return 2 * y; }
f = x²y + y³ → fx = 2xy → fxx = 2y. The y³ term has no x, so it drops.
Four numbers into a 2×2 grid — and always confirm Clairaut.
For the assessment H = [[12y, 2+12x−6y],[2+12x−6y, −6x]], find the bottom-right entry fyy at (x,y) = (−2, 7). Enter the number.
Evaluate the assessment H at (x,y) = (0, 0). All three distinct entries: fxx=12y, fxy=2+12x−6y, fyy=−6x. Enter the off-diagonal entry fxy.
For f = x² + xy + y², the Hessian is constant. Enter its fxy entry.
Write isSym(a,b,c,d) for H=[[a,b],[c,d]], returning true iff b equals c (within 1e-9).
function isSym(a, b, c, d) { return Math.abs(b - c) < 1e-9; }
Read bowl / dome / saddle straight off the matrix.
Compute det of H = [[5, 2],[2, 3]]. Enter the number.
H(1,1) = [[12, 8],[8, −6]]. Compute det H. Enter the number.
For H = [[2, 0],[0, 6]], the eigenvalues are the diagonal. Enter the larger eigenvalue.
Write classify(a,b,c) for H=[[a,b],[b,c]] returning "min", "max", "saddle", or "flat" using det = ac−b².
function classify(a, b, c) { const det = a*c - b*b; if (det < 0) return 'saddle'; if (det === 0) return 'flat'; return a > 0 ? 'min' : 'max'; }
Reverse the process: build a function whose Hessian does what you want.
You want f with a saddle at (0,0): up along x, down along y. The classic is f = x² − y². Enter fxx for this f (it's constant).
Up-curve +2 along x, down-curve −2 along y — the textbook saddle.
You want a bowl (min at origin) that is 5× steeper in x than in y, with fyy = 2. Using f = a·½x² + ½·2·y², enter the fxx you need.
Start from H = [[1, b],[b, 1]] (a = c = 1). What is the smallest |b| that makes it a saddle (det < 0)? Enter the threshold value of b above which it becomes a saddle.
Even with both diagonal entries positive, enough mixed-partial twist tears the bowl into a saddle.
You want H = [[4, b],[b, 1]] to have det = 0 (a flat eigen-direction). Enter the positive b that achieves it.
For f = x³ − 3xy² (the "monkey saddle"), the Hessian at the origin is the zero matrix. Enter det H(0,0).
The test is inconclusive here — you'd need third derivatives. A vivid reminder that det = 0 means "look closer."
Each card has a planted bug. Find the wrong line.
A student computes fx for f = 2xy + 6x²y − 3xy². Which line is wrong?
Computing fyy for −3xy²: a student writes ∂/∂y(−3xy²) = −3x. Then fyy = 0. What's wrong?
For H = [[3, 4],[4, 3]] a student writes det = 3·3 + 4·4 = 25 > 0, "minimum." Which step is the bug?
H = [[−5, 0],[0, −2]]: det = 10 > 0, so a student concludes "minimum." What did they forget?
A student writes the Newton step as xnew = x0 + H−1∇f. Why does it diverge on a bowl?
A student assembles H = [[12y, 2+12x−6y],[2+12x, −6x]] for the assessment function. Which entry is wrong?
Computing fxx for fx = 2y + 12xy − 3y², a student gets fxx = 2 + 12y − 6y. What's wrong?