Lagrange's method of multipliers
Lagrange’s method of multipliers — finding extrema of a function of several variables subject to one or more equality constraints, including conditions, geometry, formulas, and worked examples.
Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.
Why it matters
Engineers often need to optimize (minimize or maximize) a quantity while satisfying design constraints. Examples: minimum-weight truss for a required strength, maximum power transfer subject to material limits, best-fit parameters subject to conservation rules. Lagrange’s method of multipliers gives a systematic way to find such constrained extrema.
Key ideas
- Feasible set: The set of all points satisfying the constraints. Extrema are searched only in this set.
- Gradient: For a scalar function h(x₁, …, xₙ), the gradient is ∇h = (∂h/∂x₁, …, ∂h/∂xₙ). It points in the direction of the steepest increase of h.
- Geometric idea: Along the constraint surface g(x) = 0, at a constrained extremum, the level surface of f is tangent to the constraint surface. So ∇f is a linear combination of the constraint gradients: ∇f = λ₁∇g₁ + … + λ_m∇g_m.
- Lagrange multipliers: The scalars λᵢ that represent how f changes if we relax each constraint slightly. They arise from the stationarity conditions for f restricted to the constraint surface.
Exact statements commonly used in university answers
Necessary condition (equality constraints): Let f: ℝⁿ → ℝ and gᵢ: ℝⁿ → ℝ (i = 1, …, m) be continuously differentiable near a point x*, and suppose the gradients ∇g₁(x*), …, ∇g_m(x*) are linearly independent (regularity condition). If f has a local constrained extremum at x* subject to gᵢ(x) = 0 for all i, then there exist scalars λ₁, …, λ_m such that ∇f(x*) = Σᵢ λᵢ ∇gᵢ(x*) and gᵢ(x*) = 0 for all i. This is called the Lagrange multiplier (first-order) condition.
Lagrangian function: Define the Lagrangian L(x, λ) = f(x) − Σᵢ λᵢ gᵢ(x). The first-order necessary conditions are ∂L/∂xⱼ = 0 for all j, and gᵢ(x) = 0 for all i. (Using + or − sign before λ is a convention; stay consistent.)
Second-order (nature) test idea: Construct the Hessian of the Lagrangian with respect to x, HₓₓL = [∂²L/∂xⱼ∂x_k] at the stationary point (x*, λ*). Restrict its quadratic form to directions tangent to the constraint surface (directions v with ∇gᵢ(x*)·v = 0 for all i). If the restricted quadratic form is positive definite, x* is a local constrained minimum; if negative definite, a local constrained maximum. If indefinite, x* is a saddle on the constraint.
Bordered Hessian (exam-standard presentation for equality constraints): For one constraint g(x, y) = 0, define the 3×3 bordered Hessian at a stationary point (x*, y*): B = [ [0, gₓ, g_y], [gₓ, fₓₓ, fₓy], [g_y, fₓy, f_yy] ] evaluated at (x*, y*), where subscripts denote partial derivatives. A common textbook rule states: if det(B) > 0 then the constrained extremum is a maximum, and if det(B) < 0 it is a minimum. If det(B) = 0 the test is inconclusive. Note: Sign conventions vary with how the border is arranged and with using L or f; always match your course’s stated form. See “Verify”. For multiple constraints, border with all m gradients and use the course-specific rule on the signs of appropriate leading principal minors of the bordered Hessian to classify the point.
Assumptions typically required
- f and gᵢ are at least continuously differentiable (C¹) near the point of interest.
- Gradients ∇gᵢ at the stationary point are linearly independent (constraint qualification).
- To establish global maxima/minima, the feasible set should be compact (closed and bounded), or additional convexity/concavity assumptions must hold; otherwise we only certify local extrema.
Derivation (step-by-step) of the first-order condition (single constraint)
- Suppose we want to optimize f(x) subject to g(x) = 0. Let x(t) be any smooth curve lying on the constraint surface with x(0) = x* and g(x(t)) = 0 for all t.
- Since g(x(t)) ≡ 0, differentiate at t = 0: d/dt g(x(t))|₀ = ∇g(x*)·x′(0) = 0. Therefore, any allowable tangent direction v = x′(0) is orthogonal to ∇g(x*).
- If x* is a constrained extremum, then for every allowable v, the directional derivative of f along v must vanish: d/dt f(x(t))|₀ = ∇f(x*)·v = 0.
- Hence ∇f(x*) is orthogonal to every vector v orthogonal to ∇g(x*), which implies ∇f(x*) is parallel to ∇g(x*): ∇f(x*) = λ ∇g(x*) for some scalar λ. This is the Lagrange multiplier condition.
- Writing equations: fₓ = λ gₓ, f_y = λ g_y, together with g(x, y) = 0; solve for x, y, and λ.
What diagram to draw and label (typical 2D case)
- Draw the xy-plane.
- Sketch several level curves f(x, y) = c as smooth contours (label them f = c₁, f = c₂, increasing in one direction).
- Sketch the constraint curve g(x, y) = 0 (label it clearly).
- Mark the candidate point (x*, y*) where a level curve just touches (is tangent to) the constraint.
- Draw ∇f as a perpendicular (normal) to the level curve at (x*, y*), and ∇g as a perpendicular to the constraint at the same point. Show they are parallel (same or opposite directions).
Practical steps to solve problems
- Write the Lagrangian L = f − Σ λᵢ gᵢ.
- Compute partial derivatives ∂L/∂xⱼ and set them to zero; write constraints gᵢ = 0.
- Solve the system for unknowns xⱼ and λᵢ.
- Classify the stationary points: use the bordered Hessian/second-order test if your syllabus requires; or compare values of f on feasible candidates, and check endpoints or other constraints if present.
- Report the final answer clearly: the point(s) and the extreme value f.
Notes and limits
- If ∇g = 0 at a candidate point, the regularity condition fails; check by substitution or by re-parameterizing the constraint.
- Inequality constraints (like h(x) ≤ 0) are not handled by plain Lagrange multipliers; they need KKT conditions (another topic).
- If there are more constraints than variables or dependent constraints, first reduce them (solve some constraints and substitute) or check linear independence of ∇gᵢ.
Formulas
∇f(x*) = λ ∇g(x*) — Single equality constraint; necessary condition at a constrained extremum. Symbols: ∇f, ∇g are gradients (units: same as f per unit of x; typically dimensionless in math problems), λ is the Lagrange multiplier (units: same as f per unit of g). Applies when f, g are C¹ and ∇g ≠ 0.
∇f(x*) = Σᵢ λᵢ ∇gᵢ(x*) — Multiple equality constraints gᵢ(x) = 0. Assumes {∇gᵢ} are linearly independent at x*.
L(x, λ) = f(x) − Σᵢ λᵢ gᵢ(x) — Lagrangian definition (sign convention consistent within a solution). Units: same as f. Applies to equality-constrained problems.
∂L/∂xⱼ = 0, gᵢ(x) = 0 — First-order necessary conditions from the Lagrangian (equivalently, ∂L/∂λᵢ = 0 gives back the constraints).
B = [ [0, (∇g)ᵀ], [∇g, H] ] — Bordered Hessian (single constraint), where H is the Hessian of f with respect to x in this convention (some courses use HₓₓL instead; match your course). Applies at stationary points for the second-order test. See course-specific sign rule to classify max/min (verify).
HₓₓL = ∂²L/∂x∂xᵀ — Hessian of L with respect to variables, used to test definiteness on the tangent space to constraints.
Worked examples
Example 1: Minimum distance from a line (single constraint, 2 variables) Problem: Find the point on the line x + y = 1 that is closest to the origin, and the minimum distance.
- Objective: Minimize f(x, y) = x² + y² (square of distance; minimizing distance or its square gives same minimizer) subject to g(x, y) = x + y − 1 = 0.
- Lagrangian: L = x² + y² − λ(x + y − 1).
- First-order conditions:
- ∂L/∂x = 2x − λ = 0 ⇒ 2x = λ.
- ∂L/∂y = 2y − λ = 0 ⇒ 2y = λ.
- Constraint: x + y − 1 = 0.
- Solve: From 2x = λ and 2y = λ ⇒ x = y. With x + x = 1 ⇒ x = y = 1/2.
- Minimum value: f(1/2, 1/2) = (1/2)² + (1/2)² = 1/4 + 1/4 = 1/2. Distance = √(1/2).
- Nature: The objective f is strictly convex and the constraint is affine; therefore the stationary point is the unique global minimum on the line. Answer: The closest point is (1/2, 1/2). Minimum distance is √(1/2). Final minimum squared distance is 1/2.
Example 2: Maximize the product with a sum constraint (three variables) Problem: For positive real numbers x, y, z with x + y + z = 6, find the maximum of f(x, y, z) = x y z.
- Constraint: g(x, y, z) = x + y + z − 6 = 0.
- Lagrangian: L = x y z − λ(x + y + z − 6).
- First-order conditions:
- ∂L/∂x = y z − λ = 0 ⇒ y z = λ.
- ∂L/∂y = x z − λ = 0 ⇒ x z = λ.
- ∂L/∂z = x y − λ = 0 ⇒ x y = λ.
- x + y + z = 6.
- From yz = xz = xy, and x, y, z > 0, it follows x = y = z.
- Then 3x = 6 ⇒ x = y = z = 2.
- Maximum value: f = 2·2·2 = 8.
- Nature: On the positive orthant with an affine constraint, f is strictly concave in log variables; symmetry and the second-order test (or AM–GM inequality) confirm this is the unique maximum. Answer: The maximum occurs at (2, 2, 2). The maximal product is 8.
Example 3: Extremize quadratic subject to two constraints (multiple multipliers) Problem: Find the point(s) on the intersection of the plane x + y + z = 1 and the plane x − y = 0 that minimize f(x, y, z) = x² + y² + z². Geometrically: the point of the line (from intersection of planes) closest to the origin.
- Constraints: g₁(x, y, z) = x + y + z − 1 = 0; g₂(x, y, z) = x − y = 0.
- Lagrangian: L = x² + y² + z² − λ₁(x + y + z − 1) − λ₂(x − y).
- First-order conditions:
- ∂L/∂x = 2x − λ₁ − λ₂ = 0.
- ∂L/∂y = 2y − λ₁ + λ₂ = 0.
- ∂L/∂z = 2z − λ₁ = 0.
- Constraints: x + y + z = 1; x − y = 0.
- From x − y = 0 ⇒ x = y.
- Subtract the first two equations: (2x − λ₁ − λ₂) − (2y − λ₁ + λ₂) = 0 ⇒ 2(x − y) − 2λ₂ = 0 ⇒ −2λ₂ = 0 ⇒ λ₂ = 0.
- Then from ∂L/∂x: 2x − λ₁ = 0 ⇒ x = λ₁/2; from ∂L/∂y: 2y − λ₁ = 0 ⇒ y = λ₁/2; from ∂L/∂z: 2z − λ₁ = 0 ⇒ z = λ₁/2.
- Using x + y + z = 1 ⇒ 3(λ₁/2) = 1 ⇒ λ₁ = 2/3. Thus x = y = z = (2/3)/2 = 1/3.
- Check constraints: x − y = 0 holds; x + y + z = 1 holds.
- Value: f = 3·(1/3)² = 3·1/9 = 1/3. This is the unique minimum along the feasible line by convexity of f. Answer: The minimizing point is (1/3, 1/3, 1/3) with minimum value 1/3.
Example 4: Classifying with a bordered Hessian (single constraint, 2 variables) Problem: Find and classify constrained extrema of f(x, y) = x² + y² subject to g(x, y) = x² − y = 0.
- Lagrangian: L = x² + y² − λ(x² − y).
- First-order conditions:
- ∂L/∂x = 2x − 2λx = 2x(1 − λ) = 0.
- ∂L/∂y = 2y + λ = 0 ⇒ y = −λ/2.
- Constraint: x² − y = 0 ⇒ y = x².
- Cases:
- If x = 0 ⇒ y = 0 (from constraint). Then from 2y + λ = 0 ⇒ λ = 0.
- If 1 − λ = 0 ⇒ λ = 1 ⇒ y = −1/2, but constraint y = x² requires x² = −1/2, impossible in ℝ. So only candidate is (0, 0).
- Classify via bordered Hessian (using the f-based border): At (0, 0), compute derivatives: fₓₓ = 2, f_yy = 2, fₓy = 0; gₓ = 2x = 0; g_y = −1. B = [ [0, 0, −1], [0, 2, 0], [−1, 0, 2] ]. det(B) = 0·(2·2 − 0) − 0·(0·2 − (−1)·0) + (−1)·(0·0 − 2·(−1)) = (−1)·(0 − (−2)) = −2.
- With this bordering convention, det(B) < 0 indicates a constrained minimum at (0, 0) (see Verify for sign rule). Also, along the constraint y = x², f = x² + x⁴ ≥ 0 with equality only at x = 0, confirming minimum. Answer: The only constrained extremum is at (0, 0), a minimum, with minimum value 0.
Common mistakes
- Treating the stationary point as a global extremum without checking other feasible candidates or the nature test; especially risky when the feasible set is unbounded.
- Ignoring the regularity condition: if ∇g = 0 at the candidate point, the usual multiplier condition may fail; re-parameterize or analyze directly on the constraint.
- Misapplying the bordered Hessian sign rule by using the Hessian of L vs. f inconsistently, or by changing the border order; always match your course’s exact definition.
- Dropping solutions that violate hidden assumptions (e.g., assuming variables must be positive when not stated, or ignoring domain restrictions like square roots requiring non-negative arguments).
For your exam
- Short notes: geometric interpretation (level surfaces tangent to constraint), meaning of λ, regularity condition, multiple constraints.
- Derivations: first-order necessary condition; derivation using curve on constraint; bordered Hessian construction and classification rule (state assumptions clearly).
- Numericals/theory: Solve constrained optimization problems with one or more equality constraints; compute candidate points; classify using second-order test or by comparing values; interpret λ.
- Diagrams: level curves with constraint curve; gradients shown normal and parallel at the extremum; properly labeled.
- What a full-marks answer includes:
- Clear statement of f and gᵢ, the Lagrangian, all partial derivatives set to zero, and the constraint equations.
- Correct algebra leading to candidate points and multipliers.
- A correct and clearly stated classification method (bordered Hessian/second-order test) with justification, or a rigorous comparison of values when applicable.
- Final answers boxed/clearly highlighted with the extremum point(s) and the extreme value of f.
- Any domain restrictions or assumptions stated explicitly.
Quick check
- State the Lagrange multiplier condition for maximizing f(x, y, z) subject to g(x, y, z) = 0.
- What does the multiplier λ tell you, intuitively, about the constraint?
- On the line x + y = c, which point is closest to the origin?
Answers: (1) ∇f = λ ∇g with g = 0 at the extremum and ∇g ≠ 0. (2) The rate at which the optimal value of f would change per unit relaxation of the constraint; sensitivity or shadow price. (3) (c/2, c/2).
Finished this topic? Mark it so your progress, study plan and readiness keep up.
Stuck on something here?