First Year Engineering is coming soon

We're still writing and reviewing the notes for every subject, and we'll open it when every subject is ready. Meanwhile you can practise aptitude, ask Aria about any subject, or see the branches that are ready.

Lagrange's method of multipliers

Lagrange’s method of multipliers — finding extrema of a function of several variables subject to one or more equality constraints, including conditions, geometry, formulas, and worked examples.

Drafted with Aria, reviewed by the AiCanCode.org team · Updated . How we write and check our notes. Spotted an error? Use Give Feedback at the bottom of the page.

Why it matters

Engineers often need to optimize (minimize or maximize) a quantity while satisfying design constraints. Examples: minimum-weight truss for a required strength, maximum power transfer subject to material limits, best-fit parameters subject to conservation rules. Lagrange’s method of multipliers gives a systematic way to find such constrained extrema.

Key ideas

  • Feasible set: The set of all points satisfying the constraints. Extrema are searched only in this set.
  • Gradient: For a scalar function h(x₁, …, xₙ), the gradient is ∇h = (∂h/∂x₁, …, ∂h/∂xₙ). It points in the direction of the steepest increase of h.
  • Geometric idea: Along the constraint surface g(x) = 0, at a constrained extremum, the level surface of f is tangent to the constraint surface. So ∇f is a linear combination of the constraint gradients: ∇f = λ₁∇g₁ + … + λ_m∇g_m.
  • Lagrange multipliers: The scalars λᵢ that represent how f changes if we relax each constraint slightly. They arise from the stationarity conditions for f restricted to the constraint surface.

Exact statements commonly used in university answers

  1. Necessary condition (equality constraints): Let f: ℝⁿ → ℝ and gᵢ: ℝⁿ → ℝ (i = 1, …, m) be continuously differentiable near a point x*, and suppose the gradients ∇g₁(x*), …, ∇g_m(x*) are linearly independent (regularity condition). If f has a local constrained extremum at x* subject to gᵢ(x) = 0 for all i, then there exist scalars λ₁, …, λ_m such that ∇f(x*) = Σᵢ λᵢ ∇gᵢ(x*) and gᵢ(x*) = 0 for all i. This is called the Lagrange multiplier (first-order) condition.

  2. Lagrangian function: Define the Lagrangian L(x, λ) = f(x) − Σᵢ λᵢ gᵢ(x). The first-order necessary conditions are ∂L/∂xⱼ = 0 for all j, and gᵢ(x) = 0 for all i. (Using + or − sign before λ is a convention; stay consistent.)

  3. Second-order (nature) test idea: Construct the Hessian of the Lagrangian with respect to x, HₓₓL = [∂²L/∂xⱼ∂x_k] at the stationary point (x*, λ*). Restrict its quadratic form to directions tangent to the constraint surface (directions v with ∇gᵢ(x*)·v = 0 for all i). If the restricted quadratic form is positive definite, x* is a local constrained minimum; if negative definite, a local constrained maximum. If indefinite, x* is a saddle on the constraint.

  4. Bordered Hessian (exam-standard presentation for equality constraints): For one constraint g(x, y) = 0, define the 3×3 bordered Hessian at a stationary point (x*, y*): B = [ [0, gₓ, g_y], [gₓ, fₓₓ, fₓy], [g_y, fₓy, f_yy] ] evaluated at (x*, y*), where subscripts denote partial derivatives. A common textbook rule states: if det(B) > 0 then the constrained extremum is a maximum, and if det(B) < 0 it is a minimum. If det(B) = 0 the test is inconclusive. Note: Sign conventions vary with how the border is arranged and with using L or f; always match your course’s stated form. See “Verify”. For multiple constraints, border with all m gradients and use the course-specific rule on the signs of appropriate leading principal minors of the bordered Hessian to classify the point.

Assumptions typically required

  • f and gᵢ are at least continuously differentiable (C¹) near the point of interest.
  • Gradients ∇gᵢ at the stationary point are linearly independent (constraint qualification).
  • To establish global maxima/minima, the feasible set should be compact (closed and bounded), or additional convexity/concavity assumptions must hold; otherwise we only certify local extrema.

Derivation (step-by-step) of the first-order condition (single constraint)

  1. Suppose we want to optimize f(x) subject to g(x) = 0. Let x(t) be any smooth curve lying on the constraint surface with x(0) = x* and g(x(t)) = 0 for all t.
  2. Since g(x(t)) ≡ 0, differentiate at t = 0: d/dt g(x(t))|₀ = ∇g(x*)·x′(0) = 0. Therefore, any allowable tangent direction v = x′(0) is orthogonal to ∇g(x*).
  3. If x* is a constrained extremum, then for every allowable v, the directional derivative of f along v must vanish: d/dt f(x(t))|₀ = ∇f(x*)·v = 0.
  4. Hence ∇f(x*) is orthogonal to every vector v orthogonal to ∇g(x*), which implies ∇f(x*) is parallel to ∇g(x*): ∇f(x*) = λ ∇g(x*) for some scalar λ. This is the Lagrange multiplier condition.
  5. Writing equations: fₓ = λ gₓ, f_y = λ g_y, together with g(x, y) = 0; solve for x, y, and λ.

What diagram to draw and label (typical 2D case)

  • Draw the xy-plane.
  • Sketch several level curves f(x, y) = c as smooth contours (label them f = c₁, f = c₂, increasing in one direction).
  • Sketch the constraint curve g(x, y) = 0 (label it clearly).
  • Mark the candidate point (x*, y*) where a level curve just touches (is tangent to) the constraint.
  • Draw ∇f as a perpendicular (normal) to the level curve at (x*, y*), and ∇g as a perpendicular to the constraint at the same point. Show they are parallel (same or opposite directions).

Practical steps to solve problems

  1. Write the Lagrangian L = f − Σ λᵢ gᵢ.
  2. Compute partial derivatives ∂L/∂xⱼ and set them to zero; write constraints gᵢ = 0.
  3. Solve the system for unknowns xⱼ and λᵢ.
  4. Classify the stationary points: use the bordered Hessian/second-order test if your syllabus requires; or compare values of f on feasible candidates, and check endpoints or other constraints if present.
  5. Report the final answer clearly: the point(s) and the extreme value f.

Notes and limits

  • If ∇g = 0 at a candidate point, the regularity condition fails; check by substitution or by re-parameterizing the constraint.
  • Inequality constraints (like h(x) ≤ 0) are not handled by plain Lagrange multipliers; they need KKT conditions (another topic).
  • If there are more constraints than variables or dependent constraints, first reduce them (solve some constraints and substitute) or check linear independence of ∇gᵢ.

Formulas

∇f(x*) = λ ∇g(x*) — Single equality constraint; necessary condition at a constrained extremum. Symbols: ∇f, ∇g are gradients (units: same as f per unit of x; typically dimensionless in math problems), λ is the Lagrange multiplier (units: same as f per unit of g). Applies when f, g are C¹ and ∇g ≠ 0.

∇f(x*) = Σᵢ λᵢ ∇gᵢ(x*) — Multiple equality constraints gᵢ(x) = 0. Assumes {∇gᵢ} are linearly independent at x*.

L(x, λ) = f(x) − Σᵢ λᵢ gᵢ(x) — Lagrangian definition (sign convention consistent within a solution). Units: same as f. Applies to equality-constrained problems.

∂L/∂xⱼ = 0, gᵢ(x) = 0 — First-order necessary conditions from the Lagrangian (equivalently, ∂L/∂λᵢ = 0 gives back the constraints).

B = [ [0, (∇g)ᵀ], [∇g, H] ] — Bordered Hessian (single constraint), where H is the Hessian of f with respect to x in this convention (some courses use HₓₓL instead; match your course). Applies at stationary points for the second-order test. See course-specific sign rule to classify max/min (verify).

HₓₓL = ∂²L/∂x∂xᵀ — Hessian of L with respect to variables, used to test definiteness on the tangent space to constraints.

Worked examples

Example 1: Minimum distance from a line (single constraint, 2 variables) Problem: Find the point on the line x + y = 1 that is closest to the origin, and the minimum distance.

  1. Objective: Minimize f(x, y) = x² + y² (square of distance; minimizing distance or its square gives same minimizer) subject to g(x, y) = x + y − 1 = 0.
  2. Lagrangian: L = x² + y² − λ(x + y − 1).
  3. First-order conditions:
    • ∂L/∂x = 2x − λ = 0 ⇒ 2x = λ.
    • ∂L/∂y = 2y − λ = 0 ⇒ 2y = λ.
    • Constraint: x + y − 1 = 0.
  4. Solve: From 2x = λ and 2y = λ ⇒ x = y. With x + x = 1 ⇒ x = y = 1/2.
  5. Minimum value: f(1/2, 1/2) = (1/2)² + (1/2)² = 1/4 + 1/4 = 1/2. Distance = √(1/2).
  6. Nature: The objective f is strictly convex and the constraint is affine; therefore the stationary point is the unique global minimum on the line. Answer: The closest point is (1/2, 1/2). Minimum distance is √(1/2). Final minimum squared distance is 1/2.

Example 2: Maximize the product with a sum constraint (three variables) Problem: For positive real numbers x, y, z with x + y + z = 6, find the maximum of f(x, y, z) = x y z.

  1. Constraint: g(x, y, z) = x + y + z − 6 = 0.
  2. Lagrangian: L = x y z − λ(x + y + z − 6).
  3. First-order conditions:
    • ∂L/∂x = y z − λ = 0 ⇒ y z = λ.
    • ∂L/∂y = x z − λ = 0 ⇒ x z = λ.
    • ∂L/∂z = x y − λ = 0 ⇒ x y = λ.
    • x + y + z = 6.
  4. From yz = xz = xy, and x, y, z > 0, it follows x = y = z.
  5. Then 3x = 6 ⇒ x = y = z = 2.
  6. Maximum value: f = 2·2·2 = 8.
  7. Nature: On the positive orthant with an affine constraint, f is strictly concave in log variables; symmetry and the second-order test (or AM–GM inequality) confirm this is the unique maximum. Answer: The maximum occurs at (2, 2, 2). The maximal product is 8.

Example 3: Extremize quadratic subject to two constraints (multiple multipliers) Problem: Find the point(s) on the intersection of the plane x + y + z = 1 and the plane x − y = 0 that minimize f(x, y, z) = x² + y² + z². Geometrically: the point of the line (from intersection of planes) closest to the origin.

  1. Constraints: g₁(x, y, z) = x + y + z − 1 = 0; g₂(x, y, z) = x − y = 0.
  2. Lagrangian: L = x² + y² + z² − λ₁(x + y + z − 1) − λ₂(x − y).
  3. First-order conditions:
    • ∂L/∂x = 2x − λ₁ − λ₂ = 0.
    • ∂L/∂y = 2y − λ₁ + λ₂ = 0.
    • ∂L/∂z = 2z − λ₁ = 0.
    • Constraints: x + y + z = 1; x − y = 0.
  4. From x − y = 0 ⇒ x = y.
  5. Subtract the first two equations: (2x − λ₁ − λ₂) − (2y − λ₁ + λ₂) = 0 ⇒ 2(x − y) − 2λ₂ = 0 ⇒ −2λ₂ = 0 ⇒ λ₂ = 0.
  6. Then from ∂L/∂x: 2x − λ₁ = 0 ⇒ x = λ₁/2; from ∂L/∂y: 2y − λ₁ = 0 ⇒ y = λ₁/2; from ∂L/∂z: 2z − λ₁ = 0 ⇒ z = λ₁/2.
  7. Using x + y + z = 1 ⇒ 3(λ₁/2) = 1 ⇒ λ₁ = 2/3. Thus x = y = z = (2/3)/2 = 1/3.
  8. Check constraints: x − y = 0 holds; x + y + z = 1 holds.
  9. Value: f = 3·(1/3)² = 3·1/9 = 1/3. This is the unique minimum along the feasible line by convexity of f. Answer: The minimizing point is (1/3, 1/3, 1/3) with minimum value 1/3.

Example 4: Classifying with a bordered Hessian (single constraint, 2 variables) Problem: Find and classify constrained extrema of f(x, y) = x² + y² subject to g(x, y) = x² − y = 0.

  1. Lagrangian: L = x² + y² − λ(x² − y).
  2. First-order conditions:
    • ∂L/∂x = 2x − 2λx = 2x(1 − λ) = 0.
    • ∂L/∂y = 2y + λ = 0 ⇒ y = −λ/2.
    • Constraint: x² − y = 0 ⇒ y = x².
  3. Cases:
    • If x = 0 ⇒ y = 0 (from constraint). Then from 2y + λ = 0 ⇒ λ = 0.
    • If 1 − λ = 0 ⇒ λ = 1 ⇒ y = −1/2, but constraint y = x² requires x² = −1/2, impossible in ℝ. So only candidate is (0, 0).
  4. Classify via bordered Hessian (using the f-based border): At (0, 0), compute derivatives: fₓₓ = 2, f_yy = 2, fₓy = 0; gₓ = 2x = 0; g_y = −1. B = [ [0, 0, −1], [0, 2, 0], [−1, 0, 2] ]. det(B) = 0·(2·2 − 0) − 0·(0·2 − (−1)·0) + (−1)·(0·0 − 2·(−1)) = (−1)·(0 − (−2)) = −2.
  5. With this bordering convention, det(B) < 0 indicates a constrained minimum at (0, 0) (see Verify for sign rule). Also, along the constraint y = x², f = x² + x⁴ ≥ 0 with equality only at x = 0, confirming minimum. Answer: The only constrained extremum is at (0, 0), a minimum, with minimum value 0.

Common mistakes

  • Treating the stationary point as a global extremum without checking other feasible candidates or the nature test; especially risky when the feasible set is unbounded.
  • Ignoring the regularity condition: if ∇g = 0 at the candidate point, the usual multiplier condition may fail; re-parameterize or analyze directly on the constraint.
  • Misapplying the bordered Hessian sign rule by using the Hessian of L vs. f inconsistently, or by changing the border order; always match your course’s exact definition.
  • Dropping solutions that violate hidden assumptions (e.g., assuming variables must be positive when not stated, or ignoring domain restrictions like square roots requiring non-negative arguments).

For your exam

  • Short notes: geometric interpretation (level surfaces tangent to constraint), meaning of λ, regularity condition, multiple constraints.
  • Derivations: first-order necessary condition; derivation using curve on constraint; bordered Hessian construction and classification rule (state assumptions clearly).
  • Numericals/theory: Solve constrained optimization problems with one or more equality constraints; compute candidate points; classify using second-order test or by comparing values; interpret λ.
  • Diagrams: level curves with constraint curve; gradients shown normal and parallel at the extremum; properly labeled.
  • What a full-marks answer includes:
    • Clear statement of f and gᵢ, the Lagrangian, all partial derivatives set to zero, and the constraint equations.
    • Correct algebra leading to candidate points and multipliers.
    • A correct and clearly stated classification method (bordered Hessian/second-order test) with justification, or a rigorous comparison of values when applicable.
    • Final answers boxed/clearly highlighted with the extremum point(s) and the extreme value of f.
    • Any domain restrictions or assumptions stated explicitly.

Quick check

  1. State the Lagrange multiplier condition for maximizing f(x, y, z) subject to g(x, y, z) = 0.
  2. What does the multiplier λ tell you, intuitively, about the constraint?
  3. On the line x + y = c, which point is closest to the origin?

Answers: (1) ∇f = λ ∇g with g = 0 at the extremum and ∇g ≠ 0. (2) The rate at which the optimal value of f would change per unit relaxation of the constraint; sensitivity or shadow price. (3) (c/2, c/2).

Finished this topic? Mark it so your progress, study plan and readiness keep up.

Stuck on something here?