Difficulty: Intermediate to Advanced | Prerequisites: Critical points and second derivative test (§14.7), partial derivatives, basic optimisation ideas
Big picture
This material covers two flavours of optimisation. First, unconstrained optimisation applied to a specific entropy function (the Shannon index), where you reduce variables using a constraint and then find the maximum the standard way. Second, Lagrange multipliers, which handle constrained optimisation without eliminating variables: you set the gradient of the objective proportional to the gradient of the constraint and solve. The Lagrange multiplier itself turns out to have a physical or economic meaning (marginal value of relaxing the constraint).
TL;DR
The Shannon index is maximised when all species proportions are equal, giving maximum entropy. Lagrange multipliers let you optimise a function subject to a constraint by solving ∇f = λ∇g alongside the constraint. The multiplier λ measures how much the optimum changes per unit change in the constraint.
Shannon index (Shannon-Wiener index, Shannon-Weaver index)
A measure of diversity in an ecosystem: H = –Σ p_i ln(p_i), where p_i is the proportion of species i. Higher H means more disorder (more evenly distributed species).
Think of it as a number that tells you how "mixed" a community is: all one species gives H = 0, equal proportions gives the highest H.
Entropy
A measure of disorder or uncertainty. In information theory, it quantifies how unpredictable a distribution is. The Shannon index is a form of entropy.
Lagrange multiplier (λ)
A scalar that appears when you optimise f(x, y, z) subject to g(x, y, z) = k by solving ∇f = λ∇g. It measures the rate of change of the optimal value with respect to changes in the constraint constant k.
In simple terms, λ tells you how much more of the objective you could squeeze out if the constraint were relaxed by one unit.
Utility function
In economics, a function U(x, y, z) that models the satisfaction a consumer gets from purchasing quantities x, y, z of different goods. A common form is the Cobb-Douglas utility U = x^a y^b z^c.
Marginal utility per dollar
The additional satisfaction gained per additional dollar of budget. In Lagrange multiplier language, this is exactly λ when maximising utility subject to a budget constraint.
Q3(a): Reduce to two variables
Since p_1 + p_2 + p_3 = 1 (the three proportions must sum to 1, because every organism belongs to exactly one species), substitute p_3 = 1 – p_1 – p_2:
H(p_1, p_2) = –p_1 ln(p_1) – p_2 ln(p_2) – (1 – p_1 – p_2) ln(1 – p_1 – p_2)
Q3(b): Domain
All three proportions must be positive (ln is undefined at 0) and sum to 1:
Domain = {(p_1, p_2) : p_1 > 0, p_2 > 0, p_1 + p_2 < 1}
This is the open interior of a triangle in the p_1-p_2 plane with vertices at (0,0), (1,0), and (0,1).
Q3(c): Find the maximum
Take partial derivatives and set them to zero:
∂H/∂p_1 = –ln(p_1) – 1 + ln(1 – p_1 – p_2) + 1 = ln((1 – p_1 – p_2)/p_1) = 0
So (1 – p_1 – p_2)/p_1 = 1, meaning p_3 = p_1.
Similarly, ∂H/∂p_2 = 0 gives p_3 = p_2.
Therefore p_1 = p_2 = p_3 = 1/3.
H_max = –3 · (1/3) ln(1/3) = ln(3) ≈ 1.099.
Justification that this is a maximum
The second derivative test confirms it (D > 0, H_{p_1 p_1} < 0). Alternatively, as the boundary of the domain is approached (any p_i → 0), H → a value less than ln(3), since one term –p_i ln(p_i) → 0 while the remaining species share all the proportion. Since H is continuous on the closed triangle and achieves its maximum in the interior, (1/3, 1/3, 1/3) is the global maximum.
Intuition
Maximum entropy occurs when all species are equally represented. This is the most "disordered" or "uncertain" distribution: if you pick an organism at random, you have the least possible information about which species it belongs to.
Setup
A plane with positive intercepts (a, 0, 0), (0, b, 0), (0, 0, c) has equation x/a + y/b + z/c = 1. The volume it cuts in the first octant is V = abc/6. The plane must pass through (1, 2, 3), so 1/a + 2/b + 3/c = 1.
Q4(a): Without Lagrange multipliers
Solve the constraint for c: 3/c = 1 – 1/a – 2/b, so c = 3/(1 – 1/a – 2/b).
Substitute into V = abc/6 and minimise as a function of a and b. Set ∂V/∂a = 0 and ∂V/∂b = 0.
By the symmetry of the method (each intercept is proportional to its corresponding coordinate), the minimum occurs at a = 3, b = 6, c = 9. (Each intercept is 3 times the corresponding coordinate of the point.)
The plane equation: x/3 + y/6 + z/9 = 1, and the minimum volume is V = 3 · 6 · 9 / 6 = 27.
Q4(b): With Lagrange multipliers
Minimise V = abc/6 subject to g(a, b, c) = 1/a + 2/b + 3/c – 1 = 0.
∇V = λ∇g gives:
bc/6 = λ(–1/a²)
ac/6 = λ(–2/b²)
ab/6 = λ(–3/c²)
From the first two equations: (bc/6)/(–1/a²) = (ac/6)/(–2/b²), which simplifies to –a²bc/6 = –ab²c/12, giving 2a = b.
From the first and third: similarly, 3a = c.
With b = 2a, c = 3a: 1/a + 2/(2a) + 3/(3a) = 1/a + 1/a + 1/a = 3/a = 1, so a = 3, b = 6, c = 9.
Same answer: the plane is x/3 + y/6 + z/9 = 1.
Setup
Maximise U(x, y, z) = x^a y^b z^c subject to x + y + z = I, where s = a + b + c.
Q5(a): Lagrange multiplier solution
Set ∇U = λ∇g where g = x + y + z – I:
ax^(a–1) y^b z^c = λ
bx^a y^(b–1) z^c = λ
cx^a y^b z^(c–1) = λ
From the first two: a/x = b/y, so y = (b/a)x.
From the first and third: a/x = c/z, so z = (c/a)x.
Substitute into the constraint: x + (b/a)x + (c/a)x = x(a + b + c)/a = xs/a = I, so x = aI/s.
Similarly y = bI/s, z = cI/s.
The maximum utility:
U_max = (aI/s)^a (bI/s)^b (cI/s)^c = (a^a b^b c^c / s^s) · I^s
The multiplier: λ = a(aI/s)^(a–1)(bI/s)^b(cI/s)^c = (a^a b^b c^c / s^s) · s · I^(s–1)
Or more cleanly: λ = s · U_max / I.
Q5(b): λ as marginal utility per dollar
Since U_max = (a^a b^b c^c / s^s) · I^s, differentiate with respect to I:
dU_max/dI = s(a^a b^b c^c / s^s) · I^(s–1) = λ
So dU_max = λ dI. If the budget increases by dI dollars, the maximum achievable utility increases by approximately λ dI. The Lagrange multiplier is the marginal utility per dollar of budget.
This result holds for any objective and constraint, not just this particular utility function. It is one of the most important interpretive results in constrained optimisation.
H = -\sum_{i=1}^{n} p_i \ln p_i\text{Lagrange conditions: } \nabla f = \lambda \nabla g \text{ and } g(x,y,z) = kU_{\max} = \frac{a^a b^b c^c}{s^s} \cdot I^s, \quad s = a+b+c\lambda = \frac{dU_{\max}}{dI} \quad \text{(marginal utility per dollar)}\text{Plane through } (x_0, y_0, z_0): \quad \frac{x}{a} + \frac{y}{b} + \frac{z}{c} = 1, \quad V = \frac{abc}{6}The Shannon index is used in ecology to measure biodiversity and in information theory to measure the information content of a message source. The equal-proportion maximum is the same result that underpins the principle of maximum entropy in statistical mechanics.
Lagrange multipliers and the interpretation of λ as a shadow price are central to economics. A firm's λ from a production constraint tells management exactly how much more output one extra dollar of budget would buy, which drives resource-allocation decisions.
Students often set up Lagrange multipliers with the wrong sign convention: ∇f = λ∇g and ∇f = –λ∇g both work, but you must be consistent. The sign of λ flips between the two conventions.
When reducing variables using a constraint (as in Q3), students forget to adjust the domain accordingly. The new domain is the open triangle, not all of R².
A common error in the utility problem is writing the constraint as x + y + z = 0 instead of x + y + z = I, or forgetting that a, b, c are given constants.
Students sometimes treat λ = 0 as impossible or as an error. It is possible and meaningful: it means relaxing the constraint would not change the optimum (the constraint is not binding at that point).
⚠️ The interpretation of λ (dU_max/dI = λ, shadow price, marginal value of the constraint) is a favourite exam question. Be ready to state it in words.
⚠️ The Shannon index maximum at equal proportions, with H = ln(n) for n species, is a standard result. Know it and the intuition behind it.
⚠️ Expect a problem asking you to solve the same optimisation both with and without Lagrange multipliers, as in Q4.
True or False: The Shannon index is maximised when one species dominates. (Answer: False; it is maximised when all species are equally represented.)
Fill in the blank: The Lagrange condition is ∇f = ______ ∇g. (Answer: λ)
True or False: If λ = 5 in a budget-constrained utility problem, an extra dollar of budget yields approximately 5 more units of utility. (Answer: True)
Q: What is the maximum of H = –p_1 ln(p_1) – p_2 ln(p_2) when p_1 + p_2 = 1 (two species)?
A: Maximum at p_1 = p_2 = 1/2, giving H = ln(2) ≈ 0.693.
Q: A plane with intercepts (a, 0, 0), (0, b, 0), (0, 0, c) passes through (2, 1, 1). Find the minimum first-octant volume.
A: Constraint: 2/a + 1/b + 1/c = 1. By the pattern from Q4 (each intercept = 3 times the coordinate divided by... let us solve properly). Using Lagrange multipliers or substitution, the minimum occurs at a = 6, b = 3, c = 3, giving V = 6 · 3 · 3 / 6 = 9.
Q: In the utility problem U = x^2 y^3 z with budget x + y + z = 120, what are the optimal spending amounts?
A: s = 2 + 3 + 1 = 6. x = (2/6)(120) = 40, y = (3/6)(120) = 60, z = (1/6)(120) = 20.
Q: In words, what does the Lagrange multiplier λ represent?
A: The rate at which the optimal value of the objective function changes per unit increase in the constraint constant. In economics, the marginal value of relaxing the constraint by one unit.
The Shannon index connects to information theory (EECS courses) and to statistical mechanics (PHYSICS 340/451), where the Boltzmann entropy S = k_B ln(W) is the physics counterpart. Lagrange multipliers reappear in linear algebra (constrained quadratic forms), in machine learning (support vector machines), and throughout economics (consumer and producer theory). The shadow-price interpretation of λ is a cornerstone of operations research and management science.
Related Terms / Search Tags
Shannon index, Shannon-Wiener, Shannon-Weaver, entropy maximisation, Lagrange multipliers, constrained optimisation, shadow price, marginal utility per dollar, Cobb-Douglas utility, budget constraint, first octant volume plane, MATH 215 HW5 Q3 Q4 Q5, Calculus III §14.7 §14.8, University of Michigan