APEX Calculus: for University of Lethbridge

Let us continue our discussion of constrained optimization begun in Section 14.5. Theorem 14.5.19 tells us that the Extreme Value Theorem extends to functions of two variables; in fact, this is true for a function of any number of variables: if a real-valued function \(f\) is continuous on a closed, bounded subset of \(\mathbb{R}^n\text{,}\) then it is guaranteed to have an absolute maximum and minimum.

However, as the number of variables increases, the job of finding these absolute extrema becomes more and more complicated. We saw one approach in Section 14.5: given a continuous function on a closed, bounded region \(D\text{,}\) we first consider critical values on the interior of \(D\text{.}\) We then restrict our function \(f\) to points on the boundary of \(D\text{,}\) and attempt to reduce the problem to optimization in one variable.

In many cases, this approach is best accomplished by parametrizing the boundary. We learned how to define parametric curves in the plane in Section 9.2.

The above method works well, when it’s straightforward to set up. The advantage is that it reduces the problem of optimization along the boundary to an optimization problem in one variable, which is something we mastered long ago.

One downside is that it is not always easy to come up with a parametrization for a curve. In the above example, the boundary \(x^2+y^2=4\) is a level curve: it’s of the form \(g(x,y)=c\text{.}\) When we’re trying to optimize subject to a constraint of this form, there is another approach, called the method of Lagrange multipliers.

This tells us that the gradient \(\nabla f\) should be orthogonal to the constraint curve \(g(x,y)=c\) at the point \((x_0,y_0)=(x(t_0),y(t_0))\text{.}\) But we know another gradient that is orthogonal to this curve: \(\nabla g\text{!}\) Recall from Theorem 14.3.9 that \(\nabla g(x,y)\) is always orthogonal to the level curve \(g(x,y)=c\) at points along the curve.

Let’s sum up: the vectors \(\nabla f(x_0,y_0)\) and \(\nabla g(x_0,y_0)\) are both orthogonal to the vector \(\vrp(t_0)\text{.}\) We assume that \(\nabla f(x_0,y_0)\neq \vec{0}\text{,}\) since critical points of \(f\) have already been checked. We also assume that \(c\) is a regular value of \(g\text{,}\) meaning that there are no critical points of \(g\) along the curve \(g(x,y)=c\text{,}\) so \(\nabla g(x_0,y_0)\neq\vec{0}\) as well.

Note that there are two possibilities: either \(\lambda=0\text{,}\) in which case \((x_0,y_0)\) is a critical point of \(f\text{,}\) or \(\lambda\neq 0\text{,}\) in which case the level curve of \(f\) that passes through \((x_0,y_0)\) must be tangent to the curve \(g(x,y)=c\) at that point. Putting Theorem 14.7.3 to use is a matter of solving a system of equations.

The method of Lagrange multipliers seems rather arcane at first glance, but it’s actually not hard to understand geometrically why it works.

Consider Figure 14.7.7. The constraint curve \(x^2+y^2=4\) is the dashed circle. We also see the three level curves (solid) that were obtained as solutions to the Lagrange multiplier equations:

We see that all three curves are tangent to the constraint curve, as we expect from the requirement that the gradients \(\nabla f\) and \(\nabla g\) be parallel where the curves intersect.

Additional level curves \(f(x,y)=c\) are plotted as well, with dashed-dotted lines. For values of \(c\) with \(c\gt20\) (greater than the maximum) or \(c\lt-16\) (less than the minimum), the level curve \(f(x,y)=c\) does not intersect the constraint curve at all.

For values of \(c\) with \(-16\lt c\lt 20\text{,}\) the curve \(f(x,y)=c\) intersects the constraint curve, but the intersection is what’s called transversal: at these points of intersection, the two curves are not tangent, and the gradients are not parallel.

In Figure 14.7.7, you can imagine that increasing or decreasing the value of \(c\) has the effect of shifting the level curve one way or the other, until it just touches the circle. Any bigger than the maximum, or smaller than the minimum, and the curves no longer intersect. Of course, saying that the curves “just touch” amounts to saying that they are tangent at their point of intersection, just as Theorem 14.7.3 predicts.

The method of Lagrange multipliers is not restricted to functions of two variables or to single constraints. We can similarly determine the extrema of a function \(f\) of three variables on a closed bounded subset of \(\mathbb{R}^3\text{.}\)

As the above examples show, Lagrange multiplier problems are often easy to set up, but hard to solve by hand. So why is the method useful? One reason is that it can be used to establish useful theoretical results. But more practically, the method of Lagrange multipliers is useful because it is easy to program into a computer: we simply provide the function and the constraint(s), and the computer solves the resulting equations. There is no need for the same degree of problem-solving employed when we first tackled optimization problems in one variable, back in Chapter 4. To emphasize this, we consider one more example: a reprise of one of the optimization problems from Section 4.3.