COWLES FOUNDATION FOR RESEARCH IN ECONOMICS

AT YALE UNIVERSITY

Box 2125, Yale Station

New Haven, Connecticut 06520

·COWLES FOUNDATION DISCUSSION PAPER NO. 80* (Reissue)

(Cowles Commission Discussion Paper: Mathematics 403)

November 7, 1950

Note: Cowles Foundation Discussion Papers are preliminary

materials circulated to' stimulate discussion and
critical comment. Requests for single copies of a
paper will be filled by the Cowles Foundation within
the limits of supply. References in publications to
discussion papers (other than mere acknowledgement
by a writer that he has access to such unpublished
material) should be cleared with the author to protect
the tentative character of these papers.

LAGRANGE MULTIPLIERS REVISITED

Morton Slater

*CFDP 80, distributed in 1959, was an unchanged reissue of

the Cowles Foundation, CCDP: Mathematics 403. It is being
reprinted because of its historical interest and continuing
de::1and,

November 1, 1980

G. Giorgi and T.H. Kjeldsen (eds.), Traces and Emergence of Nonlinear Programming, 293
DOI 10.1007/978-3-0348-0439-4 14, © Springer Basel 2014
 COWLES CO!>lMISSION DISCUSSION PAPER: lVITI~·lATICS: 403

NOTE: Cowles Commission Discussion Papers are preliminary

materials circu+ated privately to stimulate private
discussion and are not ready for critical comment
or appraisal in publications. References in publi·
cations to Discussion Papers {other than mere ac-
knowledgement by a writer that he has had access to
such unpublished material) should be cleared with
the author to protect the tentative character of
these papers.

LAGRANGE MUI.'l'IPLIERS REVISITED

A Contribution to Non-Linear Progra~~ing

by Morton Slater

November 7, 1950

1. Introdaction

The present paper was inspired by the work of Kuhn and Tucker [1] 1

These authors transformed a certain cla:s of constrained maximum pro-

blems into equivalent saddle value (minimax) problems.

Their work seems to hinge on the consideration of still a third

type of problem. A very simple but illustrative form of this problec is

the following: let x E positive orthant of some finite dimensional

Euclidean space, and let f and g be real valued fUnctions of x with

the property that whenever f ~ 01 then also g ~ D; under what conditions

can one then conclude that 3 a non-negative constant u such that

uf ;; g for all x ~ 0?

Kuhn and Tucker showed that if f is concave and differentiable, if

g is convex and differentiable, and if the set [x: f(x) ~ 0} satisfies

certain regularity restrictions, then there does indeed exist such a u.

Two directions for generalization are presented:

First of all, the Kuhn-Tucker ar~~~nt rests heavily on the

1. Nl.U!lerals in brackets refer to the list of references at the end.

294
 -2-

differentiability of the functions, although they express the opinion

that their theorems are true without this assumption. Is this the case?

Secondly, the inequality uf ~ g may be thought of as a relation

between f and g. From this point of view, it would appear that a best

possible theorem which concludes that uf ~ g would make assumptions about

f only in relation to g, and vice versa.

In this paper it is shown how the second generalization may be part-

ly achieved, and that even with this generalization, differentiability

may be dispensed with entirely.

The neY.t section will give a detailed account of the generalized

theorem hinted at above. The last section will be devoted to an appli-

oation of this theorem to transforming a class of constrained minimum

problems into equivalent saddle value problems.

2. The Main Theorem

Throughout this section, x, u, and v will be points in the positive

orthants of n, k, and h dimensional Euclidean spaces respectively.

Theorem 1. Let and gl' ... , ~· be continuous real

valued functions of x ~ (x1 , ... , X )

n
for x ~ 0 with the follow-

ing properties:

1° If at any point x all r 1 (x) ~ 0, then for that

x some gj(x) ~ 0
2° jx such that all r 1 (x) > 0

3° If for some x1 and x2 ~A 1 ~ 0 i ~ 1, •.. , k, Bj a 0

j~1, ,,, 1 h and C of arbitrary sign such that

E A1 f 1 (x 1 ) + c: r Bj
2
and E A1 r1 (x) + c; r Bj gj

then for all X : Gx 1 + (1-G) x2 0 < ~ < 1

295
 -3-

Assertion: 3u1 , ••. , ~ ~ O, v1 , ' ' " vh ~ 0 1

such that for all x ~ 0

~ U; f 1 (x) < Z v. g. (x)

• = J J

Before proceeding with the sequence of lemmas necessary to prove

theorem l some discussion of the hypotheses of that theorem is in order.

1° is clearly essential for the truth of the theorem.

2° corresponds to the condition of regularity of the constraint

set in the Kuhn-Tucker treatment. It may possibly be weakened, but

certainly not dispensed with altogether as the example

f (x) = -(x - 1) 2 g (x) =l - x

shows. Here f (x) is concave, g (x) convex, and 1° is satisfied.

Nevertheless uf f g for some u ~ 0 is impossible as is easily verified.

3° is of course the most controversial nypothesis of all. On the

debit side is the fact that any non-nega~ive linear combination of the
r1, say f, must, as a consequence of 3°, be quasi-concave (i.e. for

all real ~~ [x : f (x) ~ ~} is convex), while any non-negative linear

combination of the say g, must be quasi-convex (i.e. for all real

a, {x : g (x) ;ii a} is convex). This is on the debit .side because it

puts conditions on the r 1 which are independent of the gj.

Parenthetically we observe that a we~~er version of 3° in which

"Ai ~ O" and "Bj -': 0" are replaced by "Ai > o" and "Bj > O" which
a priori appears to avoid the above difficulty, actually implies 3°.

(This can easily be shown.)

296
 On the credit side is that if all the fi are concave and all the

gj convex (the Kuhn-TUcker case), then 3° is automatically satisfied.

What this amounts to is that if a concave function interpolates a convex

function at two points, then the concave function dominates the convex

function in between.

Moreover, 3° is satisfied by still other functions. Let f and g

be strictly increasing functions of 8 single real variable x, continuous

for x ~ 0 and having continuous first and second derivatives for x > D.
If in addition f'(x) > 0 for all x > 01 then by 8 theorem of M. M. Peixoto

[1 , f and g satisfy 3° if and only if

g "()
x ~ d~
f'(x) f "(x) for all x > 0.

Using this theorem, the following ex~~ples were easily constructed:

(1) f convex, g convex

2
f(x) = x2 + x-2 g(x) u =3
(2) f concave, g concave

f(x) = fX -1 g(x) fX + x-2 u 3

(3) Neither f nor g convex or concave

f(x) c 2x - cos x- (2rr + 1),

2
g(x) " x + Bin X • X COS X - (7T + 7T)

The main tool in the proof of theorem 1 is the generalized minimax

theorem of von Neumann [1) and Kakutani [1], which we shall take as

lemma 1.

297
 -5-

Lemma l: Let ~(s,~) be a continuous real valued function defined for

~ € K and ~ € L where K and L are arbitrary bounded closed convex

sets of Euclidean spaces Rp and Rq respectively. If for every ~ 0 € K

and every real a1 the set of all 11 € L such that 'f (~ 0 , 11) ~a: is con-

vex, and if for every 11° € L and every real ~' the set of all ~ € K

1 ( ~'
such that "' >
~ o) ., ~ is convex, then 3(,'o' no)
., such that

Max 'P ( ~, 11°) = min 'f' ( ~ 0 , TJ)

~€K 11€1

Throughout section 2 1 the functions f1 and gj will be the functions

given in the hypotheses of theorem 1.

Lemma 2: Let L = {x : for all i r 1 (x) ~ 0}.

Assertion: jv 'i: 0, L: vj 1 such that

L:vj gj (x) ~ o for all x € L

~: Since the fi(x) are quasi concave, L is closed and

convex.

Let K • (v v ~ 0, L: v j -= 1} ,

~ "' [x x E L, L: xi ~ N}

and 'f(v, x) • L: vj gj (x)

By hypothesis 3° of theorem l we may apply lemma 1 to

'f(v, x) for v € K and x € ~· Hence j (v 0 , x0 ) such that

Max 'f(v, x 0 ) =min lf'(v 0 , x)

VEK xE~

Using the left hand side of the equality, we see that to prove

'f(v0 , x) ~ 0 for all X € ~ we need only show that for any X E

~
jvEK such that 'f(v, x) ~ o. By hypothesis 1° of theorem 1 this is

clearly the case. Hence f(v 0 , x) ~ 0 for all X e: ~·

298
 -6-

The v0 thus obtained depends on N. Choose N

n
t ~ with n and

an associated sequence [v 0 (n)} such that 'P(v 0 (n) 1 x) ~ 0 for all

X E ~ • Since K is compact, some subsequence [v 0 (n.)}

l
converges
n
to v, say. Since ~ 'I 1 with i and '-9 is a continuous function
ni
of v we have

f(v , x) ~ o for all ~ € L q.e.d.

Notation: The function ~(~, x) : L ~j g(x) defined in lemma 2 will be

denoted by 'g(x)" for the remainder of section 2.

1
Len:u;;a 3: Let for i = 1, ... , k and g(x ) < 0.

Assertion: such that fi (x 1 ) < 0 and

0
r 1 (x 0 ) g (x1 ) - f (x 1) g(x 0 ) ~ 0
o 1o
Proof: We restrict our attention to the line segment joining

x0 to x 1 ; a fortiori 1° and 3° are satisfied on the segment. We suppose

(for definiteness) that it is oriented thus:

By 1°, g(x 0 ) ~ 0. Let x be the right-most zero of g(x).

(x0 £x <: x1 ). By 1° and the continuity of the r 1 (x), ji: i0 such

that fi (i) ~ 0.
·o
We show first that f. (xl) < o. Suppose false; then j X such that
l

o ~ f1 <xl 0
< f'1 (x ) '
0
and g(xl < o. Hence jA > 0 and
0 0
B < 0 such that

299
 -7-

Af. (xo) + B = g(xo)

+o
Af. (~) + B = g(~)
1
0

0
so that by 3 , Af. (x) + B ~ g(x) for X
0
.:iX fi X·
1
0

In particular Af. (x) + B £ O, so that

1
0

B
fi (x) ~ - X> o, a contradiction.
0

Hence r 1 (x 1 ) < 0, and for

0

g(xo) - g(xl) f.
1
(xo)g(xl) - g(xo)fi
c = ---------;- > 0' D=
0 0

f. (xo) - fi (xl) fi (xo) - fi (xl)

1
0 0 0 0

we have c f. (xo) + D = g(x 0 )

1
0

c fi (xl) + D = g(x 1 )
0

1
so that c f. (x) + D~ g(x) for X
0
~X ~X •
10
In particular

c fi (X) + D ~ 0
0

so that D= - c r 1 (xl ~ o, and since

0

r 1 (x 0 )g{x1 ) - g(x 0 )f 1 (x1 ) ~ 0 q.e.d.

0 0

300
 -8-

Lemma 4; Let lf(x, u) = u1 f 1 {x) + ••• + ~ fk(x) - g(x)

Assertion: j a finite positive constant M such that for

all x ~ 0 3u e ~ = [u : u ~ o, E ui :ii M}

such that 'f>(x, u) ~ 0.

Proof: If g(x1 ) ;;: 0, 'f(x\ 0) ~ 0

If g(x 1 ) < 0, proceed as follows: let x0 be the point
of the hypothesis 2° at which all f.(x) > 0. Apply lemma 3 to
1
1
select i such that f. (x ) < 0 and
0 1
0

( 0 1 1 0
fi x )g(x ) - fi (x )g(x ) ~ 0
0 0

Let u 1 "'0 when 1 f i

0

u =~
10 fi (xo)
0

1 1
Then fi (x ) - g(x )
0

0 1 0 l
r 1 (x )g(x } - g(x )f1 (x )
.- 0
--~---------------------- :i.O
fi (xo)
0

M = ~(~
0
Thus i f we take ) where A = glb r 1 (x0 ) the lemma is proved.

The next lemma will not be needed for the proof of theorem l, but it is

convenfent to prove it now.

Lemma 5: Let r 1 (x1 ) = ,,, = f~(x 1 ) = 0 1 f~+1 (x1 ),

g(x1 ) = o.
Assertion: If r1 (x2 ), 2
,,,, f~(x);;: o, then
2
g(x) Oi: 0

~; As in lemma 3 we consider the functions on the segment.

joining x1 to x2 ;

301
 -9-

~---------------1
1 2
X X

If g(x2 ) < 0, it must have a rightmost zero x < x2 . We will show

that this is impossible.

First of all, by quasi-concavity, fi(x) ~ 0 for all x in the seg-

ment and i ~ ~· If any f 1 (x 2 ) ~ 0 for i > ~. then again by quasi-con-
2
cavity fi(x) ~ 0 on the whole segment. Finally, suppose some fp(x ) < 0

for p > ~· Then JA > 0 such that

1
A f (x1 ) + B ~ g(x )
p
2
A f (i) + B ~ g(x ) so that by 3°
p
1 2
A f p (x) + B ~ g(x) for X ~X ~X ,

In particular A fP(xl +B~O so that f (xl >

p
o.
Hence by continuity, f (x) ~ 0 in some right neighborhood of i.
p
Combining all this information we see that 3 x> x such that all
f (x) ~ o, But by 1°, this implies g(x) ~ o, a contradiction. q.e.d.

Corolla!Y: If all fi(x1 ) > 0 and g(x1 ) co, then g(x) ~ 0 for all x.
The proof of theorem l is now easy.
~: Choose M as in lemma 4 and N > 0 arbitrarily. Consider

the function

·over xe and .r.xl N}

~- (x X~ 0 Iii

ue ~= [u u;:; 0 and Eu1 Iii M}

By lemma 1 and lemma 4 :l X0 € ~,u € ~

0 such that

Max 'f(x, uo) =min 'f(xo' u) s: 0

xe:~ ue:~

302
 -10-

Thus 'f (x, u0 ) :; 0 for all x E ~·

The same kind of compactness argument as in lemma 2 is now used to
complete the proof:

Choose N0 j ~ and an associated sequence (u0 (n)} c ~ (it is

essential to observe that M is independent of N) such that

lf(x, u0 (n)) :; 0 for all x E KN

n

By the compactness of ~ j a subsequence (u0 (n ) ) converging

to u0 , say. Since f(x, u) is a continuous function of u and

l1i t P = (x : x ~ 0] we have 'P(x, u0 ) ~ 0 for all x ~ 0 q.e.d.
ni
3· Applications
Definition: Let g1 (x), ••. , gh(x) be any set of functions defined

on a set K. A point x0 E K will be called a mini.Jnal point of g1 , ..• , s;11

over K if for all x E K it is false that for all j gj(x) < gj{x0 ).

A point x0 will.be called an essential minimal point of g1 , ••• , ~ over

K if it is minimal and if the deletion of any gj{x) will cause it to fail
to be minimal. A point x0 will be called a strictly minimal point of

g1 , ••• , Bh over K if for all x, if all gj(x) ~ gj(x0 ) then all

gj{x) = gj(x 0 ),
Theorem 2. Let r 1 (x), ••• , fk(x) and g1 (x), ••• , ~(x) be real
valued continuous functions Which satisfy conditions 2° and 3° of theorem 1.
Let K be the set [x: fi(x) ~ 0 i =l 1 ••• ,·k}. Let x0 be an essential

minimal point of g1 {x), ••• , Bh(x) over K.

Assertion: x 0 is a strictly minimal point of g1 (x), ••• , ~{x)

over K.

303
 -11-

~: The functions fi (xL i = 1, ••. , k and gj(x) - g (xo)

j "' 1, •• ' J h satisfy all the hypotheses of theorem 1. Hence by lemma 2

~ vl, ... , yh ~ 0, LVi =1 such that

If for some j, vj "'0 1 then th&t gj(x) may be deleted and x0 will,

by the above inequality, remain minimal. Hence all vj > 0. But then,

again by the above inequality, if for any x E K all gj(x) ~ gj(x 0 ),

q_.e.d.

We now proceed to the last theorem, the equivalence of a constrained

minimum problem and a certain saddle value problem.

Theorem 3: Let f 1 (x), ..• , fk(x) and g1 (x), ••• , Bt(x) be con-

tinuous real valued functions of x which satisfy conditions 2° and 3°

of theorem 1. Let K = [x : f. (x) ;, 0

l
i = 1, ···J k}
0
Assertion: X is a minimal point for gl (x)' ' . 'J gh{x) over K
0 0 0 0
i f and only if 3v~, •• • , vh f: o, .l:vj = 1, and ~J .. • , ' \ ~ 0 such

that the function

(1)

for all x ~ 0. and u ~ 0, In other words, 'f(x, u) has a saddle

point at (x0 , u 0 ).

~: Suppose x0 is a minimal point for the gj(x) over K. Then

the functions fi(x) i = 1, ••• , k and gj(x) - gj(x 0 ) j = 1, •.. , h

By lemma 2 we choose

304
 -12-

so that tv~(gj(x) - gj(x 0 )) ~ 0 for all x E K. Write g(x)

tv~gj(x).
Now suppose f 1 (x 0 ) = ... = f~(x 0 ) = 0 and f~+ 1 (x 0 ), ••• , fk(x 0 ) > 0.

By lemma 5 g(x) ~ g(x 0 ) for all x such that f 1 (x), ... , f~(x) £ 0. (In

particular, i f all f.(x 0 ) > 0, then g(x) ~ g(x0 ) for all x). Hence by
J.

theorem 1 ju~, •. ,, u~;;;. 0 such that

Since f.(x 0 ) = 0 for i = 11 ••• , ~ this may be rewritten as

l.

= ~ = o.
0 0
for all x ~ 0 and all u 1 ;, 0, i = 11 ••• , ~· Now take u~+l

Then

for all x ~ 0 and all u "' (u 1 ••• 1 u ) ~ 0 q.e.d.

Conversely, suppose (1) is satisfied for some v0 and u° From

the first and last members of the inequality, we find, setting u =0

Hence if x e K, gj(x) < gj(x 0 ) for all j is impossible• This completes

the proof,

305
 -13-

References

Kakutani, S., 1. "A Generalization of Brouwer's Fixed Point Theorem,"

Duke Mathematical Journal, Volume 8 (1941), pp 457-459.

Kuhn, H. E., and A. W. Tucker l. "Non-Linear Progranuning," RAl'll Publi-

cation LOGS 107 (August 28, 1950).

von Neumann, J. 1 l. "Uber ein okonomisches Gleichungs-system und eine

Verallgemeinerung des Brouwerschen Fixpunktsatzes,"
Erg. Math. Kollclau., Volwne B (1937), pp 73-83.

Peixoto, M. M., 1. "Generalized Convex Functions and Second Order

Differential Inequalities," Bulletin of American
Math~natical Society, Volume 55 No.6 (June 1~9).

306