Andreani2010 - Constant-Rank Condition and Second-Order Constraint Qualification
Andreani2010 - Constant-Rank Condition and Second-Order Constraint Qualification
DOI 10.1007/s10957-010-9671-8
1 Introduction
We are concerned with the general nonlinear programming problems with equality
and inequality constraints in the form
Communicated by F. Giannessi.
This research was supported by PRONEX-Optimization 76.79.1008-00, FAPESP (Grant
01-04597-4) and CNPq.
R. Andreani
Department of Applied Mathematics, IMECC-UNICAMP, University of Campinas, CP 6065,
13081-970 Campinas, SP, Brazil
e-mail: [email protected]
C.E. Echagüe
Department of Mathematics, FCE, University of La Plata, CP 172, 1900 La Plata, Bs. As., Argentina
e-mail: [email protected]
m
p
∇f (x̂) + λi ∇hi (x̂) + μi ∇gi (x̂) = 0,
i=1 i=1
μi ≥ 0, μi gi (x̂) = 0, i = 1, . . . , p. (2)
m
p
l(x, λ, μ) = f (x) + λi hi (x) + μi gi (x).
i=1 i=1
not change. There are simple examples showing that CRCQ neither implies nor is
implied by MFCQ; see [1]. More recently, the constant positive linear dependence
(CPLD) condition was defined in [7] and it was established as a first-order constraint
qualification in [8]. CPLD is weaker than CRCQ and MFCQ, see [8]. We can find
other first-order constraint qualifications in the literature. Some of them are, in order
of weakness: quasinormality defined by Hestenes in [9], pseudonormality [10, 11],
Abadie [12] and Guignard [13]. Guignard condition is the weakest first-order con-
straint qualification for differentiable problems as shown in [14]. When a first-order
constraint qualification holds, it is possible to think in terms of KKT multipliers and
efficient algorithms based on duality ideas can be defined. The discovery of new
and weaker first-order constraint qualifications and necessary optimality conditions
is an open issue in nonlinear optimization. Recently [15, 16], a necessary optimality
condition and a constraint qualification were defined by means of the image space
analysis [17].
Second-order necessary optimality conditions are important because they take into
account the curvature of the Lagrangian function over the set of feasible directions.
The desirable second-order condition that combines the objective function with the
constraints is the KKT condition plus the strong second-order necessary condition
(SSONC): given a feasible point x̂ of the problem (1), there are vectors λ ∈ Rm , μ ∈
Rp such that condition (2) holds at x̂ and SSONC:
m
p
d ∇ f (x̂) +
T 2
λi ∇ hi (x̂) +
2
μi ∇ gi (x̂) d ≥ 0,
2
(3)
i=1 i=1
where
A+ (x̂) = {j ∈ A(x̂) : μj > 0}, A0 (x̂) = {j ∈ A(x̂) : μj = 0}.
Condition (3) says that the Hessian of the Lagrangian function at (x̂, λ, μ), restricted
to the tangent subspace T̃ (x̂), is positive semidefinite.
Unfortunately, the combination KKT + SSONC is not always a second-order nec-
essary optimality condition. It is true that, under an appropriate constraint qualifica-
tion, it is possible to show that a minimum point of the problem (1) verifies KKT +
SSONC.
Observe that, if a feasible point x̂ is a KKT point, then
Thus, the tangent subspace in SSONC depends on the objective function of the prob-
lem.
J Optim Theory Appl
Some of the second-order practical algorithms (see for example [18]) take into ac-
count the analysis of the Hessian of the Lagrangian function in the following tangent
subspace:
Clearly, for a given feasible point x, T (x) ⊆ T̃ (x) and T (x) does not depend on the
objective function. By considering T (x), we can define another second-order condi-
tion that combines the objective function and the constraints—KKT condition plus
the weak second-order necessary condition (WSONC)—given a feasible point x̂ of
the problem (1), there are vectors λ ∈ Rm , μ ∈ Rp such that condition (2) holds at x̂
and WSONC:
m
p
d T ∇ 2 f (x̂) + λi ∇ 2 hi (x̂) + μi ∇ 2 gi (x̂) d ≥ 0,
i=1 i=1
before, the only condition that can be a strong or a weak second-order constraint
qualification is the constant-rank condition. Here, we prove that CRCQ is, in fact,
a strong second-order constraint qualification. This is the main goal of the present
work.
In the article where the CRCQ was defined [1], the condition was used in the con-
text of nonconvex mathematical programming problems under general perturbation.
In the last years, CRCQ has been used to obtain theoretical results with practical rel-
evance in the context of bilevel problems; see for example [23]. In [24], CRCQ has
been used to achieve global and superlinear convergence of an infeasible interior-
point algorithm for monotone variational inequality problems. In [25], an augmented
Lagrangian method with convergence under a weaker constraint qualification (CPLD)
was defined. This convergence result implies convergence under CRCQ. In [26], the
CRCQ was used to investigate the properties of the parametric set defined by the
equality and inequality constraints. The author of [26] shows that, in the absence of
parameters, the CRCQ implies that the Mangasarian-Fromovitz constraint qualifica-
tion holds in some alternative expression of the feasible set.
As mentioned before, it is important, from the practical point of view, to define
weak second-order constraint qualifications. We consider the weak constant-rank
condition introduced in [27] to define a new weak second-order constraint qualifi-
cation. We believe that this new second-order constraint qualification can be used in
the future in the study of problems having complementarity constraints.
This paper is organized as follows. In Sect. 2, we state the main definitions. In
Sect. 3, we prove that the constant-rank constraint qualification is a strong second-
order constraint qualification. Also in this section, we prove that a generalization of
the second-order constraint qualification introduced in [27] implies WSONC for any
KKT multiplier vector. The conclusions are given in Sect. 4.
2 Definitions
Definition 2.1 (Ref. [1]) Given a family of differentiable functions {fi (x) : i =
1, . . . , r}, fi : Rn → R, we say that the constant-rank condition holds at x ∗ if and
only if, for any subset K ⊂ {i ∈ {1, . . . , r} : fi (x ∗ ) = 0}, the family of gradients
{∇fi (x)}i∈K
Definition 2.2 (Ref. [1]) Given the family of differentiable functions {hi (x) : i =
1, . . . , m; gi (x) : i = 1, . . . , p} associated with problem (1), we say that a feasible
point x ∗ ∈ satisfies the constant-rank constraint qualification (CRCQ) if and only
if the constant-rank condition holds at x ∗ .
J Optim Theory Appl
3 Main Results
Proposition 3.1 (Ref. [1]) Let {fi (x); i ∈ K} be a family of differentiable functions
on Rn such that the Jacobian matrix (∇fi (x))i∈K is of constant rank in a neighbor-
hood of x ∗ . Define the linear subspace
E = {y ∈ Rn : ∇fi (x ∗ )T y = 0, i ∈ K}.
Remark 3.1 As it was mentioned in [1], this result is a special case of the constant-
rank theorem in Malliavin [28]. Observe that the hypothesis required is not the
constant-rank condition but the constant rank of the Jacobian matrix in a neighbor-
hood of the point. This is the hypothesis that appears in the constant-rank theorem in
Malliavin.
Proposition 3.2 is a consequence of Proposition 3.1 and it was used in [1] to prove
that CRCQ is a first-order constraint qualification. It will be rewritten in a new form
that will be more useful for us to prove the strong second-order necessary condi-
tion. The second part of the proposition was not stated in [1] and it can be deduced
straightforwardly from the proof made in it. We will explain it here for completeness.
there exists some arc t → ξ(t), t ∈ (0, t¯), t¯ > 0, such that ξ(t) ⊂ and
∗
(a) limt→0+ ξ(t) = x ∗ , limt→0+ ξ(t)−x
t = y,
(b) for all j ∈ A(x ∗ ) such that ∇gj (x ∗ )T y = 0 then gj (ξ(t)) = 0.
Proof Let us suppose that A(x ∗ ) = {1, . . . , r} and define fi (x) = hi (x),
∀i = 1, . . . , m, fm+i (x) = gi (x), ∀i = 1, . . . , r. Take K = {i ∈ {1, . . . , m + r} :
∇fi (x ∗ )T y = 0} and consider the linear subspace E = {d ∈ Rn :
∇fi (x ∗ )T d = 0, i ∈ K}. Since the constant-rank constraint qualification holds at x ∗ ,
according to the hypothesis, we can use Proposition 3.1. Thus, there exists some local
diffeomorphism φ verifying items (i), (ii) and (iii) of Proposition 3.1.
Define the arc ξ(t) by
ξ(t) = φ −1 (x ∗ + ty),
for t > 0 sufficiently small. Then, by continuity and conditions (i) and (ii) of Propo-
sition 3.1, we have that
ξ(t) − x ∗
lim ξ(t) = x ∗ , lim = y.
t→0+ t→0+ t
fj (ξ(t)) = fj (φ −1 (x ∗ + ty)) = fj (φ −1 (x ∗ )) = fj (x ∗ ) = 0.
Thus, if j ∈ A(x ∗ ) is such that ∇gj (x ∗ )T y = 0, then we have that gj (ξ(t)) = 0 for
t > 0 small enough. This proves that (b) holds.
Let us prove that the arc is feasible. For i ∈ A(x ∗ )\K, we have that ∇fi (x ∗ )T d <
0. Thus,
A similar argument, based on the continuity of each fi (x) would show that, for i ∈
{1, . . . , p}\A(x ∗ ), there exists t¯2 > 0 sufficiently small such that
Thus, for t¯ = min{t¯1 , t¯2 }, we have that ξ(t) ⊂ , 0 < t < t¯.
In the next theorem, we prove that, if a minimum point verifies the constant-rank
constraint qualification, then the strong second-order necessary condition holds for
any KKT multiplier.
J Optim Theory Appl
Theorem 3.1 Let x ∗ be a minimum point of problem (1) that verifies the constant-
rank constraint qualification. Then, for any KKT multiplier vector (λ∗ , μ∗ ) ∈ Rm+p ,
x ∗ verifies the strong second-order necessary optimality condition.
Proof The existence of a set of KKT multipliers at x ∗ was clearly established in [1].
Let us prove that the strong second-order necessary condition is verified for any of
those multipliers. Let (λ, μ) ∈ Rm+p be any fixed KKT multiplier.
Take a direction y ∈ T̃ (x ∗ ). Then, y verifies
∇hi (x ∗ )T y = 0, i = 1, . . . , m,
∇gj (x ∗ )T y = 0, j ∈ A+ (x ∗ ),
∇gj (x ∗ )T y ≤ 0, j ∈ A0 (x ∗ ).
Thus,
m T
∇f (x ∗ ) + λi ∇hi (x ∗ ) + μi ∇gi (x ∗ ) y = 0
i=1 i∈A+ (x ∗ )
ξ(t) − x ∗
ξ(0) = x ∗ , ξ (0) = lim = y, gi (ξ(t)) = 0, ∀i ∈ A+ (x ∗ ).
t→0+ t
Define ϕ(t) = f (ξ(t)). Since x ∗ is a minimum point of f in and since
ϕ (0) = ∇f (x ∗ )T y = 0,
then
d2
ϕ (0) = f (ξ(t))|t=0 = y T ∇ 2 f (x ∗ )y + ∇f (x ∗ )T ξ (0) ≥ 0. (5)
dt 2
Furthermore, by the feasibility of the arc and using gi (ξ(t)) = 0, ∀i ∈ A+ (x ∗ ), 0 ≤
t < t¯, we have that
m
R(t) = λi hi (ξ(t)) + μi gi (ξ(t)) = 0, 0 ≤ t < t¯.
i=1 i∈A(x ∗ )
i=1 i∈A(x ∗ )
m T
+ λi ∇hi (x ∗ ) + μi ∇gi (x ∗ ) ξ (0) = 0. (6)
i=1 i∈A(x ∗ )
J Optim Theory Appl
i=1 i∈A(x ∗ )
Remark 3.2 Observe that, given a direction y ∈ T̃ (x ∗ ), it can happens that not all the
gradients of the inequality active constraints verify ∇gi (x ∗ )T y = 0. Because of this,
the hypothesis of constant-rank constraint qualification is necessary.
The existence of the feasible arc ξ(t) shows that the constant-rank constraint qual-
ification can be seen as a natural generalization of the linear independence constraint
qualification. Theorem 3.1 shows that CRCQ captures correctly the geometry of the
tangent subspace T̃ (x ∗ ), just as LICQ does. Observe that, under the CRCQ, the
SSONC condition holds for any KKT multiplier.
Due to the counterexample in [21], most of the new second-order constraint qual-
ifications have the form: MFCQ + some condition, which gives the validity of the
(strong or weak) second-order necessary condition. In this scope, we can mention the
constraint qualifications defined in [29] and [27]. Both conditions imply the existence
of some KKT multiplier vector for which a second-order necessary condition holds.
In [29], the authors proved that there is a KKT multiplier for which the SSONC
condition holds at x ∗ , if x ∗ is a minimum point of (1) verifying the following second-
order constraint qualification:
(i) MFCQ holds at x ∗ .
(ii) The rank of the gradients of the equality and inequality active constraints at x ∗
is m + q − 1, where q is the number of active inequality constraints at x ∗ .
(iii) There exists at most only one index i0 ∈ A(x ∗ ) such that, if (λ, μ) is a KKT
multiplier, then μi0 = 0.
The third condition is a complementarity condition and the authors conjecture that it
is not necessary.
In [27], the authors proved that there is a KKT multiplier for which the WSONC
condition holds at x ∗ , if x ∗ is a minimum point of (1) verifying the following second-
order constraint qualification:
(i) MFCQ holds at x ∗ .
(ii) The weak constant-rank condition (WCR) holds at x ∗ : The rank of the Jacobian
matrix made of the gradients {∇hi (x)}i=1,...,m ∪ {∇gi (x)}i∈A(x ∗ ) does not change
in a neighborhood of x ∗ .
The proof that the last condition is a weak second-order constraint qualification was
achieved using penalty ideas and this constraint qualification was used in the conver-
gence analysis of the second-order augmented Lagrangian method defined in [27].
Using Proposition 3.2, we are able to prove the following theorem that generalizes
the result obtained in [27].
J Optim Theory Appl
Theorem 3.2 Let x ∗ be a minimum point of problem (1) that verifies the KKT con-
dition and the weak constant-rank condition. Then, for any KKT multiplier vector
(λ∗ , μ∗ ) ∈ Rm+p , the weak second-order necessary condition is verified.
Remark 3.3 In Theorem 3.2, we prove that the weak second-order necessary con-
dition holds at a minimum point, under any first-order constraint qualification and
the weak constant-rank condition. The result of this theorem is interesting when we
consider the following special class of mathematical programming problems with
complementarity constraints:
In this kind of problems, the origin (0, 0, 0) is a problematic point for most of the non-
linear programming algorithms, since it only verifies the Guignard constraint qualifi-
cation [13]. We observe that this point verifies the new weak second-order constraint
qualification defined by Guignard + WRC.
Theorem 3.3 Let x ∗ be a minimum point of the following problem with only equality
constraints:
min f (x), s.t. h(x) = 0.
Suppose that x ∗ verifies the weak constant-rank condition. Then, x ∗ verifies the
KKT condition and the second-order necessary condition for any KKT multiplier.
Remark 3.4 Observe that, when the problem has only equality constraints, T̃ (x ∗ ) =
T (x ∗ ) and the weak second-order necessary condition is equivalent to the strong
one. The weak constant-rank condition for equality problems is weaker than the
Mangasarian-Fromovitz constraint qualification (which is, in this case, equivalent to
the linear independence constraint qualification) and weaker than the constant-rank
constraint qualification. Therefore, the constant-rank constraint qualification can be
seen as a generalization of those conditions.
J Optim Theory Appl
4 Conclusions
The constant-rank constraint qualification seems to be a useful tool for the analysis of
convergence of some nonlinear programming methods; see for example [23–25]. The
status of the constant-rank as a first-order constraint qualification was clearly proved
in [1]. In this work, we proved that CRCQ is in fact a strong second-order constraint
qualification. We proved that, under this constraint qualification, a minimum point
verifies the strong second-order necessary condition for any KKT multiplier.
Given the practical importance of the weak second-order necessary condition,
we generalized the second-order constraint qualification defined in [27]. We proved
that, on having a minimum point that verifies any first-order constraint qualification
and the weak constant-rank condition, then the WSONC holds for any KKT mul-
tiplier. WCR as well as MFCQ, were useful for obtaining the convergence of the
second-order augmented Lagrangian method defined in [27]. Throughout this work,
we proved that WCR is a first-order and a second-order constraint qualification when
we consider equality constraint problems. We believe that this fact can be useful
in the future, since WCR can be seen as a generalization of the regularity and the
constant-rank constraint qualification for this kind of problems.
It is still an open issue the challenge of finding new and weaker constraint qual-
ifications, not only of first-order but also of second-order. It is desirable that those
conditions be easily verifiable and possibly be associated from a practical point of
view with the convergence analysis of nonlinear optimization algorithms.
Acknowledgements We are grateful to the Associate Editor Franco Giannessi whose comments helped
us to improve the first version of this paper.
References
1. Janin, R.: Direction derivative of the marginal function in nonlinear programming. Math. Program.
Stud. 21, 127–138 (1984)
2. Karush, W.: Minima of functions of several variables with inequalities as side conditions. Master’s
Thesis, University of Chicago (1939)
3. Kuhn, H.W., Tucker, A.W.: Nonlinear programming. In: Neyman, J. (ed.) Proceedings of the Second
Berkeley Symposium on Mathematical Statistics and Probability, pp. 481–492. University of Califor-
nia Press, Berkeley (1951)
4. Bertsekas, D.P.: Nonlinear Programming, 2nd edn. Athena Scientific, Belmont (1999)
5. Mangasarian, O.L.: Nonlinear Programming. McGraw Hill, Bombay (1969)
6. Mangasarian, O.L., Fromovitz, S.: The Fritz-John necessary optimality conditions in presence of
equality and inequality constraints. J. Math. Analysis Appl. 17, 37–47 (1967)
7. Qi, L., Wei, Z.: On the constant positive linear dependence condition and its application to SQP
methods. SIAM J. Optim. 10, 963–981 (2000)
8. Andreani, R., Martínez, J.M., Schuverdt, M.L.: On the relation between the Constant Positive Linear
Dependence condition and quasinormality constraint qualification. J. Optim. Theory Appl. 125, 473–
485 (2005)
9. Hestenes, M.R.: Optimization Theory—The finite-dimensional case. Wiley, New York (1975)
10. Bertsekas, D.P., Ozdaglar, A.E.: Pseudonormality and a Lagrange multiplier theory for constrained
optimization. J. Optim. Theory Appl. 114, 287–343 (2002)
11. Bertsekas, D.P., Ozdaglar, A.E.: The relation between pseudonormality and quasiregularity in con-
strained optimization. Optim. Methods Softw. 19, 493–506 (2004)
12. Abadie, J.: On the Kuhn-Tucker Theorem. In: Abadie, J. (ed.) Nonlinear Programming. North-
Holland, Amsterdam (1967)
J Optim Theory Appl
13. Guignard, M.: Generalized Kuhn-Tucker conditions for mathematical programming problems in a
Banach space. SIAM J. Control 7, 232–241 (1969)
14. Gould, F.J., Tolle, J.W.: A necessary and sufficient qualification for constrained optimization. SIAM
J. Appl. Math. 20, 164–172 (1971)
15. Moldovan, A., Pellegrini, L.: On regularity for Constrained Extremum Problem. Part 1: Sufficient
optimality conditions. J. Optim. Theory Appl. 142, 147–163 (2009)
16. Moldovan, A., Pellegrini, L.: On regularity for Constrained Extremum Problem. Part 2: Necessary
optimality conditions. J. Optim. Theory Appl. 142, 165–183 (2009)
17. Giannessi, F.: Constrained Optimization and Image Space Analysis, Separation of Sets and Optimality
Conditions, vol. 1. Springer, New York (2005)
18. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: Second-order negative-curvature meth-
ods for box-constrained and general constrained optimization. Comput. Optim. Appl. (2009).
doi:10.1007/s10589-009-9240-y
19. Gould, N.I.M., Toint, Ph.L.: A note on the convergence of barrier algorithms for second-order neces-
sary points. Math. Program. 85, 433–438 (1999)
20. Nocedal, J., Wright, S.J.: Numerical Optimization. Springer, New York (1999)
21. Arutyunov, A.V.: Perturbations of extremum problems with constraints and necessary optimality con-
ditions. J. Sov. Math. 54, 1342–1400 (1991)
22. Anitescu, M.: Degenerate nonlinear programming with a quadratic growth condition. SIAM J. Optim.
10, 1116–1135 (2000)
23. Dempe, S.: Foundations of Bilevel Programming, Nonconvex Optimization and its Applications Se-
ries, vol. 61. Kluwer, Amsterdam (2002)
24. Ralph, D., Wright, S.: A superlinear infeasible-interior-point algorithm for monotone complementar-
ity problems. Math. Oper. Res. 21(4), 815–838 (1996)
25. Andreani, R., Birgin, E.G., Martínez, J.M., Schuverdt, M.L.: On augmented Lagrangian methods with
general lower-level constraints. SIAM J. Optim. 18, 1286–1309 (2007)
26. Lu, S.: Implications of the constant rank constraint qualification. Math. Program. A (2009).
doi:10.1007/s10107-009-0288-3
27. Andreani, R., Martínez, J.M., Schuverdt, M.L.: On second-order optimality conditions for nonlinear
programming. Optimization 56, 529–542 (2007)
28. Malliavin, P.: Geometrie differentielle intrinsique. Herman, Paris (1974)
29. Baccari, A., Trad, A.: On the classical necessary second-order optimality conditions in the presence
of equality and inequality constraints. SIAM J. Optim. 15, 394–408 (2004)
30. Penot, J.P.: A new constraint qualification condition. J. Optim. Theory Appl. 48(3), 459–468 (1986)