Short Tutorial: Getting Started With Ipopt in

90 Minutes
Andreas Wachter1
IBM T.J. Watson Research Center
Department of Business Analytics and Mathematical Sciences
1101 Kitchawan Road, Yorktown Heights, NY 10598, USA
[email protected]

Abstract. Ipopt is an open-source software package for large-scale nonlinear optimization. This tutorial gives a short introduction that should
allow the reader to install and test the package on a UNIX-like system,
and to run simple examples in a short period of time.
Keywords. Nonlinear Optimization, Tutorial, Ipopt, COIN-OR

Introduction

Ipopt is an open-source software package for large-scale nonlinear optimization.

It can be used to address general nonlinear programming problems of the form
min f (x)

(1a)

s.t. g L g(x) g U
xL x xU ,

(1b)
(1c)

xRn

where x Rn are the optimization variables with lower and upper bounds,
xL (R {})n and xU (R {+})n , f : Rn R is the objective
function, and g : Rn Rm are the constraints. The functions f (x) and g(x)
can be linear or nonlinear and convex or non-convex, but should be sufficiently
smooth (at least once, ideally twice continuously differentiable). The constraints,
g(x), have lower and upper bounds, g L (R {})m and g U (R {+})m .
Note that equality constraints of the form gi (x) = gi can be specified by setting
giL = giU = gi .
Such optimization problems arise in a number of important engineering, financial, scientific, and medical applications, ranging from the optimal control of
industrial processes (e.g., [1]) and the design of digital circuits (e.g., [2]) to portfolio optimization (e.g., [3]), from parameter identification in systems biology
(e.g., [4]) to hyperthermia cancer treatment planning (e.g., [5]).
Ipopt implements an interior-point line-search filter method; the mathematical details of the algorithm can be found in several publications [6,7,8,9,10].
This approach makes Ipopt particularly suitable for large problems with up

Dagstuhl Seminar Proceedings 09061

Combinatorial Scientific Computing
http://drops.dagstuhl.de/opus/volltexte/2009/2089

A. W
achter

to millions of variables and constraints, assuming that the Jacobian matrix of

constraint function is sparse, but also small and dense problems can be solved
efficiently. It is important to keep in mind that the algorithm is only trying to
find a local minimizer of the problem; if the problem is nonconvex, many stationary points with different objective function values might exist, and it depends
on the starting point and algorithmic choices which particular one the method
converges to.
In general, the computational effort during the optimization with Ipopt is
typically concentrated in the solution of linear systems, or in the computation
of the problem functions and their derivatives, depending on the particular application. With respect to both these tasks, research in Combinatorial Scientific
Computing is of central importance: The KKT system is a saddle point problem; see [11] for a survey of the recent developments in this area, as well as
progress in weighted graph matchings [12,13,14] and parallel partitioning tools,
such as ParMetis[15], Scotch[16], and Zoltan[17]. Furthermore, the computation of derivatives can be facilitated by automatic differentiation tools, such as
ADIFOR[18], ADOL-C[19] and OpenAD[20].
Optimization problems can be given to Ipopt either by using a modeling language, such AMPL1 or GAMS2 , which allow one to specify the mathematical
problem formulation in an easily readable text format, or by writing programming code (C++, C, Fortran 77, Matlab) that computes the problem functions
f (x) and g(x) and their derivatives.
This document provides only a short introduction to the Ipopt package. Detailed information can be found on the Ipopt home page www.coin-or.org/Ipopt
and from the Ipopt documentation available with the source code3 . The instruction here are for Linux (or UNIX-like systems, including Cygwin); if you want
to use Ipopt with the Windows Developer Studio instructions see the Ipopt
documentation.
The remainder of this tutorial is structured as follows: After presenting a
motivating challenging example application in Section 2, easy-to-follow installation instruction are provided in Section 3, which allow the reader to start
experimenting with the code in Section 4. Section 5 gives some mathematical
background of the underlying algorithm, providing the basis to explain the output (Section 6) and algorithmic options (Section 7). Some advise regarding good
modeling practices is given in Section 8. Finally, Section 9 discusses how Ipopt
can be used from programming code, and a coding exercise is provided for the
interested reader.

Example Application: PDE-Constrained Optimization

One class of optimization problems that give rise to very large and sparse nonlinear optimization problems is the optimization of models described by partial1
2
3

www.ampl.com
www.gams.com
Also available online: www.coin-or.org/Ipopt/documentation

Ipopt in 90 Minutes

Fig. 1. Heated Tumor (heated in red) In Hyperthermia Treatment Planning.

differential equations (PDEs). Here, the unknowns are functions defined on

a subset of R2 or R3 that are required to satisfy one or more PDEs, such as
a temperature profile obeying a heat transfer equation. The degrees of freedom
stem from a finite set of control parameters (e.g., intensities of a small number of
microwave antennas) or from another function defined over in the full domain of
the PDE or its boundary (e.g., material properties within the body conducting
the heat, or a controllable temperature profile at the boundary).
There are a number of ways to tackle such problems. Ignoring many subtle
details, we consider here the reformulation of the originally infinite-dimensional
problem into a discretized finite-dimensional version. In this process, the domain
of the PDE is discretized into a finite number of well-distributed points, and
the new optimization variables are the values of the original functions at those
points. Furthermore, the original PDE is then expressed as a set of constraints
for each such point, where the differential operators are approximated based on
the function values at neighboring points, e.g., by means of finite differences.
In this way, each of those constraints involves only a small number of variables,
leading to sparse derivative matrices. At the same time, the approximation error
made by discretizing the original problem is reduced by increasing the number of
discretization points, and it is therefore very desirable to solve large instances of
the reformulation. This makes this application very suitable for an interior-point
algorithm such as Ipopt.
A specific medical example application is hyperthermia treatment planning:
Here, a patient is exposed to microwave radiation, emitted by several antennas,
in order to heat a tumor; see Figure 1. The purpose of this procedure is to
increase the tumors susceptibility to regular radiation or chemotherapy.

A. W
achter

Mathematically, the problem can be formulated as this PDE-constrained optimization problem:

Z
Z
min
(T T ther )2 d +
(T T health )2 d
(2a)
xt

x6t

s.t. (T ) + cb (T T b ) =
2


2
X



u i Ei 




in

(2b)

n T = qconst on
T |/t T lim .

(2c)
(2d)

Here, is the considered region of the patients body, t the domain of

tumor tissue, and T the temperature profile. The constant is the heat diffusion
coefficient, cb the specific heat of blood, w(T ) the temperature-dependent perfusion, T b the arterial blood temperature, qconst the human vessel blood flux,
the electrical conductivity, ui the complex-valued control of antenna i, and
Ei the corresponding electrical field. The PDE is given in (2b) with a Neumann
boundary condition (2c). The goal of the optimization is to find optimal controls
ui in order to minimize the deviation from the desired temperature inside and
outside the tumor (T ther and T health , respectively), as expressed in the objective function (2a). To avoid excessively high temperature in the healthy tissue,
the bound constraint (2d) is explicitly included.
Christen and Schenk[21] recently solved an instance of this problem with real
patient data and 12 antennas, resulting in a discretized NLP with n = 854, 499
variables. The KKT matrix (see (6) below), which is factorized to compute the
optimization steps, had more than 27 million nonzero elements. This is a considerably large nonlinear optimization problem, which required 182 Ipopt iterations
and took 48 hours on a 8-core Xeon Linux workstation, using the parallel linear
solver Pardiso4 .

Installation

The Ipopt project is hosted by the COIN-OR Foundation5 , which provides a

repository with a number of different operations-research related open-source
software packages. The Ipopt source code is distributed under the Common
Public License (CPL) and can be used for commercial purposes (check the license
for details).
In order to compile the Ipopt package, you will need to obtain the Ipopt
source code from the COIN-OR repository. This can be done by downloading a
compressed archive6 or preferably by using the subversion7 repository management tool, svn; the latter option allows for a convenient upgrade of the source
4
5
6
7

www.pardiso-project.org
www.coin-or.org
look for the latest version in www.coin-or.org/download/source/Ipopt
subversion.tigris.org

Ipopt in 90 Minutes

1
2
3
4
5
6
7
8
9
10

>
>
>
>
>
>
>
>
>
>

cd $MY_IPOPT_DIR
svn co https://project.coin-or.org/svn/Ipopt/stable/X.Y Ipopt-source
cd Ipopt-source/ThirdParty/Blas
./get.Blas
cd ../Lapack
./get.Lapack
cd ../Mumps
./get.Mumps
cd ../ASL
./get.ASL

>
>
>
>
>
>
>

cd $MY_IPOPT_DIR
mkdir build
cd build
../Ipopt-source/configure
make
make test
make install

11
12
13
14
15
16
17
18

Fig. 2. Installing Ipopt (Basic Version)

code to newer versions. In addition, Ipopt requires some third-party packages,

namely
BLAS (Basic Linear Algebra Subroutines)
LAPACK (Linear Algebra PACKage)
A symmetric indefinite linear solver (currently, interfaces are available to the
solvers MA27, MA57, MUMPS, Pardiso, WSMP)
ASL (AMPL Solver Library)
Refer to the Ipopt documentation for details of the different components.
Assuming that the shell variable MY IPOPT DIR contains the name of a directory that you just created for your new Ipopt installation, Figure 2 lists
the commands to obtain a basic version of all the required source code and to
compile the code; you will need to replace the string X.Y with the current
stable release number which you find on the Ipopt homepage (e.g., 3.6). The
command in the second line downloads the Ipopt source files, including documentation and examples, to a new subdirectory, Ipopt-source. The commands
in lines 310 visit several subdirectories and run provided scripts for downloading third-party source code components8 . The commands in lines 1215 run the
configuration script, in a directory separate from the source code (this allows
you to compile different versions, such as in optimized and debug mode, and
to start over easily); make sure it says configure: Main configuration of
8

Note that it is your responsibility to make sure you have the legal right to use this
code; read the INSTALL.* files in the ThirdParty subdirectories.

A. W
achter

Ipopt successful at the end. Finally, the last three lines compile the Ipopt
code, try to execute it (check the test output for complaints!), and finally install
it. The AMPL solver executable ipopt will be in the bin subdirectory, the
library in lib and the header files under include/ipopt.
Note that this will give you only a basic version of Ipopt, sufficient for small
to medium-size problems. For better performance, you should consider getting
optimized BLAS and LAPACK libraries, and choose a linear solver that fits
your application. Also, there are several options for the configure script that
allow you to choose different compilers, compiler options, provide precompiled
third-party libraries, etc.; check the documentation for details.

First Experiments

Now that you have compiled and installed Ipopt, you might want to play with
it for a little while before we go into some algorithmic details and the different
ways to use the software.
The easiest way to run Ipopt with example problems is in connection with
a modeling language, such as AMPL. For this, you need to have a version of the
AMPL language interpreter (an executable called ampl). If you do not have
one already, you can download a free student version from the AMPL website9 ;
this version has a limit on the problem size (max. 300 variables and constraints),
but it is sufficient for getting familiar with Ipopt and solving small problems.
Make sure that both the ampl executable, as well as the ipopt solver executable
that you just compiled (located in the bin subdirectory) are in your shells PATH.
Now you can go to the subdirectory Ipopt/tutorial/AmplExperiments of
$MY IPOPT DIR/build. Here you find a few simple AMPL model files (with the
extension .mod). Have a look at such a file; for example, hs71.mod formulates
Problem 71 of the standard Hock-Schittkowski collection[22]:


min4 x(1) x(4) x(1) + x(2) + x(3)
xR

s.t. x(1) x(2) x(3) x(4) 25

(x(1) )2 + (x(2) )2 + (x(3) )2 + (x(4) )2 = 40
1x5
Even though explaining the AMPL language[23] is beyond the scope of this
tutorial, you will see that it is not difficult to understand such a model file since
the syntax is quite intuitive.
In order to solve an AMPL model with Ipopt, start the AMPL interpreter by
typing ampl in the directory where the model files are located. Figure 3 depicts
a typical AMPL session: The first line, which has to be entered only once per
AMPL session, tells AMPL to use Ipopt as the solver. Lines 23 load a model
file (replace the string FILE with the correct file name, such as hs71). The
9

www.ampl.com/DOWNLOADS

Ipopt in 90 Minutes

1
2
3
4
5
6

ampl:
ampl:
ampl:
ampl:

option solver ipopt;

reset;
model FILE.mod;
solve;
[...IPOPT output...]
ampl: display x;

Fig. 3. A typical AMPL session

fourth line runs Ipopt, and you will see the output of the optimizer, including
the EXIT message (hopefully Optimal Solution Found). Finally, you can use
the display command to examine the final values of the variables (replace x
by the name of a variable in the model file). You can repeat steps 26 for different
or modified model files. If AMPL complains about syntax error and shows you
the prompt ampl?, you need to enter a semicolon (;) and start over from line
2. Now you have all the information to find out which of the components in the
final solution for hs71.mod is not correct in x = (1, 5, 3.82115, 1.37941). . . ?
You can continue exploring on your own, using and modifying the example model files; AMPL knows the standard intrinsic functions (such as sin,
log, exp). If you are looking for more AMPL examples, have a look at the
MoreAmplModels.txt file in this directory which has a few URLs for website
with more NLP AMPL models.

The Algorithm

In this section we present informally some details of the algorithm implemented

in Ipopt. The main goal is to convey enough information to explain the output
of the software and some of the algorithmic options available to a user. Rigorous
mathematical details can be found in the publications cited in the Introduction.
Internally, Ipopt replaces any inequality constraint in (1b) by an equality
constraint and a new bounded slack variable (e.g., gi (x) si = 0 with giL si
giU ), so that bound constraints are the only inequality constraints. To further
simplify the notation in this section, we assume here that all variables have only
lower bounds of zero, so that the problem is given as
min f (x)

(3a)

s.t. c(x) = 0
x 0.

(3b)
(3c)

xRn

A. W
achter

As an interior point (or barrier) method, Ipopt considers the auxiliary barrier
problem formulation
minn (x) = f (x)

xR

n
X

ln(xi )

(4a)

i=1

s.t. c(x) = 0,

(4b)

where the bound constraints (3c) are replaced by a logarithmic barrier term
which is added to the objective function. Given a value for the barrier parameter
> 0, the barrier objective function (x) goes to infinity if any of the variables
xi approach their bound zero. Therefore, an optimal solution of (4) will be in
the interior of the region defined by (3c). The amount of influence of the barrier
term depends on the size of the barrier parameter , and one can show, under
certain standard conditions, that the optimal solutions x () of (4) converge
to an optimal solution of the original problem (3) as 0. Therefore, the
overall solution strategy is to solve a sequence of barrier problems (4): Starting
with a moderate value of (e.g., 0.1) and a user-supplied starting point, the
corresponding barrier problem (4) is solved to a relaxed accuracy; then is
decreased and the next problem is solved to a somewhat tighter accuracy, using
the previous approximate solution as a starting point. This is repeated until a
solution for (3), or at least a point satisfying the first-order optimality conditions
up to user tolerances, has been found. These first-order optimality conditions for
the barrier problem (4) are given as
f (x) + c(x)y z = 0
c(x) = 0
XZe e = 0
x, z 0

(5a)
(5b)
(5c)
(5d)

with = 0, where y Rm and z Rn are the Lagrangian multipliers for the

equality and bound constraints, respectively. Furthermore, we introduced the
notation X = Diag(x), Z = Diag(z) and e = (1, . . . , 1)T .
It is important to note that not only minimizers for (3), but also some maximizers and saddle points satisfy (5), and that there is no guarantee that Ipopt
will always converge to a local minimizer of (3). However, the Hessian regularization described below aims to encourage the algorithm to avoid maximizers
and saddle points.
It remains to describe how the approximate solution of (4) for a fixed
is
computed. The first-order optimality conditions for (4) are given by (5) with
=
, and a Newton-type algorithm is applied to the nonlinear system of
equations (5a)(5c) to generate a converging sequence of iterates that always
strictly satisfy (5d). Given an iterate (xk , yk , zk ) with xk , zk > 0, the Newton
step (xk , yk , zk ) for (5a)(5c) is computed from





(xk ) + c(xk )yk
xk
Wk + Xk1 Zk + x I c(xk )
=

(6)
c(xk )
yk
c(xk )T
0

Ipopt in 90 Minutes

with x = 0 and zk = Xk1 e zk Xk1 Zk xk . Here, Wk is the Hessian of

the Lagrangian function, i.e.,
2

Wk = f (xk ) +

m
X

(j)

yk 2 c(j) (xk ).

(7)

j=1

After the Newton step has been computed, the algorithm first computes maximum step sizes as the largest kx,max , kz,max (0, 1] satisfying
xk + kx,max xk (1 )xk ,

zk + kz,max zk (1 )zk

with = min{0.99, }; this fraction-to-the-boundary rule ensures that the new

iterate will again strictly satisfy (5d). Then a line search with trial step sizes
x
x
k,l
= 2l kx,max , l = 0, 1, . . . is performed, until a step size k,l
is found that
results in sufficient progress (see below) toward solving the barrier problem (4)
compared to the current iterate. Finally, the new iterate is obtained by setting
x
xk+1 = xk + k,l
xk ,

x
yk+1 = yk + k,l
yk ,

zk+1 = zk + kz,max zk

and the next iteration is started. Once the optimality conditions (5) for the
barrier problem are sufficiently satisfied, is decreased.
In order to decide if a trial point is acceptable as new iterate, Ipopt uses
a filter method to decide if progress has been made toward the solution of
the barrier problem (4). Here, a trial point is deemed better than the current
iterate, if it (sufficiently) decreases either the objective function (x) or the
norm of the constraint violation kc(x)k1 . In addition, similar progress has to be
made also with respect to a list of some previous iterates (the filter) in order
to avoid cycling. Overall, under standard assumptions this procedure can be
shown to make the algorithm globally convergent (i.e., at least one limit point
is a stationary point for (4)).
Two crucial points are important in connection with the line search: First,
in order to guarantee that the direction xk obtained from (6) is indeed a
descent direction (e.g., resulting in a degrees of (x) at a feasible iterate), the
projection of the upper left block of the matrix in (6) onto the null space of
c(xk )T should be positive definite, or, equivalently, the matrix in (6) should
have exactly n positive and m negative eigenvalues. The latter condition can
easily be verified after a symmetric indefinite factorization, and if the number
of negative eigenvalues is too large, Ipopt perturbs the matrix by choosing a
x > 0 by a trial-and-error heuristic, until the inertia of this matrix is as desired.
Secondly, it may happen that no acceptable step size can be found. In this
case, the algorithm switches to a feasibility restoration phase which temporarily ignores the objective function, and instead solves a different optimization
problem to minimize the constraint violation kc(x)k1 (in a way that tries to
find the feasible point closest to the point at which the restoration phase was
started). The outcome of this is either that the a point is found that allows the
return to the regular Ipopt algorithm solving (4), or a local minimizer of the
`1 -norm constraint violation is obtained, indicating to a user that the problem
is (locally) infeasible.

10

A. W
achter
Table 1. Ipopt iteration output
col #
1
2
3
4
5
6
7
8
9
10

header
iter
objective
inf pr
inf du
lg(mu)
||d||
lg(rg)
alpha du
alpha pr
ls

meaning
iteration counter k (r: restoration phase)
current value of objective function, f (xk )
current primal infeasibility (max-norm), kc(xk )k
current dual infeasibility (max-norm), kEqn. (5a)k
log10 of current barrier parameter
max-norm of the primal search direction, kxk k
log10 of Hessian perturbation x (--: none, x = 0)
dual step size kz,max
primal step size kx
number of backtracking steps l + 1

Ipopt Output

When you ran the AMPL examples in Section 4, you already saw the Ipopt output: After a short notice about the Ipopt project, self-explanatory information
is printed regarding the size of the problem that is solved. Then Ipopt displays
some intermediate information for each iteration of the optimization procedure,
and closes with some statistics concerning the computational effort and finally
the EXIT message, indicating whether the optimization terminated successfully.
Table 1 lists the columns of the iteration output. The first column is simply
the iteration counter, whereby an appended r indicates that the algorithm
is currently in the restoration phase; the iteration counter is not reset after an
update of the barrier parameter or when the algorithm switched between the
restoration phase and the regular algorithm. The next two columns indicate the
value of the objective function (not the barrier function, (xk )) and constraint
violation. Note that these are not quite the quantities used in the filter line
search, and that therefore you might sometimes see an increase in both from one
iteration to the next. The fourth column is a measure of optimality; remember
that Ipopt aims to find a point satisfying the optimality conditions (5). The
last column gives an indication how many trial points had to be evaluated.
In a typical optimization run, you would want to see that the objective function is going to the optimal value, and the constraint violation and the dual
infeasibility, as well as the size of the primal search direction are going to zero
in the end. Also, as the value of the barrier parameter is going to zero, you will
see a decrease in the number listed in column 5; if the algorithm switches to the
restoration phase a different value of might be chosen. Furthermore, the larger
the step sizes in columns 8 and 9, the better is the usually the progress. Finally,
a nonzero value of x indicates that the iterates seem to be in a region where the
original problem is not strictly convex. If you see nonzero values even at the very
end of the optimization, it might indicate that the algorithm terminated at a
critical point that is not a minimizer (but still satisfies (5) up to some tolerance).

Ipopt in 90 Minutes

11

Ipopt Options

There is a large number of options that can be set by a user to change the
algorithmic behavior and other aspects of Ipopt. Most of them are described
in the Ipopt Options section of the Ipopt documentation. It is also possible to see a list of the options by running the AMPL solver executable as
ipopt --print-options | more. Options can be set in an options file (called
ipopt.opt) residing in the directory where Ipopt is run. The format is simply
one option name per line, followed by the chosen value; anything in a line after
a hash (#) is treated as a comment and ignored. A subset of options can also
be set directly from AMPL, using
option ipopt options "option1=value1 option2=value2 ...";
in the AMPL script before running the solver. To see which particular Ipopt
options can be set in that way, type ipopt -= in the command line.
An effort has been made to choose robust and efficient default values for
all options, but if the algorithm fails to converge or speed is important, it is
worthwhile to experiment with different choices. In this section, we mention
only a few specific options that might be helpful:
Termination: In general, Ipopt terminates successfully if the optimality error, a scaled variant of (5), is below the value of tol. The precise definition of
the termination test is given in [8, Eqn. (5)]. Note that it is possible to control
the components of the optimality error individually (using dual inf tol,
constr viol tol, and compl inf tol). Furthermore, in some cases it might
be difficult to satisfy this desired termination tolerance (due to numerical
issues), and Ipopt might terminate then at a point satisfying looser criteria
that can be controlled with the acceptable * tol options. Finally, Ipopt
will stop if the maximum number of iterations (default 3000) is exceeded;
you can change this using max iter.
Output: The amount of output written to standard output is determined
by the value of print level with default value 5. To switch off all output,
choose 0; to see even the individual values in the KKT matrix, choose 12. If
you want look at detailed output, it is best to create an output file with the
output file option together with file print level.
Initialization: Ipopt will begin the optimization by setting the x components of the iterates to the user-supplied starting point (AMPL will assume
that 0 is the initial point of a variable unless it is explicitly set!). However, as
an interior point method, Ipopt requires that all variables lie strictly within
their bounds, and therefore it modifies the user-supplied point if necessary
to make sure none of the components are violating or are too close to their
bounds. The options bound frac and bound push determine how much distance to the bounds is enforced, depending on the difference of the upper
and lower bounds or the distance to just one of the bounds, respectively (see
[8, Section 3.6]).
The z values are all set to the value of bound mult init val, and the initial
y values are computed as those that minimize the 2-norm of Eqn. (5a) (see

12

A. W
achter

also constr mult init max). Furthermore, mu init determines the initial
value of the barrier parameter .
Problem modification: Ipopt seems to perform better if the feasible set of
the problem has a nonempty relative interior. Therefore, by default, Ipopt
relaxes all bounds (including bounds on inequality constraints) by a very
small amount (on the order of 108 ) before the optimization is started. In
some cases, this can lead to problems, and this features can be disabled by
setting bound relax factor to 0.
Furthermore, internally Ipopt might look at a problem in a scaled way: At
the beginning, scaling factors are computed for all objective and constraint
functions to ensure that none of their gradients is larger than nlp scaling max gradient,
unless a non-default option is chosen for nlp scaling method. The objective
function can be handled separately, using obj scaling factor; it often it
worthwhile experimenting with this last option.
Further options: Ipopts options are tailored to solving general nonlinear optimization problems. However, switching mehrotra algorithm to yes
might make Ipopt perform much better for linear programs (LPs) or convex quadratic programs (QPs) by choosing some default option differently.
One such setting, mu strategy=adaptive, might also work well in nonlinear
circumstances; for many problems, this adaptive barrier strategy seems to
reduce the number of iterations, but at a somewhat higher computational
costs per iteration.
If you are using AMPL, the modeling software computes first and second
derivatives of the problem function f (x) and g(x) for you. However, if Ipopt
is used within programming code, second derivatives (for the Wk matrix
in (7)) might not be available. For this purpose, Ipopt includes the possibility to approximate this matrix by means of a limited-memory quasiNewton method (hessian approximation=limited-memory). This option
makes the code usually less robust than if exact second derivatives are used.
In addition, Ipopts termination tests are not very good at determining in
this situation when a solution is found, and you might have to experiment
with the acceptable * tol options.

Modeling Issues

For good performance, an optimization problem should be scaled well. While it is

difficult to say what that means in general, it seems advisable that the absolute
values of the non-zero derivatives typically encountered are on the same order of
magnitude. In contrast to linear programming where all derivative information
is known at the start of the optimization and does not change, it is difficult
for a nonlinear optimization algorithm to automatically determine good scaling
factors, and the modeler should try to avoid formulations where some non-zero
entries in the gradients are typically very small or very large. A rescaling can
be done by multiplying an individual constraint function by some factor, and by
replacing a variable xi by x
i = c xi for a constant c 6= 0.

Ipopt in 90 Minutes

13

Linear problems are easier to solve than nonlinear problems, and convex
problems are easier to solve than non-convex ones. Therefore, it makes sense to
explore different, equivalent formulations to make it easier for the optimization
method to find a solution. In some cases it is worth introducing extra variables
and constraints if that leads to fewer nonlinearities or sparser derivative matrices.
An obvious example of better modeling is to use y = c x instead of
y/x = c if c is a constant. But a number of other tricks are possible. As a
demonstration consider the optimization problem
minn

x,pR

s.t.

n
X
i=1
n
Y
i=1

xi
pi 0.1;

xi
i

pi
10n

ni=1 ;

x 0,

0 p 1.

Can you find a better formulation, leading to a convex problem with much
sparser derivative matrices? (See Ipopt/tutorial/Modeling/bad1* for the solution)
Ipopt needs to evaluate the function values and their derivatives at all iterates and trial points. If a trial point is encountered where an expression cannot
be calculated (e.g., the argument of a log is a non-positive number), the step
is further reduced. But it is better to avoid such points in the model: As an
interior point method, Ipopt keeps all variables within their bounds (possibly
somewhat relaxed, see the bound relax factor option). Therefore, it can be
useful to replace the argument of a function with a limited range of definition
by a variable with appropriate bounds. For example, instead of log(h(x)), use
log(y) with a new variable y  (with a small constant  > 0) and a new
constraint h(x) y = 0.

Ipopt With Program Code

Rather than using a modeling language tool such as AMPL or GAMS, Ipopt can
also be employed for optimization problem formulations that are implemented in
a programming language, such as C, C++, Fortran or Matlab. For this purposes,
a user must implement a number of functions/methods that provide to Ipopt
the required information:

Problem size [get nlp info] and bounds [get bounds info];
Starting point [get starting point];
Function values f (xk ) [eval f] and g(xk ) [eval g];
First derivatives f (xk ) [eval grad f] and c(xk ) [eval jac g];
P (j)
Second derivatives f 2 f (xk ) + j k 2 c(j) (xk ) [eval h];
Do something with the solution [finalize solution].

14

A. W
achter

In square brackets are the names of the corresponding methods of the TNLP
base class for the C++ interface, which is described in detail in the Ipopt
documentation; the other interfaces are similar. Example code is provided in the
Ipopt distribution. If you used the installation procedures described in Figure 2,
you will find it in the subdirectories of $MY IPOPT DIR/build/Ipopt/examples,
together with Makefiles tailored to your system.
Here is some quick-start information:
For C++, the header file include/ipopt/IpTNLP.hpp contains the base
class of a new class that you must implement. Examples are in the Cpp example,
hs071 cpp and ScalableProblems subdirectories.
For C, the header file include/ipopt/IpStdCInterface.h has the prototypes for the Ipopt functions and the callback functions. An example is in
the hs071 c subdirectory.
For Fortran, the function calls are very similar to the C interface, and example code is in hs071 f.
For Matlab, you first have to compile and install the Matlab interface. To
do this, go into the directory
$MY IPOPT DIR/build/Ipopt/contrib/MatlabInterface/src
and type make clean (there is no dependency check in that Makefile)
followed by make install. This will install the required Matlab files into
$MY IPOPT DIR/build/lib, so you need to add that directory into your Matlab path. Example code is in
$MY IPOPT DIR/build/Ipopt/contrib/MatlabInterface/examples
together with a startup.m file that sets the correct Matlab path.
9.1

Programming Exercise

After you had a look at the example programs, you can try to solve the following
coding exercise included in the Ipopt distribution. In
$MY IPOPT DIR/build/Ipopt/tutorial/CodingExercise you find an AMPL
model file for the scalable example problem
minn

xR

n
X

(xi 1)2

i=1

s.t. (x2i + 1.5xi ai ) cos(xi+1 ) xi1 = 0

1.5 x 0

for

i = 2, . . . , n 1

and subdirectories for each modeling language. In each case, you find directories
1-skeleton: This has the main code structure, but no code for function
evaluation, etc. This code will not compile or run;
2-mistake: This has a running version of the code with all functions implemented. However, mistakes are purposely included;
3-solution: This has a correct implementation of the exercise.

Ipopt in 90 Minutes

15

To do the exercise, copy the content of the 1-skeleton directory to a new

directory as a starting point for your implementation.
This is a scalable example with n 3; it will be easier to start debugging
and checking the results with a small instance. The AMPL model can help to
determine the correct solution for a given n, so that it is easy to verify if your
code is correct. A very useful tool is also Ipopts derivative checker; see the
derivative test option and the corresponding section in the Ipopt documentation. Also, for C++ and C, a memory checker (such as valgrind for Linux)
comes in very handy.
The following is a suggested procedure to tackle the exercise; it refers to
method names in the C++ interface, but also applies to the other programming
languages.
1. Work out derivatives, including the sparsity structure.
2. Implement
problem information (size, bounds, starting point):
get nlp info, get bounds info, get starting point;
code for the objective function value and its gradient:
eval f, eval grad f;
code for the constraint values and the (dense) Jacobian structure:
eval g, eval jac g (structure only - in this first step, pretend it is dense
to make it easier).
3. Run a small instance (e.g., n = 5) with the following options and check the
solution:
jacobian approximation=finite-difference-values
hessian approximation=limited-memory
tol=1e-5.
4. Implement
code for constraint derivatives: eval jac g (now with correct sparsity
pattern and values).
5. Check the first derivatives and verify the solution:
derivative test=first-order
hessian approximation=limited-memory
6. Implement
code for the Hessian of Lagrangian: eval h
7. Check the second derivatives and verify the solution:
derivative test=second-order
If you want to take a short-cut, you may start with the code in 2-mistake
and look for the mistakes. Here, using the derivative checker will be very helpful.
Also, until the derivatives are correct, there is no point in running Ipopt for
many iterations, so you want to set max iter to 1.

16

10

A. W
achter

Not Covered

There are a number of features and options not covered in this short tutorial.
The Ipopt documentation describes some of them, and there is also additional
information on the Ipopt home page. Furthermore, a mailing list is available to
pose question regarding to the use of Ipopt, as well as the bug tracking system.
Links to those resources are available at the Ipopt home page.

Acknowledgments
The author thanks Olaf Schenk and Jon Lee for their comments that improved
the presentation of this tutorial.

References
1. Biegler, L.T., Zavala, V.M.: Large-scale nonlinear programming using IPOPT: An
integrating framework for enterprise-wide dynamic optimization. Computers and
Chemical Engineering 33 (2009) 575582
2. W
achter, A., Visweswariah, C., Conn, A.R.: Large-scale nonlinear optimization in
circuit tuning. Future Generation Computer Systems 21 (2005) 12511262
3. Gondzio, J., Grothey, A.: Solving non-linear portfolio optimization problems with
the primal-dual interior point method. European Journal of Operational Research
181 (2007) 10191029
4. Lu, J., Muller, S., Machne, R., Flamm, C.: SBML ODE solver library: Extensions for inverse analysis. In: Proceedings of the Fifth International Workshop on
Computational Systems Biology, WCSB. (2008)
5. Schenk, O., W
achter, A., Weiser, M.: Inertia-revealing preconditioning for largescale nonconvex constrained optimization. SIAM Journal on Scientific Computing
31 (2008) 939960
6. Nocedal, J., W
achter, A., Waltz, R.A.: Adaptive barrier strategies for nonlinear
interior methods. SIAM Journal on Optimization 19 (2009) 16741693
7. W
achter, A.: An Interior Point Algorithm for Large-Scale Nonlinear Optimization
with Applications in Process Engineering. PhD thesis, Carnegie Mellon University,
Pittsburgh, PA, USA (2002)
8. W
achter, A., Biegler, L.T.: On the implementation of a primal-dual interior point
filter line search algorithm for large-scale nonlinear programming. Mathematical
Programming 106 (2006) 2557
9. W
achter, A., Biegler, L.T.: Line search filter methods for nonlinear programming:
Motivation and global convergence. SIAM Journal on Optimization 16 (2005) 131
10. W
achter, A., Biegler, L.T.: Line search filter methods for nonlinear programming:
Local convergence. SIAM Journal on Optimization 16 (2005) 3248
11. Benzi, M., Golub, G.H., Liesen, J.: Numerical solution of saddle point problems.
Acta Numerica 14 (2005) 1137
12. Olschowka, M., Neumaier, A.: A new pivoting strategy for Gaussian elimination.
Linear Algebra and Its Applications 240 (1996) 131151
13. Duff, I.S., Pralet, S.: Strategies for scaling and pivoting for sparse symmetric
indefinite problems. SIAM Journal on Matrix Analysis and Applications 27 (2005)
313340

Ipopt in 90 Minutes

17

14. Schenk, O., Gartner, K.: On fast factorization pivoting methods for sparse symmetric indefinite systems. Electronic Transactions on Numerical Analysis 23 (2006)
158179
15. Karypis, G., Kumar, V.: Parallel multilevel k-way partitioning for irregular graphs.
SIAM Review 41 (1999) 278300
16. Pellegrini, F.: SCOTCH 5.0 Usess guide. Technical report, LaBRI, Universite
Bordeaux I (2007) Available at http://www.labri.fr/pelegrin/scotch.
17. Devine, K., Boman, E., Heaphy, R., Hendrickson, B., Vaughan, C.: Zoltan data
management services for parallel dynamic applications. Computing in Science and
Engineering 4 (2002) 9096
18. Bischof, C., Carle, A., Hovland, P., Khademi, P., Mauer, A.:
ADIFOR 2.0 Users Guide.
Technical Report ANL/MCS-TM-192, Argonne National Laboratory, Argonne, IL, USA (1995) Available at
http://www.mcs.anl.gov/research/projects/adifor.
19. Griewank, A., Juedes, D., Utke, J.: ADOL-C: A package for the automatic differentiation of algorithms written in C/C++. ACM Transactions on Mathematical
Software 22 (1996) 131167
20. Utke, J., Naumann, U., Fagan, M., Tallent, N., Strout, M., Heimbach, P., Hill, C.,
Wunsch, C.: OpenAD/F: A Modular, Open-Source Tool for Automatic Differentiation of Fortran Codes. Technical Report AIB-2007-14, (Department of Computer
Science, RWTH Aachen) Available at http://www.mcs.anl.gov/OpenAD.
21. Christen, M., Schenk, O.: (2009) Personal communication.
22. Hock, W., Schittkowski, K.: Test examples for nonlinear programming codes. Lecture Notes in Economics and Mathematical Systems 187 (1981)
23. Fourer, R., Gay, D.M., Kernighan, B.W.: AMPL: A Modeling Language For Mathematical Programming. Thomson Publishing Company, Danvers, MA, USA (1993)