Routine Implementing The Simplex Method - NUMERICAL RECIPES
Routine Implementing The Simplex Method - NUMERICAL RECIPES
Our routine implementing the simplex method is based on the algorithm given
in [1]. The constraint matrix A is input as an m n sparse matrix sa of type
NRsparseMat (÷2.7). Only the coefficients in the equations like (10.10.3) and (10.10.4)
are input; the unit vectors corresponding to the m slack and artificial variables are
generated internally. In addition to nonnegative and zero variables, the routine can
also handle free variables. These are variables that are unrestricted, taking on any
value in . 1; 1/. A free variable should always be a basic variable in the optimal
solution, since you can improve the value of the objective function by exchanging
some restricted variable for an unrestricted one. The routine moves all the free vari-
ables into the basis as part of a phase zero. The type of the variables is input in
the array initvars[1..n+m]. A value of 0 signifies a zero variable (typically an
artificial variable for an equality constraint, but zero variables among the original
independent variables are also allowed); a value of 1 denotes a nonnegative variable
(independent variable or slack variable for an inequality); and 1 is used for free
variables. The coefficients of the objective function are input in obj[0..n+m]. A
constant c0 can be included in (10.10.16), and this is stored in obj[0]. In most
problems, obj[n+1..n+m] will be zero. The solution is output in u[0..n+m]:
u[0] contains the optimal value of the objective function, u[1..n] the values of
x1 ; : : : ; xn at the minimum, and u[n+1..n+m] the values of the slack variables. In-
termediate results are printed when the variable verbose is set to true, which can
be helpful for diagnosing problems. Set it to false to suppress this printing.
For efficient access, the routine initially copies the constraint matrix sa into
an array a of its n columns. Again, only the nonzeros are stored, together with
the corresponding row numbers. Every NREFAC steps the LU decomposition of the
basis is carried out from scratch. This is because the Bartels-Golub updates are
actually stored as a sequence of transformations to the original LU decomposition.
Applying the accumulated updates becomes expensive after too many steps, and also
the accuracy can deteriorate. The default is NREFAC D 50, which is satisfactory for
most problems.
Note that the Bartels-Golub update requires not the incoming column ak , but
L 1 ak . Accordingly, the routine computes the quantity w D AB 1 ak D U 1
L 1 ak of equation (10.10.25) in two steps: first L 1 ak is computed and stored
for the update, and then w is computed by multiplying by U 1 .
The routine incorporates implicit scaling. Column scale factors scale[1..n]
and row scale factors scale[n+1..n+m] are first determined so that the maximum
norm is one for each row and each column of the matrix
1
Copyright 2007 Numerical Recipes Software
2 Routine Implementing the Simplex Method
scaleŒi
a
xik D aik i D 1; : : : ; m k D 1; : : : ; n (1)
scaleŒn C i
Then the pivot selection rules are modified so that applying the modified rules to the
unscaled matrix gives the same results as the unmodified rules to the scaled matrix.
Most of the time, performance is improved with scaling. However, since no perfect
scaling algorithm is known, occasionally no scaling works better. To turn off scaling,
just set scale[1..n+m] D 1 in function scaleit.
For coding ease, the routine implements the minimum ratio test (10.10.25) by
computing the inverse of the step length, i.e., the quantity .AB 1 ak /i =xBi . The
various tests for zero quantities in the pivot selection and in other parts of the code are
replaced by tests with appropriate tolerances that depend on the machine precision.
You invoke the routine with a fragment like the following:
Simplex s(m,n,initvars,b,obj,A,false);
s.solve();
Setting the boolean argument to true prints intermediate diagnostics. The constraint
matrix A is of type NRsparseMat. The output can be accessed with a fragment like
switch (s.ierr) {
case 0:
for (j=1;j<=n;j++)
cout << j << " " << s.u[j] << endl;
cout << "iterations " << s.nsteps << endl;
cout << "minimum " << s.u[0] << endl;
break;
case 1: cout << "phase 0 failed" << endl;
break;
case 2: cout << "no feasible solution (phase 1)" << endl;
break
...
(The rest of the error returns are listed in the function header comment.)
The routine in simplex is reasonably robust: it solves all the Netlib problems [2]
that do not involve explicit bounds on the variables other than the usual nonnegativ-
ity constraints. Typical “medium-size” problems involving . 104 constraints and
variables are solved in a few seconds on a 2006 workstation. Very degenerate or
very dense problems take longer, of course. However, the routine is nowhere close
to state-of-the-art, either in performance or in features. The main text discusses al-
ternatives that may be better for your application.
Note that simplex codes almost universally number rows and columns starting at
1, not 0. Often the coefficients of the objective function are stored in row 0 (although
we do not do so).
Doub EPS,EPSSMALL,EPSARTIF,EPSFEAS,EPSOPT,EPSINFEAS,EPSROW1,EPSROW2,EPSROW3;
Parameters: NMAXFAC specifies the maximum number of iterations as NMAXFAC*max(m,n);
NREFAC is the number of iterations between refactorizations. The various EPS tolerances are
individually adjustable. In particular, EPSSMALL governs the size of terms that are assumed
to be zero in the sparse matrix procedures. Difficult problems may require it to be either
increased or decreased.
Int nm1,nmax,nsteps;
Bool verbose;
NRvector<NRsparseCol *> a;
Input matrix sa is stored as a vector of sparse columns in a.
NRlusol *lu; Interface to LUSOL routines.
EPSROW3=EPS;
}
void Simplex::solve()
Solves problem after constructor has been called.
{
initialize();
scaleit();
phase0();
if (verbose)
cout << " at end of phase0,iter= " << nsteps << endl;
if (ierr != 0) {
delete lu;
for (Int i=0;i<n;i++)
delete a[i+1];
return;
}
phase1();
if (verbose)
cout << " at end of phase1,iter= " << nsteps << endl;
if (ierr != 0) {
delete lu;
for (Int i=0;i<n;i++)
delete a[i+1];
return;
}
phase2();
prepare_output();
delete lu;
for (Int i=0;i<n;i++)
delete a[i+1];
}
void Simplex::initialize() {
VecInt irow(2); Used to input unit vector to basis. Element 0 is ig-
VecDoub value(2); nored.
nsteps=0; Number of iterations.
ierr=0;
nm1=n+m+1;
nmax=NMAXFAC*MAX(m,n);
for (Int i=1;i<=n;i++)
if (initvars[i] == 0) A zero variable is never eligible to enter the basis.
ad[i]=0;
else
ad[i]=1;
for (Int i=n+1;i<=n+m;i++)
ad[i]=-1;
for (Int i=1;i<=m;i++) Start with the basis of logical variables, i.e., slack
if (initvars[n+i] >= 0) and artificial variables.
ord[i]=n+i;
else Keep track of free logical variables (if any) by sub-
ord[i]=-m+i-1; tracting n C m C 1.
for (Int i=0;i<n;i++) { Copy input into columns. Neither column 0 nor row
Int nvals=sa.col_ptr[i+1]-sa.col_ptr[i]; index 0 in each column is used.
a[i+1]=new NRsparseCol(m+1,nvals+1);
Int count=1;
for (Int j=sa.col_ptr[i]; j<sa.col_ptr[i+1]; j++) {
Int k=sa.row_ind[j];
a[i+1]->row_ind[count]=k+1;
a[i+1]->val[count]=sa.val[j];
count++;
}
}
lu=new NRlusol(m,sa.col_ptr[n]); Initialize as m m. Number of nonzeros is
sa.col_ptr[n].
value[0]=0.0;
value[1]=1.0;
irow[0]=0;
for (Int i=1;i<=m;i++) { Load unit vectors.
irow[1]=i;
lu->load_col(i,irow,value);
}
lu->factorize(); Initial factorization of basis (unit matrix).
}
void Simplex::scaleit()
Store scale factors in scale[1:n+m].
{
for (Int i=1;i<=m;i++)
scale[n+i]=0.0;
for (Int k=1;k<=n;k++) {
x=getcol(k);
Doub h=0.0;
for (Int i=1;i<=m;i++)
if (abs(x[i]) > h)
h=abs(x[i]);
if (h == 0.0)
scale[k]=0.0;
else
scale[k]=1.0/h;
for (Int i=1;i<=m;i++)
scale[n+i]=MAX(scale[n+i],abs(x[i])*scale[k]);
}
for (Int i=1;i<=m;i++)
if (scale[n+i] == 0.0)
scale[n+i]=1.0;
}
void Simplex::phase0()
Phase 0 ends when all free variables are in the basis, and all zero variables are out.
{
Int ind,ip,kp;
Doub piv;
for (kp=1;kp<=n;kp++) { Loop over all columns.
if (initvars[kp] < 0) { Found a free variable.
x=lx(kp); x D L 1 ak .
w=lu->uinv(x); w D AB 1 ak .
ip=rowpiv(w,0,kp,0.0); Find column ip to leave.
ind=ord[ip];
transform(x,ip,kp); Interchange columns.
ord[ip] -= nm1; Mark as a free variable.
if (initvars[ind] == 0) If a zero variable went out, keep it out.
ad[ind]=0;
}
}
for (ip=1;ip<=m;ip++) { Loop over basis.
ind=ord[ip];
if (ind < 0 || initvars[ind] != 0) Skip free variables and nonzero variables.
continue;
for (Int i=1;i<=m;i++) v[i]=0.0;
v[ip]=1.0; Column ip will leave.
kp=colpiv(v,0,piv); Find entering column kp.
if (abs(piv) < EPSSMALL) {
xb=lu->solve(b); xB D AB 1 b.
if (abs(xb[ip]) > EPSARTIF*maxnorm(xb)) {
ierr=1; Zero artificial variable left in basis with nonzero rhs.
return;
} else {
if (verbose)
cout << " artificial variable remains: ip " << ip << endl;
continue;
}
}
x=lx(kp); x D L 1 ak .
transform(x,ip,kp); Interchange columns.
if (ad[ind] == 1)
ad[ind]=0; Zero variable removed and no longer eligible.
}
}
void Simplex::phase1()
Phase 1: find a feasible basis.
{
Int ip,kp;
Doub piv;
for (;;) {
xb=lu->solve(b); xB D AB 1 b.
Doub xbmax=maxnorm(xb);
Bool done=true;
for (Int i=1;i<=m;i++) {
if (ord[i] > 0 && xb[i] < -EPSFEAS*xbmax) {
v[i]=1.0/scale[ord[i]]; The auxiliary objective function (scaled).
done=false;
}
else
v[i]=0.0;
}
if (done) Done: go to phase 2.
break;
kp=colpiv(v,1,piv); Find entering column kp.
if (ierr != 0)
return;
Bool first=true;
for (;;) {
x=lx(kp); x D L 1 ak .
w=lu->uinv(x); w D AB 1 ak .
ip=rowpiv(w,1,kp,xbmax);
if (ierr == 0) Found column ip to leave.
break;
if (!first) { Failure on second attempt: error.
ierr=6;
return;
}
Maybe accumulated error of updates is too large. Try refactorizing.
if (verbose)
cout << " attempt to recover" << endl;
ierr=0;
first=false;
refactorize();
}
transform(x,ip,kp); Interchange columns.
if (nsteps >= nmax) {
ierr=3;
return;
}
}
}
void Simplex::phase2()
Find optimal feasible basis.
{
Int ip,kp;
Doub piv;
for (;;) {
for (Int i=1;i<=m;i++) {
if (ord[i] > 0) Load minus coefficients of objective function into v.
v[i]=-obj[ord[i]];
else
v[i]=-obj[ord[i]+nm1];
}
kp=colpiv(v,2,piv); Find entering column kp.
if (piv > -EPSOPT) The feasible solution has been optimized successfully.
break;
Bool first=true;
for (;;) {
x=lx(kp); x D L 1 ak .
w=lu->uinv(x); w D AB 1 ak .
xb=lu->solve(b); xB D AB 1 b.
Doub xbmax=maxnorm(xb);
ip=rowpiv(w,2,kp,xbmax);
if (ierr == 0) Found column ip to leave.
break;
if (!first)
return; Unbounded objective function.
if (verbose) Maybe accumulated error of updates is too large.
cout << " attempt to recover" << endl; Try refactorizing.
ierr=0;
first=false;
refactorize();
}
transform(x,ip,kp); Interchange columns.
if (verbose) {
prepare_output();
cout << " in phase2,iter,obj. fn. " << nsteps << " " << u[0] << endl;
}
if (nsteps >= nmax) {
prepare_output();
cout << " in phase2,iter,obj. fn. " << nsteps << " " << u[0] << endl;
ierr=4;
return;
}
}
}
piv=h1;
kp=k;
}
}
}
if (phase == 1) { If no pivot is found in phase 1, the problem is infea-
h1=0.0; sible.
for (Int k=1;k<=m;k++)
h1 += abs(x[k])*scale[n+k];
if (piv > -EPSINFEAS*h1)
ierr=2;
}
return kp;
}
VecDoub Simplex::getcol(Int k)
Returns column k of A.
{
VecDoub temp(m+1,0.0);
for (Int i=1;i<a[k]->nvals;i++) {
temp[a[k]->row_ind[i]]=a[k]->val[i];
}
return temp;
}
void Simplex::refactorize()
Refactorize basis by sparse LU decomposition.
{
Int count=0;
Doub sum=0.0;
VecInt irow(2);
VecDoub value(2);
value[0]=0.0;
value[1]=1.0;
irow[0]=0;
lu->clear();
for (Int i=1;i<=m;i++) {
Int ind=ord[i];
if (ind < 0)
ind += nm1;
if (ind > n) { Basis vector is a unit vector.
irow[1]=ind-n;
lu->load_col(i,irow,value);
count++;
sum += 1.0;
}
else { Basis vector is a column of A.
lu->load_col(i,a[ind]->row_ind,a[ind]->val);
for (Int j=1;j<a[ind]->nvals;j++) {
count++;
sum += abs(a[ind]->val[j]);
}
}
}
Doub small=EPSSMALL*sum/count; Set size of LU factors to treat as zero.
lu->LUSOL->parmlu[LUSOL_RP_SMALLDIAG_U] =
lu->LUSOL->parmlu[LUSOL_RP_EPSDIAG_U] = small;
lu->factorize();
}
void Simplex::prepare_output()
Compute basic solution. On output, u[0] is the value of the objective function, u[1..n] con-
tains the values of the variables at the minimum, while u[n+1..n+m] gives the values of the
slack variables (i.e., the amount by which the corresponding inequality is satisfied).
{
Int indv;
Doub sum=obj[0];
xb=lu->solve(b); xB D AB 1 b.
for (Int i=0;i<=m+n;i++) u[i]=0.0;
for (Int i=1;i<=m;i++) {
if (ord[i] > 0)
indv=ord[i];
else
indv=ord[i]+nm1;
u[indv]=xb[i]; Nonzero components of solution.
sum += xb[i]*obj[indv];
}
u[0]=sum; c0 C c x.
}
The simplex routine makes use of the external package LUSOL, and so re-
quires the following interface, which is is NRlusol.h.
struct NRlusol
Interface between Numerical Recipes routine Simplex and the required external package LUSOL.
{
LUSOLrec *LUSOL;
Int inform;
void NRlusol::factorize()
{
LU1FAC(LUSOL, &inform );
if (inform > LUSOL_INFORM_SERIOUS) {
cout << " Error:" << endl << LUSOL_informstr(LUSOL, inform) << endl;
throw("LUSOL exiting");
}
}
void NRlusol::clear()
{
LUSOL_clear(LUSOL, TRUE);
}
NRlusol::~NRlusol()
{
LUSOL_free(LUSOL);
}