Revised Simplex

Domenico Salvagnin
November 2, 2020

1 Introduction
Let us consider a linear program in standard form:

min z = cT x (1.1)
Ax = b (1.2)
x≥0 (1.3)

where A ∈ Rm×n is full rank with m < n. Let N = {1, . . . , n} be the index set of all variables.
A vector x that satisfies (1.2) is called a solution, and it is feasible solution if it also satisfies (1.3).

2 Bases
A basis is defined as a set of m linearly independent columns of A. We will denote with B the ordered set
of indices corresponding to the basic columns:

B = {k1 , . . . , km } (2.1)

and with R the set of indices associated with the non-basic columns. The partition of the columns (and thus
variables) into basic and non-basic allows us to rewrite the system (1.2) as

BxB + RxR = b (2.2)

where B is the set of basic columns and R the set of non-basic columns. Given that B is by definition
invertible, we can finally rewrite (2.2) as

xB = B −1 b − B −1 RxR (2.3)

from which it is clear that the value of basic variables xB is uniquely determined by the values assigned to
the non-basic ones xR . In particular, if all non-basic variables are assigned the value zero we obtain the
solution:
x = (xB , xN ) = (B −1 b, 0) (2.4)
which is called a basic solution. If such solution also satisfies the non negativity constraints (1.3), then it is
a basic feasible solution.
Two bases of A are adjacent if they differ by just one column. Let B = [b1 , . . . , bp−1 , bp , bp+1 , . . . , bm ] be
a basis of A and consider the adjacent basis Ba = [b1 , . . . , bp−1 , a, bp+1 , . . . , bm ], where we replaced the p-th
column bp with a. Since B is a basis, we can express a as a linear combination of the columns of B, i.e.:
m
X
a= αi bi = Bα (2.5)
i=1

1
From linear algebra we know that Ba is non singular iff αp 6= 0 and thus we can write:

1 X αi
bp = a− bi = Ba η (2.6)
αp αp
i6=p

where  
α1 αp−1 1 αp+1 αm
η = − ,...,− , ,− ,...,− (2.7)
αp αp αp αp αp
If we define the elementary transformation matrix E as

η1
 
1
 .. .. 

 . . 

E = [e1 , . . . , ep−1 , η, ep+1 , . . . , em ] = 
 ηp 
 (2.8)
 .. .. 
 . . 
ηm 1

we can express the columns of B as a function of the columns of Ba as B = Ba E. By taking the inverse and
multiplying by E we also get:
Ba−1 = EB −1 (2.9)
The above relation shows that it is possible to obtain the inverse of Ba by just multiplying the inverse of B
by E on the left. The matrix E is easily obtained by the vector η, which in turn is easily computed from
α = B −1 a.

3 Primal Simplex
The main idea behind the primal simplex is as follows: we start from a feasible basis B of the linear
problem, and iteratively move to an adjacent basis whose cost is no worse, until we reach the optimal basis.
To formalize the algorithm we need to specify:
• how to recognize an optimal basis (optimality conditions);
• how to move to a better adjacent basis.

3.1 Optimality conditions

Starting from (2.3), we can partition the objective function as:

z = cTB xB + cTR xR
= cTB B −1 b + (cTR − cTB B −1 R)xR (3.1)

Thus, the objective function is now expressed as a constant term plus a linear function of the non-basic
variables only. The vector
π T = cTB B −1 (3.2)
is called vector of multipliers and it conveniently allows the definition of the coefficients dj in the objective
function (3.1). Such coefficients are called reduced costs and are defined as:

dj = cj − π T aj (3.3)

It is easy to show that the reduced costs of basic variables are always zero.

2
 The reduced costs of the non-basic variables give an optimality condition for the current basis. Let us
assume to set one of the non-basic variables xj , j ∈ R to a strictly positive value. From (3.1), we can deduce
that the objective function z will increase or decrease depending on whether dj > 0 or dj < 0. We can thus
conclude that if
dj ≥ 0 ∀j ∈ R (3.4)
then no non-basic variable can move away from zero and improve the objective value, and thus the current
basis is optimal. Note that the condition above is only a sufficient condition for optimality, but it is possible
to show that there always exists an optimal basis that satisfies it.

3.2 Finding a better basis

If the current basis B does not satisfy the optimality condition (3.4), then there exist a non-basic variable
xq with negative reduced cost dq < 0. We can thus try to increase xq while still satisfying equations (2.2)
and the non negativity of basic variables, and improve the objective value as much as possible.
Let t ≥ 0 be the (displacement) of variable xq . Given that all other non-basic variables stay at zero, we
can rewrite equations (2.2) as:
BxB (t) + taq = b (3.5)
from which

xB (t) = B −1 b − tB −1 aq
= β − tαq (3.6)

introducing the notation β = B −1 b and αq = B −1 aq . Since we want to maintain the feasibility of the basic
variables, it must hold that xiB (t) = βi − tαqi ≥ 0 for all i = 1, . . . , m, and this set of m inequalities defines
the maximum value for t. Given
I = {i : αqi > 0} (3.7)
we can compute the maximum value θ for t as
 
βp βi
θ= = min (3.8)
αqp i∈I αqi

This operation is the so-called ratio test. The p-th basic variabile is the blocking variable and the element
αqp is called pivot element (note that the pivot element is not necessarily unique). For t = θ the blocking
variable becomes null (like a non-basic variable): we can thus say that variable xkp exits the basis and is
replaced by variable xq . The new objective value is:

z = z + θdq (3.9)

To obtain the inverse of the new basis we just need to apply the formula (2.9), where the η vector is defined
as in (2.7). If I = ∅, then there is no restriction on t and thus t = +∞. In this case the objective function
value z can decrease indefinitely and the problem is (unbounded ). Note also that in case of degeneracy, it
may happen that βp = 0, and thus we have a zero step length θ: in this case we change the basis but the
basic feasible solution stays the same.

3.3 Algorithmic Description

We can thus define the revised primal simplex algorithmically as follows:

Step 0 (init) Given a feasible basis B, compute B −1 , β = B −1 b and z = cTB xB .

Step 1 Compute the vector of multipliers π T = cTB B −1 .

3
Step 2 (pricing) Compute the reduced costs dj = cj − π T aj ∀j ∈ R. If dj ≥ 0 ∀j then the current basis
is optimal and we can stop. Otherwise choose an entering variable xq with dq < 0.
Step 3 Compute αq = B −1 aq .
Step 4 (pivot step) Define I = {i : αqi > 0}. If I = ∅ the problem is unbounded. Otherwise apply the
ratio test (3.8), computing θ and the leaving variable xkp .
Step 5 (update) Update the basis B and its inverse. Then update the quantities β and z

βi = βi − θαqi i 6= p (3.10)
βp = θ (3.11)
z = z + θdq (3.12)

Go to Step 1.

4 Dual Simplex
To a given linear program in standard form (1.1 – 1.3) we can always associate the dual linear programming
problem:

max Z = bT y (4.1)
T
A y≤c (4.2)

Introducing auxiliary variables d ≥ 0 we obtain the form:

max bT y (4.3)
T
A y+d=c (4.4)
d≥0 (4.5)

We can now define a different version of the simplex algorithm, that still tries to solve the primal problem
(1.1 – 1.3), but does so using the dual form (4.3 – 4.5).
Let B any basis of A, non necessarily feasible. We can rewrite the dual equations (4.4) as dT = cT − y T A
and pick y T = cTB B −1 (this is always possible as variables y are free). Then we can obtain the dual variables
d as

dTB = cTB − cTB B −1 B = 0 (4.6)

dTR = cTR − cTB B −1 R ≥0 (4.7)

We thus have that the dual variables dB associated to the basic primal variables are null, while to have dual
feasibility it must hold that dR ≥ 0. Let us assume that the current basis is dual feasible, i.e., that (4.7)
is satisfied. Note that this condition is nothing but the primal optimality condition: thus, if the current
basis were also primal feasible it would be optimal. Dual simplex thus starts from a dual feasible basis and
iteratively moves to an adjacent one until it finds one that is also primal feasible, and thus optimal.

4.1 Finding a better dual basis

Consider equations (4.6) and, as done with the primal simplex, let us try to increase a variable dkp by the
amount t ≥ 0. As usual, we want to move a variable while satisfying the equations in the model (in this

4
case, the dual equations), and thus we compute the vector of multipliers y(t) as a function of t and impose
the conditions (4.6):

y T B + dTB = cTB (4.8)

B T y + dB = cB (4.9)
T T
B y(t) + tep = cB = B y (4.10)
−T
y(t) = y − tB ep (4.11)

Note that B −T ep = (ep B −1 )T is nothing but the p-th row of B −1 . Given y(t) we can compute the change
in the objective function z as a function of t:

Z(t) = bT y(t)
= bT y − tbT B −T ep
= Z − tβp (4.12)

Since we want to maximize the dual objective, and t ≥ 0, a criterion for the choice of the variable dkp to
increase is βp < 0 (in other words, it is convenient to pick a primal variable which is infeasible). This variable
is the leaving variable.
Let us now see what happens to the non-basic dual variables dR as t increases. For all j ∈ R we have:

dj (t) = cj − aTj y(t)

= cj − aTj y + taTj B −T ep
= dj + tαjp ≥ 0 (4.13)

Similarly to what we did for the primal case, we can define the set of indices:

J = {j ∈ R : αjp < 0} (4.14)

and compute the maximum displacement θD as

( )
dq dj
θD = = min (4.15)
−αqp j∈J −αjp

For t = θD , the reduced cost of variable xq becomes null, and xq enters the basis. Note that in the primal
simplex we first pick the entering variable and the pick a leaving one with the ratio test, while in the dual
simplex it is the exact opposite. If J = ∅, then t can grow arbitrarily large and thus the dual problem is
unbounded (and the primal problem is infeasible).

4.2 Algorithmic Description

We can thus define the revised dual simplex algorithmically as follows:

Step 0 (init) Given a dual feasible basis B, compute B −1 , β = B −1 b, π T = cTB B −1 , dTR = cTR − y T R and
Z = y T b.
Step 1 (dual pricing) If β ≥ 0 then the current basis is optimal. Otherwise pick a leaving variable xkp
with βp < 0.
p
Step 2 Compute αR = eTp B −1 R.
Step 3 (dual pivot step) Define J = {j ∈ R : αjp < 0}. If J = ∅ the problem is infeasible. Otherwise
apply the dual ratio test (4.15), and determine θD and the entering variable xq .

5
Step 4 (update) Update dual variables y and d, and the objective function Z

y = y − θD B −T ep (4.16)
dj = dj + θD αjp j∈R (4.17)
dkp = θD (4.18)
Z = Z − θ D βp (4.19)
βp
Compute the vector αq = B −1 aq and the primal step θ = αp
q
, and update β. Update the basis and its
inverse. Go to Step 1.

5 Bounded Simplex
So far we have described the simplex algorithm for a linear program in standard form. An arbitrary linear
program can be written as:

min cT x
Ax ∼ b (5.1)
l≤x≤u

where ∼∈ {≤, ≥, =} and some lower/upper bound can be +/ − ∞. Introducing artificial variables as needed
for the linear inequalities we can obtain the general form:

min cT x
Ax = b (5.2)
l≤x≤u

Depending of the values of the lower and upper bounds, we can, barring some shifting, split the variables
into the following categories:

ID Type Bounds
FX fixed xj = 0
P non negative xj ≥ 0
N non positive xj ≤ 0
BD bounded 0 ≤ xj ≤ uj
FR free xj

Let us now see how the primal simplex can be extended to deal with this more general form. For ease of
exposition, we will assume that there are no N variables (we can always get rid of them by complementation,
and in any case the extension of the concepts is trivial).

5.1 Feasibility
A solution is feasible if each variable is within its bounds. For non-basic variables, those are no longer at
zero, as they might be at their upper bound. Thus, we define the set U of non-basic variables at their upper
bound and compute the basic variables as:
X
xB = B −1 (b − aj uj ) = B −1 bu (5.3)
j∈U

6
5.2 Optimality conditions
The definition of reduced costs does not change, but we need to update the optimality conditions, as some
non-basic variables might have a negative displacement. The update conditions are:

ID Value Reduced cost

FX 0 any
P 0 dj ≥ 0
BD 0 dj ≥ 0
BD uj dj ≤ 0
FR 0 dj = 0

Note that a fixed variable never enters the basis (there are no conditions on its reduced cost as the
displacement is always zero), while for a free variable the reduced cost must be null, as it can move in both
directions.

5.3 Basis change

Depending on the type of the entering variable, we might have dq < 0 (t ≥ 0) or dq > 0 (t ≤ 0). Let us
consider the case dq < 0 first. Equations (3.5) are rewritten as:

BxB (t) + taq = bu (5.4)

xB (t) = B −1 bu − tB −1 aq (5.5)
= β − tαq (5.6)

which is not really different from the standard case. However, the conditions on the maximum displacement
t need some update. In particular:
• some basic variables might have a upper bound, thus we might have a limit on t even for αqi < 0.
• if a basic bariable is free, then there is not constraint on t (a free variable never leaves the basis).
• the entering variable xq might itself have a finite upper bound uq , and it might reach it before all basic
variables.
Formally, we need to define two sets of indices:

I + = {i : αqi > 0 type(xki ) ∈ {F X, P, BD}} (5.7)

−
I = {i : αqi <0 type(xki ) ∈ {F X, BD}} (5.8)

compute the quantities:

 
βi
θ+ = min (5.9)
+ i∈I αqi
 
− βi − uki
θ = min (5.10)
− i∈I αqi

and finally obtain the maximum value for t as

θ = min{θ+ , θ− , uq } (5.11)

The case in which θ = uq is called a bound flip and is extremely convenient: in this case the iteration is
for sure non-degenerate (thus the algorithm makes a strictly positive move toward optimality) and it is very
computationally cheap as there is no basis change.

7
 The case dq > 0 is analogous. In this case we define the index sets as:

I + = {i : αqi > 0 tipo(xki ) ∈ {F X, BD}} (5.12)

I − = {i : αqi < 0 tipo(xki ) ∈ {F X, P, BD}} (5.13)

from which we derive

 
βi − uki
θ+ = max (5.14)
i∈I + αqi
 
βi
θ− = max (5.15)
i∈I − αqi

and finally compute the maximum value for t as

θ = max{θ+ , θ− , −uq } (5.16)

Note that in this case we take the maximum as the displacement t is ≤ 0.