FOUNDATIONS OF OPTIMIZATION: IE6001

Simplex Algorithm

Napat Rujeerapaiboon
Semester I, AY2019/2020
 Simplex Tableaux

• Consider a basic representation of the form

r T xN = cBT B −1 b

x0 −
−1 (∗)
xB + B NxN = B −1 b

where r = cN − N T B −T cB is the reduced cost vector.

• To simplify notation we represent (∗) as a tableau

BV x0 xBT xNT RHS

x0 1 0 T −r T cB B −1 b
T

xB 0 I −1
B N B −1 b

where I is the m × m identity matrix.

 Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Explicit formulation:
2 1
x0 − 5 x3 − 5 x4 = −8
1 2
x2 − 5 x3 + 5 x4 = 5
3 1
− 5 x3 + 5 x4 + x5 = 1
3 1
x1 + 5 x3 − 5 x4 = 3
 Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Removing variables and mathematical operators:

1 − 25 − 15 −8
1 − 15 2
5 5
− 35 1
5 1 1
3
1 5 − 15 3
 Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Appending zeroes:

1 0 0 − 25 − 15 0 −8
0 0 1 − 15 2
5 0 5
0 0 0 − 35 1
5 1 1
0 1 0 35 − 15 0 3
 Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Introducing labels:

BV x0 x1 x2 x3 x4 x5 RHS
x0 1 0 0 − 25 − 15 0 −8
x2 0 0 1 − 15 2
5 0 5
x5 0 0 0 − 35 1
5 1 1
3
x1 0 1 0 5 − 15 0 3

Rows are indexed by the respective basic variables.

 Properties of the Simplex Tableau

• x0 appears only in objective row and has coefficient 1;

• each basic variable appears in only one other row and has
coefficient 1;
• only one basic variable can occur in each row;
• the RHS of the objective row is the objective value of the
current BS
• the RHS’s of the other rows are the values of the basic
variables at the current BS;
• the coefficients of the nonbasic variables in the objective
row are the negative reduced costs.
Note: The current BS is feasible iff the RHS’s are
nonnegative in all rows (except objective row).
 Notational Conventions

Tableau for an admissible index set I (with p ∈ I, q ∈

/ I):
BV x0 x1 ··· xp ··· xq ··· xn RHS
x0 1 β1 ··· βp ··· βq ··· βn β0
.. .. .. .. .. .. ..
. . . . . . .
xp 0 yp1 ··· ypp ··· ypq ··· ypn yp0
.. .. .. .. .. .. ..
. . . . . . .

• yii = 1 ∀i ∈ I and yi 0 i = 0 ∀i ∈ I, i 0 ∈ I\{i}

• βi = −ri ∀i ∈/ I, i 6= 0 (negative reduced cost)
• βi = 0 ∀i ∈ I
 Sequences of BFSs

Simplex Algorithm: if BFS x is not optimal, then there is some

neighbouring BFS x 0 with better objective value.

Consider the sequence of BFS’s when solving Example 1. We

found the following sequence of index sets It of BV’s:

{3, 4, 5} → {3, 4, 1} → {2, 4, 1} → {2, 5, 1}

↑ ↑ ↑ ↑
I1 I2 I3 I4
 Neighbouring BFS’s

It+1 is obtained from It by removing one element and replacing

it by a new element:

|It \It+1 | = |It+1 \It | = 1.

Definition: Two index sets I and I 0 with B(I) and B(I 0 )

non-singular, are said to be neighbours if

|I\I 0 | = |I 0 \I| = 1.
 Pivoting

Given a basic representation, suppose a basic variable is to be

made nonbasic and a nonbasic variable is to be made basic.
What is the new basic representation?

Given: ∗ Two neighbouring index sets I and I 0

∗ Basic representation for I with tableau (yij , βj )

Sought: ∗ Basic representation for I 0 with tableau (yij0 , βj0 )

The basic representation for I 0 is constructed via pivoting.

 Pivoting (cont)

Suppose that I 0 = (I\{p}) ∪ {q}, i.e.,

• the basic variable xp becomes nonbasic;
• the nonbasic variable xq becomes basic.

We say that
• xp leaves the basis;
• xq enters the basis.
 Pivoting (cont)
To swap xp and xq we
1. divide row p by the pivot element ypq :
ypj
yqj0 = ∀j = 0, . . . , n (1)
ypq
yiq
2. subtract the ypq -fold multiple of row p from row i ∈ I\{p}:

yiq
yij0 = yij − ypj ∀j = 0, . . . , n (2)
ypq
βq
3. subtract the ypq -fold multiple of row p from objective row:

βq
βj0 = βj − ypj ∀j = 0, . . . , n (3)
ypq
 Pivoting (Remarks)

• The formulas (1)–(3) are termed pivot equations.

• The pivot equations ensure that:

• the qth column of the new tableau contains only 0’s except
for its qth entry, which is 1;
• the other basic columns remain unchanged.

• Row p is relabelled as row q after pivoting.

• Pivoting is possible iff ypq 6= 0.

 Pivoting (Example)

Basic representation for I = {1, 2, 3}:

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1

Find the neighbouring basic representation for I 0 = {4, 2, 3}.

 Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1

Pivot Element: y14 = 1

 Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0
x4 0 1 0 0 1 1 −1 5
x2
x3

Equation (1) ⇒ row for x1 remains unchanged,

but now relabelled with x4 .
 Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3

Equation (2) ⇒ subtract row for x1 twice from row for x2 ,

no relabelling necessary.
 Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3 0 1 0 1 0 3 −2 4

Equation (2) ⇒ add row for x1 to row for x3 ,

no relabelling necessary.
 Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0 1 −1 0 0 0 1 2 −4
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3 0 1 0 1 0 3 −2 4

Equation (3) ⇒ subtract row for x1 from objective row,

no relabelling necessary.
 Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0 1 −1 0 0 0 1 2 −4
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3 0 1 0 1 0 3 −2 4

Pivot equations ensure that the other basic columns and the
objective column remain unchanged!
 Pivot Selection

Consider the BS for I = {1, 2, 4} in Example 1.

x2
We can pivot on:
• (xp , xq ) = (x4 , x5 )
⇒ I 0 = {1, 2, 5}
I' = {1, 2, 3}
I' = {1, 2, 5} • (xp , xq ) = (x4 , x3 )
⇒ I 0 = {1, 2, 3}
I = {1, 2, 4} • (xp , xq ) = (x2 , x3 )
⇒ I 0 = {1, 4, 3}
• (xp , xq ) = (x2 , x5 )
I' = {1, 5, 4} ⇒ I 0 = {1, 5, 4}
I' = {1, 4, 3}
x1
 Pivot Selection (cont)

When choosing a pivot, we impose the following rules:

1. Non-Inferiority: The new BS must have a better

objective value than the current BS, i.e., β00 ≤ β0 .
0 ≥0
2. Feasibility: The new BS must be feasible, i.e., yi0
for all i ∈ I 0 .
• Rule 1 is used to choose the nonbasic variable xq ,
q∈
/ I, which enters the basis.
• Rule 2 is used to choose the basic variable xp , p ∈ I,
which leaves the basis.
 Choosing xq

The objective row of the simplex tableau is equivalent to:

n
X X
x0 + βi xi = β0 ⇐⇒ x0 = β 0 − βi xi
i=1 i ∈I
/

By definition, we have
• βi = 0 for all i ∈ I (basic variables);
• βi = −ri for all i ∈
/ I (nonbasic variables).
 Choosing xq (cont)

Two cases:
• if βi ≤ 0 ∀i ∈
/ I, then the current BFS is optimal.
X
x0 = β0 − βi xi ≥ β0 ∀xi ≥ 0
i ∈I
/

⇒ No other FS achieves a lower objective value.

• if ∃q ∈
/ I with βq > 0, then increasing the nonbasic variable
xq decreases the current objective value.

⇒ Any nonbasic xq with βq > 0 can enter the basis.

Note: Decreasing nonbasic variables is not allowed.

 Choosing xp

• Increase some nonbasic xq with βq > 0 while fixing all

other nonbasic variables at zero, i.e., xi = 0 ∀i ∈ / I ∪ {q}.
• The rows for the basic variables xi , i ∈ I, in the simplex
tableau then imply
X
xi + yij xj = yi0 and xi = yi0 − yiq xq ∀i ∈ I.
j ∈I
/

• Aim: increase xq as much as possible while ensuring that

all basic variables xi , i ∈ I, remain nonnegative.
 Choosing xp (cont)

• We must choose the value of xq > 0 such that

xi = yi0 − yiq xq ≥ 0 ∀i ∈ I. (∗)

Otherwise, some basic variable(s) become infeasible.

• Each i ∈ I gives a constraint on xq :

 xq ≤ x , yi0

iq if yiq > 0,
xi = yi0 − yiq xq ≥ 0 ⇐⇒ yiq
 x ≤x ,∞ if yiq ≤ 0.
q iq

⇒ The feasibility requirement (∗) is equivalent to

xq ≤ min x iq . (∗∗)
i∈I
 Choosing xp (cont)

Two cases:

• Suppose min x iq = ∞ (∀i ∈ I : yiq ≤ 0), then

i∈I
• xq can grow indefinitely.
• As βq > 0, the objective value x0 = β0 − βq xq can drop
indefinitely. ⇒ The LP is unbounded below.

• Suppose min x iq < ∞ (∃i ∈ I : yiq > 0), then

i∈I
• The best solution is obtained by making xq as large as
possible, i.e., we set xq = min x iq .
i∈I
• ∃p ∈ arg min x iq ⇒ xq = x pq = yyp0
i∈I pq

• Thus, xp = yp0 − ypq xq = 0 becomes nonbasic!

 Pivot Selection (Summary)

• If βi ≤ 0 for all i ∈
/ I, then the current BFS is optimal.
⇒ We are done! Otherwise. . .

• Any xq with q ∈
/ I and βq > 0 can enter the basis.
⇒ Choose a suitable xq . . .

• If yiq ≤ 0 for all i ∈ I, then the LP is unbounded.

⇒ We are done! Otherwise. . .

• Any xp with p ∈ arg mini∈I x iq can leave the basis.

⇒ Choose a suitable xp . . .

 yi0

if yiq > 0,
Recall: x iq , yiq
 ∞ if yiq ≤ 0.
 Pivoting Conventions

• If there are several q ∈

/ I with βq > 0, then choose q with

βq = max βj . (∗)
j ∈I
/

Such a q produces the maximum decrease in x0 per unit of

increase in xq .
• If several q ∈
/ I satisfy (∗), choose the one with the smallest
index.
• If there are several p ∈ arg mini∈I x iq , choose the one with
the smallest index.
 Simplex Algorithm (Minimisation)

• Step 0: Find initial BFS and its basic representation.

• Step 1: If βi ≤ 0 for all i ∈

/ I:
STOP — the current BFS is optimal.

• Step 2: If ∃j ∈
/ I with βj > 0 and yij ≤ 0 for all i ∈ I:
STOP — no finite minimum exists.

• Step3: Choose xq with βq > 0

Entry criterion — xq enters the basis.

• Step 4: Choose p ∈ arg mini∈I x iq

Exit criterion — xp leaves the basis.

• Step 5: Pivot on ypq and go to STEP 1.

 Example (Simplex Algorithm)

minimize:

x0 = −4x1 − 2x2 + x3

Subject to:

x1 +x2 +x3 ≤ 4
x1 −x2 −2x3 ≤ 3
3x1 +2x2 +x3 ≤ 12

x1 , x2 , x3 ≥ 0.
 Example (Simplex Algorithm)

Adding slack variables x4 , x5 , x6 we get:

minimize:

x0 = −4x1 − 2x2 + x3

Subject to:

x1 +x2 +x3 +x4 =4

x1 −x2 −2x3 +x5 =3
3x1 +2x2 +x3 +x6 = 12

x1 , x2 , x3 , x4 , x5 , x6 ≥ 0.
We have thus an initial BFS for I = {4, 5, 6}.
 Example (Simplex Algorithm)

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 4 2 −1 0
x4 1 1 1 1 4
x5 1 −1 −2 1 3
x6 3 2 1 1 12
x0 1 6 7 −4 −12
x4 2 3 1 −1 1
x1 1 −1 −2 1 3
x6 5 7 −3 1 3
 Example (Simplex Algorithm)

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 6 7 −4 −12
x4 2 3 1 −1 1
x1 1 −1 −2 1 3
x6 5 7 −3 1 3
4
x0 1 3 − 73 5
−3 − 43
3
2 1 1 1
x3 3 1 3 −3 3
1 2 1 11
x1 1 3 3 3 3
1
x6 3 − 73 − 23 1 2
3
 Example (Simplex Algorithm)

BV x0 x1 x2 x3 x4 x5 x6 RHS
4
x0 1 3 − 37 − 53 − 43
3
2 1
x3 3 1 3 − 13 1
3
1 2 1 11
x1 1 3 3 3 3
1 7
x6 3 − 3 − 23 1 2
3
x0 1 −2 −3 −1 −15
3 1
x2 1 2 2 − 12 1
2
1 1 1 7
x1 1 −2 2 2 2
x6 − 21 − 52 − 12 1 1
2

Solution: x1 = 72 , x2 = 21 , and x3 = 0.
 Degenerate BS’s

Feasible set of Example 1:

x2

demand constraint:
x1 6 4

x1
 Degenerate BS’s (cont)

Consider now a variant of Example 1:

x2

modified demand constraint:

x1 6 3

x1
 Degenerate BS’s (cont)
The highlighted BS corresponds to the index sets
I = {1, 2, 3}, I = {1, 2, 4} and I = {1, 2, 5}.

x2

x5 = 0

Degenerate BFS with

x 3 = x4 = x5 = 0

x4 = 0

x3 = 0

x1
 Degenerate BFS’s: Definition

Definition: A BS is called degenerate if more than n − m

of its components are zero.

⇒ A degenerate BS has at least one zero-valued basic variable

(next to the n − m zero-valued nonbasic variables).

⇒ For a degenerate BS with index set I there exists at least

one i ∈ I with yi0 = 0.

Definition: A BS is called non-degenerate if all of its

basic variables are different from zero.
 Finite Termination

Theorem: If all BS’s are non-degenerate, then the simplex

algorithm must terminate after a finite number of steps with

• either an optimal solution

• or a proof that the problem is unbounded.
 Finite Termination (Proof)

• At each step we have yi0 > 0 ∀i ∈ I (non-degeneracy).

• Unless optimality or unboundedness is detected in STEP 1
β
or 2, we find β00 = β0 − ypqq yp0 < β0 .
• Thus, the sequence of objective values obtained by the
algorithm is strictly decreasing.

β0 > β00 > β000 > · · ·

No basic solution will ever be repeated!

• There are ≤ m n


basic solutions. Thus, the process cannot
continue indefinitely and must terminate at STEP 1 or 2
after a finite number of iterations.