0% found this document useful (0 votes)
28 views42 pages

Simplex Algorithm

The document describes the simplex tableau representation of a linear program. It defines the structure and properties of the tableau, including that it contains the objective function and constraints in a matrix-like form. The document also provides an example to illustrate pivoting, which is the process of exchanging a basic and non-basic variable when applying the simplex algorithm. Pivoting transforms the current basic feasible solution into a neighboring solution through a series of well-defined pivot equations.

Uploaded by

Steve Yeo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
0% found this document useful (0 votes)
28 views42 pages

Simplex Algorithm

The document describes the simplex tableau representation of a linear program. It defines the structure and properties of the tableau, including that it contains the objective function and constraints in a matrix-like form. The document also provides an example to illustrate pivoting, which is the process of exchanging a basic and non-basic variable when applying the simplex algorithm. Pivoting transforms the current basic feasible solution into a neighboring solution through a series of well-defined pivot equations.

Uploaded by

Steve Yeo
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 42

FOUNDATIONS OF OPTIMIZATION: IE6001

Simplex Algorithm

Napat Rujeerapaiboon
Semester I, AY2019/2020
Simplex Tableaux

• Consider a basic representation of the form

r T xN = cBT B −1 b

x0 −
−1 (∗)
xB + B NxN = B −1 b

where r = cN − N T B −T cB is the reduced cost vector.


• To simplify notation we represent (∗) as a tableau

BV x0 xBT xNT RHS


x0 1 0 T −r T cB B −1 b
T

xB 0 I −1
B N B −1 b

where I is the m × m identity matrix.


Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Explicit formulation:
2 1
x0 − 5 x3 − 5 x4 = −8
1 2
x2 − 5 x3 + 5 x4 = 5
3 1
− 5 x3 + 5 x4 + x5 = 1
3 1
x1 + 5 x3 − 5 x4 = 3
Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Removing variables and mathematical operators:

1 − 25 − 15 −8
1 − 15 2
5 5
− 35 1
5 1 1
3
1 5 − 15 3
Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Appending zeroes:

1 0 0 − 25 − 15 0 −8
0 0 1 − 15 2
5 0 5
0 0 0 − 35 1
5 1 1
0 1 0 35 − 15 0 3
Example: Simplex Tableaux

Basic representation (4) from Example 1 (I = {1, 2, 5}).

Introducing labels:

BV x0 x1 x2 x3 x4 x5 RHS
x0 1 0 0 − 25 − 15 0 −8
x2 0 0 1 − 15 2
5 0 5
x5 0 0 0 − 35 1
5 1 1
3
x1 0 1 0 5 − 15 0 3

Rows are indexed by the respective basic variables.


Properties of the Simplex Tableau

• x0 appears only in objective row and has coefficient 1;


• each basic variable appears in only one other row and has
coefficient 1;
• only one basic variable can occur in each row;
• the RHS of the objective row is the objective value of the
current BS
• the RHS’s of the other rows are the values of the basic
variables at the current BS;
• the coefficients of the nonbasic variables in the objective
row are the negative reduced costs.
Note: The current BS is feasible iff the RHS’s are
nonnegative in all rows (except objective row).
Notational Conventions

Tableau for an admissible index set I (with p ∈ I, q ∈


/ I):
BV x0 x1 ··· xp ··· xq ··· xn RHS
x0 1 β1 ··· βp ··· βq ··· βn β0
.. .. .. .. .. .. ..
. . . . . . .
xp 0 yp1 ··· ypp ··· ypq ··· ypn yp0
.. .. .. .. .. .. ..
. . . . . . .

• yii = 1 ∀i ∈ I and yi 0 i = 0 ∀i ∈ I, i 0 ∈ I\{i}


• βi = −ri ∀i ∈/ I, i 6= 0 (negative reduced cost)
• βi = 0 ∀i ∈ I
Sequences of BFSs

Simplex Algorithm: if BFS x is not optimal, then there is some


neighbouring BFS x 0 with better objective value.

Consider the sequence of BFS’s when solving Example 1. We


found the following sequence of index sets It of BV’s:

{3, 4, 5} → {3, 4, 1} → {2, 4, 1} → {2, 5, 1}


↑ ↑ ↑ ↑
I1 I2 I3 I4
Neighbouring BFS’s

It+1 is obtained from It by removing one element and replacing


it by a new element:

|It \It+1 | = |It+1 \It | = 1.

Definition: Two index sets I and I 0 with B(I) and B(I 0 )


non-singular, are said to be neighbours if

|I\I 0 | = |I 0 \I| = 1.
Pivoting

Given a basic representation, suppose a basic variable is to be


made nonbasic and a nonbasic variable is to be made basic.
What is the new basic representation?

Given: ∗ Two neighbouring index sets I and I 0


∗ Basic representation for I with tableau (yij , βj )

Sought: ∗ Basic representation for I 0 with tableau (yij0 , βj0 )

The basic representation for I 0 is constructed via pivoting.


Pivoting (cont)

Suppose that I 0 = (I\{p}) ∪ {q}, i.e.,


• the basic variable xp becomes nonbasic;
• the nonbasic variable xq becomes basic.

We say that
• xp leaves the basis;
• xq enters the basis.
Pivoting (cont)
To swap xp and xq we
1. divide row p by the pivot element ypq :
ypj
yqj0 = ∀j = 0, . . . , n (1)
ypq
yiq
2. subtract the ypq -fold multiple of row p from row i ∈ I\{p}:

yiq
yij0 = yij − ypj ∀j = 0, . . . , n (2)
ypq
βq
3. subtract the ypq -fold multiple of row p from objective row:

βq
βj0 = βj − ypj ∀j = 0, . . . , n (3)
ypq
Pivoting (Remarks)

• The formulas (1)–(3) are termed pivot equations.

• The pivot equations ensure that:


• the qth column of the new tableau contains only 0’s except
for its qth entry, which is 1;
• the other basic columns remain unchanged.

• Row p is relabelled as row q after pivoting.

• Pivoting is possible iff ypq 6= 0.


Pivoting (Example)

Basic representation for I = {1, 2, 3}:

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1

Find the neighbouring basic representation for I 0 = {4, 2, 3}.


Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1

Pivot Element: y14 = 1


Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0
x4 0 1 0 0 1 1 −1 5
x2
x3

Equation (1) ⇒ row for x1 remains unchanged,


but now relabelled with x4 .
Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3

Equation (2) ⇒ subtract row for x1 twice from row for x2 ,


no relabelling necessary.
Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3 0 1 0 1 0 3 −2 4

Equation (2) ⇒ add row for x1 to row for x3 ,


no relabelling necessary.
Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0 1 −1 0 0 0 1 2 −4
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3 0 1 0 1 0 3 −2 4

Equation (3) ⇒ subtract row for x1 from objective row,


no relabelling necessary.
Pivoting (Example)

Exchange: x1 becomes nonbasic, x4 becomes basic.

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 0 0 0 1 2 1 1
x1 0 1 0 0 1 1 −1 5
x2 0 0 1 0 2 −3 1 3
x3 0 0 0 1 −1 2 −1 −1
x0 1 −1 0 0 0 1 2 −4
x4 0 1 0 0 1 1 −1 5
x2 0 −2 1 0 0 −5 3 −7
x3 0 1 0 1 0 3 −2 4

Pivot equations ensure that the other basic columns and the
objective column remain unchanged!
Pivot Selection

Consider the BS for I = {1, 2, 4} in Example 1.


x2
We can pivot on:
• (xp , xq ) = (x4 , x5 )
⇒ I 0 = {1, 2, 5}
I' = {1, 2, 3}
I' = {1, 2, 5} • (xp , xq ) = (x4 , x3 )
⇒ I 0 = {1, 2, 3}
I = {1, 2, 4} • (xp , xq ) = (x2 , x3 )
⇒ I 0 = {1, 4, 3}
• (xp , xq ) = (x2 , x5 )
I' = {1, 5, 4} ⇒ I 0 = {1, 5, 4}
I' = {1, 4, 3}
x1
Pivot Selection (cont)

When choosing a pivot, we impose the following rules:

1. Non-Inferiority: The new BS must have a better


objective value than the current BS, i.e., β00 ≤ β0 .
0 ≥0
2. Feasibility: The new BS must be feasible, i.e., yi0
for all i ∈ I 0 .
• Rule 1 is used to choose the nonbasic variable xq ,
q∈
/ I, which enters the basis.
• Rule 2 is used to choose the basic variable xp , p ∈ I,
which leaves the basis.
Choosing xq

The objective row of the simplex tableau is equivalent to:


n
X X
x0 + βi xi = β0 ⇐⇒ x0 = β 0 − βi xi
i=1 i ∈I
/

By definition, we have
• βi = 0 for all i ∈ I (basic variables);
• βi = −ri for all i ∈
/ I (nonbasic variables).
Choosing xq (cont)

Two cases:
• if βi ≤ 0 ∀i ∈
/ I, then the current BFS is optimal.
X
x0 = β0 − βi xi ≥ β0 ∀xi ≥ 0
i ∈I
/

⇒ No other FS achieves a lower objective value.


• if ∃q ∈
/ I with βq > 0, then increasing the nonbasic variable
xq decreases the current objective value.

⇒ Any nonbasic xq with βq > 0 can enter the basis.

Note: Decreasing nonbasic variables is not allowed.


Choosing xp

• Increase some nonbasic xq with βq > 0 while fixing all


other nonbasic variables at zero, i.e., xi = 0 ∀i ∈ / I ∪ {q}.
• The rows for the basic variables xi , i ∈ I, in the simplex
tableau then imply
X
xi + yij xj = yi0 and xi = yi0 − yiq xq ∀i ∈ I.
j ∈I
/

• Aim: increase xq as much as possible while ensuring that


all basic variables xi , i ∈ I, remain nonnegative.
Choosing xp (cont)

• We must choose the value of xq > 0 such that

xi = yi0 − yiq xq ≥ 0 ∀i ∈ I. (∗)

Otherwise, some basic variable(s) become infeasible.


• Each i ∈ I gives a constraint on xq :

 xq ≤ x , yi0

iq if yiq > 0,
xi = yi0 − yiq xq ≥ 0 ⇐⇒ yiq
 x ≤x ,∞ if yiq ≤ 0.
q iq

⇒ The feasibility requirement (∗) is equivalent to

xq ≤ min x iq . (∗∗)
i∈I
Choosing xp (cont)

Two cases:

• Suppose min x iq = ∞ (∀i ∈ I : yiq ≤ 0), then


i∈I
• xq can grow indefinitely.
• As βq > 0, the objective value x0 = β0 − βq xq can drop
indefinitely. ⇒ The LP is unbounded below.

• Suppose min x iq < ∞ (∃i ∈ I : yiq > 0), then


i∈I
• The best solution is obtained by making xq as large as
possible, i.e., we set xq = min x iq .
i∈I
• ∃p ∈ arg min x iq ⇒ xq = x pq = yyp0
i∈I pq

• Thus, xp = yp0 − ypq xq = 0 becomes nonbasic!


Pivot Selection (Summary)

• If βi ≤ 0 for all i ∈
/ I, then the current BFS is optimal.
⇒ We are done! Otherwise. . .

• Any xq with q ∈
/ I and βq > 0 can enter the basis.
⇒ Choose a suitable xq . . .

• If yiq ≤ 0 for all i ∈ I, then the LP is unbounded.


⇒ We are done! Otherwise. . .

• Any xp with p ∈ arg mini∈I x iq can leave the basis.


⇒ Choose a suitable xp . . .

 yi0

if yiq > 0,
Recall: x iq , yiq
 ∞ if yiq ≤ 0.
Pivoting Conventions

• If there are several q ∈


/ I with βq > 0, then choose q with

βq = max βj . (∗)
j ∈I
/

Such a q produces the maximum decrease in x0 per unit of


increase in xq .
• If several q ∈
/ I satisfy (∗), choose the one with the smallest
index.
• If there are several p ∈ arg mini∈I x iq , choose the one with
the smallest index.
Simplex Algorithm (Minimisation)

• Step 0: Find initial BFS and its basic representation.

• Step 1: If βi ≤ 0 for all i ∈


/ I:
STOP — the current BFS is optimal.

• Step 2: If ∃j ∈
/ I with βj > 0 and yij ≤ 0 for all i ∈ I:
STOP — no finite minimum exists.

• Step3: Choose xq with βq > 0


Entry criterion — xq enters the basis.

• Step 4: Choose p ∈ arg mini∈I x iq


Exit criterion — xp leaves the basis.

• Step 5: Pivot on ypq and go to STEP 1.


Example (Simplex Algorithm)

minimize:

x0 = −4x1 − 2x2 + x3

Subject to:

x1 +x2 +x3 ≤ 4
x1 −x2 −2x3 ≤ 3
3x1 +2x2 +x3 ≤ 12

x1 , x2 , x3 ≥ 0.
Example (Simplex Algorithm)

Adding slack variables x4 , x5 , x6 we get:

minimize:

x0 = −4x1 − 2x2 + x3

Subject to:

x1 +x2 +x3 +x4 =4


x1 −x2 −2x3 +x5 =3
3x1 +2x2 +x3 +x6 = 12

x1 , x2 , x3 , x4 , x5 , x6 ≥ 0.
We have thus an initial BFS for I = {4, 5, 6}.
Example (Simplex Algorithm)

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 4 2 −1 0
x4 1 1 1 1 4
x5 1 −1 −2 1 3
x6 3 2 1 1 12
x0 1 6 7 −4 −12
x4 2 3 1 −1 1
x1 1 −1 −2 1 3
x6 5 7 −3 1 3
Example (Simplex Algorithm)

BV x0 x1 x2 x3 x4 x5 x6 RHS
x0 1 6 7 −4 −12
x4 2 3 1 −1 1
x1 1 −1 −2 1 3
x6 5 7 −3 1 3
4
x0 1 3 − 73 5
−3 − 43
3
2 1 1 1
x3 3 1 3 −3 3
1 2 1 11
x1 1 3 3 3 3
1
x6 3 − 73 − 23 1 2
3
Example (Simplex Algorithm)

BV x0 x1 x2 x3 x4 x5 x6 RHS
4
x0 1 3 − 37 − 53 − 43
3
2 1
x3 3 1 3 − 13 1
3
1 2 1 11
x1 1 3 3 3 3
1 7
x6 3 − 3 − 23 1 2
3
x0 1 −2 −3 −1 −15
3 1
x2 1 2 2 − 12 1
2
1 1 1 7
x1 1 −2 2 2 2
x6 − 21 − 52 − 12 1 1
2

Solution: x1 = 72 , x2 = 21 , and x3 = 0.
Degenerate BS’s

Feasible set of Example 1:

x2

demand constraint:
x1 6 4

x1
Degenerate BS’s (cont)

Consider now a variant of Example 1:

x2

modified demand constraint:


x1 6 3

x1
Degenerate BS’s (cont)
The highlighted BS corresponds to the index sets
I = {1, 2, 3}, I = {1, 2, 4} and I = {1, 2, 5}.

x2

x5 = 0

Degenerate BFS with


x 3 = x4 = x5 = 0

x4 = 0

x3 = 0

x1
Degenerate BFS’s: Definition

Definition: A BS is called degenerate if more than n − m


of its components are zero.

⇒ A degenerate BS has at least one zero-valued basic variable


(next to the n − m zero-valued nonbasic variables).

⇒ For a degenerate BS with index set I there exists at least


one i ∈ I with yi0 = 0.

Definition: A BS is called non-degenerate if all of its


basic variables are different from zero.
Finite Termination

Theorem: If all BS’s are non-degenerate, then the simplex


algorithm must terminate after a finite number of steps with

• either an optimal solution


• or a proof that the problem is unbounded.
Finite Termination (Proof)

• At each step we have yi0 > 0 ∀i ∈ I (non-degeneracy).


• Unless optimality or unboundedness is detected in STEP 1
β
or 2, we find β00 = β0 − ypqq yp0 < β0 .
• Thus, the sequence of objective values obtained by the
algorithm is strictly decreasing.

β0 > β00 > β000 > · · ·

No basic solution will ever be repeated!


• There are ≤ m n

basic solutions. Thus, the process cannot
continue indefinitely and must terminate at STEP 1 or 2
after a finite number of iterations.


You might also like