Simplex Method

(Chapter 4)
MSCI 603 Section 1
Instructor: Mehrdad Pirnia
Introduction
 Last class, we solved a two-variable problem graphically
 It had a unique (one) solution
 What if there is more than one solution? No solution?

 Solving a two-variable problem can be done graphically

 What about a three-variable problem?
 n-variable?

 Simplex Method allows us to tackle bigger problems

MSCI 603 2
How Simplex Works
Recall: extreme point (corner-point feasible) solutions

1. Begin with an initial extreme point solution

 How to find this initial point?

2. Move in the direction of improvement to the next extreme

point solution
 How to identify the direction of improvement?
 How far to move in direction of improvement?

3. Keep moving until no further improvement can be

achieved
 How do we know when to stop?

MSCI 603 3
How Simplex Works
9
 Example:
max z  3 x1  5 x2 8

s.t. x1 4 7

3 x1  2 x2  18 6

2 x2  12 5

x1 , x2  0 4

1 2 3 4 5 6

MSCI 603 4
Simplex Solution Concepts
1. Focus solely on corner-point feasible (CPF) solutions
2. Simplex is iterative
Stop
Yes

Is Current
No
Initialization Solution Iteration
Optimal?

3. Choose origin to be initial CPF solution, when possible

4. Quicker to gather information on adjacent CPF solutions
5. Simplex chooses direction of largest rate of improvement in z
6. Optimality test checks only if any positive rate of improvement in z is
possible
MSCI 603 5
Solving a System of Linear Equations
 Recall: Gauss-Jordan elimination
 An m x n (m rows, n columns) system of linear equations has
either
 A unique solution
 No solution
 Multiple solutions

 Example:
 Find the solution to the following system of equations
x1  2 x2  5 Note: We are assuming that all
2 x1  x2  0 variables must be non-negative

MSCI 603 6
 Note: We are assuming that all
More examples variables must be non-negative

 Unique Solution: x1  2 x2  x3  4
2 x1  x2  3x3  3
x1  x2  x3  3

 Infeasible: x1  2 x2  x3  4
2 x1  x2  2 x3  1
2 x1  3x2  3x3  2

MSCI 603 7
Standard Form of an LP
 Constraints must be written as equalities
 Add slack variable for each ≤ constraint

max z  3x1  5 x2 max z  3 x1  5 x2

s.t. x1 4 s.t. x1  s1 4
3x1  2 x2  18 3 x1  2 x2  s2  18
2 x2  12 2 x2  s3  12
x1 , x2  0 x1 , x2 , s1 , s2 , s3  0

 Subtract excess variable for each ≥ constraint

MSCI 603 8
Example 1: Simplex Method
 Solve using the simplex method.
 First, introduce the variable, z, and re-write the objective
function

max z
s.t. z  3 x1  5 x2 0 Row 0

x1  s1 4 Row 1

3 x1  2 x2  s2  18 Row 2

2 x2  s3  12 Row 3

x1 , x2 , s1 , s2 , s3  0

MSCI 603 9
Basic and non-basic variables
 When we find a solution to a system of equations, we are
defining a basis
 E.g. (x1, x2, x3) = (1, 0, 4)

 A basic variable is a variable that is in the basis, i.e. it takes

on a non-zero value
 In the example above, x1 and x3 are basic variables

 A non-basic variable is not in the basis, i.e. it has a value of

zero
 In the example above, x2 is a non-basic variable
MSCI 603 10
Simplex Method
1. Initialization
 Write LP in standard form (max, all RHS ≥ 0) and determine
initial basic and non-basic variables

2. Optimality Test
 Current basic feasible solution is optimal if and only if each
reduced cost (coefficient in row 0) is non-negative.
Otherwise, go to Step 3.

3. Iteration
 Swap a basic variable for a non-basic variable and calculate
the new basic feasible solution (see next slide for details)
MSCI 603 11
Simplex Method (continued)
3. Iteration (detailed)
a) Determine the entering variable
 The non-basic variable with the most negative reduced cost (objective
function coefficient) is the pivot column
b) Determine the leaving variable
 Perform the minimum ratio test using only the positive, non-zero
coefficients in the pivot column
 The row with the minimum ratio is the leaving variable
c) Calculate the new basic feasible solution
 The pivot column is now an “identity” column
 Perform EROs to update the simplex tableau

4. Go to Step 2 (Optimality Test)

MSCI 603 12
Important!
 All constraints must be equality (=)
 For a ≤ constraint, add a positive slack
 For a ≥ constraint, subtract a positive slack (or excess)

 All variables must be non-negative

 For x unrestricted in sign (urs), replace with x = x1 – x2 ,
where x1 , x2 ≥ 0.

 All right hand sides must be non-negative

 For negative rhs multiply both sides by -1

 To solve a minimization problem, transform to a max problem

 Multiply objective function by -1
 Consider the variable with most positive coefficient in row zero

MSCI 603 13
Min Problems
 Method 1(Recommended):
transform to max problem
multiply objective function by -1

MSCI 603 14
Min Problem
 Method 2:
Choose the variable with the
most positive coefficient in row zero

MSCI 603 15
Alternate Solutions
 Happens when a non-basic variable has a coefficient of zero in
row zero.
 Consider that non-basic variable as an entering basic variable and go through
iterations

 Example 2: Find 2 alternate solutions for the following problem:

max 𝑧 = −3𝑥1 + 6𝑥2

s. t.
5𝑥1 + 7𝑥2 ≤ 35
−𝑥1 + 2𝑥2 ≤ 2
𝑥1 , 𝑥2 ≥ 0

MSCI 603 16
Unbounded LP

 Happens when there is non-basic variable with negative

coefficient in row zero, and non-positive coefficients in
other rows

 Example:

17
Unbounded LP

18
Degeneracy
 Nondegenerate LP
 Each basic feasible solution has basic variables that are all > 0
 Optimal solution will be found in a finite number of iterations
 Objective function will improve with each iteration

 An LP is degenerate if it has at least one bfs in which a BV

is equal to zero
 Will cycle through solutions
 Often, if there is a tie in the minimum ratio test, the next
simplex tableau will exhibit degeneracy

MSCI 603 Lecture 4: More Simplex Method 19

Try at home
 If there is a tie in mrt, choose row 1 over row 2

max z  2 x1  3 x2  x3  12 x4
s.t.  2 x1  9 x2  x3  9 x4  0
1 x  x  1 x  2x  0
3 1 2 3 3 4

x1 , x2 , x3 , x4  0

20
Unrestricted (urs) variables

 For each urs variable 𝑥𝑖 define two new variables 𝑥𝑖′ and 𝑥𝑖′′

 Substitute the variable 𝑥𝑖 with 𝑥𝑖′ − 𝑥𝑖′′

 Add two constraints 𝑥𝑖′ ≥ 0 and 𝑥𝑖′′ ≥ 0

21
Example: URS variables

22
 ≥ and = constraints
min z  0.4 x1  0.5 x2
s.t. 0.3 x1  0.1x2  2.7
No initial basis because
0.5 x1  0.5 x2  6 of coefficient on e3
0.6 x1  0.4 x2  6
x1 , x2  0
max - z  0.4 x1 - 0.5 x2
Convert to
standard form s.t. 0.3 x1  0.1x2  s1  2 .7
0.5 x1  0.5 x2 6
10 x2  e3  6
x1 , x2 , s1 , e3  0
23
Add artificial variables
 Find an initial bfs by introducing artificial variables, ai, to
the “problematic” constraints

 Must modify the LP to ensure all artificial variables equal 0

in the optimal solution

 Two possible approaches:

1. Big M Method
2. Two-Phase Method (not covered in this class)

24
Big M Method
1. Modify constraints so that all RHS are non-negative
a) Identify all ≥ and = constraints

2. Convert problem to standard form

3. For ≥ and = constraints, add artificial variable ai (ai ≥ 0)

4. Let M denote a very large positive number.

a) For min problem, add Mai to objective function
b) For max problem, subtract Mai from objective function
c) M is usually 100 times larger than the largest coefficient

5. Using EROs, eliminate all artificial variables from objective function

row before beginning simplex algorithm

25
Infeasibility
 Detected when artificial variables gain value in optimal
solution.

26
Next Class
 Sensitivity analysis
 Graphical Method
 Simplex

MSCI 603 27