Duality Theory

Duality Theory
• Every linear programming problem has associated with it another linear
programming problem called the Dual. The original problem is called
Primal.
The watch maker’s Problem
Smith Family: Father (F) and Son (S)

Decision Watch Working hours required of

Variable Type Profit Per Unit ($) F S
x1 1 60 2 3
x2 2 40 1 4
x3 3 80 4 2
Maximum Working Hours available per week 50 60
Production plan to maximize total profit from 3 type of watches
Max W = 60 x1 + 40 x2 + 80 x3
S.t. 2 x1 + x2 + 4 x3 ≤ 50
3 x1 + 4 x2 + 2 x3 ≤ 60
x1, x2, x3 ≥ 0 2
John Blake
Neighbor and owner of similar product line company (competitor)
Wants to hire Father and Son full time

How to convince them (Son and Father) to join him (John Blake)?
Comparable wages with respect to before : “at least” as good as before
Hire them at the minimum possible cost: “just enough” to effect the switch over

F’s wages – $ y1 per hour

S’s wages – $ y2 per hour

Min B = 50 y1 + 60 y2
S.t. 2 y1 + 3 y2 ≥ 60
y1 + 4 y2 ≥ 40
4 y1 + 2 y2 ≥ 80
y1, y2 ≥ 0
Solution: y1 = 15, y2 =10, Objective function B = $1350 per week
3
 Solution of John Blake Problem
y2
• Obj B is min at
(0,40)
y1 = 15, y2 =10 3
• Objective function
B = $1350 per week
(0,20) 1

(15,10)
(0,10)
(24,4)
2 y1
(20,0) (30,0) (40,0)
4
Will the solution of John Blake help to solve Watch maker’s problem?
From watch maker’s constraint
2 x1 + x2 + 4 x3 ≤ 50 (1)
3 x1 + 4 x2 + 2 x3 ≤ 60 (2)

Now, (1)x15 + (2) x 10

60 x1 + 55 x2 + 80 x3 ≤ 1350

Also, 60 x1 + 55 x2 + 80 x3 ≤ 1350 ≤ 50 y1 + 60 y2 (for any feasible y1 and y2)

60 x1 + 40 x2 + 80 x3 : Watch maker’s (Smith Family) objective function

Hence, the best Smith Family could do is $1350 per week.
How can they make it?
Only by chance if x2= 0
2 x1 + 4 x3 = 50
3 x1 + 2 x3 = 60 => x1 = 70/4 and x3 = 15/4
Objective function value W = 60 x 70/4 + 80 x 15/4 = $1350
John Blake Problem -> Watch maker’s problem & vice-versa 5
Another Example: Diet Problem
n
Min Z   c j x j Where,
j 1 cj : cost per unit of food j
n
aij: amount of nutrient i available in per unit of food j
S.t. a x
j 1
ij j  bi , i  1,2,..., m
bi: minimum nutritional requirement for nutrient i
xj: amount of food to be included in the diet.
x j  0, j  1,2,...,n

Let us consider a hypothetical dual problem to the diet problem

Salesman: sales pure nutrient pills (i.e. only iron or only vitamins etc.)
 wants to sell pills to the dietician in order to switch completely from foods to pills
 Price pills subject to
(1) A switch takes place – competitive prices in terms of cost of foods
(2) Total revenue to salesman is maximized if minimum requirements are sold

Dual Problem
m
Max W   bi yi Where, yi: price of ith nutrient pill containing one unit of nutrient i
i 1
m
S.t. a
i 1
ij yi  c j , j  1,2,...,n (Competitive pricing w.r.t. foods)

yi  0, i  1,2,...,m 6
How to get Dual (D) from Primal (P)?
Max Z = c1x1 + c2x2 + … + cnxn Dual Variable
P
S.t. a11x1 + a12x2 + … + a1n xn ≤ b1 y1
a21x1 + a22x2 + … + a2n xn ≤ b2 y2 (Optimal Allocation of Resources)
. .
. .
. .
am1x1 + am2x2 + … + amn xn ≤ bm ym
and xj ≥ 0, for j = 1, 2, …, n

Min W = b1y1 + b2y2 + … + bmym

D
S.t. a11y1 + a21y2 + … + am1 ym ≥ c1
a12y1 + a22y2 + … + am2 ym ≥ c2 (Optimal Pricing of Resources)
.
.
.
a1ny1 + am2y2 + … + amn yn ≥ cn
and yi ≥ 0, for i = 1, 2, …, m 7
 Matrix Form of Primal and Dual Problems
P Max Z = cx Dual variable D Min W = yb
S.t. Ax ≤ b y S.t. yA ≥ c
x≥0 y≥0 y = [y1 y2 … ym]

Summary of P-D Relations (Rules for constructing the dual problem)

Primal Problem ↔ Dual Problem To convert into dual
Objective: Maximization ↔ Objective: Minimization  If objective
Cost coefficient ↔ RHS function is Max
type, see the
RHS ↔ Cost coefficient table from left to
Constraints Variables right.
≤ ↔ ≥0
= Unrestricted in sign  If objective
≥ ≤0 function is Min
Variables Constraints type, see the
≥0 ↔ ≥ table from right to
Unrestricted in sign = left.
≤0 ≤ 8
 Demonstration of Primal-Dual Rule: Numerical Examples

Example 1: Maximization Type

Max Z = x1+4x2+3x3
Dual Variable Min W = 2y1+y2+4y3
S.t. 2x1+3x2−5x3 ≤ 2 y1 S.t. 2y1+3y2+ y3 ≥ 1
3x1 − x2+6x3 ≥ 1 y2 Dual
3y1 − y2+ y3 ≤ 4
x1 + x2+ x3 = 4 y3 − 5y1 + 6y2+ y3 = 3
x1≥ 0, x2≤ 0, x3unrestricted y1≥ 0, y2≤ 0, y3unrestricted

Example 2: Minimization Type

Min Z = 2x1+x2 − x3
Dual Variable Max W = y1+ 2y2+4y3
S.t. x1 + x2 − x3 = 1 y1 S.t. y1+ y2 ≤ 2
x1 − x2 + x3 ≥ 2 y2 y1 − y2+ y3 ≥ 1
Dual
x2 + x3 ≤ 4 y3 − y1 + y2+ y3 = -1
x1≥ 0, x2≤ 0, x3unrestricted y1unrestricted, y2 ≥ 0, y3 ≤ 0
 Duality Theorems
Theorem 1: The dual problem of the dual is the primal
Proof:
D
Min 𝐲𝐛 => Max − 𝐲𝐛
S.t. 𝐲𝐀 ≥ 𝐜 S.t. −𝐲𝐀 ≤ −𝐜
𝐲≥0 𝐲≥0

D’ P
Min − 𝐜𝐱 => Max 𝐜𝐱
S.t. −𝐀𝐱 ≥ −𝐛 S.t. 𝐀𝐱 ≤ 𝐛
𝐱≥0 𝐱≥0

10
 Primal [P] Dual [D]
Min W = yb
Max Z = cx Dual variable
S.t. yA ≥ c
S.t. Ax ≤ b y
y≥0
x≥0

Theorem 2: Weak Duality Theorem (WDT)

If x is a f.s. to [P] and y is a f.s. to [D], then cx ≤ yb.
(i.e. maximum profit is constrained by the value of resources)

Proof: x is feasible to [P] => Ax ≤ b (1)

y is feasible to [D] => yA ≥ c (2)

Pre-multiply (1) by y and post multiply (2) by x, then

cx ≤ yAx ≤ yb => cx ≤ yb
11
 Observations from WDT (cx ≤ yb)
Note: Primal refers to the maximization problem and dual
refers to the corresponding minimization problem
1) The Objective function value of the primal for its
any feasible solution is a lower bound to the
minimum value of the dual problem
2) Similarly, the Objective function value of the dual
for its any feasible solution is an upper bound to the
maximum value of the primal problem
3) If the primal is unbounded (i.e., Max Z->+∞), then
the dual problem is infeasible.
4) If the dual is unbounded (i.e., Min W->-∞), then the
primal is infeasible. 12
Theorem 3: Strong Duality Theorem (SDT)

If x* and y* are feasible to [P] and [D] respectively and

cx* = y*b, then x* and y* are optimal to [P] and [D],
respectively.

Proof: From WDT, cx ≤ yb for any feasible x and y.

Then cx ≤ y*b for any feasible x and y*.
But cx* = y*b (Given)
Then cx ≤ cx* for any feasible x
=> x* is optimal to [P] {maximization problem}

Similarly, y* is optimal to [D]

13
Simplex Tableau for Primal at any iteration

BV xT xsT RHS
xB B-1A B-1 B-1b
Z cBB-1A - c cBB-1 cBB-1b

Proposition: If B is the optimal primal basis, then optimal solution of

the dual problem is y* = cBB-1
Proof:
From SDT, cx*=y*b
=> cBxB= y*b
=> cBB-1b = y*b
=> y* = cBB-1
14
 Economic interpretation of Duality (dual variable)
 b1 
b 
If B is the optimal primal basis  2
 
At optimality, XB = B-1b and So Z= CBB-1b = CB B 1  b 
 i 
 
 

 m
b 
Suppose optimal basis B does not change by changing the ith resource from bi to bi + ∆bi.
 b1 
 b2 
 
  So Z’= CBB-1b’
b'   
 bi  bi 
 
 

 bm 


The Change in objective function,

 0   0 
 0   0 
   
   
Z  C B B1  
 y1* y*2 y*i y*m 
  b   y*i b i
 bi   i
   
   
 0   0  m1
=>∆Z/∆bi= yi*= Shadow Price or Dual Price
15
 Complementary Slackness Theorem (CST)
• Suppose
Primal [P] Max Z=cx s.t. Ax≤b, x ≥ 0
Dual [D] Min W=yb s.t. yA ≥ c, y≥0
• If x and y are feasible solution to P and D,
respectively. Then x and y are optimal to their
respective problems iff
(yA-c)x + y(b-Ax) = 0

16
 Complementary Slackness Theorem (CST)
Proof: Ax + u = b (i), yA - v = c (ii)
Where u, v ≥ 0  u1 
u 
u=  2  v=  v1 v2 vn 
 
 
u m 
Pre-multiply (i) by y and post-multiply (ii) by x, we get
yAx + yu = yb (iii) and yAx - vx = cx (iv)
Now, (iii) – (iv),
yu + vx = yb – cx =>y(b-Ax)+(yA-c)x = yb – cx
and from the given condition y(b-Ax)+(yA-c)x = 0
 yb – cx = 0 => yb = cx, which implies optimality of x and y (from
SDT)
The proof is complete when the other side of statement is also proved,
i.e. if x and y are optimal (=> yb = cx) it must be yu + vx = 0, which is
straightforward. 17
 Implications of CST
(i) Complementary slackness conditions
Since x, u, y, v ≥ 0, if the sum of nonnegative terms equals
zero, then each term is zero.
yu = 0 => y(b – Ax) = 0
and vx = 0 => (yA – c)x = 0

(ii) Using duality interpretations and the condition

If yi > 0 => ui = 0
Thus, if another party is willing to +ve prices, then it
must be the case that the resource must have been
utilized fully. (Valuable resource)
If ui > 0 => yi = 0 (resource not used fully)
(iii) CST conditions can be used to find an optimal primal
solution from an optimal dual solution, and vice-versa 18
 Example
[P]
Min W = 2y1 + 3y2+5y3+2y4+3y5 Dual variable Surplus variable
Subject to y1 + y2 + 2y3 + y4 + 3y5 ≥ 4 x1 v1
2y1 - 2y2 + 3y3 + y4 + y5 ≥ 3 x2 v2
and y1, y2, y3, y4, y5 ≥ 0
[D]
Max Z = 4x1+3x2 Slack variable
Subject to x1+2x2 ≤ 2 (1) u1
x1-2x2 ≤ 3 (2) u2 Optimal Solution:
2x1+3x2 ≤ 5 (3) u3 x1 = 4/5, x2 = 3/5 and Z = 5
x1+x2 ≤ 2 (4) u4
3x1+x2 ≤ 3 (5) u5
x 1, x 2 ≥ 0
For which constraints, slack > 0 => u2, u3, u4 > 0. From CST, primal variable y’s
corresponding to these constraints = 0 => y2= y3= y4 = 0
Also, from CST, since x1, x2 > 0 => primal constraint should be satisfied as equality,
i.e. v1= v2 = 0
Now, [P] reduces to y1 + 3y5 = 4 and 2y1 + y5 = 3 => y1 = 1, y5 = 1 and W = 5
 Dual Simplex Method
• Primal Problem:
Maximize 𝑍 = 𝐜𝐱
Subject to 𝐀𝐱 ≤ 𝐛
𝐱≥0
• Basis B is primal feasible if and only if 𝐱 𝐁 = 𝐁 −𝟏 𝐛 ≥0
• Suppose this basis B is also feasible for the dual problem (called dual feasible), then
𝐲 = 𝐜𝐁 𝐁 −𝟏 ≥ 0
and 𝐲𝐀 ≥ 𝐜
⇒ 𝐜𝐁 𝐁 −𝟏 𝐀 − 𝐜 ≥ 0
⇒ 𝑧𝑗 − 𝑐𝑗 ≥ 0, ∀𝑗
which are optimality conditions for the primal problem
• Thus, if a Basis B is dual feasible, then it is also primal optimal
• Since, this is primal optimal, using SDT (cx = yb 𝐚𝐬 𝐱 𝐁 = 𝐁 −𝟏 𝐛 and 𝐲 = 𝐜𝐁 𝐁 −𝟏 ) ⇒ it is
also dual optimal
• Thus, the main crux in the LP is to find the basis B which is both primal and dual feasible
→ Optimality
• (Primal) simplex method: Moves from one primal feasible basis to
another till it achieves dual feasibility, i.e.

𝑧𝑗 −𝑐𝑗 ≥ 0 optimality

• Dual simplex method: Starts with a dual feasible Basis and moves
towards primal feasibility

• Difference from primal simplex

– Entering variable / leaving rule

– Optimality
 Algorithm: Dual Simplex

1) The problem should maximization type with ≤ constraints

2) Updated objective function coefficients are ≥ 0 ⇒ Dual feasibility

3) At least one RHS is negative ⇒ Primal infeasibility

4) Leaving variable: The basic variable with most (-ve) RHS

5) Entering variable: Maximum ratio rule, that xj for which the ratio

𝑧𝑗 −𝑐𝑗
: 𝑎𝑗𝑘 < 0 is maximum, where xk is leaving variable
𝑎𝑗𝑘

6) Perform elementary row operations as usual

 Example: Dual simplex
• Min 𝑍 = 3𝑥1 + 2𝑥2
Subject to 3𝑥1 + 𝑥2 ≥ 3
4𝑥1 + 3𝑥2 ≥ 6
𝑥1 + 𝑥2 ≤ 3
𝑥1 , 𝑥2 ≥ 0

• Convert into maximization type with all the constraints ≤ type

M𝑎𝑥 𝑍 ′ = −𝑍 = −3𝑥1 − 2𝑥2
Subject to
−3𝑥1 − 𝑥2 ≤ −3
−4𝑥1 − 3𝑥2 ≤ −6
𝑥1 + 𝑥2 ≤ 3
𝑥1 , 𝑥2 ≥ 0

• Augmented form
M𝑎𝑥 𝑍 ′ = −𝑍 = −3𝑥1 − 2𝑥2
Subject to
−3𝑥1 − 𝑥2 + 𝑥3 = −3
−4𝑥1 − 3𝑥2 + 𝑥4 = −6
𝑥1 + 𝑥2 + 𝑥5 = 3
𝑥1 , 𝑥2 , 𝑥3 , 𝑥4 , 𝑥5 ≥ 0
 Solving using Dual simplex
Basis RHS

-3 -1 1 0 0 -3
Iteration 0
-4 -3 0 1 0 -6
1 1 0 0 1 3
3 2 0 0 0 0
Ratio -3/4 -2/3 - - -
-5/3 0 1 -1/3 0 -1
Iteration 1
4/3 1 0 -1/3 0 2
-1/3 0 0 1/3 1 1
1/3 0 0 2/3 0 -4
Ratio -1/5 - - -2 --
1 0 -3/5 1/5 0 3/5
Iteration 2
0 1 4/5 -3/5 0 6/5
0 0 -1/5 2/5 1 6/5
0 0 1/5 3/5 0 -21/5

Dual feasible and optimal: 𝑥1 = 3 5, 𝑥2 = 6 5 , 𝑍 = −𝑍 ′ = 21 5

 Graphical Interpretation

2 B

C
1

0 A D
1 2 3