CSCI5654 (Linear Programming, Fall 2013) Duality

Lecture-7

Lecture 7 Slide# 1 Lecture 7 Slide# 2

Linear Program (standard form) Example Problem

maximize x1 + 2x2
maximize c1 x1 + · · · + cn xn s.t. x1 ≤ 3
s.t. aj1 x1 + · · · + ajn xn ≤ bj j ∈ {1, 2, . . . , m} x2 ≤ 3
x1 , . . . , xn ≥ 0 −x1 + x2 ≤ 1
x1 + x2 ≤ 5
Problem has n variables and m constraints. x1 , x2 ≥ 0

Lecture 7 Slide# 3 Lecture 7 Slide# 4

Bounds on Optimal Value of LP Bounds

maximize x1 + 2x2 Lower Bound: Find a feasible solution ~x .

s.t. x1 ≤ 3 Then ~x is a lower bound on optimum. (why?).
x2 ≤ 3
−x1 + x2 ≤ 1 Eg. (x1 : 3, x2 : 2) is feasible.
x1 + x2 ≤ 5
x1 , x2 ≥ 0 Therefore, z = x1 + 2x2 ≥ 7 .

Upper Bounds: Can we find upper bounds to optimum?

Goal: Find lower and upper bounds on the solution to the LP.

Lecture 7 Slide# 5 Lecture 7 Slide# 6

Example (continued) Example (continued)

maximize x1 + 2x2 ←z maximize x1 + 2x2 ←z

s.t. x1 ≤ 3 ← C1 s.t. x1 ≤ 3 ← C1
x2 ≤ 3 ← C2 x2 ≤ 3 ← C2
−x1 + x2 ≤ 1 ← C3 −x1 + x2 ≤ 1 ← C3
x1 + x2 ≤ 5 ← C4 x1 + x2 ≤ 5 ← C4
x1 , x2 ≥ 0 x1 , x2 ≥ 0

Note: Note:
x1 ≤ 3 ∧ x2 ≤ 3 ⇒ x1 + 2x2 ≤??? . x1 ≤ 3 ∧ x2 ≤ 3 ⇒ x1 + 2x2 ≤ 3 + 2 · 3 ≤ 9 .
| {z } | {z } | {z } | {z } | {z } | {z }
C1 C2 z
C1 C2 z

Q: Using the inequality above what can you say about the solution to the Q: Using the inequality above what can you say about the solution to the
LP? LP?
A: 9 is an upper bound to the solution (i.e., z ≤ 9).

Lecture 7 Slide# 7 Lecture 7 Slide# 8

Finding Upper Bounds Upper Bounds to LP Solution
Strategy: Multiply constraint rows by positive quantities.
maximize x1 + 2x2 ←z
maximize c1 x1 + ··· + cn xn z
s.t. x1 ≤ 3 ← C1
s.t. a11 x1 + ··· + a1n xn ≤ b1 ← C1
x2 ≤ 3 ← C2
a21 x1 + ··· + a2n xn ≤ b2 ← C2
−x1 + x2 ≤ 1 ← C3 .. ..
x1 + x2 ≤ 5 ← C4 . .
x1 , x2 ≥ 0 am1 x1 ··· + amn xn ≤ bm ← Cm
x1 , . . . , xn ≥ 0
I C 1 + 2 · C 2 ⇒ z ≤ 9. Consider y1 C1 + y2 C2 + · · · + ym Cm.
I 2 · C 4 ⇒ z ≤ 10.
Q: What are the conditions on y1 , . . . , ym so that this combination upper
I Question #1: −1 · C 3 + 3 · C 2 ⇒ z ≤? bounds z?
I Question #2: 2 · C 3 + C 1 ⇒ z ≤?
I Question #3: Can you think of other combinations?

Lecture 7 Slide# 9 Lecture 7 Slide# 10

Upper Bounds to LP Solution Best upper bound on LP Solution

Conditions:
a11 y1 + a21 y2 + · · · + am1 ym ≥ c1 minimize b1 y1 + b2 y2 + · · · + bm ym
a12 y1 + a22 y2 + · · · + am2 ym ≥ c2 s.t. a11 y1 + a21 y2 + · · · + am1 ym ≥ c1
..
. a12 y1 + a22 y2 + · · · + am2 ym ≥ c2
a1n y1 + a1n y2 + · · · + amn ym ≥ cn ..
.
y1 , y2 , . . . , ym ≥ 0 a1n y1 + a1n y2 + · · · + amn ym ≥ cn
Upper Bound: b1 y1 + . . . + bm ym y1 , y2 , . . . , ym ≥ 0
Note: This is called the dual problem.
Q: What values of y1 , . . . , ym can yield best bound on optimum??

Lecture 7 Slide# 11 Lecture 7 Slide# 12

Dual Problem Dual Problem (matrix)
Primal LP:
max. ~c · ~x
Primal Problem: decision variables are x1 , . . . , xn . s.t. A~x ≤ ~b
~x ≥0
maximize c1 x1 + · · · + cn xn
s.t. aj1 x1 + · · · + ajn xn ≤ bj j ∈ {1, 2, . . . , m} Dual LP:
x1 , . . . , xn ≥ 0 min. ~b · ~y
At~y ≥ ~c
Dual Problem: decision variables are y1 , . . . , ym . ~y ≥ 0

minimize b1 y1 + ··· + bm ym (dual in standard form)

s.t. a1j y1 + ··· + amj ym ≥ cj j ∈ {1, 2, . . . , n} max −~b · ~y
y1 , . . . , yn ≥ 0 −At~y ≤ −~c
~y ≥ 0
Exercise: Use the information in this frame to prove:
The dual of dual problem is the same as the original primal problem.
Lecture 7 Slide# 13 Lecture 7 Slide# 14

Example Example (dual)

maximize x1 + 2x2
min. 3y1 + 3y2 + y3 + 5y4
s.t. x1 ≤ 3 ← y1
s.t. y1 − y3 + y4 ≥ 1
x2 ≤ 3 ← y2
y2 + y3 + y4 ≥ 2
−x1 + x2 ≤ 1 ← y3
y1 , y2 , y3 , y4 ≥ 0
x1 + x2 ≤ 5 ← y4
x1 , x2 ≥ 0 Dual optimal: y1 = 0, y2 = 0, y3 = 12 , y4 = 3
2 yields optimal value 8.
Q: Write down the dual problem.

Lecture 7 Slide# 15 Lecture 7 Slide# 16

Example (dictionary) Example (final dictionary)

x5 x6
x3 = 3 − x1 x1 = 2 + 2 − 2
x5 x6
x4 = 3 − x2 x2 = 3 − 2 + 2
x5 = 1 + x1 − x2 x3 = ···
x6 = 5 − x1 − x2 x4 = ···
1 3
z = 0 + x1 + 2x2 z = 8 − 2 x5 − 2 x6

Lecture 7 Slide# 17 Lecture 7 Slide# 18

Dual Certificate Dual Certificate

Situation: I ask X to solve a large LP for me. After sometime X presents

a solution ~x = (x1∗ , . . . , xn∗ ) that is claimed to be optimal.
Situation: I ask X to solve a large LP for me. After sometime X presents
Q: How do I verify that X’s solution is indeed optimal?
a solution ~x = (x1∗ , . . . , xn∗ ) that is claimed to be optimal.
A2: Provide dual variables ~y : (y1∗ , . . . , yn∗ ).
Q: How do I verify that X’s solution is indeed optimal?
A1: OK solve the LP yourself!! 1. Check primal feasibility of ~x .
(not acceptable) 2. Check dual feasibility of ~y .
3. Check that the objective values are the same.
Primal-Dual Certificate: In the presence of both, each solution can serve
as a certificate to the optimality of the other.

Lecture 7 Slide# 19 Lecture 7 Slide# 20

Primal dual correspondences

Strong Duality Theorem: If x1∗ , . . . , xm ∗ is primal optimal then there

Variable to constraint correspondence: exists a dual optimal solution y1∗ , . . . , ym

∗ that satisifies

X X
Dual variable (yj ) Primal constraint Aj ~x ≤ bj ci xi∗ = bj yj∗ .
Primal variable (xi ) Dual constraint A ·i ~y ≥ ci i j

Correspondence in dictionary: (∴ if primal has optimal solution then the dual has optimum with the
Dual variable yj Primal slack variable xn+j . same value as primal).
Primal variable (xi ) Dual dictionary slack variable ym+i . Proof: This is theorem 5.1 in Chvátal (pages 58-59).

Lecture 7 Slide# 21 Lecture 7 Slide# 22

Example Example (final dictionary)

x5 x6
x1 = 2 + 2 − 2
x5 x6
x2 = 3 − 2 + 2
maximize x1 + 2x2 x3 = ···
s.t. x1 ≤ 3 x4 = ···
x2 ≤ 3 z = 8 − 1
− 3
2 x5 2 x6
−x1 + x2 ≤ 1
x1 + x2 ≤ 5 Insight: Dual variables correspond to primal slack variables.
x1 , x2 ≥ 0
x3 ↔ y1 , x4 ↔ y2 , x5 ↔ y3 , x6 ↔ y4 .

Read off dual solution by from objective row of final dictionary:

1 3
y1 : 0, y2 : 0, y3 : , y4 : .
2 2

Lecture 7 Slide# 23 Lecture 7 Slide# 24

Proof of Strong Duality Relationship Between Primal/Dual

Final Dictionary:
P
xi = bi + j∈Independent aij xj Primal
.. Optimal Infeasible Unbounded
. Optimal Possible Impossible Impossible
z = z ∗ + c1∗ x1 + · · · + cn+m
∗ xn+m Dual Infeasible Impossible Possible Possible
Note: ci∗ = 0 if i ∈ Basis Unbounded Impossible Possible Impossible
ci∗ ≤ 0 if i ∈ Independent.
Claim: Set variable yi = −(cn+i∗ ). This is a dual feasible solution with
∗
optimal value z .

Lecture 7 Slide# 25 Lecture 7 Slide# 26

Primal and Dual Infeasible

Primal:
max. x2
s.t. x1 ≤ −1
−x2 ≤ −1
x1 , x2 ≥ 0
Dual:
min. −y1 − y2
s.t. y1 ≥ 0
−y2 ≥ 1
y1 , y2 ≥ 0

Lecture 7 Slide# 27