DM554/DM545

Linear and Integer Programming

Lecture 9
Integer Linear Programming
Modeling

Marco Chiarandini
Department of Mathematics & Computer Science
University of Southern Denmark
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

2
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

3
Discrete Optimization

• Often we need to deal with integral inseparable quantities

• Sometimes rounding can go

• Other times rounding not feasible: eg, presence of a bus on a line is 0.3...

4
Integer Linear Programming
Linear Objective The world is not linear: "OR is the art and
Linear Constraints science of obtaining bad answers to questions to
but! integer variables which otherwise worse answers would be given"

max cT x + hT y
max cT x Ax + G y ≤b
max cT x Ax ≤ b max cT x x ≥0
Ax ≤ b x≥0 Ax ≤ b y ≥0
x≥0 x ∈ {0, 1}n y integer
x integer

Linear Programming Integer (Linear) Programming Binary Integer Program Mixed Integer (Linear)
(LP) (ILP) (BIP) Programming (MILP)
0/1 Integer Programming

max f (x)
g (x) ≤ b Non-linear Programming (NLP)
x≥0

5
Recall:

• Z set of integers

• Z+ set of positive integer

• Z+ +
0 set of nonnegative integers ({0} ∪ Z )

• N0 set of natural numbers, ie, nonnegative integers {0, 1, 2, 3, 4, ...}

6
Combinatorial Optimization Problems

Definition (Combinatorial Optimization Problem (COP))

Input: Given a finite set N = {1, . . . , n} of objects,
weights cj for all j ∈ N,
a collection of feasible subsets of N, F
Task: Find a minimum weight feasible subset, ie,
 
X 
min cj | S ∈ F
S⊆N  
j∈S

Many COP can be modelled as IP or BIP. (

S n
1 if j ∈ S
Typically: incidence vector of S, x ∈ B : xjS =
0 otherwise

7
Rounding
x2 3x1 − 2x2 + 4
max 100x1 + 64x2
50x1 + 31x2 ≤ 250
3x1 − 2x2 ≥ −4 Note: rounding
x1 , x2 ∈ Z+ does not help in the
example above!
LP optimum (376/193, 950/193)
IP optimum (5, 0) x1

x1 + 0.64x2 − 4 50x1 + 31x2 − 250

feasible region convex but not continuous: Now the optimum can be on the border (vertices)
but also internal.
Possible way: solve the relaxed problem.
• If solution is integer, done.
• If solution is rational (never irrational) try rounding to the nearest integers (but may exit
feasibility region)
if in R2 then 22 possible roundings (up or down)
if in Rn then 2n possible roundings (up or down)
8
Cutting Planes

max x1 + 4x2
x1 + 6x2 ≤ 18
x1 ≤3
x1 , x2 ≥ 0
x1 , x2 integers
x2

x1 = 3

x1 + 6x2 = 18

x1 + x2 = 5
x1
x1 + 4x2 = 2

9
Branch and Bound

max x1 + 2x2
x1 + 4x2 ≤ 8
4x1 + x2 ≤ 8
x1 , x2 ≥ 0, integer

x2

x1 + 4x2 = 8
x1
x1 + 2x2 = 1
4x1 + x2 = 8

10
 4.8
x1 ≤ 1 x1 ≥ 2

x2 x2
x1 = 1

x1 + 4x2 = 8 x1 + 4x2 = 8
x1 x1
x1 + 2x2 = 1 x1 + 2x2 = 1
4x1 + x2 = 8 4x1 + x2 = 8
11
 4.8
−∞
x1 ≤ 1 x1 ≥ 2

4.5 2 x1 =2
−∞ 2 x2 =0
x2 ≤ 1 x2 ≥ 2

x1 =1 3 4 x1 =0
x2 =1 3 4 x2 =2

x2 x2

x1 + 4x2 = 8 x1 + 4x2 = 8
x1 x1
x1 + 2x2 = 1 x1 + 2x2 = 1
4x1 + x2 = 8 4x1 + x2 = 8
12
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

13
Mathematical Programming: Modeling

• Find out exactly what the decision maker needs to know:

• which investment?
• which product mix?
• which job j should a person i do?

• Define Decision Variables of suitable type (continuous, integer valued, binary) corresponding
to the needs and Known Parameters corresponding to given data.

• Formulate Objective Function computing the benefit/cost

• Formulate mathematical Constraints indicating the interplay between the different variables.

14
How to “build” a constraint

• Formulate relationship between the variables in plain words

• Then formulate your sentences using logical connectives and, or, not, implies
• Finally convert the logical statement to a mathematical constraint.

Example
• “The power plant must not work in both of two neighbouring time periods”
• on/off is modelled using binary integer variables
• xi = 1 or xi = 0
• xi = 1 implies ⇒ xi+1 = 0
• xi + xi+1 ≤ 1

15
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

16
The Assignment Problem

Problem
Common application: Assignees are being assigned to perform tasks.

Suppose we have n persons and n jobs

Each person has a certain proficiency at each job.

Formulate a mathematical model that can be used to find an assignment that maximizes the total
proficiency.

17
The Assignment Problem
Model

Decision Variables:

1 if person i is assigned job j
xij = for i, j = 1, 2, . . . , n
0 otherwise,

Objective Function:
n X
X n
max ρij xij
i=1 j=1

where ρij is person i’s proficiency at job j

18
The Assignment Problem
Model

Constraints:
Each person is assigned one job:
n
X
xij = 1 for all i
j=1

e.g. for person 1 we get x11 + x12 + x13 + · · · + x1n = 1

Each job is assigned to one person:

n
X
xij = 1 for all j
i=1

e.g. for job 1 we get x11 + x21 + x31 + · · · + xn1 = 1

19
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

20
The Knapsack Problem

Problem ..
Input: Given a set of n items, each with a value vi and weight wi (i = 1, . . . , n)
Task: determine the number of each items to include in a collection so that the total weight is less
than a given limit, W , and the total value is as large as possible.

The “knapsack” name derives from the problem faced by someone who is constrained by a
fixed-size knapsack and must fill it with the most useful items.
P
Assuming we can take at most one of any item and that i wi > W , formulate a mathematical
model to determine which items give the largest value.

Model used, eg, in capital budgeting, project selection, etc.

21
The Knapsack Problem
Decision Variables:

1 if item i is taken
xi = for i = 1, 2 . . . , n
0 otherwise,

Objective Function:
n
X
max vi xi
i=1

Constraints:
Knapsack capacity restriction:
n
X
wi xi ≤ W
i=1

22
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

23
Set Covering

Problem
Given: a set of regions, a set of possible construction locations for emergency centers, regions that
can be served in less than 8 minutes, cost of installing an emergency center in each location.
Task: decide where to install a set of emergency centers such that the total cost is minimized and
all regions are safely served
As a COP: M = {1, . . . , m} regions, N = {1, . . . , n} centers, Sj ⊆ M regions serviced by
j ∈ N in 8 min.  
X [ 
min cj | Sj = M
T ⊆N  
j∈T j∈T

regions: M = {1, . . . , 5}
centers: N = {1, . . . , 6}
cost of centers: cj = 1 ∀j = 1, . . . , 6
coverages: S1 = (1, 2), S2 = (1, 3, 5), S3 = (2, 4, 5), S4 = (3), S5 = (1), S6 = (4, 5)

24
Example

• regions: M = {1, . . . , 5}
centers: N = {1, . . . , 6}
cost of centers: cj = 1 ∀j = 1, . . . , 6
coverages: S1 = (1, 2), S2 = (1, 3, 5), S3 = (2, 4, 5), S4 = (3), S5 = (1), S6 = (4, 5)

x1 x2 x3 x4 x5 x6
S1 S2 S3 S4 S5 S6
1 1 1 0 0 1 0
 

A= 2 1 0 1 0 0 0
3 0 1 0 1 0 0
 
4 0 0 1 0 0 1
5 0 1 1 0 0 1

25
As a BIP:

Variables:
x ∈ Bn , xj = 1 if center j is selected, 0 otherwise

Objective:
n
X
min c j xj
j=1

Constraints: (
1
• incidence matrix: aij =
0
Pn
• j=1 aij xj ≥1

26
Set covering Set packing Set partitioning
cover each of M at least once cover as many of M without cover exactly once each element
1. min, ≥
overlap of M
2. all RHS terms are 1 1. max, ≤ 1. max or min, =

3. all matrix elements are 1 2. all RHS terms are 1 2. all RHS terms are 1
3. all matrix elements are 1 3. all matrix elements are 1

min cT x max cT x max cT x

Ax ≥ 1 Ax ≤ 1 Ax = 1
x ∈ Bn x ∈ Bn x ∈ Bn

Generalization: RHS ≥ 1
Application examples:

• Aircrew scheduling: M: legs to cover, N: rosters

• Vehicle routing: M: customers, N: routes
27
A good written example of how to present a model:

[from G. Desaulniers, J. Desrosiers, Y. Dumas, M.M. Solomon and F.

Soumis. Daily Aircraft Routing and Scheduling. Management Science, 1997,
43(6), 841-855]
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

29
Review

• Assignment Problem

• Set Problems: Knapsack problem, facility location problem

30
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

31
Matching
Definition (Matching Theory Terminology)
Matching: set of pairwise non adjacent edges
Covered (vertex): a vertex is covered by a matching M if it is incident to an edge in M
Perfect (matching): if M covers each vertex in G
Maximal (matching): if M cannot be extended any further
Maximum (matching): if M covers as many vertices as possible
Matchable (graph): if the graph G has a perfect matching

P
max we xe
v ∈V
P
xe ≤ 1 ∀v ∈ V
e∈E :v ∈e
xe ∈ {0, 1} ∀e ∈ E

Special case: bipartite matching ≡ assignment problems

32
Vertex Cover
Select a subset S ⊆ V such that each edge has at least one end vertex in S.
P
min xv
v ∈V
xv + xu ≥ 1 ∀u, v ∈ V , uv ∈ E
xv ∈ {0, 1} ∀v ∈ V

Approximation algorithm: set S derived from the LP solution in this way:

SLP = {v ∈ V : xv∗ ≥ 1/2}
(it is a cover since xv∗ + xu∗ ≥ 1 implies xv∗ ≥ 1/2 or xu∗ ≥ 1/2)
Proposition
The LP rounding approximation algorithm gives a 2-approximation: |SLP | ≤ 2|SOPT | (at most as
bad as twice the optimal solution)
P ∗ P
Proof: Let
P x̄ be opt to
P IP. Then xv ≤ x̄v .
∗ ∗
|SLP | = P v ∈SLP 1 ≤ ∈V 2xv since xv ≥ 1/2 for each v ∈ SLP
vP
|SLP | ≤ 2 v ∈V xv∗ ≤ 2 v ∈V x̄v = 2|SOPT |
33
Maximum Independent Set

Find the largest subset S ⊆ V such that the induced graph has no edges

P
max xv
v ∈V
xv + xu ≤ 1 ∀u, v ∈ V , uv ∈ E
xv = {0, 1} ∀v ∈ V

Optimal sol of LP relaxation sets xv = 1/2 for all variables and has value |V |/2.

What is the value of an optimal IP solution of a complete graph?

LP relaxation gives an O(n)-approximation (almost useless)

34
Traveling Salesman Problem

• Find the cheapest movement for a drilling, welding, drawing, soldering arm as, for example, in
a printed circuit board manufacturing process or car manufacturing process

• n locations, cij cost of travel

Variables:
(
1
xij =
0

Objective:
n X
X n
cij xij
i=1 j=1

35
Constraints:
•
X
xij = 1 ∀i = 1, . . . , n
j:j6=i
X
xij = 1 ∀j = 1, . . . , n
i:i6=j

• cut set constraints

XX
xij ≥ 1 ∀S ⊂ N, S 6= ∅
i∈S j6∈S

• subtour elimination constraints

XX
xij ≤ |S| − 1 ∀S ⊂ N, 2 ≤ |S| ≤ n − 1
i∈S j∈S

36
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

37
Modeling Tricks

Objective function and/or constraints do not appear to be linear?

• Absolute values
• Minimize the largest function value
• Maximize the smallest function value
• Constraints include variable division
• Constraints are either/or
• A variable must take one of several candidate values

38
Modeling Tricks I

Minimize the largest of a number of function values:

min max{f (x1 ), . . . , f (xn )}

• Introduce an auxiliary variable z:

min z
s. t. f (x1 ) ≤ z
f (x2 ) ≤ z

39
Modeling Tricks II

Constraints include variable division:

• Constraint of the form
a1 x + a2 y + a3 z
≤b
d1 x + d2 y + d3 z

• Rearrange:

a1 x + a2 y + a3 z ≤ b(d1 x + d2 y + d3 z)

which gives:

(a1 − bd1 )x + (a2 − bd2 )y + (a3 − bd3 )z ≤ 0

40
III “Either/Or Constraints”

In conventional mathematical models, the solution must satisfy all constraints.

Suppose that your constraints are “either/or”:

a1 x1 + a2 x2 ≤ b1 or
d1 x1 + d2 x2 ≤ b2

Introduce new variable y ∈ {0, 1} and a large number M:

a1 x1 + a2 x2 ≤ b1 + My if y = 0 then this is active

d1 x1 + d2 x2 ≤ b2 + M(1 − y ) if y = 1 then this is active

41
III “Either/Or Constraints”

Binary integer programming allows to model alternative choices:

• Eg: 2 feasible regions, ie, disjunctive constraints, not possible in LP.

introduce y auxiliary binary variable and M a big number:

Ax ≤ b + My if y = 0 then this is active

A0 x ≤ b 0 + M(1 − y ) if y = 1 then this is active

42
IV “Either/Or Constraints”
Generally:
a11 x1 + a12 x2 + a13 x3 + . . . + a1m xm ≤ d1
a21 x1 + a22 x2 + a23 x3 + . . . + a2m xm ≤ d2
..
.
am1 x1 + aN2 x2 + aN3 x3 + . . . + aNm xm ≤ dN

Exactly K of the N constraints must be satisfied.

Introduce binary variables y1 , y2 , . . . , yN and a large number M
a11 x1 + a12 x2 + a13 x3 + . . . + a1m xm ≤ d1 + My1
a21 x1 + a22 x2 + a23 x3 + . . . + a2m xm ≤ d2 + My2
..
.
am1 x1 + aN2 x2 + aN3 x3 + . . . + aNm xm ≤ dN + MyN

y1 + y2 + . . . yN = N − K
K of the y -variables are 0, so K constraints must be satisfied
43
IV “Either/Or Constraints”

n
P
At least h ≤ k of aij xj ≤ bi , i = 1, . . . , k must be satisfied
j=1
introduce yi , i = 1, ..., k auxiliary binary variables
n
X
aij xj ≤ bi + Myi
j=1
X
yi ≤ k − h
i

44
V “Possible Constraints Values”

A constraint must take on one of N given values:

a1 x1 + a2 x2 + a3 x3 + . . . + am xm = d1 or
a1 x1 + a2 x2 + a3 x3 + . . . + am xm = d2 or
..
.
a1 x1 + a2 x2 + a3 x3 + . . . + am xm = dN

Introduce binary variables y1 , y2 , . . . , yN :

a1 x1 + a2 x2 + a3 x3 + . . . + am xm = d1 y1 + d2 y2 + . . . dN yN

y1 + y2 + . . . yN = 1

45
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

46
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

47
Uncapacited Facility Location (UFL)
Given: Task: Which depots to open and which depots serve
• depots N = {1, . . . , n} which client

• clients M = {1, . . . , m}
• fj fixed cost to use depot j
• transport cost for all orders cij
(
1 if depot open
Variables: yj = , xij fraction of demand of i satisfied by j
0 otherwise
Objective:
XX X
min cij xij + fj yj
i∈M j∈N j∈N

Constraints:
X n
xij = 1 ∀i = 1, . . . , m
j=1
X
xij ≤ myj ∀j ∈ N
i∈M 48
Outline

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

49
Good and Ideal Formulations

Definition (Formulation)
A polyhedron P ⊆ Rn+p is a formulation for a set X ⊆ Zn × Rp if and only if X = P ∩ (Zn × Rp )

That is, if it does not leave out any of the solutions of the feasible region X .

There are infinite formulations

Definition (Convex Hull)
Given a set X ⊆ Zn the convex hull of X is defined as:
n t
X t
X
conv(X ) = x : x = λi xi , λi = 1, λi ≥ 0, for i = 1, . . . , t,
i=1 i=1
for all finite subsets {x1 , . . . , xt } of X

50
Proposition
conv(X ) is a polyhedron (ie, representable as Ax ≤ b)

Proposition
Extreme points of conv(X ) all lie in X

Hence:
max{cT x : x ∈ X } ≡ max{cT x : x ∈ conv(X )}
However it might require exponential number of inequalities to describe conv(X )
What makes a formulation better than another?
X ⊆ conv(X ) ⊆ P2 ⊂ P1
P2 is better than P1

Definition
Given a set X ⊆ Rn and two formulations P1 and P2 for X , P2 is a better formulation than P1 if
P2 ⊂ P1
51
Example
P
P1 = UFL with i∈M xij ≤ myj ∀j ∈ N
P2 = UFL with xij ≤ yj ∀i ∈ M, j ∈ N

P2 ⊂ P1

P
• P2 ⊆ P1 because summing xij ≤ yj over i ∈ M we obtain i∈M xij ≤ myj

• P2 ⊂ P1 because there exists a point in P1 but not in P2 : m = 6 = 3 · 2 = k · n

P
x10 = 1, x20 = 1, x30 = 1, Pi xi0 ≤ 6y0 y0 = 1/2
x41 = 1, x51 = 1, x61 = 1 i xi1 ≤ 6y1 y1 = 1/2

52
Resume

1. Integer Programming

2. Modeling
Assignment Problem
Knapsack Problem
Set Problems

3. More on Modeling
Graph Problems
Modeling Tricks

4. Formulations
Uncapacited Facility Location
Alternative Formulations

53