COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Federico Della Croce

Federico Della Croce Heuristics 1 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Solving Combinatorial Optimization problems (COPs)

Solution approaches for COPs, can be split into different groups but an
immediate, though rough, classification for these methods, divides them
into two main categories:
• exact algorithms
• heuristic algorithms.

Federico Della Croce Heuristics 2 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Exact Methods - Pros and Cons

Pros.
• a proven optimal (or no feasible) solution can be obtained for every
finite size instance
• information on upper/lower bounds on the optimal solution can be
drawn at any moment
• general modeling framework and general purpose open-source and
commercial solvers

Cons.
• the size of the instances that are practically solvable is limited
• the best exact algorithms are problem specific and often require an
important software development time

Federico Della Croce Heuristics 3 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristic Methods - Pros and Cons

Pros.
• heuristic algorithm of reasonable performance can be typically
developed rather quickly
• can examine a huge number of possible solutions in short
computational time
• many solution frameworks methods

Cons.
• cannot prove optimality
• typically do not provide theoretical bounds on the quality of the
solutions they return
• no general problem modeling framework nor general solvers

Federico Della Croce Heuristics 4 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

•Constructive polynomial time Heuristics

– Myopic (Greedy) algorithms;
– Simplifying the steps of an exact solution procedure
(ex. Beam Search vs Branch and Bound).
•Local Search
– Classic Local Search
• Steepest Descent
• First Improvement

Federico Della Croce Heuristics 5 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

•Improved heuristics
– Metaheuristics
• Iterated Local Search
• Variable neighborhood search
• Tabu Search, Grasp, Simulated Annealing
• Genetic algorithms
• …
– Matheuristics

Federico Della Croce Heuristics 6 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Myopic rules (greedy algorithms)

•The solution is generated in a constructive way

•At each iteration a partial solution is available

and this solution is extended by selecting one
of the possible choices available at that point.

Federico Della Croce Heuristics 7 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Myopic rules (greedy algorithms)

•Example for TSP

NN (Nearest Neighbor) rule:
select the next town to be visited as the
nearest to the current town.

Federico Della Croce Heuristics 8 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Ex. for the uncapacitated p-median problem

•The problem: locate p facilities (medians) on a

network so as to minimize the sum of all distances
from each vertex to each nearest facility.
•Hakimi theorem: the medians belong to the
vertices set.
•Can be seen as a pure mathematical problem:
given a nxm matrix D, select p columns of D in
order that the sum of minimum coefficients in each
line within these columns be the smallest possible.
•Computational complexity: NP-hard for general
values of p.

Federico Della Croce Heuristics 9 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

ILP model for the p-median problem

xij = 0/1 variable = 1 if vertex i
is allocated to vertex j.

dij = cost for allocating i to j.

V = Vertex set.

Federico Della Croce Heuristics 10 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Example for the p-median problem

P=2
Medians = {2,3} Total cost = 8
1 2 3 4 5
1 0 2 5 7 9
2 2 0 4 7 3
3 5 4 0 3 6
4 7 7 3 0 4
5 9 3 6 4 0

Federico Della Croce Heuristics 11 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Myopic rules (greedy algorithms)

•Ex. for the p-median problem

– (1-median) select as first median the nearest node
on average to all the other nodes;
– (adding medians) add a new median to the current
solution in such a way that the solution is improved
as much as possible.At each iteration a partial
solution is available and this solution is extended at
that point.

Federico Della Croce Heuristics 12 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Improving myopic rules

•“Randomization”:
– make randomly one of the best “k” choices;
– make a random choice among the choices that are at
most p% worse than the best choice (according to the
myopic rule).

•Whenever a random choice is foreseen:

Multi-start

Federico Della Croce Heuristics 13 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

BEAM SEARCH
(Truncated Branch and Bound)
•Search tree (as for Branch and Bound)
•Breadth First strategy
•At each level of the search tree keep only the
best k nodes (k = Beam width).
•How to evaluate the nodes:
– Greedy algorithms;
– Optimal solution of a relaxed problem;
– Weighted sum of 1) and 2).

Federico Della Croce Heuristics 14 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

COMPLEXITY OF BEAM SEARCH

•D = # search tree levels

•E = # children of a generic node
•k = Beam width
Hence
•k˜E = # nodes per level
•J = O(D˜k˜E) = total number of opened nodes
(to be multiplied by the complexity for
evaluating a single node)

Federico Della Croce Heuristics 15 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL (NEIGHBORHOOD) SEARCH

•In general, a local search consists in searching for
a solution xj belonging to the neighborhood N(xi) of
xi, according to some predefined rules.
•Neighborhood: to each x& is associated N(x)Ž&.
•Example: & = set of binary vectors
xk = current solution with xk &.
•N(xk) = set of solutions x& that can be obtained
from xk by swapping a single component from 0 to 1
or viceversa.

Federico Della Croce Heuristics 16 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL (NEIGHBORHOOD) SEARCH

•We assume that xk N(xk).

•Given an initial arbitrary solution x1X, at each

step k we search for a new solution xk+1N(xk).

•The local search proceeds by iterations

searching at each iteration to find a new
solution belonging to the neighborhood of the
current solution xk and that is better than xk.

Federico Della Croce Heuristics 17 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of a local search

1.Initialization: selection of an initial (feasible)

solution x1 as current solution and computation
of its objective function value.
2.Neighborhood generation: construction of a
(totally feasible) neighborhood N(xi) of the
current solution xi and selection of a new
solution xi+1 N(xi).

Federico Della Croce Heuristics 18 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of a local search

3. Acceptance test: check whether solution xi+1

can be accepted; in the negative case the
current solution remains xi; in the positive
case the current solution must be updated: xi+1
becomes the new current solution.
4. Stopping test: if the test is positive, STOP, the
best solution found represents the output of
the procedure, else GO TO step 2.

Federico Della Croce Heuristics 19 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH: INITIALIZATION

•The initial solution is typically computed by

means of a polynomial time (greedy) heuristic
procedure.
If the problem is so “hard” that no simple
procedures are available to get a feasible
solution
– THEN
•Local search applied to the feasibility phase:
– the objective function to be minimized becomes the
sum of infeasibilities.

Federico Della Croce Heuristics 20 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION

•Problem representation.

•Definition of the neighborhood and

computation of its size.

•Complexity of computing a neighbor solution.

Federico Della Croce Heuristics 21 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)

•Solution representation: string

2 3

1 4 1 2 3 4 5 6

6 5

Federico Della Croce Heuristics 22 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)

•Neighborhood: swap two arcs (the solution

remains feasible) 2 3

1 4

6 5

Federico Della Croce Heuristics 23 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)

•The arcs swap corresponds to

|1|2|3|4|5|6| Æ |1|2|5|4|3|6|

Federico Della Croce Heuristics 24 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(SYMMETRIC TSP)

•The arcs swap corresponds to

|1|2|3|4|5|6| Æ |1|2|5|4|3|6|

•a pairwise interchange between two elements of the

string while applying a twist on the subsequence which
is between the two elements.

Federico Della Croce Heuristics 25 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Complexity of computing a neighbor

solution

•Constant # of sums for any size of the problem

•f(xi+1) = f(xi) + c3,6 + c2,5 – c2,3 – c5,6

where ci,j = distance between cities i and j

Federico Della Croce Heuristics 26 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)

n
•Solution representation:
min ¦  c j ˜ x j
j 1

¦w
j 1
j xj dW

xij^0,1`

x1 x2 x3 x4 x5 x6 x7 x8

0 0 1 0 1 1 0 1

Federico Della Croce Heuristics 27 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)

•Neighborhood:

– swap the value of a variable;

– Swap xi with xj if xi + xj = 1;
– Non feasible solutions may be generated.
– Neighborhood size: O(n2).
– Complexity of computing a neighbor solution :
constant.

Federico Della Croce Heuristics 28 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)

•Another representation for the 0/1 KNAPSACK

Given any sequence of the xi variables, the variables are
set to 1 whenever it is possible.

Federico Della Croce Heuristics 29 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)

•For the sequence | x1 | x2 | x3 | x4 | x5 | x6 | x7 | x8 |

we get:

x1 = 1 IF w1 d W ELSE x1 = 0
x2 = 1 IF w1 ˜ x1 + w2 d W ELSE x2 = 0
x3 = 1 IF w1 ˜ x1 + ... + wi-1 ˜ xi-1 + wi d W ELSE x3 = 0
etc.

Federico Della Croce Heuristics 30 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)

•New neighborhood:

– Swap a pair of variables, e.g. x2 and x7, inducing the

new sequence | x1 | x7 | x3 | x4 | x5 | x6 | x2 | x8 |.
We get:
x1 = 1 IF w1 d W ELSE x1 = 0
x7 = 1 IF w1 ˜ x1 + w7 d W ELSE x7 = 0
x3 = 1 IF w1 ˜ x1 + w7 ˜ x7 + w3 d W ELSE x3 = 0
etc.

Federico Della Croce Heuristics 31 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(0/1 KNAPSACK)

•New neighborhood size = O(n2).

•Complexity of computing a neighbor solution: O(n).

•Why the solutions space moves from 2n to n! ?

Federico Della Croce Heuristics 32 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP)

• K salesmen partition the cities in K subtours in

order to minimize the distance.
• First representation: two-levels decomposition.
– Assign the cities to the salesmen;
– Solve K TSP problems.

Federico Della Croce Heuristics 33 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP – first representation)

•Two-level local search:

– Level 1: assignment of the cities to the salesmen.

– Level 2: solution of K TSP deduced from the assignment

at level 1.

Federico Della Croce Heuristics 34 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP – second representation)

•Another representation:
– introduce k fictituous cities all with the same location (the
departure city of the salesmen) and solve a unique aggregated
TSP

Federico Della Croce Heuristics 35 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(K-TSP – second representation)

•Example with 3 salesmen and 8 cities.

•Introduce 3 fictituous cities A, B, e C.

•The sequence | A|1|2|3|B|4|5|6|C|7|8| indicates that

salesman A will visit cities 1,2,3 in that order and then get
back to his home city; same occurs to salesman B with cities
4,5,6 and salesman C with cities 7,8.

Federico Della Croce Heuristics 36 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(Graph Coloring)

•A non-oriented graph G(N,E) is said r-cromatic if it is

possible to color its vertices with r colors in such a way
that no adjacent vertices have the same color.
•The smallest value of r such that G(N,E) is r-cromatic is
defined as the cromatic number J(G).
•The Graph Coloring problem consists in searching for
J(G).

Federico Della Croce Heuristics 37 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(Graph Coloring)
•Solution representation:
– The Graph Coloring problem can be tackled by
iterating on the number J of available colors and
searching for a feasible solution for each value of J.
– For any given value of J
• search for a J-partition of G minimizing the total
number of crossing edges (edges belonging to
the same partition);
• Every J-partition with no crossing edges is a J-
coloring of G;

Federico Della Croce Heuristics 38 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

LOCAL SEARCH:
NEIGHBORHOOD GENERATION
(Graph Coloring)

•The neighborhood N(x) of this J-partitioning problem is

given by all J-partitions reachable from the J-partition x
moving a bad vertex (a vertex which is adjacent to at least
another vertex of the same partition) from a partition to
another.

Federico Della Croce Heuristics 39 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

CLASSIC LOCAL SEARCH:

STEEPEST DESCENT STRATEGY

1.Initialization: selection of an initial solution x1 &

– x’ = current solution = x1
2.Generation of the Neighborhood N(x’) of the current
solution x’ and evaluation of each neighbor. Let x* N(x’)
the best neighbor solution.
3.Acceptance Test:
– IF x* improves upon the current solution x’ THEN
– set x’= x* and GO TO 2, ELSE GO TO 4.
4.Stop.

Federico Della Croce Heuristics 40 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

CLASSIC LOCAL SEARCH:

FIRST IMPROVEMENT STRATEGY

1.Initialization: selection of an initial solution x1 &

– x’ = current solution = x1
2.Generation of the Neighborhood N(x’) of the current
solution: N(x’) = ^ x2, x3, ..., xn`.
3.Neighborhood evaluation:
– FOR i = 2, ..., n DO:
– IF xi improves upon the current solution x’ THEN :
– set x’= xi and GO TO 2.
4.Stopping test: IF no improving solution exists in the
neighborhood THEN STOP.

Federico Della Croce Heuristics 41 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Metaheuristics
Iterated Local Search
• Iteratively applies neighborhood search steps in order to reach
different local minima.

Each time a new local minimum is reached, a perturbation is applied

to the corresponding solution in order to get a new solution to
which re-apply the neighborhood search.

• The perturbation is typically obtained by randomly generating a

neighbor of the current local minimum within a much wider
neighborhood than the one considered in the local search.

• An acceptance test determines whether to accept the new local

minimum or get back to the old one.

• A stopping test determines the end of the procedure.

Federico Della Croce Heuristics 42 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of the Iterated Local Search

1. Initialization:
– Select an initial solution x1 &;
– Apply a local search procedure to the initial solution x1
generating a local minimum that represents the current
solution.
2. Repeat:
– Perturbation of the current solution and corresponding
generation of a new solution x’;
– Application to solution x’ of a local search procedure yelding
a new local minimum x’’;
– Acceptance test: if the answer is positive, x’’ becomes the
new current solution;
Until Stopping test is satisfied.

Federico Della Croce Heuristics 43 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Metaheuristics
Variable Neighborhood Search

• Can be seen as a special kind of Iterated Local

Search.

• Foresees the use a series of neighborhood

structures handled in a hierarchical way in the
perturbation phase plus a (typically) unique
neighborhood structure for the local search phase.

• The neighborhood structures of the perturbation

phase are typically sorted by increasing
neighborhood size.

Federico Della Croce Heuristics 44 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Metaheuristics
Variable Neighborhood Search
• The local search step is typically of a first
improvement type.

• A new local minimum is accepted only if it improves

upon the current solution.

• Each time there is no improvement, the

neighborhood considered in the perturbation phase
is modified (getting larger).

• The procedure stops once all neighborhoods of the

perturbation phase have been considered.

Federico Della Croce Heuristics 45 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of the

Variable Neighborhood Search

1. Initialization:
– Select an initial solution x1 &;
– Select a set of neighborhood structures Nk with
k=1,…, kmax to be used in the procedure;
– k = 1; x* = current solution = x1.

Federico Della Croce Heuristics 46 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of the

Variable Neighborhood Search

2. Repeat:
– Generate randomly a solution x’ belonging to the
neighborhood Nk of the current solution x*;
– Apply a local search procedure to solution x’
yelding a new local minimum x’’;
– Acceptance test: IF x’’ improves upon x*, THEN
x*=x’’; k = 1; LOOP;
– ELSE k = k +1; LOOP;
Until k = kmax OR stopping test is satisfied.

Federico Della Croce Heuristics 47 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Metaheuristics
Variable Neighborhood Descent

• Represents an extension/generalization of Variable

Neighborhood Search.

• Foresees the use a series of neighborhood

structures handled in a hierarchical way in the
perturbation phase: these structures are however
used directly in the local search phase.

Federico Della Croce Heuristics 48 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of the

Variable Neighborhood Descent
1. Initialization:
– Select an initial solution x1 ∈Χ;
– Select a set of neighborhood structures Nk with k=1,…,kmax to be used
in the procedure;
– k = 1; x* = current solution = x1.
2. Repeat:
– Generate a random solution x’ belonging to the neighborhood Nk of x*.
– Apply a local search procedure to solution x’ yelding a new local
minimum x’’;
– IF x’’ improves upon x*, THEN x* = x’’; k = 1; LOOP;
ELSE k = k+1; LOOP;
Until k = kmax OR stopping test satisfied.

Federico Della Croce Heuristics 49 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Metaheuristics
Greedy Randomized Adaptive Search Procedure

• It represents an evolution of the classic local search with multistart;

• At each iteration, the local search is preceded by a greedy

constructive procedure where the greedy choices are executed with
some randomness (greedy randomized type procedure).

• “Greedy randomized”: at each step of the greedy procedure the best k

choices (according to the greedy procedure) are selected and one of
them is randomly applied.

• Only two parameters are required:

– The value of k (how many choices at each step?)

– # multistart iterations

Federico Della Croce Heuristics 50 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of the

Greedy Randomized Adaptive Search Procedure

1. x* = current solution = +infty;

2. FOR k = 1,…,kmax DO
• Generate a solution xk by means of a greedy randomized
constructive procedure;
• Apply a local search procedure to the initial solution xk
yelding solution xk’;
• If xk’ is better than x*, then x* = xk’.

Federico Della Croce Heuristics 51 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Metaheuristics
Simulated Annealing
• First Improvement like Local Search: tries to avoid local minima.

• Analogy with the physical process of Annealing, namely the slow

cooling of a metal to reach a solid state with minimum energy.

• Same principle in Simulated Annealing:

• Initially both better and worse solutions (w.r.t. the current solution)
are accepted (so-called phase of “high temperature”) and this allows a
good exploration of the solutions space.

• Then the “temperature” is gradually reduced and the selection of new

moves becomes more and more selective.

• At the end just improving moves are accepted.

Federico Della Croce Heuristics 52 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of
Simulated Annealing
1) Initialization:
Select an initial solution x1∈Χ
Set the current solution: x’ = x1
Set to 0 the iterations counter: I = 0.
2) Neighborhood generation N(x’) of the current solution.
3) Select randomly a solution x’’ from N(x’);
Update the iterations counter: I = I + 1;
IF x’’ is better than or equal to x’ THEN x’ = x’’,
ELSE: −
F ( x '') − F ( x ')

Compute probability p = e T (I )
Select a random real value p∈[0,1]
IF p< p
THEN x’ = x’’,
ELSE keep the current solution unchanged
4) Stopping test: IF the stopping test is positive STOP,
ELSE GO TO 2.

Federico Della Croce Heuristics 53 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Values of the temperature T(I)

The temperature has a typical behaviour as the one depicted in the figure
below.
T (I)

T0

T1

T
T2(I)
T3

L 2L 3L 4L I

• T (k⋅L) = Tk = αk ⋅ T0
• T0 = initial temperature: usually the value is computed in such a way to
have an acceptance probability w.r.t. an average neighbor solution.
• α ∈[0,1] = cooling coefficient (e.g. α = 0.98)
• L = length of the “plateau” (typically proportional to the size of N()).

Federico Della Croce Heuristics 54 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Stopping test
Typically the following alternatives are considered:

– The solution did not improve more than ε1% for a given
value of consecutive iterations (e.g. k⋅L).
– The total number of accepted moves is below ε2% in the
last k2⋅L iterations.
– A given threshold S* of has been reached.
– An overall CPU time limit τ* has been reached.

The calibration of such parameters

must be derived based on experimental testing.

Federico Della Croce Heuristics 55 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Metaheuristics
Tabu Search
• Steepest Descent like local search: tries to avoid
local minima.
• At each iteration all neighbor solutions (or a proper
subset of the considered neighborhood) are
evacuate.
• All decisions are deterministic.
• Main features
– “Tabu moves”
– Tabu List
– Aspiration Criteria (e.g.: an overall improving solution is always
accepted)
– Intensification / diversification in the solutions space

Federico Della Croce Heuristics 56 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Main steps of Tabu Search

1) Initialization:
Select an initial solution x1∈Χ; Set the current solution: x’ = x1;
Empty the Tabu List. TL = {}; Set to 0 the iterations counter: I = 0.
2) Select the neighborhood N(x’) of the current solution and evaluate N(x’) or a subset
N'(x’).
Let x* be the best found solution in N(x’) such that:
x* improves upon the best currently available feasible solution
(aspiration criterion)
or
m(x’,x*) does not belong to the Tabu List TL, that is the move from x’ to x* is not
a Tabu move
THEN
set x’ = x*;
insert the move m(x*,x’) in the Tabu list, that is TL=TL ∪ m(x*,x’);
IF the Tabu List size has reached a given threshold,
THEN eliminate from TL the “oldest” move;
Update the iterations counter: I = I + 1.
3) Stopping test: IF the stopping test is positive STOP, ELSE GO TO 2.

Federico Della Croce Heuristics 57 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Implementation of a
Tabu Search procedure
To implement a Tabu Search procedure the following
items must be considered:
– Definition of a neighborhood or of a subset of such
neighborhood and evaluation of all its components.
– Selection of the moves to be inserted in the Tabu List and
representation of such moves.
– Length of the Tabu List.
– Selection of the aspiration criterion (criteria).
– Selection of some stopping test.
– Intensification / diversification techniques.

Federico Della Croce Heuristics 58 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Genetic algorithms

• Developed: USA in the 1970’s

• Early names: J. Holland, K. DeJong, D. Goldberg
• Now part of a more complex framework: the
Evolutionary Algorithms
• Attributed features:
• not too fast
• good heuristics for combinatorial problems but…
usually not the best
• Easy to implement, “applicable to every problem”

Federico Della Croce Heuristics 59 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Genetic algorithms

• Imitation of natural evolutionary processes.

• Heuristic combinatorial optimization approach that is based on

the following evolutionary operators

– selection
– reproduction
– mutation

Federico Della Croce Heuristics 60 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Genetic algorithms

Genetic algorithms possess the following characteristics:

• The search is not entrapped in local minima.

• They are naturally suitable to be applied on parallel

architectures parallele.

• Easy hybridization with other heuristics.

Federico Della Croce Heuristics 61 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Genetic algorithms for C.O.

1. The search space of all possible solutions is mapped onto a set of finite
strings; the alphabet of symbols is of finite dimensions, in a typical case
it includes two symbols (0 and 1). Each string (chromosome) has a
corresponding point in the search space.

2. An iteration threshold is chosen, typically a number of generations.

3. A set of solutions is selected (usually at random) and this constitutes the

initial population, i.e. the first generation.

4. A fitness is computed for each of the individuals; fitness is a parameter

which reflects the quality of the solution represented by the individual.

5. A set of individuals is selected to reproduce itself, the selection takes

place in a random way but with a probability proportional to fitness.

Federico Della Croce Heuristics 62 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Genetic algorithms for C.O.

6. From the selected set some progeny is generated by applying different
genetic operators.

7. Some of the old individuals are eliminated from the population to allow
the entrance of the new ones.

8. The new individuals' fitness is computed and they are included in the
population. This step marks the end of a generation.

9. IF the iteration threshold is reached, THEN the process is finished and

the best chromosome is returned, ELSE GO TO step 5.

Federico Della Croce Heuristics 63 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Constructing a Genetic Algorithm

The main phases in the construction of a genetic algorithm are the following:

• Coding
• Initial population
• Fitness function definition
• Genetic operators
– Crossover
– Mutation
– Inversion
• Genetic operators selection
• Reproduction
• Parameters calibration

Federico Della Croce Heuristics 64 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Coding

• Consists in defining a representation of the feasible solutions in terms of

chromosomes.

• strictly depends from the problem. For a given problem many options of
coding are possible (traditionally a binary alphabet is used for the
chromosomes), but the genetic operators definition is linked to the used
coding.

Federico Della Croce Heuristics 65 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Initial population

• random generation of several of chromosomes

• initial population elements generated according to

simple heuristics applied to the considered problem

Federico Della Croce Heuristics 66 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Fitness

• The fitness is a parameter that measures the quality of the solution of

any given chromosome;

• it is used to compute chromosome reproduction probability (the larger

the fitness function, the larger the probability to select that chromosome
for reproduction);
• example:
– fitness = f(x) in a maximization problem
– fitness = 1/f(x) in a minimization problem;

Federico Della Croce Heuristics 67 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Classic genetic operators

• STANDARD CROSSOVER
Operates on a couple of chromosomes: the “children" are originated from the
“parents” swapping the genes between two cutting sites ramdomly
chosen.

parent 1 1 0 1 | 0 1 1 1 | 1 0
parent 2 0 0 0 | 1 1 0 1 | 0 1
^ ^ cutting sites
child 1 1 0 1 | 1 1 0 1 | 1 0
child 2 0 0 0 | 0 1 1 1 | 0 1

Federico Della Croce Heuristics 68 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Classic genetic operators

ORDER CROSSOVER

• Two cutting sites are ramdomly chosen.

• Choose a a subsequence of a list from one parent and preserve

the relative order of the items from the other.

• Each child is obtained by selecting the crossing section of a

parent and filling the remaining elements with the missing
symbols according to the relative order of the other parent.

Federico Della Croce Heuristics 69 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Classic genetic operators

ORDER CROSSOVER (example)

Parent 1 = (1 2 3 | 5 4 6 7 | 8 9) Parent 2 = (4 5 2 | 1 8 7 6 | 9 3)
Child 1 = (x x x | 5 4 6 7 | x x) Child 2 = (x x x | 1 8 7 6 | x x)

The loading in parent 1, starting from its second cut point, is

9 o 3 o 4 o 5 o 2 o 1 o 8 o 7 o 6.

Remove the items already in child 1, obtaining the partial tour

9 o 3 o 2 o 1 o 8.

Insert this partial list after the second cut point of child 1, obtaining
Child 1 = (2 1 8 | 5 4 6 7 | 9 3 ). Similarly, child 2 = (3 4 5 | 1 8 7 6 | 9 2).

Federico Della Croce Heuristics 70 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Classic genetic operators

MUTATION

Operates on a single chromosome: a gene is ramdomly substituted by

another symbol of the alphabet. It generally is applied to a minimal part of
the chromosomes (ramdomly chosen).

1 0 1 0 1 1 1 1 0
^
1 0 0 0 1 1 1 1 0

Federico Della Croce Heuristics 71 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Classic genetic operators

INVERSION

Operates on a single chromosome: the order between the cutting sites is

inverted

1 0 1 | 0 1 1 1 | 1 0
^ ^
1 0 1 | 1 1 1 0 | 1 0

Federico Della Croce Heuristics 72 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Genetic operators selection

• If the algorithm uses more than one operator, also the genetic operators
selection can be executed in a probabilistic way.

• The operators selection probabilities may be constant or evolve suring

the process.

• Mutation and inversion typically have low probability to be used.

However they may become very useful whenever the population is quite
uniform.

Federico Della Croce Heuristics 73 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Reproduction
• Generational replacement (the new generation fully substitutes the old
one).

• Elitist technique (the best element is always kept in the new generation;
as a variant several “best elements” are kept).

• Steady state technique (most of the element are kept and just a few
elements are substituted; typically, the worst element in probability are
eliminated).

Federico Della Croce Heuristics 74 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Heuristics

Parameters calibration
• Various parameters determine the behavior of a genetic algorithm. The
main ones are the following:

• Population size;

• Percentage of elements of a population substituted in the following

population;

• Number of generations;

• Probability in applying the different genetic operators.

Federico Della Croce Heuristics 75 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Hybridization

• Combination (hybridization) of techniques is a popular and well

established practice in the metaheuristic field.
• The popularity, success and importance of that specific line of
research is well documented by the huge number of successful
reported applications.
• So, combining techniques with complementary advantages, such as
exact and metaheuristic approaches appears to be a winning chance.
• Recently, the hybridization of exact methods and metaheuristic has
originated the so-called Matheuristics1 .

• Hereafter two slides stolen from M. Fischetti one of the inventors of

matheuristics.
1
Matheuristics 2006, 1st International Workshop on Mathematical Contributions to
Metaheuristics, 27-30 August 2006, Bertinoro (FC)
Federico Della Croce Heuristics 76 / 106
 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristics

MIP solvers for hard optimization problems

• Mixed-integer linear programming (MIP) plays a central role in modelling difficult-to-solve

(NP-hard) combinatorial problems

• General-purpose (exact) MIP solvers are very sophisticated tools, but in some hard cases they
are not adequate even after clever tuning

• One is therefore tempted to quit the MIP framework and to design ad-hoc heuristics for the
specific problem at hand, thus loosing the advantage of working in a generic MIP framework

• As a matter of fact, too often a MIP model is developed only “to better describe the problem” or,
in the best case, to compute bounds for benchmarking the proposed ad-hoc heuristics

Can we devise an alternative use of a general-purpose MIP solver, e.g.,

to address important steps in the solution process?

2
Federico Della Croce Heuristics 77 / 106
 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristics

MIPping critical sub-tasks in the design of specific algorithms

We teach engineers to use MIP models for solving their difficult problems
(telecom, network design, scheduling, etc.)

Be smart as an engineer!

Model the most critical steps in the design of your own algorithm
through MIP models, and solve them (even heuristically) through a
general-purpose MIP solver…

6
Federico Della Croce Heuristics 78 / 106
 COMBINATORIAL OPTIMIZATION: heuristic approaches

Framework Description

• The over guiding framework of our approach is the Local Search

approach.
• Problem tackled are all formulated as MIPs.
• We rely on (partial) starting solutions that can be generated by a
MIP solver or by any problem specific heuristic.
• We generate exponential size neighborhoods and we handle them by
means of a MIP solver2 .
• Neighborhoods are defined in such way that they are “linear
constraints” of the corresponding MIP formulation of the problem.

2
Notice that also other exact methods can be applied
Federico Della Croce Heuristics 79 / 106
 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristic applications

• Nurse rostering problem.

• Multi-dimensional knapsack problem.

Federico Della Croce Heuristics 80 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Nurse Rostering Problem – Personnel Shifts

MARCH 2016 1 2 3 4 5 6 7 8 9 10 11 12 13 14
Tu We Th Fr Sa Su Mo Tu We Th Fr Sa Su Mo
HEAD NURSE H H O O O O
Nurse 1 N N O O M A N N O O A A A N
Nurse 2 A O M M O M M N N O O M M A
Nurse 3 H A H N H H M A N N O O O O
Nurse 4 O A A A N O O M M N N O O M
Nurse 5 O M N O O M A O A A O N O O
Nurse 6 N N O O M A A A O M M A N O
Nurse 7 M A O A A N N O O M M A A O
Nurse 8 M O N N O O A A A O M N N O
Nurse 9 O M A O A A O M M A A O O M
Nurse 10 (50%) M O O O N N O O O O O M M N
Nurse 11 A O M M O M M N N O O M M A
Nurse 12 N O O A A N N O O A A O A A
Nurse 13 O M A N N O O M A N N O O M
Nurse 14 (Mat.) O O M M M O O H H H H H N N
Nurse 15 H H H H H O O H H H H H O O
Nurse 16 A N N O H O O H M M N N O O

LEGENDA:
M = Morning; A = Afternoon; N = Night
H = Holiday; O = Off/Rest;

Federico Della Croce Heuristics 81 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Nurse Rostering Problem(s) (NRP)

• The problem considered is based on a real situation encountered in a

ward of a private hospital.
• At the end of each month, the nurses’ shifts for the following month
must be scheduled.
• The ward is made up of a given number of hired nurses, but the
timetable of these nurses is not always sufficient to cover the legal
minimum of personnel presence in every shift.
• The hospital management can make use of freelance nursing staff to
cope with the minimum demand of personnel.
• Freelance staff has a cost.
• The problem consists in assigning a working shift or a day off to each
nurse, on each day, in order to reduce the outsourced work.

Federico Della Croce Heuristics 82 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP What is needed?

1 Problem description (head nurse).

• Problem specs.
• Legal regulations.
• Special regulations (commons, etc.).
• ...
2 Problem formulation.
• Constraints.
• Objective Function.

3 Solution Approach.
• Solution algorithm.

Federico Della Croce Heuristics 83 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Problem Description

• Labor Contracts rules:

• no more than 2 consecutive Night shifts
• no sequence Night–Morning shift.
• no complete freelance assignment for shifts.
• ...
• Operational constraints
• minimum personnel (ex. Morning 3, Afternoon 3, Night 2)
• shifts number and type balance among nurses
• holidays requests.
• ...
• Special constraints
• maternity
• part–time
• ...

Federico Della Croce Heuristics 84 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Problem formulation – Variables and Objective

Function

• The considered problem can be formulated as an IP model.

• With n nurses and m days in a month, it is sufficient to introduce a
set of 0/1 variables denoted xi,j,k (i = 1, ...n, j = 1, ...m, k = 1, ...t)
indicating if nurse i is assigned to shift k on day j.
• Correspondingly, a positive integer variable yj,k is introduced
indicating the number of freelance nurses needed in working shift k
on day j.
• Objective Function: min m
P Pt
j=1 k=1 yj,k

Federico Della Croce Heuristics 85 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Problem formulation – Constraints

• no sequence Night–Morning shift

xi,j,N + xi,j+1,M ≤ 1 ∀i = 1, . . . , n, j = 1, . . . , m − 1

• no more than 2 consecutive Night shifts

P2
s=0 xi,j+s,N ≤2 ∀i = 1, . . . , n, j = 1, . . . , m − 2

• minimum personnel
Pn
i=1 xi,j,k ≥ min persj,k ∀j = 1, . . . , m, k = 1, . . . , 3

• holiday requests
xi,j,H = 1 ∀i, j ∈ holiday requestsi,j

• at least 4 and at most 8 Afternoon shifts for each nurse

Pm
4≤ j=1 xi,j,A ≤8 ∀i = 1, . . . , n

Federico Della Croce Heuristics 86 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach

• The first attempt is done to verify the problem model against the
real–life situation so as to check coherence and correctness.
• We can model and solve the problem by means of a MIP solver.

Solving....

Federico Della Croce Heuristics 87 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach

• The first attempt is done to verify the problem model against the
real–life situation so as to check coherence and correctness.
• We can model and solve the problem by means of a MIP solver.

Solving....

NO FEASIBLE SOLUTION EXISTS!!!

Federico Della Croce Heuristics 87 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach – Possible mistakes 1

1 Bugs into the model.

2 Constraints violations: given the constraints it is impossible to

generate a feasible roster.

Federico Della Croce Heuristics 88 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach – Possible mistakes 2

• The head nurse has generated rosters since many years.

• How it is possible?

• Check of a head nurse’s solution

Federico Della Croce Heuristics 89 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach – Possible mistakes 2

• The head nurse has generated rosters since many years.

• How it is possible?

• Check of a head nurse’s solution

• UNFEASIBLE!!!

Federico Della Croce Heuristics 89 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach – Possible mistakes 2

• The head nurse has generated rosters since many years.

• How it is possible?

• Check of a head nurse’s solution

• UNFEASIBLE!!!

• This means that the head nurse violates some given constraints while
generating rosters.

Federico Della Croce Heuristics 89 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach – how to go on?

• Identification of the subset of constraints usually (smoothly) violated

by the head nurse.
• Those constraints are then relaxed so as to find feasible solutions.

Relax means:
• accept (small) violations
ex. no more than 4 consecutive working days but a fifth day is accepted once a
month.
• enlarge feasible solution space
ex. minimum number of Afternoons shifts from 4 to 3.

Federico Della Croce Heuristics 90 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Approach – Recap

Having relaxed constraints we now have:

• new problem description,
• new constraint set,
• guarantee of feasible solutions.

Federico Della Croce Heuristics 91 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Algorithm – 1

• It is known that solvers are not particularly efficient on large size

nurse rostering problems.
• One hour of CPU time is not sufficient to find the optimal solution or
even good solutions.

• We want to improve solutions quality given the same CPU time (1 h.)
• Matheuristic algorithm. This means that:
• we can reuse the MIP model
• we can take advantage of solvers
• we have to design neighborhoods and implement a local search phase.

Federico Della Croce Heuristics 92 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Algorithm – 2

Given a feasible solution X̄ we define the following neighborhoods:

1 Hamming distance
N1 (X̄ , h1 ) sets the number of variables xi,j,k that can chance value w.r.t the
current solution is less or equal than h1
2 Days slot
N2 (X̄ , h2 ) defines all subset of size h2 of consecutive days that can change
value.
3 Nurse slot
N3 (X̄ , h3 ) defines all subset of size h3 of consecutive nurses that can change
value..

Federico Della Croce Heuristics 93 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Algorithm – 3

• All described constraints can be modeled as linear constraints thus

any MIP solver is able to solve them.

• Neighborhoods define portions of solutions that remain constant while

trying to optimize portions were we let variables free.

• We have generated subproblems very similar to the global problem

but of a smaller (controlled) size.

Federico Della Croce Heuristics 94 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Algorithm – An Example

Federico Della Croce Heuristics 95 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Algorithm – Pseudocode

Begin
(1) choose an initial feasible solution z
(2) initialize parameters h1, h2 and h3
(3) iter := 1
(4) while iter <=3 do
(5) add neighborhood constraint to the model -- N_iter
(6) set the variables not belonging to the considered neighborhood
to their corresponding values in z
(7) solve subproblems within local time limit
(8) if f(x) < f(z) then do
(9) update solution
(10) iter:=1
(11) else do
(17) iter:=iter+1
(19) end if
(20) if the overall CPU time is greater than 3600 s. then
(21) EXIT
(22) end if
(24) end-do
End

Federico Della Croce Heuristics 96 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Algorithm – Results 1

Federico Della Croce Heuristics 97 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Solution Algorithm – Results 2

Federico Della Croce Heuristics 98 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

NRP Conclusions

1 We have built a MIP model for creating rosters.

2 We had to refine the model so as to guarantee feasibility.

3 We have obtained starting feasible solutions by means of a MIP

solver .

4 We have developed an ad–hoc algorithm to improve solution quality.

5 We have heuristically applied and exact method.

Federico Della Croce Heuristics 99 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristics for MKP

• MKP: well known tough NP-Hard model that represents several real
problems.
• There exist plenty of heuristics and meta-heuristics as much as exact
approaches for this problem.
• OR-Library benchmark problems with 100 - 250 - 500 variables and 5
- 10 - 30 constraints.
• Typically 0/1 multi-dimensional knapsack problems with 30 - 35
variables can be solved (by XPRESS-CPLEX etc.) to optimality
within few seconds.
• Most of the benchmark problems with 500 variables and 30
constraints are currently open with respect to optimality

Federico Della Croce Heuristics 100 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristics for MKP: ILP model and notation

n
X
max pj xj
j=1

n
X
wij xj ≤ ci i = 1, ...m
j=1
xj ∈ {0, 1} j = 1, ...n
that is
• a set of n items with profits pj > 0
• a set of m resources with capacities ci > 0 ;
• each item j consumes an amount wij ≥ 0 from each resource i ;
• the selected items must not exceed the resource capacities ci ;
• the 0 − 1 decision variables xj indicate which items are selected ;
• the goal is to choose a subset of items with maximum total profit.
Federico Della Croce Heuristics 101 / 106
 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristics for MKP: notation

• Z ∗ = the value of the continuous relaxation solution X ∗

• Z = the value of the optimum X
• rj = the reduced cost of the non basic variable xj∗
• uj = the upper pseudo-cost of the basic variable xj∗
• lj = the lower pseudo-cost of the basic variable xj∗
• parameters rj , uj , lj are available from the continuous relaxation
solution.

Federico Della Croce Heuristics 102 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

MKP: properties of the continuous relaxation solution X ∗

• For all non-basic 0-1 xj variables such that xj∗ = 0,

IF xj = 1 then Z ≥ Z ∗ + |rj |.
• For all non-basic 0-1 xj variables such that xj∗ = 1,
IF xj = 0 then Z ≥ Z ∗ + |rj |.
• In other words, the larger |rj |, the higher the probability that xj∗ = xj .
• For all basic 0-1 xj variables,
IF xj = 1 then Z ≥ Z ∗ + |uj |(1 − xj∗ );
IF xj = 0 then Z ≥ Z ∗ + |lj |xj∗ .
• In other words, if |uj |(1 − xj∗ ) < |lj |xj , it is more likely to have xj∗ = 1
and viceversa.
• IF Z ∗ is reliable, the number of variables where xj∗ = 1 − xj is limited.

Federico Della Croce Heuristics 103 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

MKP: A basic core problem approach

• Compute Z ∗ .
• Fix the n − k non-basic xj variables with largest |rj | to their xj∗ value.
• Solve to optimality (or with a time limit) the remaining core problem
with k variables where no more than γ variables are such that
xj∗ = 1 − xj .
• Fair values of k and γ are 50 and 10.

Such a simple approach dominates the best performing metaheuristics

(among others, a genetic algorithm)

Federico Della Croce Heuristics 104 / 106

 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristics: discussion (1)

• The most promising way to apply matheuristics appears to be based

on a neighborhood search slave procedure realized by a MIP solver (or
an exact method).
• It seems reasonable to “mip3 ” all metaheuristics that rely in some
part of the algorithm of features based on local search; for example a
Math - Iterated Local Search or a Math - Genetic algorithm.
• It seems that the typical intensification phase of a method when
tackled by means of an exact method is well performing because:
1 The solutions achieved by pure metaheuristics are often not local
minima for combinatorial neighborhoods.
2 MIP local minima are more robust than the metaheuristic ones. They
are, in fact, also local minima for a corresponding metaheuristic
procedure.
3
transform a local search procedure with polynomial size neighborhood, into a very
large scale neighborhood procedure where an exponential size neighborhood is handled
by means of an exact method
Federico Della Croce Heuristics 105 / 106
 COMBINATORIAL OPTIMIZATION: heuristic approaches

Matheuristics: discussion (2)

• The definition of a neighborhoods structure in an exact procedure

exploiting the MIP formulation of the problem is often
straightforward.
• Many kind of different neighborhoods can be defined for solving a
combinatorial optimization problem.
• Not only general purpose methods (e.g. branch and bound) but also
dedicated exact methods can be embedded into a matheuristic in
order to cope with problem-specific features.
• Also the diversification phase can be nested in the matheuristic
framework in order to escape from local minima.
• Last but not least, the use of generic solvers, commercial or open
source will guarantee, in our opinion, a continuous development of
matheuristic approaches.

Federico Della Croce Heuristics 106 / 106