Applications of Operations Research

Study Set 4: Lecture 10

1. Consider the following instance of the Traveling Salesperson Problem with initial so-
lution T = (1, 2, 3, 4, 5, 1) with length l(T ) = 19.

4
4 1

4 1
6 5 2

2 5
3 2
6

As neighborhood for a simulated annealing approach, we consider the insertion move

seen in Lecture 6, i.e., a node v is removed from the tour and inserted in the remaining
tour.
Assume a current temperature of T = 80 and that node 5 is picked for the next move.
What is the acceptance probability for the insertion in the edge {1, 2}, {2, 3} or {3, 4},
respectively?

′
Solution: Let lT ({i, j}) denote the length of the tour obtained by inserting node 5
between nodes i and j. Then, we obtain:
′ −δ 22−19
• lT ({1, 2}) = 1 + 5 + 6 + 6 + 4 = 22. The acceptance probability is e T = e− 80 =
3
e− 80 = 96.32%.
′ −δ 19−19
• lT ({2, 3}) = 2 + 5 + 2 + 6 + 4 = 19. The acceptance probability is e T = e− 80 =
e0 = 100%.
′
• lT ({3, 4}) = 2 + 6 + 2 + 4 + 4 = 18. As it is an improving move, it is always
accepted and has acceptance probability 100%.

2. Which of the following answers on tabu search are correct? Explain.

1
 a) What is the purpose of the tabu list in tabu search?
1. To store the cost of each solution explored during the search.
2. To store the best solution found so far
3. To prohibit revisiting recently explored solutions
4. To guide the search towards promising regions of the solution space
b) How does tabu search choose a neighbor solution to explore?
1. By selecting the solution with the lowest cost from a fixed number of neigh-
bors.
2. By applying a fixed sequence of predefined moves to the current solution
3. By randomly selecting a solution from the neighborhood
4. By considering all solutions that can be obtained through the local search
move and selecting the best neighbor solution.
c) What is a key advantage of tabu search compared to other local search algorithms?
1. It guarantees finding the global optimum solution
2. It has a faster convergence rate
3. It is less sensitive to the choice of initial solution
4. It requires less computational resources

Solution:

a) 1. False, that is usually not done.

2. False, that needs to be done, but is not the purpose of the tabu list.
3. Correct.
4. False, we do not exactly guide the search in a specific direction (we always
look at the best neighbor), we only want to avoid going back.
b) 1. False, in general, we want to find the best neighbor of the entire neighborhood
(or one of the best neighbors if there are multiple equal ones).
2. False, we compare different possible moves of the same type.
3. False, that is what is done in simulated annealing.
4. Correct. Note that sometimes not all neighbors need to be calculated explic-
itly, but can be quickly discarded by efficient searching/mathematical argu-
ments (e.g. if some arcs in the TSP are too expensive and will never lead to
better solutions than the other neighbors).
c) What is a key advantage of tabu search compared to other local search algorithms?
1. False, it can still happen that global optima are not found.
2. False, it potentially explores more solutions that a simple local search.
3. Correct, it has higher chances of escaping local optima surrounding the intial
solution.
4. False, keeping the tabu list increases the memory usage.

2
3. Find the maximum element in the following array A of integers using a Variable Neigh-
borhood Search approach:

A = [5, 9, 3, 7, 12, 6, 8, 10, 1]

Clearly define the neighborhood Nk and start your search at the left or right end of
the array.

Solution: One possibility is to define the k-th neighborhood of the entry A[i] as fol-
lows: Nk (i) := A[i − k, . . . , i + k], i.e., all elements in the array which are at most k
away from the index of the current best solution.

Suppose we start from the left with the initial solution of A[0] = 5. The iterations of
the VNS approach would look like the following:

1) The N1 neighborhood would look as follows: N1 (0) = [5, 9]. Since taking the
value 9 would improve upon our current best solution, we replace our current
best solution with A[1] = 9.
2) The N1 neighborhood for 9 is [5, 9, 3]. Since no neighbor improves our current
best solution we continue and look at the next neighborhood.
3) The N2 neighborhood for 9 is [5, 9, 3, 7]. Since no neighbor improves our current
best solution we continue and look at the next neighborhood.
4) The N3 neighborhood for 9 is [5, 9, 3, 7, 12]. Since taking the value 12 would
improve upon our current best solution, we replace our current best solution.
5) The N1 neighborhood for 12 is [7, 12, 6]. Since no neighbor improves our current
best solution we continue and look at the next neighborhood.
6) The N2 neighborhood for 12 is [3, 7, 12, 6, 8]. Since no neighbor improves our
current best solution we continue and look at the next neighborhood.
7) The N3 neighborhood for 12 is [9, 3, 7, 12, 6, 8, 10]. Since no neighbor improves
our current best solution we continue and look at the next neighborhood.
8) The N4 neighborhood for 12 is [5, 9, 3, 7, 12, 6, 8, 10, 1]. Since no neighbor improves
our current best solution, we look at the next neighborhood. However, since there
no next neighborhood and no neighbor improves our current best solution, the
VNS approach returns 12 as the best solution.

Note that, since in this case the last neighborhood included all possible solutions, we
obtained also the global optimum and not just a local (non-global) optimum like in
the general case.

3
 4. Consider the following 2-dimensional knapsack problem.

max 12x1 + 7x2 + 9x3 + 8x4

s.t. 3x1 + x2 + x3 + x4 ≤ 3
x3 + x4 ≤ 1
xj ∈ {0, 1}, j = 1, 2, 3, 4

a) Apply tabu search starting from the initial feasible solution x = (0, 0, 0, 1). Stop
after five iterations.
Move: switch one 0-component to 1 or one 1-component to 0.
Tabu rule: If a component xj has been complemented, make it tabu for the next
2 iterations.
b) Apply simulated annealing starting from the same solution.
Take T = 10 and decrease by 2 units at each iteration. Use the random numbers
0.8, 0.1, 0.6, 0.2, 0.9, 0.72, 0.4, 0.92, 0.6, 0.24 ...1
Stop after 5 iterations.

Solution:

a)

t best sol current sol neighbor 1 neighbor 2 neighbor 3 neighbor 4 tabu list
1 (0,0,0,1) (0,0,0,1) (1,0,0,1) (0,1,0,1) (0,0,1,1) (0,0,0,0) {2}
8 8 inf 15 inf 0
2 (0,1,0,1) (0,1,0,1) (1,1,0,1) - (0,1,1,1) (0,1,0,0) {2,4}
15 15 inf inf 7
3 (0,1,0,1) (0,1,0,0) (1,1,0,0) - (0,1,1,0) - {4,3}
15 7 inf 16
4 (0,1,1,0) (0,1,1,0) (1,1,1,0) (0,0,1,0) - - {3,2}
16 16 inf 9
5 (0,1,1,0) (0,0,1,0) (1,0,1,0) - - (0,0,1,1) {2 }
16 9 inf inf

Since no feasible move is left, stop. Report x̂ = (0,1,1,0) with value 16.
b) In choosing a random neighbour, we divide the interval [0,1] between feasible
neighbours. If there are three feasible neighbours and if the random number is in
1
Use the random numbers for the decision between different neighbors, e.g., if there are three neighbors,
assign them to the intervals [0, 1/3], (1/3, 2/3] and (2/3, 1] and pick the neighbor according to where the
random number lies. A move with acceptance probability p is accepted if p is larger than the random
number.

4
 [0,1/3], we pick the first neighbour, if it is in (1/3,2/3], we pick the second one
and the third one otherwise.

iter/ random random decision/ random number/

best solution current solution
T number neighbor probability decision
1 (0,0,0,1) (0,0,0,1) 0.8 (0,0,0,0) accept with prob 0.1
10 8 8 0 e−8/10 = 0.45 accept
2 (0,0,0,1) (0,0,0,0) 0.6 (0,0,1,0) accept
8 8 0 9
3 (0,0,1,0) (0,0,1,0) 0.2 (0,1,1,0) accept
6 9 9 16
4 (0,1,1,0) (0,1,1,0) 0.9 (0,1,0,0) accept with prob 0.72
4 16 16 7 e−9/4 = 0.11 reject
5 (0,1,1,0) (0,1,1,0) 0.4 (0,0,1,0) accept with prob 0.92
2 16 16 9 e−7/2 = 0.03 reject

Report x̂ = (0,1,1,0) with value 16.

5. Suppose a genetic algorithm uses chromosomes of the form x = abcdef gh with a fixed
length of eight genes. Each gene can be any digit between 0 and 9. Let the fitness of
individual x be calculated as:

f (x) = (a + b) − (c + d) + (e + f ) − (g + h)

and let the initial population consist of four individuals with the following chromo-
somes:

x1 = 65413532
x2 = 87126601
x3 = 23921285
x4 = 41852094

a) Evaluate the fitness of each individual and arrange them in order with the fittest
first and the least fit last.
b) Cross the fittest two individuals using one-point crossover at the middle point.
c) Cross the second and third fittest individuals using a two-point crossover.
d) Suppose the new population consists of the four offspring individuals received
by the crossover operations in the above steps. Evaluate the fitness of the new
population. Has the overall fitness improved?

5
Solution:

a)

f (x1 ) = (6 + 5) − (4 + 1) + (3 + 5) − (3 + 2) = 9
f (x2 ) = (8 + 7) − (1 + 2) + (6 + 6) − (0 + 1) = 23
f (x3 ) = (2 + 3) − (9 + 2) + (1 + 2) − (8 + 5) = −16
f (x4 ) = (4 + 1) − (8 + 5) + (2 + 0) − (9 + 4) = −19

The order is x2 , x1 , x3 and x4 .

b)
x2 = 8712 | 6601 O1 = 87123532
⇒
x1 = 6541 | 3532 O2 = 65416601
c)
x1 = 65 | 4135 | 32 O3 = 65921232
⇒
x3 = 23 | 9212 | 85 O4 = 23413585
d) The new population is:
O1 = 87123532
O2 = 65416601
O3 = 65921232
O4 = 23413585
Now apply the fitness function:

f (O1 ) = (8 + 7) − (1 + 2) + (3 + 5) − (3 + 2) = 15
f (O2 ) = (6 + 5) − (4 + 1) + (6 + 6) − (0 + 1) = 17
f (O3 ) = (6 + 5) − (9 + 2) + (1 + 2) − (3 + 2) = −2
f (O4 ) = (2 + 3) − (4 + 1) + (3 + 5) − (8 + 5) = −5

The overall fitness has improved.