Backtracking

Terminology
● Live node is a node that has been generated but whose
children have not yet been generated.
● E-node is a live node whose children are currently being
explored. In other words, an E-node is a node currently being
expanded.
● Dead node is a generated node that is not to be expanded or
explored any further. All children of a dead noden have
already been expanded.
● Branch-and-bound refers to all state space search methods in
which all children of an E-node are generated before any other
live node can become the E-node.
■ Used for state space search
■ In BFS, exploration of a new node cannot begin until the node
currently being explored is fully explored
 What is backtracking?

● Useful technique for optimizing search under some

constraints
● Express the desired solution as an n-tuple (x , . . . , x )
1 n
where each xi є Si, Si being a finite set
● The solution is based on finding one or more vectors
that maximize, minimize, or satisfy a criterion
function P(x1, . . . , xn)
● Sorting an array a[n]
■ Find an n-tuple where the element x is the index of ith
i
smallest element in a
■ Criterion function is given by a[x ] ≤ a[x ] for 1 ≤ i < n
i i+1
 What is backtracking?
● Brute force approach
■ Let the size of set Si be mi
■ There are m = m1m2 · · ·mn n-tuples that satisfy the criterion function P
■ In brute force algorithm, you have to form all the m n-tuples to
determine the optimal solutions
● Backtrack approach
■ Form a solution (partial vector) and check at every step if this has any
chance of success
■ If the solution at any point seems not-promising, ignore it
■ If the partial vector (x1, x2, . . . , xi) does not yield an optimal solution,
ignore mi+1 · · ·mn possible test vectors even without looking at them
■ Determine problem solution by systematically searching the solution
space for the given problem instance using a tree organization for
solution space
 Explicit & Implicit Constraints
● All the solutions are required to satisfy a set of
constraints divided into two categories:
explicit and implicit constraints
 Explicit Constraints
● Explicit constraints are rules that restrict each x i to
take on values only from a given set.
● Explicit constraints depend on the particular instance
I of problem being solved
● All tuples that satisfy the explicit constraints define a
possible solution space for I
● Examples of explicit constraints
■ xi ≥ 0, or all nonnegative real numbers
■ xi = {0, 1}
■ li ≤ xi ≤ ui
 Implicit Constraints
● Implicit constraints are rules that determine
which of the tuples in the solution space of I
satisfy the criterion function.
● Implicit constraints describe the way in which
the xis must relate to each other.
 Backtracking
● Two versions of backtracking algorithms
■ Solution needs only to be feasible (satisfy
problem’s constraints)
○ sum of subsets
○ N- queens problem
■ Solution needs also to be optimal
○ knapsack
 N-Queens Problem
● N-Queens dates back to the 19th century
(studied by Gauss)
● Classical combinatorial problem, widely used
as a benchmark because of its simple and regular structure
● Problem involves placing N queens on an N  N chessboard such
that no queen can attack any other
● Queen attacks other at the same row, column or diagonal line
● Benchmark code versions include finding the first solution and
finding all solutions
A Solution for 6-Queen
 8-queens problem
● Identify data structures to solve the problem
■ We may define the solution to be an 8 × 8 array
■ However, since each queen is in a different row,
define the chessboard solution to be an 8-tuple
(x1, . . . , x8), where xi is the column for ith queen
i= 1 2 3 4 5
x [] 1 3 5 2 4

Q
Q

Q
Q
Q
 8-queens problem

● Identify explicit constraints

■ Explicit constraints using 8-tuple formulation are S = {1, 2,
3, 4, 5, 6, 7, 8}
■ Solution space of 88 8-tuples
● Identify implicit constraints
■ No two xi can be the same, or all the queens must be in
different columns
■ All solutions are permutations of the 8-tuple (1, 2, 3, 4, 5,
6, 7, 8)
■ Reduces the size of solution space from 88 to 8! tuples
■ No two queens can be on the same diagonal
 The Backtracking Method
● A given problem has a set of constraints and
possibly an objective function
● The solution optimizes an objective function,
and/or is feasible.
● We can represent the solution space for the
problem using a state space tree
■ The root of the tree represents 0 choices,
■ Nodes at depth 1 represent first choice
■ Nodes at depth 2 represent the second choice, etc.
■ In this tree a path from a root to a leaf represents a
candidate solution
Terms Related to Solution Space Trees

● Problem state is each node in the depth-first search tree

● State space is the set of paths from root node to other nodes
● Solution states are the problem states s for which the path
from the root node to s defines a tuple in the solution space
■ Partitioned into disjoint sub-solution spaces at each internal node
● Answer states are those solution states for which the path
from root node to s defines a tuple that is a member of the set
of solutions
■ These states satisfy implicit constraints
● State space tree is the tree organization of the solution space
● Static trees are ones for which tree organizations are
independent of the problem instance being solved
● Dynamic trees are ones for which organization is dependent
on problem instance
 BACKTRACKING (Contd..)
● The problem may be solved by systematically
generating the problem states determining which are
solution states, and determining the answer states.
● Let us see the following terminology
● LIVE NODE A node which has been generated and
all of whose children are not yet been generated .
● E-NODE (Node being expanded) - The live node
whose children are currently being generated .
● DEAD NODE - A node that is either not to be
expanded further, or for which all of its children have
been generated.
 BACKTRACKING (Contd..)

● DEPTH FIRST NODE GENERATION- In

this, as soon as a new child C of the current
E-node R is generated, C will become the
new E-node.
R will become E-node again when C has
been fully explored.
● BOUNDING FUNCTION - will be used to
kill live nodes without generating all their
children.
 BACKTRACKING (Contd..)

● BACTRACKING
■ Depth – first node generation with bounding functions.
■ New nodes are placed in to a stack. The last node added is
the first to be explored.
● BRANCH-and-BOUND
■ Breadth – first node generation method in which E-node
remains E-node until it is dead.
■ Each new node placed in a queue. The front of the queue
becomes the new E-node.
 BACKTRACKING (Contd..)

Example : 4 Queens problem

1 1 1 1
. . 2 2 2
3
. . . .

1 1
2
3
. , 4
 BACKTRACKING (Contd..)

1
x1 = 1 x1=2
2 18
x2=2 3 4 x2=1 x2=3 x2 = 4
B 3 8 13 19 24 29
x3=3 x3=4 2 3 B B
4 6 14 16 x3 = 1
x4=4 3 B 30
5 7 15 x4 = 3
B 31
 No two queens on the same diagonal
If two queens are placed at (i,j) & (k,l), then they are on the same diagonal
only if
i – j = k – l => j – l = i – k
Or
i + j = k + l => j – l = k – i
This means that two queens are at the same diagonal iff
Abs (j – l) = Abs (i – k)

Condition for testing if queen

k placed on column i is on the
same diagonal as queen j

abs(x[j]-i) == abs(j-k)
 N-queens Backtracking Algorithm
Nqueens(k, n) place(k, i)
{ {
for i = 1 to n
{ if place(k, i) then for j= 1 to k-1
{ {
x[k] = i if (x[j] == i) or
if (k==n) then (abs(x[j]-i) == abs(j-
write (x[1..n])
k))
else Nqueens(k+1, n);
} then return false
} }
} return true
}
 Sum of subsets
● Problem: Given n positive integers w1, ... wn and
a positive integer m. Find all subsets of w1, ... wn
that sum to m.
● Example:
n = 4, w = (11, 13, 24, 7), and m = 31
● Solutions:
(11, 13, 7) and (24, 7)
Solution Vectors - Fixed size tuples
● A simple formulation of the problem is where the
solution subset is represented by an n-tuple (x 1, . . . ,
xn) such that xi = 0 if wi is not chosen and xi = 1 if wi
is chosen.
● Explicit constraint - xi є {0, 1}
● The above solutions are then represented by (1, 1, 0,
1) and (0, 0, 1, 1)
● Implicit constraint – For all xj =1, Σ wj = m
● The solution space is 2n distinct tuples
 State SpaceTree for 3 items using
fixed tuple formulation

w1 = 2, w2 = 4, w3 = 6 and m = 6
0
X1= 1 X1= 0

2 0
X2= 1 X2= 1 X2= 0
X2= 0

6 2 4 0
X3= 1 X3= 0
X3= 0 X3= 1 X3= 1 X3= 0 X3= 1 X3= 0

12 6 8 2 10 4 6 0

The sum of the included integers is shown at the node.

Draw State SpaceTree for 4 items using
fixed tuple formulation

w1 = 2, w2 = 4, w3 = 6, w4 = 8 and m = 12
Solution Vectors - Variables size tuples

● The solution vectors can also be represented by the

indices of the numbers as (1, 2, 4) and (3, 4)
■ All solutions are k-tuples, 1≤ k ≤ n
● Explicit constraints
■ x є {j | j is an integer and 1 ≤ j ≤ n}
i
● Implicit constraints
■ No two x can be the same
i
■ Σx =m
i
■ x < x , 1 ≤ i < k (total order in indices) – this avoids
i i+1
generating multiple instances of the same set e.g. (1,2,4)
and (2,4,1)
● The solution space is 2n distinct tuples
 State SpaceTree for 3 items using
variable tuple formulation

w1 = 2, w2 = 4, w3 = 6 and m = 6
0
X1= 1 X1= 3

2 X1= 2 6
X2= 2 4
X2= 3

6 8 X2= 3

X3= 3
10

12

The sum of the included integers is shown at the node.

Draw State SpaceTree for 4 items using
variable tuple formulation

w1 = 2, w2 = 4, w3 = 6, w4 = 8 and m = 12
 When is a node “promising”?
● Sort the weights in non-decreasing order
● Let weight be the subtotal from root to node i
at level i.
● A node is non-promising if
■ weight +w[i+1] > W - any descendant of node i
will be nonpromising because w[i+1] is the lightest
weight remaining.
■ weight + weight of all remaining items < W - any
descendant of node i will be nonpromising
 A Pruned State Space Tree (find all solutions)
w1 = 3, w2 = 4, w3 = 5, w4 = 6; m = 13
There are only 15 nodes in the pruned state space tree
The full state space tree has 31 nodes
0
3 0

3 0
4 0 4 0

7 3 4 0

5 0 5 0 5 0

12 7 8 3 9 4
6 0

13 7
 Sum of Subsets – Algorithm
sumOfSubsets (weight, i, totalLeft )
{
if ((weight + totalLeft  m) and (weight + w[i + 1]  m or
weight = m ))
if ( weight = m )
then print x[ 1 ] to x[ i ] //found solution
else
{
x [ i+1 ] = 1 //try including
sumOfSubsets (weight + w[i+1], i + 1, totalLeft - w[i+1] )
x [ i+1 ] = 0 //try excluding i+1
sumOfSubsets (weight, i + 1, totalLeft - w[i+1] )
}
n
}
Initial call sumOfSubsets(0, 0,  w i )
i 1
n=6, w[1:6]={5,10,12,13,15,18}, m=30