DESIGN AND ANALYSIS OF

ALGORITHMS

MODULE 5
 Topics

 Backtracking:

 General method,

 N-Queens problem,

 Sum of subsets problem,

 Graph colouring,

 Hamiltonian cycles Problems.

 Branch and Bound:

 Basic concepts,

 Assignment Problem,

 Travelling Sales Person problem,

 0/1 Knapsack problem.

 NP-Complete and NP-Hard problems:

 Basic concepts,

 Non- deterministic algorithms,

 P, NP, NP-Complete, and NP-Hard classes.

Backtracking

 The exhaustive-search technique suggests generating all candidate solutions and then identifying the one

(or the ones) with desired property.

 Backtracking is a more intelligent variation of this approach.

 The name backtrack was first coined by D. H. Lehmer in the 1950.

 The principal idea is to construct solutions one component at a time and evaluate such partially

constructed candidates as follows.

 If a partially constructed solution can be developed further without violating the problem’s constraints, it is done by

taking the first remaining legitimate option for the next component.

 If there is no legitimate option for the next component, no alternatives for any remaining component need to be

considered. In this case, the algorithm backtracks to replace the last component of the partially constructed
solution with its next option.
Backtracking

 It is implemented by constructing a tree of choices being made, called the state-space tree.

 Its root represents an initial state before the search for a solution begins.

 The nodes of the first level in the tree represent the choices made for the first component of a solution,

 The nodes of the second level represent the choices for the second component, and so on.

 A node in a state-space tree is said to be promising if it corresponds to a partially constructed solution that may still

lead to a complete solution; otherwise, it is called non-promising.

 Leaves represent either non-promising dead ends or complete solutions found by the algorithm.
Backtracking

 In the majority of cases, a state-space tree for a backtracking algorithm is constructed in the manner of depth

first search.
 If the current node is promising, its child is generated by adding the first remaining legitimate option for the next

component of a solution, and the processing moves to this child.

 If the current node turns out to be non-promising, the algorithm backtracks to the node’s parent to consider the next

possible option for its last component;

 If there is no such option, it backtracks one more level up the tree, and so on.

 Finally, if the algorithm reaches a complete solution to the problem, it either stops (if just one solution is required) or

continues searching for other possible solutions.

THE GENERAL METHOD

 In many applications of the backtrack method, the desired solution is expressible as an n-tuple (x1,….,.xn),

where the xi are chosen from some finite set Si.

 Suppose mi is the size of set Si. Then there are m = m1m2 … mn n- tuples that are possible candidates for

satisfying the function P.

 The brute force approach would be to form all these n-tuples evaluate each one with P, and save those which yield the

optimum.

 The backtrack algorithm, yield the same answer with far fewer than m trials. Its basic idea is to build up the solution

vector one component at a time and to use modified criterion functions Pi (x1,….,.xn) (bounding functions) to test

whether the vector being formed has any chance of success.

 The major advantage of this method is this, if it is realized that the partial vector (x1,….,.xn), can in no way lead to an

optimal solution, then mi + 1,….,.mn , possible test vectors can be ignored entirely.
THE GENERAL METHOD

 Many of the problems solved using backtracking require that all the solutions should satisfy a complex set of

constraint ( explicit and implicit ).

 Explicit constraints are rules that restrict each xi to take on values only from a given set

 Ex 1,

 Ex 2, 4- Queens Problem, How many queens to be considered.

 Implicit constraints, are rules that determine which of the tuples in the solution space of ‘i’ satisfy the

criterion function.
 That is implicit constraints describe the way in which the xi must relate to each other.

 Ex, 4- Queens Problem: No two queens can be attacked

Sol Sol
TERMINOLOGIES

 Backtracking algorithms determine problem solutions by systematically searching for the solutions.

 Each node in the tree is called a Problem state.

 All paths from the root to the other nodes define the state space of the problem.

 The Solutions states are those problems states for which the path from root to s define a tuple in the

solution space.

 Answer states, these are the leaf nodes which correspond to element in the set of solution.

 Live node, A node which is been generated and whose a children have not been generated is called live node

 A Dead node which is generated which is not expanded further or all whose a children have been generated.
Backtracking
APPLICATIONS

 n-Queens Problem

 Sum of subsets problem,

 Graph coloring,

 Hamiltonian cycles Problems.

n-Queens Problem
 The problem is to place n queens on an n × n chessboard so that no two queens attack each other by
being in the same row or in the same column or on the same diagonal.

 For n = 1, the problem has a trivial solution, and it is easy to see that there is no solution for n = 2 and
n = 3.

 Now let us consider the four-queens problem and solve it by the backtracking technique. Since each of
the four queens has to be placed in its own row, all we need to do is to assign a column for each queen on
the board.
n-Queens Problem
 First start with the empty board and then place queen 1 in the first possible position of its row, which is in
column 1 of row 1.

 Then place queen 2, after trying unsuccessfully columns 1 and 2, in the first acceptable position for it,
which is square (2, 3), the square in row 2 and column 3.
 This proves to be a dead end because there is no acceptable position for queen 3.

 So, the algorithm backtracks and puts queen 2 in the next possible position at (2, 4).

 Then queen 3 is placed at (3, 2), which proves to be another dead end.

 The algorithm then backtracks all the way to queen 1 and moves it to (1, 2).

 Queen 2 then goes to (2, 4), queen 3 to (3, 1), and queen 4 to (4, 3), which is a solution to the problem.

 The state-space tree of this search is shown in Figure 12.2.

Sol Sol
n-Queens Problem
Subset-Sum Problem
 Find a subset of a given set A = {a1, . . . , an } of n positive integers whose sum is equal to a given positive

integer d. Example, for A = {1, 2, 5, 6, 8} and d = 9, there are two solutions: {1, 2, 6} and {1, 8}.

 Sort the set’s elements in increasing order, a1 < a2 < . . . < an.

 The state-space tree can be constructed as a binary tree, for the instance A = {3, 5, 6, 7} and d = 15.

 The root of the tree represents the starting point, with no decisions about the given elements made as yet.

 Its left and right children represent, respectively, inclusion and exclusion of a1 in a set being sought. Similarly,

going to the left from a node of the first level corresponds to inclusion of a2 while going to the right corresponds to its
exclusion, and so on.

 Thus, a path from the root to a node on the ith level of the tree indicates which of the first i numbers have been

included in the subsets represented by that node.

Subset-Sum Problem
 The sum of these numbers s, in the node is recorded,.

 If s is equal to d, solution is obtained to the problem. This solution can be either reported and stop or, if all the

solutions need to be found, continue by backtracking to the node’s parent.

 If s is not equal to d, terminate the node as non-promising if either of the following two inequalities holds:
Subset-Sum Problem
Hamiltonian Circuit Problem
 Let G = (V, E) be a connected graph with n vertices. A Hamiltonian cycle (suggested by Sir William

Hamilton)is a round-trip path along n edges of G that visits every vertex once and returns to its starting
position.

 Ex, Using the alphabet order to break the three-way tie among the vertices adjacent to a, we select vertex b.

 From b, the algorithm proceeds to c, then to d, then to e, and finally to f, which proves to be a dead end.

 So the algorithm backtracks from f to e, then to d, and then to c, which provides the first alternative

 Going from c to e eventually proves useless, and the algorithm has to backtrack from e to c and then to b.

 From there, it goes to the vertices f , e, c, and d, from which it can legitimately return to a, yielding the Hamiltonian

circuit a, b, f , e, c, d, a.

 If wanted to find another Hamiltonian circuit, continue this process by backtracking from the leaf of the solution

found.
Hamiltonian Circuit Problem
Hamiltonian Circuit Problem
Graph Coloring
 Let G be a graph and m be a given positive integer. We have to discover whether the nodes of G can be

colored in such a way that no two adjacent nodes have the same color yet only m colors are used. This is
termed the m-colorability decision problem.

 Example, the graph can be colored with three colors 1, 2 and 3.The color of each node is indicated next to it.

It can also be seen that three colors are needed to color this graph and hence this graph's chromatic number is
3.
Graph Coloring
 A graph is said to be planar iff it can be drawn in a plane in such a way that no two edges crosses each other.

A famous special case of the m-colorability decision problem is the 4-color problem for planar graphs.

 This problem asks the following question: given any map, can the regions be colored in such a way that no

two adjacent regions have the same color yet only four colors are needed

 Each region of the map becomes a node, and if two regions are adjacent, then the corresponding nodes are

joined by an edge. Figure shows a map with five regions and its corresponding graph.
Graph Coloring
Graph Coloring
Branch-and-Bound
 Branch-and-Bound an algorithm design technique, primarily for solving optimization problems. Guarantees

that the optimal solution will be found.

 Does not necessarily guarantee worst case polynomial time complexity. But tries to ensure faster time on

most instances.

 Basic Idea

 Model the entire solution space as a tree

 Search for a solution in the tree systematically, eliminating parts of the tree from the search intelligently.

 Important technique for solving many problems for which efficient algorithms (worst case polynomial time) are not

known
Branch-and-Bound
 Branch-and-Bound an algorithm design technique, primarily for solving optimization problems. Guarantees

that the optimal solution will be found.

 Does not necessarily guarantee worst case polynomial time complexity. But tries to ensure faster time on

most instances.

 Basic Idea

 Model the entire solution space as a tree

 Search for a solution in the tree systematically, eliminating parts of the tree from the search intelligently.

 Important technique for solving many problems for which efficient algorithms (worst case polynomial time) are not

known
Branch-and-Bound
 The principal idea of the branch-and-bound technique,

 The value of the best solution seen so far.

 Provide, for every node of a state-space tree, a bound on the best value of the objective function on any solution

that can be obtained by adding further components to the partially constructed solution represented by the node.

 If this information is available, compare a node’s bound value with the value of the best solution seen so far,

 If the bound value is not better than the value of the best solution seen so far—

 i.e., not smaller for a minimization problem and

 not larger for a maximization problem—

 then the node is non-promising and can be terminated (branch is “pruned”).

Branch-and-Bound
 In general, search path at the current node in a state-space tree of a branch-and-bound algorithm is

terminated for any one of the following three reasons:

 The value of the node’s bound is not better than the value of the best solution seen so far.

 The node represents no feasible solutions because the constraints of the problem are violated.

 The subset of feasible solutions represented by the node consists of a single point (and hence no further choices can be

made).
Assignment Problem
 The problem is to assigning n people to n jobs so that the total cost of the assignment is as small as

possible.

 Assignment problem is specified by an n × n cost matrix C, state the problem as follows:

 select one element in each row of the matrix so that no two selected elements are in the same column and their

sum is the smallest possible.

 Demonstration:

 Problem can be solved using the branch-and-bound technique by considering the same small instance of the

problem.
Assignment Problem
 Find a lower bound.

 Solution: Include smallest elements in each of the matrix’s rows. For the instance here, this sum is 2 + 3+ 1+ 4

= 10. This is lower bound, is denoted by lb.

 Next the tree nodes will be generated,

 Rather than generating a single child of the last promising node (by comparing the lower bounds of the live nodes)

generate all the children of the most promising node. Consider a node with the best bound as most promising, although
the optimal solution may belong to a different branch of the state-space tree.

 This variation of the strategy is called the best-first branch-and-bound.

Assignment Problem
Assignment Problem
Assignment Problem
Knapsack Problem
 Given n items of known weights wi and values vi , i = 1, 2, . . . , n, and a knapsack of capacity W, find the

most valuable subset of the items that fit in the knapsack.

 Order the items of a given instance in descending order by their value-to-weight ratios.

 v1/w1 ≥ v2/w2 ≥ . . . ≥ vn/wn.

 Demonstration:

 Construct state-space tree.

 Each node on the ith level of this tree, 0 ≤ i ≤ n, represents all the subsets of n items that include a particular

selection made from the first i ordered items.

 A branch going to the left indicates the inclusion of the next item, and a branch going to the right

indicates its exclusion.

Knapsack Problem
 Record total weight w and the total value v of this selection in the node, with upper bound ub on the value

of any subset, that can be obtained by adding zero or more items to this selection.

 to compute the upper bound ub is,

 add to v, the total value of the items already selected, the product of the remaining capacity of the knapsack W − w and

the best per unit payoff among the remaining items, which is vi+1 / w i+1

 Example,
Knapsack Problem
Traveling Salesman Problem
 Applying the branch-and-bound technique to instances of the traveling salesman problem, find reasonable

lower bound on tour lengths.

 Find the smallest element in the intercity distance matrix D and multiplying it by the number of cities n.

 For each city i, 1≤ i ≤ n, find the sum si of the distances from city i to the two nearest cities; compute the sum

s of these n numbers, divide the result by 2, and, if all the distances are integers, round up the result to the
nearest integer:

 Example,
Traveling Salesman Problem
 Basic Concepts
• For many of the problems we know and study, the best algorithms for their
solution have computing times can be clustered into two groups;
– Solutions are bounded by the polynomial- Examples include Binary search
O(log n), Linear search O(n) or in generalO(nk) where k is a constant.
– Solutions are bounded by a non-polynomial - Examples include travelling
salesman problem O(n22n) & knapsack problem O(2n/2).
As the time increases exponentially, even moderate size problems
cannot be solved.
 Non deterministic algorithms
• We also need the idea of two models of computer (Turing machine):
deterministic and non-deterministic.
• A deterministic computer is the regular computer we always thinking
of.
• When the result of every operation is uniquely defined then it is
called deterministic algorithm.
• A non-deterministic computer is one that is just like we’re used to
except that it has unlimited parallelism, so that any time you come to a
branch, you spawn a new “process” and examine both sides.
 Non deterministic algorithms
• When the outcome is not uniquely defined but is limited to a specific
set of possibilities, we call it non deterministic algorithm.
 P, NP
• NP stands for Non-deterministic Polynomial time.
• Definition: P is a set of all decision problems solvable by a deterministic
algorithm in polynomial time.
• Definition: NP is the set of all decision problems solvable by a
nondeterministic algorithm in polynomial time.

• This also implies P ⊆ NP

• i.e. Problems known to be in P are trivially in NP
83
 P
• P is a complexity class that represents the set of all decision problems that
can be solved in polynomial time.
• That is, given an instance of the problem, the answer yes or no can be
decided in polynomial time.
• Example
Given a connected graph G, can its vertices be coloured using two colours so
that no edge is monochromatic?
– Algorithm: start with an arbitrary vertex, color it red and all of its neighbours
blue and continue. Stop when you run out of vertices or you are forced to make
an edge have both of its endpoints be the same color.
 NP
• NP is a complexity class that represents the set of all decision problems
for which the instances where the answer is "yes" have proofs that can
be verified in polynomial time.
• This means that if someone gives us an instance of the problem and a
certificate (sometimes called a witness) to the answer being yes, we can
check that it is correct in polynomial time.