CHRIST

Deemed to be University

DESIGN AND ANALYSIS OF ALGORITHMS

CS533P
Limitations of Algorithm Power: Decision Trees, P, NP and NP Complete Problems, Challenges
in Numerical Algorithms.
Backtracking: n-Queen’s Problem, Hamiltonian Circuit problem and Subset-Sum problem.
Branch and Bound: Assignment problem, Knapsack problem and Traveling salesman problem.

Dr. Jeno Lovesum

Associate Professor, Department of CSE, School of Engineering and
Technology
CHRIST (Deemed to be University), Bangalore, India

Contact No.: +919606600986|[email protected]

MISSION VISION CORE VALUES
CHRIST is a nurturing ground for an individual’s Excellence and Service Faith in God | Moral Uprightness
holistic development to make effective contribution to Love of Fellow Beings 1
the society in a dynamic environment Excellence and Service Social Responsibility | Pursuit of Excellence
 CHRIST
Deemed to be University

CHRIST (Deemed to be University), Bangalore

Vision and Mission

VISION : Excellence and Service

MISSION : CHRIST (Deemed to be University) is a nurturing ground for an

individual's holistic development to make effective contribution to society in a
dynamic environment.

Core Values:
• Faith in God
• Moral Uprightness
• Love of Fellow Beings
• Social Responsibility
• Pursuit of Excellence

2
Excellence and Service
 CHRIST
Deemed to be University

SCHOOL OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Department’s Vision and Mission

VISION : “To fortify Ethical Computational Excellence”

MISSION :
1. Imparts core and contemporary knowledge in the areas of Computation and
Information Technology
2. Promotes the culture of research and facilitates higher studies
3. Acquaints the students with the latest industrial practices, team building and
entrepreneurship
4. Sensitizes the students to serve for environmental, social & ethical needs of
society through lifelong learning.

3
Excellence and Service
 CHRIST
Deemed to be University

SCHOOL OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
Bachelor of Technology (Computer Science and Engineering - Artificial
Intelligence and Machine Learning)

Programme Educational Objective

• Professional Acumen : Understand, analyze and design solutions with

professional competency for the real world problems
• Critical Analysis : Develop software/embedded solutions for the requirements,
based on critical analysis and research.
• Team work : Function effectively in a team and as an individual in a
multidisciplinary/multicultural environment.
• Life Long Learning : Accomplish holistic development comprehending
professional responsibilities

4
Excellence and Service
 CHRIST
Deemed to be University

SCHOOL OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
Bachelor of Technology (Computer Science and Engineering - Artificial
Intelligence and Machine Learning)
Programme Outcome
• An ability to apply Engineering knowledge of computing, mathematics, science, and computer science & engineering
fundamentals for Problem solving.
• An ability to think critically to identify, formulate, and solve complex computer science & engineering problems by
developing models, evaluating validity and accuracy of solutions in terms of computer science and engineering validity
measures.
• An ability to analyse, design of complex problems, implement, and evaluate a computer-based system, to meet expected
needs with appropriate considerations such as economic / environmental/societal.
• An ability to conduct experiments to investigate problems based on changing requirements, analyze and interpret
results.
• An ability to create, select, adapt appropriate techniques and use of the modern computational tools, techniques and
skills, and best of engineering practices.
• To understand the impact of contextual knowledge on social aspects and cultural issues.
• An ability to understand contemporary issues related to social & environmental context for sustainable development of
engineering solutions.
• An ability to understand professional & ethical responsibility to contribute for societal and national needs.
• An ability to function and coordinate effectively as an individual, as a member or leader in diverse, multicultural&
multidisciplinary teams
• An ability to communicate effectively.
• An understanding of computer science and engineering & management principles to manage software projects.
• A recognition and realization of the need for, and an ability to engage in lifelong learning.

5
Excellence and Service
 CHRIST
Deemed to be University

SCHOOL OF ENGINEERING AND TECHNOLOGY

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Bachelor of Technology (Computer Science and Engineering - Artificial

Intelligence and Machine Learning)

Programme Specific Outcome

• Artificial Intelligence: To utilize artificial intelligence principles to solve real

world problems
• Machine Learning: To select suitable machine learning algorithms for the given
problem
• Service Learning: Analyze Social Relevant Problems and design solutions
through Service Learning

6
Excellence and Service
 CHRIST
Deemed to be University
DESIGN AND ANALYSIS OF ALGORITHMS (CS533P)
UNIT DETAILS HOURS
I INTRODUCTION AND FUNDAMENTALS OF THE ANALYSIS OF ALGORITHM EFFICIENCY: 8
Introduction, Notion of Algorithm, Fundamentals of Algorithmic Solving, Important Problem
Types, Fundamentals of the Analysis Framework, Mathematical Analysis of Non-recursive
Algorithm, Mathematical Analysis of Recursive Algorithm and examples, Empirical Analysis of
Algorithms and Algorithm Visualization.
II ALGORITHM DESIGN TECHNIQUES: Brute Force and Exhaustive Search: Selection Sort, Bubble 11
Sort, Sequential Search and Brute-force string matching, Travelling Salesman Problem, Knapsack
Problem, Assignment Problem.
Decrease and Conquer: Insertion Sort and Topological Sorting and Fake Coin Problem, Russian
Peasant Multiplication, Josephus Problem.
III ALGORITHM DESIGN TECHNIQUES: Divide and conquer: Merge sort, Quick Sort, Binary Tree 10
Traversals and Related Properties and Multiplication of Large Integers and Strassen’s Matrix
Multiplication.
Transform and Conquer: Pre-sorting, Notion of Heap and Heapsort, Horner’s Rule and Binary
Exponentiation.
IV ALGORITHM DESIGN TECHNIQUES: Space and Time Trade - Offs: Sorting by Counting, 8
Horspool’s and Boyer - Moore Algorithm for String Matching, Hashing.
Dynamic Programming: Knapsack Problem, Warshall’s and Floyd’s Algorithm.
Greedy Techniques: Prim’s Algorithm, Kruskal’s Algorithm, Dijkstra’s Algorithm.
V ALGORITHM DESIGN TECHNIQUES: Limitations of Algorithm Power: Decision Trees, P, NP and 8
NP Complete Problems, Challenges in Numerical Algorithms.
Backtracking: n-Queen’s Problem, Hamiltonian Circuit problem and Subset-Sum problem.
Branch and Bound: Assignment problem, Knapsack problem and Traveling salesman problem.
TOTAL HOURS: 45
7
Excellence and Service
 CHRIST
Deemed to be University

Study Materials
TEXT BOOKS
T1. AnanyLevitin, “Introduction to the Design and Analysis of Algorithm”, 3/e,
Pearson Education Asia, Reprint 2012.
T2. Michael T. Goodrich, Roberto Tamassia, Michael H. Goldwasser, “Data
Structures and Algorithms in Java”, 6/e, Wiley, 2014.

REFERENCE BOOKS
R1. T. H Cormen, C E Leiserson, R L Rivest and C Stein: “Introduction to
Algorithms”, 3rd Edition, The MIT Press, 2014.
R2. Ellis Horowitz, SartajSahni and SanguthevarRajasekaran, Computer Algorithms,
Second Edition, Universities Press, 2007.
R3. Richard Neapolitan, “Foundations of Algorithms”, 5/e, Jones & Bartlett
Learning, 2014.
R4. Richard Johnsonbaugh, Marcus Schaefer, “Algorithms”, Pearson Education, 2009.
R5. Clifford A Shaffer, “Data Structures and Algorithm Analysis in Java”, 3rd
Edition, Courier Corporation, 2014.

OTHER RESOURCES
● NPTEL Course: https://onlinecourses.nptel.ac.in/noc22_cs71/preview

8
Excellence and Service
 CHRIST
Deemed to be University

COURSE PREREQUISITES

• Computer Programming
• Data Structure and Algorithms

9
Excellence and Service
 CHRIST
Deemed to be University

Limitations of Algorithm Power

Decision Trees, P, NP and NP Complete Problems, Challenges in
Numerical Algorithms.

10
Excellence and Service
 CHRIST
Deemed to be University

Limitations of Algorithm Power

There are many algorithms for solving a variety of different problems.
They are very powerful instruments, especially when they are executed
by modern computers.
The power of algorithms is limited because of the following reasons:
● There are some problems cannot be solved by any algorithm.
● There are some problems can be solved algorithmically but not in
polynomial time.
● There are some problems can be solved in polynomial time by some
algorithms, but they are usually lower bounds on their efficiency.

Algorithms limits are identified by the following:

● Lower-Bound Arguments
● Decision Trees
● P, NP and NP-Complete Problems

11
Excellence and Service
 CHRIST
Deemed to be University

Lower Bounds
Lower bound: an estimate on a minimum amount of work needed to
solve a given problem

Examples:
● number of comparisons needed to find the largest element in a set of
n numbers
● number of comparisons needed to sort an array of size n
● number of comparisons necessary for searching in a sorted array
● number of multiplications needed to multiply two n-by-n matrices

Excellence and Service 12

 CHRIST
Deemed to be University

Lower Bounds
 We can look at the efficiency of an algorithm two ways. We can
establish its asymptotic efficiency class (say, for the worst case) and
see where this class stands with respect to the hierarchy of
efficiency classes.
● For example, selection sort, whose efficiency is quadratic, is a
reasonably fast algorithm,
● whereas the algorithm for the Tower of Hanoi problem is very
slow because its efficiency is exponential.
 Lower bounds means estimating the minimum amount of work
needed to solve the problem. We present several methods for
establishing lower bounds and illustrate them with specific
examples.
1. Trivial Lower Bounds
2. Information-Theoretic Arguments
3. Adversary Arguments
13
4. Problem Reduction Excellence and Service
 CHRIST
Deemed to be University

Lower Bounds (cont.)

● Lower bound can be

○ an exact count
○ an efficiency class ()

● Tight lower bound: there exists an algorithm with the same efficiency
as the lower bound

Problem Lower bound Tightness

sorting (nlog n) yes
searching in a sorted array (log n) yes
element uniqueness (nlog n) yes
n-digit integer multiplication (n) unknown
multiplication of n-by-n matrices (n2) unknown

Excellence and Service 14

 CHRIST
Deemed to be University

Methods for Establishing Lower Bounds

● trivial lower bounds

● information-theoretic arguments (decision trees)

● adversary arguments (SKIP)

● problem reduction

Excellence and Service 15

 CHRIST
Deemed to be University

Trivial Lower Bounds

Trivial lower bounds: based on counting the number of items that must be
processed in input and generated as output

Examples
● finding max element
● polynomial evaluation
● sorting
● element uniqueness
● Hamiltonian circuit existence

Conclusions
● may and may not be useful
● be careful in deciding how many elements must be processed (min-max)

Excellence and Service 16

 CHRIST
Deemed to be University

Decision Trees
Decision tree —model algorithms that use comparisons:
● internal nodes represent comparisons
● leaves represent outcomes
● How many orderings of 2 elements? Of 3? Of n?
● Tree for 3-element insertion sort [insert b in a, then c into ab]
abc
yes no
a<b

abc bac
yes no yes no
b< c a<c

a<b<c acb bca

b<a<c
yes no yes no
a<c b<c

a<c<b c<a<b b<c<a c<b<a

Excellence and Service 17

 CHRIST
Deemed to be University

Decision Trees and Sorting Algorithms

● Any comparison-based sorting algorithm can be represented by a
decision tree

● Number of leaves (outcomes)  n!

● Height of binary tree with n! leaves  log2n!

● Minimum number of comparisons in the worst case  log2n! for any

comparison-based sorting algorithm
● log23! = log2 6 = 3

● log2n!  n log2n, for large n

● This lower bound is tight (mergesort)

Excellence and Service 18

 CHRIST
Deemed to be University

Adversary Arguments (SKIP)

Adversary argument: a method of proving a lower bound by
playing role of adversary that makes algorithm work the hardest
by adjusting input

Example 1: “Guessing” a number between 1 and n with yes/no

questions
Adversary: Puts the number in a larger of the two subsets
generated by last question

Example 2: Merging two sorted lists of size n

a1 < a2 < … < an and b1 < b2 < … < bn
Adversary: ai < bj iff i < j
Output b1 < a1 < b2 < a2 < … < bn < an requires 2n-1 comparisons
of adjacent elements

Excellence and Service 19

 CHRIST
Deemed to be University

Lower Bounds by Problem Reduction

If we know a LB for alg A, can we use that knowledge to find a LB for alg
B? Remember reduction:
- Minheap reduces to maxheap
- Minimum reduces to sort
- Element uniqueness for pairs reduces to closest pair

Which would be most useful?

A [(f )] reduces to B [(? )]
B [(? )] reduces to A [(f )]

[Assume the reduction itself is fast (ie O(f)) ]

Excellence and Service 20

 CHRIST
Deemed to be University

Lower Bounds by Problem Reduction

Which would be most useful?
A [(f )] reduces to B [(? )]
B [(? )] reduces to A [(f )]

Proc A [(f )] Proc B [(? )]

… …
B [(? )] A [(f )]
… …

A= [(? )] => ??? A = [(? )] => ???

Excellence and Service 21

 CHRIST
Deemed to be University

Lower Bounds by Problem Reduction

Which would be most useful?
A [(f )] reduces to B [(? )]
B [(? )] reduces to A [(f )]

Proc A [(f )] Proc B [(? )]

… …
B [(? )] A [(f )]
… …

A= [(f )] => B = [(f )] A = [(f )] => Nothing

Excellence and Service 22

 CHRIST
Deemed to be University

Lower Bounds by Problem Reduction

To show lower bounds: Reduce problem with known lower bound to
problem with unknown lower bound.

Idea: If problem P is at least as hard as problem Q, then a lower

bound for Q is also a lower bound for P.
Hence, find problem Q with a known lower bound that can
be reduced to problem P in question.

Example: P is finding MST for n points in Cartesian plane

Q is element uniqueness problem (known to be in (nlogn))

Excellence and Service 23

 CHRIST
Deemed to be University

Lower Bounds by Problem Reduction

Example: P is finding MST for n points in Cartesian plane
Q is element uniqueness problem (known to be in (nlogn))

Reduce Element Uniqueness to MST:

Unique (S : IntSet = {x1, x2, …, xn}) [Known (nlogn)]

P : PairSet = {(x1, 0), (x2, 0), …, (xn, 0)}
T : Tree = MST (P)
D : Nat = min_length (edges of T)
return D = 0

Thus, MST is (nlogn)!

A. Levitin “Introduction to the DesignExcellence

& Analysis ofand
Algorithms,”
Service3rd ed., Ch. 11 ©2012 Pearson 24
Education, Inc. Upper Saddle River, NJ. All Rights Reserved.
 CHRIST
Deemed to be University

P, NP, NP Complete

P and NP are important classes of problems.

Question: Are they the same? Is P = NP ?

Excellence and Service 25

 CHRIST
Deemed to be University

Basic Concepts

- Classifying Problems
- Tractable and non-tractable
- Optimization and Decision Problems

- Problem Classes
-P
- NP
- NP Complete

Excellence and Service 26

 CHRIST
Deemed to be University

Classifying Problem Complexity

Is the problem tractable, i.e., is there a polynomial-time (O(p(n))
algorithm that solves it?

Possible answers:
● yes (give examples)

● no
○ because it’s been proved that no algorithm exists at all
(e.g., Turing’s halting problem)
○ because it’s been be proved that any algorithm takes
exponential time

● unknown

Excellence and Service 27

 CHRIST
Deemed to be University

Problem Types: Optimization and Decision

● Optimization problem: find a solution that maximizes or minimizes
some objective function

● Decision problem: answer yes/no to a question

Many problems have decision and optimization versions.

E.g.: traveling salesman problem

● optimization: find Hamiltonian cycle of minimum length
● decision: find Hamiltonian cycle of length  m

Decision problems are more convenient for formal investigation of their

complexity.

Excellence and Service 28

 CHRIST
Deemed to be University

Class P
P: the class of decision problems that are solvable in O(p(n)) time,
where p(n) is a polynomial of problem’s input size n

Examples:
● searching

● element uniqueness

● graph connectivity

● graph acyclicity

● primality testing (finally proved in 2002)

Excellence and Service 29

 CHRIST
Deemed to be University

Class NP
NP (nondeterministic polynomial): class of decision problems whose
proposed solutions can be verified in polynomial time = solvable by a
nondeterministic polynomial algorithm

A nondeterministic polynomial algorithm is an abstract two-stage

procedure that:
● generates a random string that may solve the problem
● checks whether this solution is correct, in polynomial time
By definition, it solves the problem if it will generate and verify a
solution on one of its tries

Why this definition?

● led to development of the rich theory called “computational
complexity”

Excellence and Service 30

 CHRIST
Deemed to be University

Example: CNF satisfiability

Problem: Is a boolean expression in its conjunctive normal form
(CNF) satisfiable, i.e., are there values of its variables that makes
it true?
This problem is in NP. Nondeterministic algorithm:
● Guess truth assignment
● Substitute the values into the CNF formula to see if it evaluates to
true
Example: (A | ¬B | ¬C) & (A | B) & (¬B | ¬D | E) & (¬D | ¬E)
Truth assignments:
ABCD E
0 0 0 0 0
. . .
1 1 1 1 1
Checking phase: O(n)

Excellence and Service 31

 CHRIST
Deemed to be University

What problems are in NP?

● Hamiltonian circuit existence
● Partition problem: Is it possible to partition a set of n integers into two
disjoint subsets with the same sum?
● Decision versions of TSP, knapsack problem, graph coloring, and many
other combinatorial optimization problems. (Few exceptions include:
MST, shortest paths)

● All the problems in P can also be solved in this manner (no guessing is
necessary), so we have:
P  NP

● Big question: Is P = NP ? Or is P ⊂ NP ?
○ Answer unknown. Current approach: focus on hardest NP problems

Excellence and Service 32

 CHRIST
NP-Complete : The Hardest NP Problems Deemed to be University

A decision problem D in NP is NP-complete if it’s as hard as (ie no

easier than) any and every problem in NP. That is:
● D is in NP
● every problem in NP is polynomial-time reducible to D

NP problems

NP-complete
problem

Think: Which problems can be easier?

Excellence and Service 33
 CHRIST
Deemed to be University

Cook: The First NP-Complete Problem

A decision problem D that is in NP is NP-complete if
● D is in NP
● Every problem in NP is polynomial-time reducible to D
NP problems

NP-complete
problem

How to prove every NP problem is reducible to D? Not easy!

Cook’s theorem (1971): CNF-sat is NP-complete

Excellence and Service 34

 CHRIST
Proving Other Problems are NP-Complete Deemed to be University

Prove other problems are NP-complete by polynomial-

time reductions from a known NP-complete problem
NP problems

known
NP-complete
problem
candidate
for NP -
completeness

Pay attention to the direction of the reductions

Examples: TSP, knapsack, partition, graph-coloring and
hundreds of other problems of combinatorial nature

Excellence and Service 35

 CHRIST
Deemed to be University

P = NP ? Dilemma Revisited
● If P = NP then every problem in NP (including all NP-complete
problems) could be solved in polynomial time
● If a polynomial-time algorithm for just one NP-complete problem is
discovered, then every problem in NP can be solved in polynomial
time, i.e., P = NP
NP problems

NP-complete
problem

● Most but not all researchers believe that P  NP , i.e. P is a proper

subset of NP

Excellence and Service 36

 CHRIST
Deemed to be University

Challenges of numerical Algorithms. Study page-412

● The first major obstacle is the fact that most

numerical analysis
problems cannot be solved exactly.
● They have to be solved approximately, and this is
usually done by replacing an infinite object by a
finite approximation.
● For example, the value of ex at a given point x can
be computed by approximating its infinite Taylor’s
series about x = 0 by a finite sum of its first terms,
called the nth-degree Taylor polynomial:

Excellence and Service 37

 CHRIST
Deemed to be University

● Numerical analysis is usually described as the

branch of computer science concerned with
algorithms for solving mathematical problems.
● This description needs an important clarification:
the problems in question n are problems of
“continuous”mathematics
● solving equations and systems of equations,
evaluating such functions as sin x and ln x,
computing integrals, and so on—as opposed to
problems of discrete mathematics dealing with
such structures as graphs, trees, permutations,
and combinations.

Excellence and Service 38

 CHRIST
Deemed to be University

BackTracking

● The principal idea is to construct solutions one

component at a time and evaluate such partially
constructed candidates as follows.
● If a partially constructed solution can be developed
further without violating the problem’s constraints,
it is done by taking
● the first remaining legitimate option for the next
component. If there is no legitimate option for the
next component, no alternatives for any remaining
component need to be considered.
● In this case, the algorithm backtracks to replace
the last component of the partially constructed
solution with its next option.

Excellence and Service 39

 CHRIST
Deemed to be University

● choices being made, called the state-space tree.

Its root represents an initial state before the
search for a solution begins.
● The nodes of the first level in the tree represent
the choices made for the first component of a
solution, the nodes of the second level represent
the choices for the second component, and so on.
● A node in a state-space tree is said to be promising
if it corresponds to a partially constructed solution
that may still lead to a complete solution;
otherwise it is called nonpromising.

Excellence and Service 40

 CHRIST
Deemed to be University

N-Queens Problem

● As our first example, we use a perennial favorite of

textbook writers: the n-queens problem. The
problem is to place n queens on an n x n
chessboard so that no two queens attack each
other by being in the same row or in the same
column or on the same diagonal. For n = 1, the
problem has a trivial solution, and it is easy to see
that there is no solution for n = 2 and n = 3.
● So let us consider the four-queens problem and
solve it by the backtracking technique

Excellence and Service 41

 CHRIST
Deemed to be University

● We start with the empty board and then place

queen 1 in the first possible position of its row,
which is in column 1 of row 1. Then we place queen
2, after trying unsuccessfully columns 1 and 2, in
the first acceptable position for it, which is square
(2, 3), the square in row 2 and column 3. This
proves to be a dead end because there is no
acceptable position for queen 3. So, the algorithm
backtracks and puts queen 2 in the next possible
position at (2, 4).
● Then queen 3 is placed at (3, 2), which proves to be
another dead end.
● The algorithm then backtracks all the way to queen
1 and moves it to (1, 2). Queen 2 then goes to (2, 4),
queen 3 to (3, 1), and queen
Excellence 4 to (4, 3), which is a
and Service 42
 CHRIST
Deemed to be University

Hamiltonian cycle-Refer text book page 426-427

● Without loss of generality, we can assume that if a

Hamiltonian circuit exists, it starts at vertex a.
Accordingly, we make vertex a the root of the
state-space
● The first component of our future solution, if it
exists, is a first intermediate vertex of a
Hamiltonian circuit to be constructed. Using the
alphabet order to break the three-way tie among
the vertices adjacent to a, we select vertex b. From
b, the algorithm proceeds to c, then to d, then to e,
and finally to f, which proves to be a dead end.

Excellence and Service 43

 CHRIST
Deemed to be University

● So the algorithm backtracks from f to e, then to d,

and then to c, which provides the first alternative
for the algorithm to pursue. Going from c to e
eventually proves useless, and the algorithm has to
backtrack from e to c and then to b.
● From there, it goes to the vertices f , e, c, and d,
from which it can legitimately return to a, yielding
the Hamiltonian circuit a, b,
f , e, c, d, a. If we wanted to find another Hamiltonian
circuit, we could continue this process by
backtracking from the leaf of the solution found.
Refer Text Book

Excellence and Service 44

 CHRIST
Deemed to be University

Sum of subsets-Refer text book page 427-428

● we consider the subset-sum problem: find a subset

of a given set A = {a1, . . . , an} of n positive
integers whose sum is equal to a given positive
● integer d. For example, for A = {1, 2, 5, 6, 8} and d =
9, there are two solutions:
● {1, 2, 6} and {1, 8}. Of course, some instances of
this problem may have no
● solutions.
● It is convenient to sort the set’s elements in
increasing order. So, we will
● assume that
● a1< a2 < . . . < an.

Excellence and Service 45

 CHRIST
Deemed to be University

● The state-space tree can be constructed as a binary

tree like that in Figure 12.4 for the instance A = {3, 5, 6,
7} and d = 15. The root of the tree represents the
starting point, with no decisions about the given
elements made as yet. Its left and right children
represent respectively, inclusion and exclusion of a1 in
a set being sought.
● Similarly, going to the left from a node of the first level
corresponds to inclusion of a2 while going to the right
corresponds to its exclusion, and so on. Thus, a path
from the root to a node on the ith level of the tree
indicates which of the first i numbers have been
included in the subsets represented by that node.

Excellence and Service 46

 CHRIST
Deemed to be University

● We record the value of s, the sum of these

numbers, in the node. If s is equal to d, we have a
solution to the problem.We can either report this
result and stop or, if all the solutions need to be
found, continue by backtracking to the node’s
parent. If s is not equal to d, we can terminate the
node as nonpromising if either of the following two
inequalities holds:
-Refer Text Book

Excellence and Service 47

 CHRIST
Deemed to be University

Branch & Bound-Assignment Problem

● Let us illustrate the branch-and-bound approach by

applying it to the problem of assigning n people to
n jobs so that the total cost of the assignment is as
small as possible.
● Recall that an instance of the assignment problem
is specified by an n x n cost matrix C so that we
can state the problem as follows: select one
element in each row of the matrix so that no two
selected elements are in the same column and
their sum is the smallest possible
● Eg:Assignment problem, Knapsack problem and Traveling salesman
problem.

Excellence and Service 48

 CHRIST
Deemed to be University

Assignment Problem-Branch and Bound

● Let us illustrate the branch-and-bound approach by

applying it to the problem of assigning n people to
n jobs so that the total cost of the assignment is as
small as possible. We introduced this problem in
Section 3.4, where we solved it by exhaustive
search.
● Recall that an instance of the assignment problem
is specified by an n x n cost matrix C so that we
can state the problem as follows: select one
element in each row of the matrix so that no two
selected elements are in the same column and
their sum is the smallest possible.
● We will demonstrate how this problem can be
solved using the branch-and-bound technique
Excellence and Service 49
 CHRIST
Deemed to be University

Knapsack problem-Branch& Bound

● given n items of known weights wi and values vi , i

= 1, 2, . . . , n, and a knapsack of capacity W,
● find the most valuable subset of the items that fit
in the knapsack. It is convenient
● to order the items of a given instance in
descending order by their value-to-weight
● ratios. Then the first item gives the best payoff per
weight unit and the last one
● gives the worst payoff per weight unit, with ties
resolved arbitrarily:
● v1/w1 ≥ v2/w2 ≥ . . . ≥ vn/wn.

Excellence and Service 50

 CHRIST
Deemed to be University

● We record the total weight w and the total value v

of this
● selection in the node, along with some upper bound
ub on the value of any subset
● that can be obtained by adding zero or more items
to this selection.
● A simple way to compute the upper bound ub is to
add to v, the total value of
● the items already selected, the product of the
remaining capacity of the knapsack
● W − w and the best per unit payoff among the
remaining items, which is vi+1/wi+1:
ub = v + (W − w)(vi+1/wi+1).

Excellence and Service 51

 CHRIST
Deemed to be University

Wieght value value/weight

4 40 10
7 42 6
5 25 5
3 12 4
The knapsack’s capacity W is 10.
● At the root of the state-space tree no items have
been selected as yet. Hence, both the total weight
of the items already selected w and their total
value v are equal to 0. The value of the upper bound
computed by formula is $100. Node 1, the left child
of the root, represents the subsets that include
item 1.

Excellence and Service 52

 CHRIST
Deemed to be University

● The total weight and value of the items already

included are 4 and $40, respectively; the value of
the upper bound is 40 + (10 − 4) ∗ 6 = $76. Node 2
represents the subsets that do not include item 1.
Accordingly, w = 0, v = $0, and ub = 0 + (10 − 0) ∗ 6
= $60.
● Since node 1 has a larger upper bound than the
upper bound of node 2, it is more promising for this
maximization problem, and we branch from node 1
first. Its children—nodes 3 and 4—represent
subsets with item 1 and with and without item 2,
respectively

Excellence and Service 53

 CHRIST
Deemed to be University

● Since the total weight w of every subset

represented by node 3 exceeds the knapsack’s
capacity, node 3 can be terminated immediately.
Node 4 has the same values of w and v as its
parent; the upper bound ub is equal to 40 + (10 − 4)
∗ 5 = $70.
● Selecting node 4 over node 2 for the next branching
(why?), we get nodes 5 and 6 by respectively
including and excluding item 3. The total weights
and values as well as the upper bounds forthese
nodes are computed in the same way as for the
preceding nodes. Branching from node 5 yields
node 7, which represents no feasible solutions, and
node 8, which represents just a single subset {1, 3}
of value $65. Excellence and Service 54
 CHRIST
Deemed to be University

● The remaining live nodes 2 and 6 have smaller

upper-bound values than the value of the solution
represented by node 8. Hence, both can be
terminated making the subset {1, 3} of node 8 the
optimal solution to the problem

Excellence and Service 55

 CHRIST
Deemed to be University

Travelling Salesman Problem-Refer Text Book-pg438

● We will be able to apply the branch-and-bound

technique to instances of the traveling salesman
problem if we come up with a reasonable lower
bound on tour lengths.
● One very simple lower bound can be obtained by
finding the smallest element in the intercity
distance matrix D and multiplying it by the number
of cities n.
● But there is a less obvious and more informative
lower bound for instances with symmetric matrix
D, that we can compute a
● lower bound on the length l of any tour as follows.
For each city i, 1≤ i ≤ n, find the sum si of the
distances from city i to the two nearest cities;
compute the sum sExcellence
of these n numbers, divide the
and Service 56
 CHRIST
Deemed to be University

Transform & Conquer-Presorting

● So far, we have discussed three elementary sorting

algorithms—selection sort, bubble sort, and
insertion sort—that are quadratic in the worst and
average cases, and two advanced algorithms—
mergesort, which is always in (n log n), and
quicksort, whose efficiency is also (n log n) in the
average case but is quadratic in the worst case.
Are there faster sorting algorithms?
● no general comparison-based sorting algorithm can
have a better efficiency than n log n in the worst
case, and the same result holds for the average-
case efficiency

Excellence and Service 57

 CHRIST
Deemed to be University

EXAMPLE 1 Checking element uniqueness

in an array
● The brute-force algorithm compared pairs of the
array’s elements until either two equal elements
were found or no more pairs were left. Its worst-
case efficiency was in O(n2).
● Alternatively, we can sort the array first and then
check only its consecutive elements: if the array
has equal elements, a pair of them must be next to
each other, and vice versa.
● ALGORITHM PresortElementUniqueness(A[0..n − 1])
//Solves the element uniqueness problem by sorting the array first
//Input: An array A[0..n − 1] of orderable elements
//Output: Returns “true” if A has no equal elements, “false”
otherwise
sort the array A
for i ←0 to n − 2 do
if A[i]= A[i + 1] return false
Excellence and Service 58
return true
 CHRIST
Deemed to be University

● The running time of this algorithm is the sum of the

time spent on sorting and the time spent on
checking consecutive elements. Since the former
requires at least n log n comparisons and the latter
needs no more than n − 1 comparisons, it is the
sorting part that will determine the overall
efficiency of the algorithm. So, if we use a
quadratic sorting algorithm here, the entire
algorithm will not be more efficient than the brute-
force one

Excellence and Service 59

 CHRIST
Deemed to be University

EXAMPLE 2 Computing a mode

● A mode is a value that occurs most often in a given

list of numbers. For example, for 5, 1, 5, 7, 6, 5, 7,
the mode is 5. (If several different values occur
most often, any of them can be considered a
mode.) The brute-force approach to computing a
mode would scan the list and compute the
frequencies of all its distinct values, then find the
value with the largest frequency.
● As an alternative, let us first sort the input. Then
all equal values will be adjacent to each other. To
compute the mode, all we need to do is to find the
longest run of adjacent equal values in the sorted
array.

Excellence and Service 60

 CHRIST
Deemed to be University

● PresortMode(A[0..n − 1])
//Computes the mode of an array by sorting it first
//Input: An array A[0..n − 1] of orderable elements
//Output: The array’s mode
sort the array A
i ←0 //current run begins at position i
modefrequency ←0 //highest frequency seen so far
while i ≤ n − 1 do
runlength←1; runvalue←A[i]
while i + runlength ≤ n − 1 and A[i + runlength]= runvalue
runlength←runlength + 1
if runlength > modef requency
modef requency ←runlength; modevalue←runvalue
i ←i + runlength
return modevalue

Excellence and Service 61

 CHRIST
Deemed to be University

THANK YOU

Excellence and Service 62