ll

LECTURE NOTES ON DESIGN A Department

J N TU
AND ANALYSIS of Computer

Science OF and ALGORITHMS

World
Engineering

CONTENTS
CHAPTER 1 BASIC CONCEPTS
Algorithm Performance of Programs Algorithm Design Goals Classification of Algorithms Complexity of
Algorithms Rate of Growth Analyzing Algorithms The Rule of Sums The Rule of products The Running
time of Programs Measuring the running time of programs Asymptotic Analyzing of Algorithms Calculating
the running time of programs General rules for the analysis of programs
CHAPTER 2 Advanced Data Structures and Recurrence Relations
Priority Queue, Heap and Heap sort Heap Sort 2.3 Priority Queue implementation using heap tree
Binary Search trees Balanced Trees Dictionary Disjoint Set Operations Recurrence Relations – Iterative
Substitution Method Recursion Tree The Guess-and test The Master Theorem Method Cold Form
expression Solving Recurrence relations
CHAPTER 3 Divide And Conquer
General Method Control Abstraction of Divide and Conquer Binary Search External and Internal path
length Merge Sort Strassen’s Matrix Multiplication Quick Sort Straight Insertion Sort
CHAPTER 4 Greedy Method
4.1 General Method
Control Abstraction Knapsack Problem Optimal Storage on Tapes Job Sequencing with deadlines

All
Optimal Merge Patterns Huffman Codes

JNTU
World
I
Graph Algorithms
CHAPTER 5 Dynamic programming
Multi Storage graphs All Pairs Shortest paths Traveling Sales Person problem Optimal Binary Search
Tree 0/1 Knapsack Reliability design
CHAPTER 6 Basic Traversal and Search Techniques
Techniques for traversal of Binary tree
Techniques for graphs Representation of Graph and Digraphs Depth First and Breadth First Spanning
trees Articulation Points and bi-connected components Articulation points by Depth First Search Game
planning Alpha-Beta pruning AND/OR Graphs
CHAPTER 7 Backtracking
General method Terminology N-Queens problem Sum of Subsets Graph Coloring( for planar graphs)
Hamiltonian Cycles 0/1 Knapsack Traveling Sales Person using Backtracking
CHAPTER 8 Branch and Bound
General method Least Cost (LC) Search Control Abstraction for LC-Search Bounding The 15-Puzzle
problem LC Search for 15-Puzzle Problem Job Sequencing with deadlines Traveling Sales Person

l
problem 0/1 Knapsack
Al
JNTU

World
II

Chapter
1
Basic Concepts
Algorithm
An Algorithm is a finite sequence of instructions, each of which has a clear meaning and can be performed
with a finite amount of effort in a finite length of time. No matter what the input values may be, an algorithm
terminates after executing a finite number of instructions. In addition every algorithm must satisfy the
following criteria: Input: there are zero or more quantities, which are externally supplied;
Output: at least one quantity is produced;
Definiteness: each instruction must be clear and unambiguous;
Finiteness: if we trace out the instructions of an algorithm, then for all cases the algorithm will terminate
after a finite number of steps;
Effectiveness: every instruction must be sufficiently basic that it can in principle be carried out by a person
using only pencil and paper. It is not enough that each operation be definite, but it must also be feasible.
In formal computer science, one distinguishes between an algorithm, and a program. A program does not
necessarily satisfy the fourth condition. One important example of such a program for a computer is its
operating system, which never terminates (except for system crashes) but continues in a wait loop until
more jobs are entered.
We represent algorithm using a pseudo language that is a combination of the constructs of a
programming language together with informal English statements.
Performance of a program:
The performance of a program is the amount of computer memory and time needed to run a program. We
use two approaches to determine the performance of a program. One is analytical, and the other
experimental. In performance analysis we use analytical methods, while in performance measurement we
conduct experiments.
Time Complexity:
The time needed by an algorithm expressed as a function of the size of a problem is called the time
complexity of the algorithm. The time complexity of a program is the amount of computer time it needs to
run to completion.
The limiting behavior of the complexity as size increases is called the asymptotic time complexity. It is the
asymptotic complexity of an algorithm, which ultimately determines the size of problems that can be

All
solved by the algorithm.

JNTU
 World
1
Space Complexity:
The space complexity of a program is the amount of memory it needs to run to completion. The space
need by a program has the following components:
Instruction space: Instruction space is the space needed to store the compiled version of the program
instructions.
Data space: Data space is the space needed to store all constant and variable values. Data space has
two components:
• Space needed by constants and simple variables in program.
• Space needed by dynamically allocated objects such as arrays and class instances.
Environment stack space: The environment stack is used to save information needed to resume
execution of partially completed functions.
Instruction Space: The amount of instructions space that is needed depends on factors such as:
• The compiler used to complete the program into machine code.
• The compiler options in effect at the time of compilation
• The target computer.
Algorithm Design Goals
The three basic design goals that one should strive for in a program are:
1. Try to save Time 2. Try to save Space 3. Try to save Face
A program that runs faster is a better program, so saving time is an obvious goal. Like wise, a program that
saves space over a competing program is considered desirable. We want to “save face” by preventing the
program from locking up or generating reams of garbled data.
Classification of Algorithms
If ‘n’ is the number of data items to be processed or degree of polynomial or the size of the file to be
sorted or searched or the number of nodes in a graph etc.
1 Next instructions of most programs are executed once or at most only a few times. If all the instructions
of a program have this property, we say that its running time is a constant.
Log n When the running time of a program is logarithmic, the program gets slightly slower as n grows.
This running time commonly occurs in programs that solve a big problem by transforming it into a smaller
problem, cutting the size by some constant fraction., When n is a million, log n is a doubled. Whenever n
doubles, log n increases by a constant, but log n does not double until n increases to n 2.
 All JNTU

World
2
n When the running time of a program is linear, it is generally the case that a small amount of processing
is done on each input element. This is the optimal situation for an algorithm that must process n inputs.
n. log n This running time arises for algorithms that solve a problem by breaking it up into smaller sub-
problems, solving then independently, and then combining the solutions. When n doubles, the running time
more than doubles.
n2 When the running time of an algorithm is quadratic, it is practical for use only on relatively small problems.
Quadratic running times typically arise in algorithms that process all pairs of data items (perhaps in a double
nested loop) whenever n doubles, the running time increases four fold.
n3 Similarly, an algorithm that process triples of data items (perhaps in a triple–nested loop) has a cubic
running time and is practical for use only on small problems. Whenever n doubles, the running time
increases eight fold.
2n Few algorithms with exponential running time are likely to be appropriate for practical use, such
algorithms arise naturally as “brute–force” solutions to problems. Whenever n doubles, the running time
squares.
Complexity of Algorithms
The complexity of an algorithm M is the function f(n) which gives the running time and/or storage space
requirement of the algorithm in terms of the size ‘n’ of the input data. Mostly, the storage space required by
an algorithm is simply a multiple of the data size ‘n’. Complexity shall refer to the running time of the
algorithm.
The function f(n), gives the running time of an algorithm, depends not only on the size ‘n’ of the input data
but also on the particular data. The complexity function f(n) for certain cases are:
1. Best Case : The minimum possible value of f(n) is called the best case.
2. Average Case : The expected value of f(n).
3. Worst Case : The maximum value of f(n) for any key possible input.
The field of computer science, which studies efficiency of algorithms, is known as analysis of algorithms.
Algorithms can be evaluated by a variety of criteria. Most often we shall be interested in the rate of growth
of the time or space required to solve larger and larger instances of a problem. We will associate with the
problem an integer, called the size of the problem, which is a measure of the quantity of input data.

l JNTU
Al
World
3
Rate of Growth:
The following notations are commonly use notations in performance analysis and used to characterize the
complexity of an algorithm:
 l l
A Big–OH f(n) less Big–OMEGA f(n) than than = = or 1. 2. 3. 4. Ω(g(n))

J
O(g(n)), equal O or (Upper equal Ω to Big–OH Big–OMEGA Big–THETA Little–OH (Lower

N
(>) (pronounced (pronounced (<) Bound) that (O) that (o) Bound) of (θ) 1(Ω), ,

of g(n). and
 T U
g(n). omega), order of says or big that oh), the says growth that

W o r
rate the growth of f(n) rate is greater of f(n) is

ld
1 In 1892, P. Bachmann invented a notation for characterizing the asymptotic behavior of functions. His invention
has come to be known as big oh notation.
4
Big–THETA θ (Same order) f(n) = Θ(g(n)) (pronounced theta), says that the growth rate of f(n) equals
(=) the growth rate of g(n) [if f(n) = O(g(n)) and T(n) = Ω (g(n)].

A Little–OH T(n) the Suppose complexity we functions. Numerical The Analyzing

l l
growth want execution n 1 2 4 8 16 32 64 128 256 = O(1), o(p(n)) ‘M’ to The (o) f(n)

J
rate Comparison log2 0 1 2 3 4 5 6 7 8 is examine. O(log2 Algorithms time most an of of
 N
(pronounced n algorithm, M p(n) for n), common increases six This O(n), [if n*log2n 0 2 8

24 64 160 384 896 2048 of T(n) of is Different the and O(n. computing little as usually =

T
typical suppose O(p(n)) n log2 oh), increases. n1 4 16 64 256 1024 4096 16,384 65,536 n),

U
Algorithms done 2 says functions times and O(n‘n’ by that is 2T(n) It are: ), the

comparing is n1 8 64 512 4096 32,768 2,62,144 2,097,152 1,677,216 O(nthe is 3 usually ≠ size given 3),
 W
θ(p(n))]. growth O(2of below: the the nf(n) ), rate n! rate input 22

4
16
256
65,536
4,294,967,296
Note Note ????????
n with and of of T(n) 1

o
2 data. nincrease some n
rld
is less than
Clearly the of f(n) standard
Note1: The value here is approximately the number of machine instructions executed by a 1 gigaflop
computer in 5000 years.
5
Note 2: The value here is about 500 billion times the age of the universe in nanoseconds, assuming a
universe age of 20 billion years. Graph of log n, n, n log n, n2, n3, 2n, n! and nn

A O(log convention constants. running that earlier One functional integers ‘n’. Which

l
Suppose and running For O(nsequentially together The Then, 2example, ), the P2, way n) is than
 l
O(n rule time time. and with program read does that is to • • • • ‘O’ 3We ) will O(max this, of is

compare and of that suppose the as f(n) Linear Binary Bubble Merge notation, T 1(n) not Since sums

J
O will P1 not “f(n) but O(n. property will followed = (max(ndepend T 1(n) (nhave and also

N
O(g(n)) Big-Oh search sort never search sort 3terminate is , that the log n. suppose of

T2(n) is is throw 2any is log on , n). order later. function we that by O O(f(n)) is is O nis 3the (n n)) P2 O
 T
(nparticular O Then )) have are an f(n) (n) 2within away log is (log g(n)”. f(n) ) which which

base upper O(max the three f(n) and n) the is n) and bounded low–order of running a For is running is

U
units with bound, T2(n) certain the steps O(nf(n), g(n) O(nexample, logarithm. 3these

). of 3is g(n)), are ). times whose by time the time time. O(g(n)). terms The some functions standard

W
answer the this of of period. running running Thus the To order two while multiple is

Then simplify called first provided we defined programs The function of times computing time throw
 o
complexity two T1(n) program g(n) as the are rule on steps is of is for away fragments +

analysis, a the respectively to of T 2(n), guarantee all almost a may executed for:

r
sums. use positive Big–Oh leading

three stop
the the
the all

ld
P1
6
The rule of products
If T1(n) and T2(n) are O(f(n) and O(g(n)) respectively. Then T 1(n)*T2(n) is O(f(n) g(n)). It follows term the
product rule that O(c f(n)) means the same thing as O(f(n)) if ‘c’ is any positive constant. For example,
O(n2/2) is same as O(n2). Suppose that we have five algorithms A1–A5 with the following time
complexities:
A1 : n A2 : A3 : n log n
n2 A4: n3 A5: 2n
The time complexity is the number of time units required to process an input of size ‘n’. Assuming that one
unit of time equals one millisecond. The size of the problems that can be solved by each of these five
algorithms is:
Algorithm Time

A
Maximum complexity 1 second A1 n 1000 A2 n log n 140 A3 n2 31 A4 n3 10 A5 2n 9 The seem

matters has simulation The advantage problems, problem Suppose generation, Instead algorithm.

l l
comparison, larger; results Algorithm speed grown faster that A1 A2 A3 A4 A5 are by more

size of the efficiency of of By of it for replacing from an the with by that physical computations their is next
 J
today looking more increase replacing the computer the can our power. n generation

N
impressive in then table log complexity A4 2be n2 n3 data n into n algorithm computing n

achieved with in ever algorithm As we the are has run, speed the can A2 demanding following before. of

T
than increased is of the computer we see with computers no power. consider an the A4 can
 U
more the longer The an algorithm table with two so increase increase more solve

S1 S2 S3 S4 S5 reason becomes Virtually need much fold important. A3, the it is Maximum over 1 6

W o
4893 minute 244 x 39 15 last problem 104 thirty size years 3.6 2.0 1

1897 and 153

hour

r 65
21 x x 1010it

might
ld
But, paradoxically, efficiency why this is so is that our ambition all applications of computing speed.
are efficient algorithms to take faster and we can handle larger that determines the increase in in
computer speed.
ten times faster than the current
in size of the problem.
Time
problem size Complexity
after speed up 10 S1 ≈10 S2 for large S2 3.16 S3 2.15 S4 S5 + 3.3
effect of using a more efficient is clear that if minute as a basis for we can solve a problem six times a
problem 125 times larger. These improvement obtained by a ten fold increase in speed. If an hour is used
as the basis of comparison, the differences are even more significant. Maximum problem size before
speed up
7
We therefore conclude that the asymptotic complexity of an algorithm is an important measure of the
goodness of an algorithm.
The Running time of a program
When solving a problem we are faced with a choice among algorithms. The basis for this can be any one
of the following:
i. We would like an algorithm that is easy to understand, code and debug.
ii. We would like an algorithm that makes efficient use of the computer’s
resources, especially, one that runs as fast as possible.
Measuring the running time of a program
The running time of a program depends on factors such as:
1. The input to the program.
2. The quality of code generated by the compiler used to create the object
program.
3. The nature and speed of the instructions on the machine used to execute the
program, and
4. The time complexity of the algorithm underlying the program.
The running time depends not on the exact input but only the size of the input. For many programs, the
running time is really a function of the particular input, and not just of the input size. In that case we define
T(n) to be the worst case running time, i.e. the maximum overall input of size ‘n’, of the running time on that
input. We also consider Tavg(n) the average, over all input of size ‘n’ of the running time on that input. In
practice, the average running time is often much harder to determine than the worst case running time.
Thus, we will use worst–case running time as the principal measure of time complexity.
Seeing the remarks (2) and (3) we cannot express the running time T(n) in standard time units such as
seconds. Rather we can only make remarks like the running time of such and such algorithm is proportional
to n2. The constant of proportionality will remain un-specified, since it depends so heavily on the compiler,
the machine and other factors.
Asymptotic Analysis of Algorithms:
Our approach is based on the asymptotic complexity measure. This means that we don’t try to count the
exact number of steps of a program, but how that number grows with the size of the input to the program.
That gives us a measure that will work for different operating systems, compilers and CPUs. The asymptotic
complexity is written using big-O notation.
Rules for using big-O:
The most important property is that big-O gives an upper bound only. If an algorithm is O(n2), it doesn’t

All
have to take n2 steps (or a constant multiple of n2). But it can’t

JNTU
 World
8
take more than n2. So any algorithm that is O(n), is also an O(n2) algorithm. If this seems confusing, think
of big-O as being like "<". Any number that is < n is also < n2.
1. Ignoring constant factors: O(c f(n)) = O(f(n)), where c is a constant; e.g.
O(20 n3) = O(n3)
2. Ignoring smaller terms: If a<b then O(a+b) = O(b), for example O(n2+n)
= O(n2)
3. Upper bound only: If a<b then an O(a) algorithm is also an O(b) algorithm. For example, an O(n) algorithm
is also an O(n2) algorithm (but not vice versa).
4. n and log n are "bigger" than any constant, from an asymptotic view (that means for large enough n). So
if k is a constant, an O(n + k) algorithm is also O(n), by ignoring smaller terms. Similarly, an O(log n + k)
algorithm is also O(log n).
5. Another consequence of the last item is that an O(n log n + n) algorithm,
which is O(n(log n + 1)), can be simplified to O(n log n).
Calculating the running time of a program:
Let us now look into how big-O bounds can be computed for some common algorithms.
Example 1:
Let’s consider a short piece of source code:
x = 3*y + 2; z = z + 1;
If y, z are scalars, this piece of code takes a constant amount of time, which we write as O(1). In terms of
actual computer instructions or clock ticks, it’s difficult to say exactly how long it takes. But whatever it is, it
should be the same whenever this piece of code is executed. O(1) means some constant, it might be 5, or
1 or 1000.
Example 2:
2n2 + 5n – 6 = O (2n) 2n2 + 5n – 6 ≠ Θ (2n) 2n2 + 5n – 6 = O (n3) 2n2 + 5n – 6 ≠ Θ (n3) 2n2 + 5n – 6 = O (n2)
2n2 + 5n – 6 = Θ (n2) 2n2 + 5n – 6 ≠ O (n) 2n2 + 5n – 6 ≠ Θ (n)
2n2 + 5n – 6 ≠ Ω (2n) 2n2 + 5n – 6 = o (2n) 2n2 + 5n – 6 ≠ Ω (n3) 2n2 + 5n – 6 = o (n3) 2n2 + 5n – 6 = Ω (n2)
 All
2n2 + 5n – 6 ≠ o (n2) 2n2 + 5n – 6 = Ω (n) 2n2 + 5n – 6 ≠ o (n)

JNTU

World
9
Example 3:
If the first program takes 100n2 milliseconds and while the second takes 5n3 milliseconds, then might not
5n3 program better than 100n2 program?
As the programs can be evaluated by comparing their running time functions, with constants by
proportionality neglected. So, 5n3 program be better than the 100n2 program. 5 n3/100 n2 = n/20
for inputs n < 20, the program with running time 5n3 will be faster than those the one with running time 100
n2. Therefore, if the program is to be run mainly on inputs of small size, we would indeed prefer the program
whose running time was O(n3)
However, as ‘n’ gets large, the ratio of the running times, which is n/20, gets arbitrarily larger. Thus, as the
size of the input increases, the O(n 3) program will take significantly more time than the O(n 2) program. So
it is always better to prefer a program whose running time with the lower growth rate. The low growth rate
function’s such as O(n) or O(n log n) are always better.
Example 4:
Analysis of simple for loop
Now let’s consider a simple for loop:
for (i = 1; i<=n; i++) v[i] = v[i] + 1;
This loop will run exactly n times, and because the inside of the loop takes constant time, the total running
time is proportional to n. We write it as O(n). The actual number of instructions might be 50n, while the
running time might be 17n microseconds. It might even be 17n+3 microseconds because the loop needs
some time to start up. The big-O notation allows a multiplication factor (like 17) as well as an additive factor
(like 3). As long as it’s a linear function which is proportional to n, the correct notation is O(n) and the code
is said to have linear running time.
Example 5:
Analysis for nested for loop
Now let’s look at a more complicated example, a nested for loop:
for (i = 1; i<=n; i++)
for (j = 1; j<=n; j++)
a[i,j] = b[i,j] * x;
The outer for loop executes N times, while the inner loop executes n times for every execution of the
outer loop. That is, the inner loop executes n × n = n 2 times. The assignment statement in the inner loop
takes constant time, so the running time of the code is O(n2) steps. This piece of code is said to have

All
quadratic running time.
 JNTU

World
10
Example 6:
Analysis of matrix multiply
Lets start with an easy case. Multiplying two n × n matrices. The code to compute the matrix product C =
A * B is given below.
for (i = 1; i<=n; i++)
for (j = 1; j<=n; j++)
C[i, j] = 0; for (k = 1; k<=n; k++)
C[i, j] = C[i, j] + A[i, k] * B[k, j];
There are 3 nested for loops, each of which runs n times. The innermost loop therefore executes n*n*n =
n3 times. The innermost statement, which contains a scalar sum and product takes constant O(1) time. So
the algorithm overall takes O(n3) time.
Example 7:
Analysis of bubble sort
The main body of the code for bubble sort looks something like this:
for (i = n-1; i<1; i--)
for (j = 1; j<=i; j++)
if (a[j] > a[j+1])
swap a[j] and a[j+1];
This looks like the double. The innermost statement, the if, takes O(1) time. It doesn’t necessarily take the
same time when the condition is true as it does when it is false, but both times are bounded by a constant.
But there is an important difference here. The outer loop executes n times, but the inner loop executes a
number of times that depends on i. The first time the inner for executes, it runs i = n-1 times. The second
time it runs n-2 times, etc. The total number of times the inner if statement executes is therefore:
(n-1) + (n-2) + ... + 3 + 2 + 1

This is the sum of an arithmetic series. N ∑ -1 i =1

A The which O((nThus, l value 2)/2)

l J n(n
bubble is O((nby of (n the sort 2ignoring -n)/2). - i) sum is = an Using is

N - 2i)
T n
a O(n2 n(n-1)/2. smaller ) the algorithm. = rules 2 term, 2 So - the for
 and 2

n
U W
big-O running to time of bubble sort given earlier, this bound

orld
O(n2), by ignoring a is O(n(n-1)/2), simplifies to constant

factor.
11
Example 8:
Analysis of binary search
Binary search is a little harder to analyze because it doesn’t have a for loop. But it’s still pretty easy because
the search interval halves each time we iterate the search. The sequence of search intervals looks
something like this:
n, n/2, n/4, ..., 8, 4, 2, 1
It’s not obvious how long this sequence is, but if we take logs, it is:
log2 n, log2 n - 1, log2 n - 2, ..., 3, 2, 1, 0
Since the second sequence decrements by 1 each time down to 0, its length must be log 2 n + 1. It takes
only constant time to do each test of binary search, so the total running time is just the number of times that
we iterate, which is log2n + 1. So binary search is an O(log2 n) algorithm. Since the base of the log doesn’t
matter in an asymptotic bound, we can write that binary search is O(log n).
General rules for the analysis of programs
In general the running time of a statement or group of statements may be parameterized by the input size
and/or by one or more variables. The only permissible parameter for the running time of the whole program
is ‘n’ the input size.
1. The running time of each assignment read and write statement can usually be taken to be O(1). (There
are few exemptions, such as in PL/1, where assignments can involve arbitrarily larger arrays and in any
language that allows function calls in arraignment statements).
2. The running time of a sequence of statements is determined by the sum rule.
I.e. the running time of the sequence is, to with in a constant factor, the largest running time of any
statement in the sequence.
3. The running time of an if–statement is the cost of conditionally executed statements, plus the time for
evaluating the condition. The time to evaluate the condition is normally O(1) the time for an if–then–else
construct is the time to evaluate the condition plus the larger of the time needed for the statements executed
when the condition is true and the time for the statements executed when the condition is false.
4. The time to execute a loop is the sum, over all times around the loop, the time to execute the body and
the time to evaluate the condition for termination (usually the latter is O(1)). Often this time is, neglected
constant factors, the product of the number of times around the loop and the largest possible time for one
execution of the body, but we must consider each loop separately to make sure.

l JNTU
Al
World
12

Chapter 2
Advanced Data Structures and Recurrence
Relations
Priority Queue, Heap and Heap Sort:
Heap is a data structure, which permits one to insert elements into a set and also to find the largest element
efficiently. A data structure, which provides these two operations, is called a priority queue.
Max and Min Heap data structures:
A max heap is an almost complete binary tree such that the value of each node is greater than or equal to
those in its children.
Figure 2. 1. Max. and Min heap
A min heap is an almost complete binary tree such that the value of each node is less than or equal to
those in its children. Figure 2.1 shows the maximum and minimum heap tree.
Representation of Heap Tree:
Since heap is a complete binary tree, a heap tree can be efficiently represented using one dimensional
array. This provides a very convenient way of figuring out where children belong to.
• The root of the tree is in location 1.
• The left child of an element stored at location i can be found in location 2*i.
• The right child of an element stored at location i can be found in location
2*i + 1.

l
• The parent of an element stored at location i can be found at location floor(i/2). A 55

l J N TU
75 85 65 25 95 Max 35 heap 45 15 85
W orld
55 45 95 65

15
25
35 75
Min heap

13
For example let us consider the following elements arranged in the form of array as follows:

A The represented The Insertion This properties heap Let to data values. to reached For

l
parent’s Operations adjoin the 1. 2. 3. us illustration, elements major operation in tree, consider
 l
root This its the value, the Insertion, Deletion Merging. operations one into of parent; root.

J
node using will data heap of on can is a and the 35 the continue X[1] heap 65 till a heap

N
used in build and is tree. if heap single array to the required we the added x[8] be X[2] tree:

to tree: up 45 complete Using get (max) value between can x[4] array a insert a 30 as max heap a X[3] be
 T
to 60 repeated parent the is x[2] 40 looks tree. be thought Figure a heap, greater binary tree. two

U
right performed node X[4] 40 The as 45 x[5] 2. whose nodes parent’s follows:

insertions 2. of child tree. principle into x[1] than X[5] Heap 25 as 25 on value lying on Next, of an 65 Tree that
X[6] path value x[6] 80. a 50 existing of of heap in at is we 50 Its insertion data, a from

W
greater X[7] greater parent tree have 55 tree: value 60 heap the starting structure. x[3]

to X[8] 30
 o
then is 55 is than than newly compare tree that, compared x[7] interchange from its child’s

r
satisfying inserted A first child heap it an we with value with empty or tree node have the the the

we

ld
its is

satisfied, hence interchange as well as further comparisons are no more required.

As another illustration, let us consider the case of insertion 90 into the resultant heap tree. First, 90 will be
added as left child of 40, when 90 is compared with 40 it
14
requires interchange. Next, 90 is compared with 80, another interchange takes place. Now, our process
stops here, as 90 is now in root node. The path on which these comparisons and interchanges have taken
places are shown by dashed line.
The algorithm Max_heap_insert to insert a data into a max heap tree is as follows:
Max_heap_insert (a, n) {
//inserts the value in a[n] into the heap which is stored at a[1] to a[n-1] integer i, n; i = n; item = a[n] ; while
( (i > 1) and (a[ ⌊ i/2 ⌋ ] < item ) do {
a[i] = a[ ⌊ i/2 ⌋ ] ; // move the parent down i = ⌊ i/2 ⌋ ; } a[i] = item ; return true ; } Example:
Form a heap by using the above algorithm for the given data 40, 80, 35, 90, 45, 50, 70.
1. Insert 40:
2. Insert 80:
40

ll J
3. Insert 35: A 40 80 80 40

N
80 40 35
TU World
15
80
40
4. Insert 90:
80
90
90
80
90
40 35

40 90
5. Insert 45:
90
80 35
40 45
6. Insert 50:
50
7. Insert 70:

ll
A Deletion Any deleting follows: • • node the of can Read Replace tree • 80

35
40
 J N
a node be deleted from from heap a 50

T U
tree: heap tree. But from 80 90

35

World
35
40 45 50
the application point of view, root node has some special importance. The principle of deletion is as
the root node into a temporary storage say, ITEM.
the root node by the last node in the heap tree. Then re-heap the as stated below:
Let newly modified root node be the current node. Compare its value with the value of its two child. Let X
be the child whose value is the largest. Interchange the value of X with the value of the current node.
• Make X as the current node.
• Continue re-heap, if the current node is not an empty node.
90
80 50
40 45 35
90
70 80 50
40 45 35 70
16
90
80 70
40 45 35 50
The algorithm for the above is as follows:
delmax (a, n, x) // delete the maximum from the heap a[n] and store it in x {

A } adjust // single { } Here 26 compared 26 interchanged. shown 27 The l is and Deleting

l
the 35 heap, compared complete (a, in if { } x adjust return j item while { } a this = = (n [ 12 root figure

⌊ i, 2 with the a[1]; = 45 j 1 = n) *i (j / node true; < (a, 0) 24 a[i] node node 2 write return if if else j ; < Now, binary i =
 J
its ⌋ ((j (item 2.3. then

a[1] < 1, n) with ] 29 ; 2 with a[ = n. two < with n-1); do * (“heap is 26 false; = item; 26 n) No ⌊ j; trees // > 99. its j

N
data a[n]; child 99 / appear and a compare node data 2 (j)) children, The is with 99 ⌋ 26 (a ]

T
empty”); 45 26 then has = 26 (j) last 63 roots as a[j] 57 and left an < is break; the node namely, a 26
 U
63 // address and removed a(2*i) 63. (j move leave 57 42 + right is As Figure 1)) and 26, //

the 63 greater 57 node, then child a from 2. a(2*i it is larger position 27 3. and is greater, j After and ← hence than

W
in 35 the + 42, j child the let 1) Deletion + 12 for n tree. 1; are j as 45 or level re-heap be up

they item 24 57 less combined the a of Next level is 3. 29 node are is than larger found So, greater, is

o
interchanged. 26 with completed. 1. 63 with 99 child // at data is a(i) root so replaced 99 26 to they
r
node form This 57 Now, are by

42 a

ld
is is

17
Merging two heap trees:
Consider, two heap trees H1 and H2. Merging the tree H2 with H1 means to include all the node from H2
to H1. H2 may be min heap or max heap and the resultant tree will be min heap if H1 is min heap else it

l l
will be max heap. A Merging empty: They Applications 1. 2. 1. 2. are H1:
 J
operation two 38 Delete Insert Sorting Priority max main 59 heap 38 of the 45 the 80 queue (Heap

N
applications heap consists 92 node root 59 93 implementation. sort) tree: node, 45 x 67 into of

T
Figure 92 and two of say H1 92 heap 2. 96 steps: satisfying x, + 4. from Merging trees Resultant
U
Continue 13 H2. known: of the 67 two max Re-heap 92 property heaps. 19 heap steps 19

W o
after 93 H2. H2: of 1 13 merging min H1. and 96 heap 2 80 H1 while and H2 H2

r
is not
ld
18
HEAP SORT:
A heap sort algorithm works by first organizing the data to be sorted into a special type of binary tree called
a heap. Any kind of data can be sorted either in ascending order or in descending order using heap tree. It
does this with the following steps:
1. Build a heap tree with the given set of data.
2. a. Remove the top most item (the largest) and replace it with the last
element in the heap.
b. Re-heapify the complete binary tree.
c. Place the deleted node in the output.
3. Continue step 2 until the heap tree is empty.
Algorithm:
This algorithm sorts the elements a[n]. Heap sort rearranges them in-place in non- decreasing order. First
transform the elements into a heap.
heapsort(a, n) {
heapify(a, n); for i = n to 2 by – 1 do {
temp = a[I]; a[i] = a[1]; a[1] = t; adjust (a, 1, i – 1); } } heapify (a, n) //Readjust the elements in a[n] to form a heap. {

for i ← ⌊ n/2 ⌋ to 1 by – 1 do adjust (a, i, n); } adjust (a, i, n) // The complete binary trees with roots a(2*i) and
a(2*i+1) are combined // with a(i) to form a single heap, 1<i<n. No node has an address greater // than n or less than
1. {
j = 2 *i ; item = a[i] ; while (j < n) do {
if ((j < n) and (a (j) < a (j + 1)) then j ← j + 1;
// compare left and right child and let j be the larger child if (item > a (j)) then break;
// a position for item is found else a[ ⌊ j / 2 ⌋ ] = a[j] // move the larger child up a level j = 2 * j; } a [ ⌊ j / 2 ⌋ ] = item;
 ll JNTU
} A
World
19
Time Complexity:
Each ‘n’ insertion operations takes O(log k), where ‘k’ is the number of elements in the heap at the time.
Likewise, each of the ‘n’ remove operations also runs in time O(log k), where ‘k’ is the number of elements
in the heap at the time. Since we always have k ≤ n, each such operation runs in O(log n) time in the worst
case.
Thus, for n elements it takes O(n log n) time, so the priority queue sorting algorithm runs in O(n log n)
time when we use a heap to implement the priority queue.
Example 1:
Form a heap from the set of elements (40, 80, 35, 90, 45, 50, 70) and sort the data using heap sort.
Solution:
First form a heap tree from the given set of data and then sort by repeated deletion operation:
1. E xc h a n g e ro ot 9 0 w it h t h e la st e le me n t 3 5 of t h e array a n d re - h e a p ify
35
2. E xc h a n g e ro ot 8 0 w it h t h e la st e le me n t 5 0 of t h e array a n d re - h e a p ify

70 50
50 45 7 0
40 35 80 90
3. E xc h a n g e ro ot 7 0 w it h t h e la st e le me n t 3 5 of t h e array a n d re- h e a p ify

50 3 5
45
 80 3 5 45 35 80 70

ll JNTU
40 45 50 90 A
W
45 80

orld
40 35 70

50 90
35 50
40 70 80 90
20
70
45 50
40 35 80 90
50
45 35
40 70 80 90
4. Exchange root 50 with the last element 40 of the array and re-heapify
5. Exchange root 45 with the last element 35 of the array and re-heapify
40 35 35 40 45
50 70 80 90
6. Exchange root 40 with the last element 35 of the array and re-heapify
35

40 35
45
40 35

A Priority simplified represented complexities of As values priority so and As These that

l l
Priority array. it an heap so a can illustration, processes process on. are values. queue
 J
trees be implementation Process queue Priority to deleted 50 of using be allow having circular

N
The can enter stored consider implementation 70 top and the be an P1 higher 5 the 40

array priority implemented duplicity heap array. in 80 system the is P2 from 4 priority and possible 45 can

T
following element This P3 3 of of 90 linked be in data heap using value the rebuilt P4
 U
implementation using 4 using processes list that order in tree, will heap P5 it. 5 but

to heap circular has Elements as be get which getting P6 5 tree: listed to serviced with tree; the be array,

W
P7 can 3 next their is above the processed associated the 50 first. be therefore

advantages P8 element 2 priorities: linked heap, The at formed The 70 time sorted P9 1 first with however, list

o
heap to free 0, based Ptree is 5 80 be 10 of etc. their say. at tree simplicities processed, from

the on Another
 r
Assume priority can 90

can root; their the

ld
be be

formed considering the process priority values. The order of servicing the process is successive deletion
of roots from the heap.
45 40 40 45 35
50 70 80 90
21
40
35 45
50 70 80 90
35
40 45
50 70 80 90
Binary Search Trees:
A binary search tree has binary nodes and the following additional property. Given a node t, each node to
the left is “smaller” than t, and each node to the right is “larger”. This definition applies recursively down the
left and right sub-trees. Figure
shows a binary search tree where characters are stored in the nodes.
 A Figure tree: how Binary The root R, we node Why we l reach search node the “c”.

l
use you proceed 2.5 Tree name to will binary R, also operation a we Searching: get inorder

J N
dead to shows proceed its search a Figure list end. right starts what

originated. Figure of c a to trees? The 2.6. child. the the happens d b from 2.5. Figure Searching node left e
 T U
The Binary root child; f contents process Path if 2.6 you g node a Search

shows to if binary h do item find will R, in an tree c tree sorted the if is be inorder item greater continued

W
path order. is taken traversal less than In till than when the fact, the of value the

o r
item a searching that’s binary value in is the probably found search in node for the
 ld
or a

Binary search trees provide an efficient way to search through an ordered collection of items. Consider the
alternative of searching an ordered list. The search must proceed sequentially from one end of the list to
the other. On average, n/2 nodes must be compared for an ordered list that contains n nodes. In the worst
case, all n
f
bh
aeg
d
c Inorder: a b c d e f g h
22
nodes might need to be compared. For a large collection of items, this can get very expensive.
The inefficiency is due to the one-dimensionality of a linked list. We would like to have a way to jump into
the middle of the list, in order to speed up the search process. In essence, that’s what a binary search
tree does. The longest path we will ever have to search is equal to the height of the tree. The efficiency of

a binary A search smallest To sub have Figure Unfortunately, than example. worst When
 l l
maintain the Inserting obtain next trees as linked case nodes 2.7 tree many possible

section. the For the have shows Nodes of thus lists. balance are n smallest a Figure children trees

J
comparisons degenerate approximately depends height an being into Such Figure example

N
2.7. can of height, a as is added the become trees Binary Well 2.8. log(n). on possible,
 T
tree, – tree. of the constructed a the A the and are a tree degenerate Search an height so well-

constructed same We a same with called deleted unbalanced average must will b as of binary all

U
Tree: number be investigate binary degenerate for the levels in c of balanced, a

search a tree. n/2 linked that d search binary being of tree. comparisons For tree. nodes. they’re e methods

W
list. trees. tree. where full a search tree except Also, no both Figure holding of tree,
 o
better are balancing each possibly the needed, 2.8 it’s for left n node nodes, difficult shows

r
searching and the trees should with right last. the

an

ld
to in a

When adding nodes to a binary search tree, we must be careful to maintain the binary search tree property.
This can be done by first searching the tree to see whether the key we are about to add is already in the
tree. If the key cannot be found, a new node is allocated and added at the same location where it would go
if
e
cg
bdfh
a
23
the search had been successful. Figure 2.9 shows what happens when we add some nodes to a tree.
Figure 2.9. Inserting nodes into a binary search tree.
Deleting nodes from a Binary search tree:
Deletions from a binary search tree are more involved than insertions. Given a node to delete, we need to
consider these tree cases:
1. The node is a leaf
2. The node has only one child.
3. The node has two children.
Case 1: It is easy to handle because the node can simply be deleted and the corresponding child pointer
of its parent set to null. Figure 2.10 shows an example.
Figure 2. 10. Deleting a leaf node.
Case 2: It is almost as easy to manage. Here, the single child can be promoted up the tree to take the place
of the deleted node, as shown in figure 2.11.

l
Al
Figure 2. 11. Deleting a node that has one child.

JNTU
 World
b
ae
df
ddd

b f insert a b f insert e b f

aae
b
ac
e
df
b
ac
d
24
b
ac
Case 3: The node to be deleted has two children, is more difficult. We must find some node to take the
place of the one deleted and still maintain the binary search tree property. There are two obvious cases:
• The inorder predecessor of the deleted node.
• The inorder successor of the deleted node.
We can detach one of these nodes from the tree and insert it where the node to be deleted.
The predecessor of a node can be found by going down to the left once, and then all the way to the right
as far as possible. To find the successor, an opposite traversal is used: first to the right, and then down to
the left as far as possible. Figure 2.12 shows the path taken to find both the predecessor and successor of
a node.
Figure 2.12. Finding predecessor and successor of nodes.
Both the predecessor and successor nodes are guaranteed to have no more than one child, and they
may have none. Detaching the predecessor or successor reduces to either case 1 or 2, both of which are
easy to handle. In figure 2.13 (a), we delete a node from the tree and use its predecessor as the
replacement and in figure 2.13 (b), we delete a node from the tree and use its successor as the
 l J
replacement.
Al a (a) predecessor b After c Figure 2.13. (b) Deleting

N
a node that has two children. d i g f h deleting the of e.

TU Wo
f

bi
gj
jad
ch
 rld
After deleting the successor of e.

ee
bibi

adgjad
gj

cfhcfh
(a) Predecessor of e. (b) Successor of e.
25
Balanced Trees:
For maximum efficiency, a binary search tree should be balanced. Every un-balanced trees are referred to
as degenerate trees, so called because their searching performance degenerates to that of linked lists.
Figure 2.14 shows an example.

Figure 2.14. A degenerate binary search A The search This Figure balanced. Balancing There

l l
first pathological are 1) 2) tree. 2.15 tree two Use Allow Acts:
 J
The shows was basic tree nodes second sequence built rotations. ways what

N
to by Figure a tree have used inserting the b is 2.15 was to more often tree c

T U
keep built e Balanced d the in than used f trees 2.14-(b) keys using g

two h to Tree
balanced. ‘a’ i
 W
children. test the through above insertion the ‘i’ tree.

or
in sorted order into binary

ld
sequence. a-g-b-f-c-e-d. balancing capacity of a tree. would look like if it were
aa
bi
cb
dh
e
c
f
g
g
(a) h d (b)
f
i
e
26
Tree Rotations:
Certain types of tree-restructurings, known as rotations, can aid in balancing trees. Figure 2.16 shows two
types of single rotations.
pctb4a312p
at1b
2c
34
Figure 2.16. Single Rotations
Another type of rotation is known as a double rotation. Figure 2.17 shows the two symmetrical cases.

ll
gcpa1 A 1 2 Figure 2.17. Double Rotations

J N T
a2p btcb433 4 (a) Right Rotation. (a) Left Rotation. b
U
cbac

1234

b
Wo
a1 23 c4

rld
b
4ac
a31234
b
1 2 (a) Left - Right Rotation. a
ac1b
12342c
34
(b) Right - Left Rotation.
27
Both single and double rotations have one important feature: in order traversals of the trees before and
after the rotations are preserved. Thus, rotations help balance trees, but still maintain the binary search
tree property.
Rotations don’t guarantee a balanced tree, however for example, figure 2.18 shows

A A element two Examples: The element) Operations a when The rotate. Dictionary:

l
Dictionary right 1. 2. 3. ▪ ▪ ▪ pairs above Red-black to A which etc. A The Operating Get For Insert

l
For Delete For rotations associated do telephone word pairs in example, example, example,
 J
the list are rotation last a on is is or dictionary or dictionary of having element associated the

Dictionaries: a trees put remove two students System that collection directory with key an and

N
remove(k) get(k) put(k, examples the have makes a element associated and should the

what an is course. same b with enrolled a element e) certain returns the key of Figure with collection a c

T
kind the puts pairs have tree key. with meaning are k removes The a (equivalently, 2.18. with

word. for of collection the rules the with more a the dictionaries, of list notations the specified of a element
 U
element form Un-balanced same contains specified a of to elements; the data un-

balanced. specified the be of (k, element key. structures to key telephone word, e with used e e), do.
(CourseName, which whose is key into key. rotation. each the the where pronunciation to with from

W
The the element a permit key key determine element course, numbers key the

b
dictionary. k trick k.

o
is is k. k dictionary. RollID) two whose a into compiler lies c comprises key and and when or

the in names. and key pairs. determining

etymologies

r
more dictionary. and a is e a how k). is word,

ld
(key, the No to

28
Disjoint Set Operations
Disjoint Set Operations
Set:
A set is a collection of distinct elements. The Set can be represented, for examples, as S1={1,2,5,10}.
Disjoint Sets:
The disjoints sets are those do not have any common element. For example S1={1,7,8,9} and
S2={2,5,10}, then we can say that S1 and S2 are two disjoint sets.
Disjoint Set Operations:
The disjoint set operations are
1. Union 2. Find
Disjoint set Union:
If Si and Sj are tow disjoint sets, then their union Si U Sj consists of all the elements x such that x is in Si
or Sj.
 l
Find: A Example: S1={1,7,8,9} S1={1,7,8,9} S1 Given Example:

l J N
Then, U S2={1,2,5,7,8,9,10} the element I, find S2={2,5,10}

T U
the set containing i.
World
S2={2,5,10} s3={3,4,6}
Find(4)= S3 Find(5)=S2 Find97)=S1
Set Representation:
The set will be represented as the tree structure where all children will store the address of parent / root
node. The root node will store null at the place of parent address. In the given set of elements any element
can be selected as the root node, generally we select the first node as the root node.
Example: S1={1,7,8,9} S2={2,5,10} s3={3,4,6} Then these sets can be represented as
Disjoint Union:
To perform disjoint set union between two sets Si and Sj can take any one root and make it sub-tree of the
other. Consider the above example sets S1 and S2 then the union of S1 and S2 can be represented as any
one of the following.
29

l
A i p Find: the names. is identify use of represent Union the all an name sets. pointer
 l
To In Example: For Algorithm To array [1] -1 The sets and perform the perform presenting

J
the of The data just Find of to parent each following [2] -1 1 index root. structure by to

N
Algorithms: for find union set. the n node. Union Union values elements operation, [3] -1

sets roots So, the contains For the and we operation: SimpleUnion(i,j) represent of the [4] 3 where
 T
array require trees along Find root two representation representing n algorithms, [5] 2 value

U
with fields. the is one the nodes the the more One [6] 3 maximum tree function entry

W
(elements is we them. data structure the is will [7] 1 as ignore set takes structure

value shown To be name of represent ‘-1’. we the [8] 1 the among set) below. need and inputs set to
 o r
and store the [9] 1 to the names the the as other elements maintain sets, the the

entries [10] 2
and one set

ld
set we

roots i and j . And make the parent of i as j i.e, make the second root as the parent of first root.
Algorithm SimpleUnion(i,j) {
P[i]:=j; }
30
Algorithm for find operation: The SimpleFind(i) algorithm takes the element i and finds the root node of
i. It starts at I until it reaches a node with parent value -1.
Algorithms SimpleFind(i) {
while( P[i]≥0) do i:=P[i]; return i; }
Analysis of SimpleUnion(i,j) and SimpleFind(i):
Although the SimpleUnion(i,j) and SimpleFind(i) algorithms are easy to state, their performance
characteristics are not very good. For example, consider the sets
1234......
n
Then if we want to perform following sequence of operations Union(1,2) , Union(2,3)....... Union(n-1,n)
and sequence of Find(1), Find(2)......... Find(n).
The sequence of Union operations results the degenerate tree as below.
n n-1

l l
A Since, processed complexity We degenerate tree Weighting can with the

J
If improve the the time in of tree root number time O rule taken ( by the ∑ i=1 j, n O(n). for

i )
N = O(n
applying then performance for of Union: nodes make a And Union weighting 2).
 T
for in ‘j’ n-2 is the of the 1 the constant, union tree parent sequence rule with and for of the

Union.
find root i; of otherwise n-1 by I Find is sequence avoiding less operations make than the of the ‘i' union
creation the it number will parent can take of be
in of the
time

U World
j.

31

A To tree. then Since maintain Algorithm //Union // { } root[i]. Union(1,2), Collapsing

l l
implement P[i]=-count[i] count[i] To all temp:= if { } else { } If // // Consider (P[i]>P[j]) do

nodes count j sets i j P[i]:=j; P[j]:=temp;

P[j]:=i; P[i]:=temp; is has has WeightedUnion(i,j) Union(3,4), this a equals P[i]+P[j]; rule weighting with

J
node fewer in other fewer we the P then for field and maintain roots on to tree than nodes

N
nodes find: number the Union(5,6) p[j]=-count[j] of rule created roots i the path and
 T
“count” we root of have from by need j nodes , and as i≠j WeightedUnion() positive i field

U
negative to to Union(7,8) using in its know in tree root the numbers the number.

W
with how and root weighted root many p[i]≠root[i], on of in the i. every parent nodes
 o
sequence rule tree. (P) are then field, If there of set ‘i' 1≤i≤8. we is P[j] in the can

r
every

to
root

ld
32
 l l
A Now Find(8), If total SimpleFind() process of 24

J N
Find(8)...........................Find(8) moves the is following . used each eight

T
Find(8) find operations
 U Worl
requires going up three parent link

d
fields for a

When Collapsing find is used the first Find(8) requires going up three links and resetting three links. Each
of remaining seven finds require going up only one link field. Then the total cost is now only 13 moves.( 3
going up + 3 resets + 7 remaining finds).
33
Algorithm CollapsingFind(i) // Find the root of the tree containing element i // use the collapsing
rule to collapse all nodes from i to root. {
r:=i; while(P[r]>0) do r:=P[r]; //Find root
while(i≠r) {
 l
//reset the parent node from element i to the root s:=P[i]; P[i]:=r; i:=s; } }
Al
JNTU

World
34
Recurrence Relations
35 Recurrence Relation for a sequence of numbers S is a
formula that relates all but a finite number of terms of S to previous terms of the sequence, namely, {a 0, a1,
a2, . . . . . , an-1}, for all integers n with n ≥ n0, where n0 is a nonnegative integer. Recurrence relations are
also called as difference equations.
Sequences are often most easily defined with a recurrence relation; however the calculation of terms by
directly applying a recurrence relation can be time consuming. The process of determining a closed form
expression for the terms of a sequence from its recurrence relation is called solving the relation. Some
guess and check with respect to solving recurrence relation are as follows:
• Make simplifying assumptions about inputs
• Tabulate the first few values of the recurrence
• Look for patterns, guess a solution
• Generalize the result to remove the assumptions
Examples: Factorial, Fibonnaci, Quick sort, Binary search etc.
Recurrence relation is an equation, which is defined in terms of itself. There is no single technique or
algorithm that can be used to solve all recurrence relations. In fact, some recurrence relations cannot be
solved. Most of the recurrence relations that we encounter are linear recurrence relations with constant
coefficients.
Several techniques like substitution, induction, characteristic roots and generating function are available
to solve recurrence relations.
The Iterative Substitution Method:
One way to solve a divide-and-conquer recurrence equation is to use the iterative substitution method. This
is a “plug-and-chug” method. In using this method, we assume that the problem size n is fairly large and
we than substitute the general form of the recurrence for each occurrence of the function T on the right-
hand side. For example, performing such a substitution with the merge sort recurrence equation yields the
equation.
T(n) = 2 (2 T(n/22) + b (n/2)) + b n
= 22 T(n/22) + 2 b n
Plugging the general equation for T again yields the equation.
T(n) = 22 (2 T(n/23) + b (n/22)) + 2 b n
= 23 T(n/23) + 3 b n
The hope in applying the iterative substitution method is that, at some point, we will see a pattern that can

All
be converted into a general closed-form equation (with T only
 JNTU

World
appearing on the left-hand side). In the case of merge-sort recurrence equation, the general form is:
T(n) = 2 i T (n/2i) + i b n
Note that the general form of this equation shifts to the base case, T(n) = b, where n = 2 i, that is, when i =
log n, which implies:
T(n) = b n + b n log n.
In other words, T(n) is O(n log n). In a general application of the iterative substitution technique, we hope
that we can determine a general pattern for T(n) and that we can also figure out when the general form of
T(n) shifts to the base case.
The Recursion Tree:
Another way of characterizing recurrence equations is to use the recursion tree method. Like the iterative
substitution method, this technique uses repeated substitution to solve a recurrence equation, but it differs
from the iterative substitution method in that, rather than being an algebraic approach, it is a visual
approach.
In using the recursion tree method, we draw a tree R where each node represents a different substitution
of the recurrence equation. Thus, each node in R has a value of the argument n of the function T (n)
associated with it. In addition, we associate an overhead with each node v in R, defined as the value of the
non-recursive part of the recurrence equation for v.
For divide-and-conquer recurrences, the overhead corresponds to the running time needed to merge the
subproblem solutions coming from the children of v. The recurrence equation is then solved by summing
the overheads associated with all the nodes of R. This is commonly done by first summing values across
the levels of R and then summing up these partial sums for all the levels of R.
For example, consider the following recurrence equation:

T(n) =
⎧
⎨
b
3 T ( n / 3) + b n
A This sort sequences, sequences R overhead problem for

l
l is algorithm this the solutions associated recurrence, to recursively recurrence produce

J N
so produced that with each a sort equation sorted we it, each internal by
 T
divide which v’s version that one, children. corresponds an node we and unsorted of if if

U
get, v the n n then ≥ We has < 3 original 3 for do to illustrate three sequence the

W
example, a three-way sequence. children time the needed into by tree and merge
 o
modifying In three R equal of the recursion has a to merge as follows:

rld
the merge – sized three sorted tree size and an the sub-

36
O v erhe a d bn bn bn
The overheads of the nodes of each level, sum to bn. Thus, observing that the depth of R is log 3 n, we
have that T(n) is O(n log n).
The Guess-and-Test Method:
Another method for solving recurrence equations is the guess-and-test technique. This technique involves
first making a educated guess as to what a closed-form solution of the recurrence equation might look like
and then justifying the guesses, usually by induction. For example, we can use the guess-and-test method
as a kind of “binary search” for finding good upper bounds on a given recurrence equation. If the justification
of our current guess fails, then it is possible that we need to use a faster-growing function, and if our current
guess is justified “too easily”, then it is possible that we need to use a slower-growing function. However,
using this technique requires case careful, in each mathematical step we take, in trying to justify that a
certain hypothesis holds with respect to our current “guess”.
Example 2.10.1: Consider the following recurrence equation:
T (n) = 2 T(n/2) + b n log n. (assuming the base case T(n) = b for n < 2)
This looks very similar to the recurrence equation for the merge sort routine, so we might make the
following as our first guess:
First guess: T (n) < c n log n.
for some constant c>0. We can certainly choose c large enough to make this true for the base case, so
consider the case when n > 2. If we assume our first guesses an inductive hypothesis that is true for input
sizes smaller than n, then we have:
T (n) = 2T (n/2) + b n log n
≤ 2 (c (n/2) log (n/2)) + b n log n
≤ c n (log n – log 2) + b n log n
≤ c n log n – c n + b n log n.
But there is no way that we can make this last line less than or equal to c n log n for n ≥ 2. Thus, this first
guess was not sufficient. Let us therefore try:
Better guess: T (n) ≤ c n log2 n.
for some constant c > 0. We can again choose c large enough to make this true for the base case, so

All
consider the case when n ≥ 2. If we assume this guess as an

JNTU
 World
37
inductive hypothesis that is true for input sizes smaller than n, then we have inductive hypothesis that is
true for input sizes smaller than n, then we have:
T (n) = 2T (n/2) + b n log n
≤ 2 (c (n/2) log2 (n/2)) + b n log n
≤ c n (log2n – 2 log n +1) + b n log n
≤ c n log2 n - 2 c n log n + c n + b n log n
≤ c n log2 n
Provided c ≥ b. Thus, we have shown that T (n) is indeed O(n log 2 n) in this case. We must take care in
using this method. Just because one inductive hypothesis for T (n) does not work, that does not
necessarily imply that another one proportional to this one will not work.
Example 2.10.2: Consider the following recurrence equation (assuming the base case T(n) = b for n < 2):
T (n) = 2T (n/2) + log n
This recurrence is the running time for the bottom-up heap construction. Which is O(n). Nevertheless, if we
try to prove this fact with the most straightforward inductive hypothesis, we will run into some difficulties. In
particular, consider the following:
First guess: T (n) ≤ c n.
For some constant c > 0. We can choose c large enough to make this true for the base case, so consider
the case when n ≥ 2. If we assume this guess as an inductive hypothesis that is true of input sizes smaller
than n, then we have:
T (n) = 2T (n/2) + log n
≤ 2 (c (n/2)) + log n
= c n + log n
But there is no way that we can make this last line less than or equal to cn for n > 2. Thus, this first guess
was not sufficient, even though T (n) is indeed O (n). Still, we can show this fact is true by using:
Better guess: T (n) ≤ c (n – log n)
For some constant c > 0. We can again choose c large enough to make this true for the base case; in fact,
we can show that it is true any time n < 8. So consider the case when n ≥ 8. If we assume this guess as an
inductive hypothesis that is true for input sizes smaller than n, then we have:
T (n) = 2T (n/2) + log n
≤ 2 c ((n/2) – log (n/2)) + log n = c n – 2 c log n + 2 c + log n = c (n – log n) – c log n + 2 c + log n ≤ c (n –
log n)
Provided c ≥ 3 and n ≥ 8. Thus, we have shown that T (n) is indeed O (n) in this case. The Master
 All
Theorem Method:

JNTU

World
38
Each of the methods described above for solving recurrence equations is ad hoc and requires mathematical
sophistication in order to be used effectively. There is, nevertheless, one method for solving divide-and-
conquer recurrence equations that is quite general and does not require explicit use of induction to apply
correctly. It is the master method. The master method is a “cook-book” method for determining the
asymptotic characterization of a wide variety of recurrence equations. It is used for recurrence equations
of the form:
a T(n / b) + f (n) if n > d
T(n) = ⎨ ⎧ ⎩ c if n < d Where d > 1 is an integer constant, a > 0, c > 0, and b >
1 are real constants, and f (n) is a function that is positive for n ≥ d.
The master method for solving such recurrence equations involves simply writing down the answer based
on whether one of the three cases applies. Each case is distinguished by comparing f (n) to the special
function nlog a (we will show later why this special function is so important.
The master theorem: Let f (n) and T (n) be defined as above.
1. If there is a small constant є > 0, such that f (n) is O (nlogb a - ε ) , then T

(n) is Θ (nlogb a ).
2. If is there Θ (n logis b a a logconstant k +1 n . )

( )
K ≥ 0, such that f (n) is Θ nlogb a logk n , then T (n)

3. If there are small constant ε > 0 and δ < 1, such that f (n) is
and af(n/b) < δf(n), for n ≥ d, then T (n) is Θ (f(n)).

A
Case function, Case function, Case function. We the Example In This Example this

l l
illustrate assumption means 1 2 3 case, characterizes characterizes characterizes nand
 J
2.11.1: 2.11.2: log
that bnthe a. log that bT a usage = (n) Consider Consider T nthe the the logis

N
(n) of 2 Θ situation situation situation 4 the = (n= the c 2the ) nmaster for by 2. recurrenceT

T UW
recurrence Thus, n the where when when < Ω f

(n) is polynomial f (n) is asymptotically f (n) is polynomially method with a few examples d, for constants c
> 1 T (n) = 4 we are in case 1, for master. (n) = 2 T (n/2) b
orl
(n logb a + ε )

smaller than the special

close to the special
larger than the special
(with each taking and d > 1).
T (n/2) + n

f (n) is O (n2-ε) for ε = 1.

+ n log n

d
In this case, nlogba = nlog 2 2 Θ (n log n). This means that T = n. Thus, we (n) is Θ (n log 2n) are in case 2,
with k=1, for f (n) is
by the master method.
39
Example 2.11.3: consider the recurrence T (n) = T (n/3) + n
In this case nlogba = nlog3 =n = 1. Thus, we are in Case 3, for f (n) is Ω(n ), for ε = 1, and af(n/b) = n/3 = (1/3)
f(n). This means that T (n) is Θ (n) by the master method.
Example 2.11.4: Consider the recurrence T (n) = 9 T (n/3) + n2.5
In this case, nlogba = nlog 9 = n2. Thus, we are in Case 3, since f (n) is Ω(n 2+ε ) (for ε=1/2) and af(n/b) = 9
(n/3)2.5 = (1/3)1/2 f(n). This means that T (n) is Θ (n2.5) by the master method.
Example 2.11.5: Finally, consider the recurrence T (n) = 2T (n1/2) + log n
Unfortunately, this equation is not in a form that allows us to use the master method.
We can put it into such a form, however, by introducing the variable k = log n, which
lets us write:
T (n) = T (2k) = 2 T (2k/2) + k
Substituting into this the equation S (k) = T (2k), we get that
S (k) = 2 S (k/2) + k
Now, this recurrence equation allows us to use master method, which specifies that S (k) is O (k log k).
Substituting back for T (n) implies T (n) is O (log n log log n).
CLOSED FROM EXPRESSION
There exists a closed form expression for many summations
Sum of first n natural numbers (Arithmetic progression)

∑ ni=1

1 0 0+ε
3

All
Sum of squares i ∑ =n1

J N TU n(n +
i of = first 1 + 2 n 2+ . . . . . ..+n = 2

1)

natural numbers
World
n(n + 1)
i 2 = 1 + 4 + 9 +. . . . . . . + n2 = 6
(2n + 1)

Sum of cubes n ∑ i=1

3= n (n + 1)
22i 13 + 23 + 33 +. . . . . . . + n3 = 4 Geometric progression

∑ n
i
= 20 + 21 + 22 +. . . . . . . + 2n = 2n + 1 -1 i = 0

40

n ∑ i=0

r n+1
ri= -1
r-1

∑ ni=1

rn
ri= -1
r -1
SOLVING RECURRENCE RELATIONS
Example 2.13.1. Solve the following recurrence relation:
T(n) = ⎧2
│ 2 .T │ │
│⎩ ⎛⎝ 2n⎞ ⎠
A l l JN
Now substitute TT ⎝ ⎝ │ 22 │ ⎠ = 2.T ⎝ │

T
23 this back (n) = 22 . ⌈ │2. T ⎛ │ │ + 7 ⎠ into the n │ ⎞ + ⌊ ⎝ 23 ⎠ last T(n) ⌉ 7│ + 7 ⌋ ,

U Wo
n=1 22 ⎠

definition (last line from step 2):

rld
+7,n>1

Solution: We first start by labeling the main part of the relation as Step 1:
Step 1: Assume n > 1 then,

T(n) = 2.T ⎛ ⎝ │ n 2 ⎞ │ + 7
⎠ Step 2: Figure out what T ⎛ │ n │ ⎞ is; everywhere you see n, replace it with n .
⎛n⎞⎛n⎞
⎨
⎝2⎠2
T │ │ = 2.T │ │ + 7
⎝ 2 ⎠ ⎝ 22 ⎠
Now substitute this back into the last T (n) definition (last line from step 1):

T (n) = 2. ⌈ │2. T ⌊ ⎛ │ n │ ⎞ + ⎝ 22 ⎠ ⌉ 7│ ⌋ + 7 = 22 .T ⎛ │ n │ ⎞ + 3.7 ⎝ 22 ⎠

Step 3: let's expand the recursive call, this time, it's T ⎛ │ n │ ⎞ :

⎛n⎞⎛n⎞
41
⎛ 23 .T │ n │ ⎞ + 7.7 ⎝ 23 ⎠
From this, first, we notice that the power of 2 will always match i, current. Second, we notice that the
multiples of 7 match the relation 2i – 1 : 1, 3, 7, 15. So, we can write a general solution describing the state
of T (n).
Step 4: Do the ith substitution.
│
T(n) = 2i . T │ ⎛ n ⎞

(
+ 2i
) . 7 ⎝ 2i ⎠
-1

However, how many times could we take these "steps"? Indefinitely? No... it would stop when n has been
cut in half so many times that it is effectively 1. Then, the original definition of the recurrence relation would
give us the terminating condition T (1) = 1 and us restrict size n = 2i
n
When, 1 = ⇒ 2i 2i
=n
⇒ log2 2 = log2 n
⇒ i .log2 2 = log2 n
⇒ i = log2 n

│+ (2
Now we can substitute T (n) = this "last 2 . T i │ ⎛ n value ⎞i

-1 ) . 7
for i"

back into a our general Step 4 equation:

= ⎝ 2i ⎠

( ) .7
2log2 n . T ⎛ n │ + 2log2 n - 1

⎝ 2log2 n ⎠
= n . T(1) + (n - 1) . 7
= n . 2 + (n - 1) . 7
=9.n-7
This implies that T (n) is O (n).
Example 2.13.2. Imagine that you have a recursive program whose run time is
described by the following recurrence relation:
│
T(n) = ⎧1
⎛n⎞i

All JN
│
T U World
⎞,

2.T │ │ + 4. n , n > 1
n = 1 ⎨ │⎩ ⎝2⎠
42
Solve the relation with iterated substitution and use your solution to determine a
tight big-oh bound.
Solution:
Step 1: Assume n > 1 then,
│n
T(n) = 2 . T ⎛ ⎞
│ + 4 . n ⎝ 2 ⎠ Step 2: figure out what T ⎛ ⎝ │ n 2 │ ⎞ ⎠

is:
n
T│ │=2.T⎛⎞│⎛n│⎞

+ 4 . n ⎝ 2 ⎠ ⎝ 22 ⎠ 2
Now substitute this back into the last T (n) definition (last line from step 1):
n n
T (n) = 2 . ⌈ │2 . T ⌊ ⎛ │ ⎞ │ + ⎝ 22 ⎠ 4 . 2 ⌋ ⌉ │+4.n
n
= 22 . T ⎛ │ ⎞
│ + 2 . 4 . n ⎝ 22 ⎠
⎛n⎞
Step 3: figure out what T is:
n n
⎝ 22 ⎠ T ⎛ │ │⎞=2.T│⎛ │⎞
n
+4.
⎝ 22 ⎠ ⎝ 23 ⎠ 22
Now substitute this back into the last T (n) definition (last time from step 2):
T (n) = 22 . ⌈ ⎛ │2 . T
⎞⌉
│⌊
││
 A l l
+ Step The the definition same parameter 4: Do T(n) of analysis the

JN n
the = = 23 . T ⎛│ │⎞

TU
⎝ 23 ⎠ + ith substitution. 2i . T │ ⎛ to the as recurrence
World
n
│ │
n
+4. │ ⎝ 23 ⎠
22 │⌋
2.4.n
n i
3.4.n ⎞│+i.4.n⎝2 ⎠
recursive call in the last line will equal 1 when i = log2 n (use the previous example). In which case T (1) =
1 by the original
relation.
Now we can substitute this back into a our general Step 4 equation:
43
T(n) = 2i . T │ ⎛ n │ ⎞

+i.4.n
⎝ 2i ⎠
= 2log2 n . T ⎛ n ⎞ │ + log n . 4 . n

⎝ 2log2 n ⎠ 2
= n . T(1) + 4 . n . log2 n
= n + 4 . n . log2 n
This implies that T (n) is O(n log n).
Example 2.13.3. Write a recurrence relation and solve the recurrence relation for
the following fragment of program:
Write T(n) in terms of T for fractions of n, for example, T(n/k) or T(n - k).
int findmax (int a[ ], int start, int end) {
int mid, max1, max2;
if (start == end) // easy case (2 steps)
return (a [start]);
mid = (start + end) / 2; // pre-processing (3) max1 = findmax (a, start, mid); // recursive call: T(n/2) + 1
max2 = findmax (a, mid+1, end); // recursive call: T(n/2) + 1 if (max1 >= max2) //post-processing (2)
return (max1); else
}
│

A l l
Solution: The Solving Assume Step Recurrence 1: T the Substituting n

J N
(n) > Recurrence 1 = return then 2 Relation 2, T(n/2) T(n) n (max2); =

T U
Relation = is: n/2 + 2 8, T , n/4 (n/2) by . iteration . if n = 1 if n > 1

method:
 World
+ 8 . . for n in the relation:

Since, T (n/2) = 2 T ((n/2) /2) + 8, and T (n/4) = 2T((n/4) /2) + 8,

T (n) = 2 [ T(n/2) ] + 8 (1)
44
T (n) = 2 [ 2 T(n/4) + 8 ] + 8 (2)
T (n) = 2 [ 2 [ 2 T(n/8) + 8] + 8 ] + 8 (3)
= 2x2x2 T (n/2x2x2) + 2x2x8 + 2x8 + 8,
Step 2: Do the ith substitution:
T (n) = 2i T (n/2i) + 8 (2i-1 + . . . + 22 + 2 + 1)
k
= 2i T (n/2i) i - 1 + 8 ( ∑ k = 0 2 )
= 2i T (n/2i) + 8 (2i - 1) (the formula for geometric series)
Step 3: Define i in terms of n:
If n = 2 i, then i = log n so, T (n/2 i) = T(1) = 2
T(n) = 2i T (n/2i) + 8 (2i - 1) = n T (1) + 8 (n-1) = 2n + 8n - 8 = 10n - 8
Step 4: Representing in big-O function:
T (n) = 10n - 8 is O(n)
Example 2.13.4. If K is a non negative constant, then prove that the recurrence
T(n) = ⎧k

│
All n
JN 3 .T
⎛n⎞ T⎛⎝│ 2⎝│⎠ ⎩│
 T │ │
U
⎝2 ⎠ ⎞ = Substituting equation ,n=1

2⎠
for n in equation (1), we get
n
3T │ ⎛ ⎞
n
│+k ⎝4⎠2

(2) for T ⎛ ⎝ │ n 2 │ ⎞ ⎠ in equation (1)

Wo rld +k.n,n>1
(2)

has the following solution (for n a power of 2)

T(n) = 3 k . nlog2 3 - 2k .n
│ n n
Solution: Assuming n > 1, we have T (n) = 3T ⎛ ⎞ │ + K . n (1) Substituting n = 2
⎨
45
n
⎛ T (n) = 3 │3T │ ⎝ │ ⎛ n │ ⎞ + k n ⎞ │ ⎠ │ + k . n T(n) = 32 T │ ⎛ ⎞
3
│+ k . n + k . n (3) ⎝ 4 ⎠ 2
n for n equation (1)
Substituting n = 4
n n
T⎛│ │ ⎞ = 3T ⎛ │ ⎞
 n
│+k (4) ⎝ 4 ⎠ ⎝ 8 ⎠ 4
n in equation (3)
Substituting equation (4) for T = 4

T(n) = 32 ⎛ │3 │ ⎝ T ⎛ │ n │ ⎞ + k n │ ⎞ │ ⎠
n
+ 3 k . n + k . n 33 T │ ⎛ │ ⎞
9 3
+ k . n + k .n + k . n (5) ⎝ 8 ⎠ 4 2
Continuing in this manner and substituting n = 2i, we obtain
n 3 i-1
T(n) = 3i T ⎛ │ │⎞+
9kn 3
k .n + . . . . . . . + + k . n + k .n ⎝ 2i ⎠ 2i - 1 4 2

││⎛3│
⎞i ⎞
│
⎛⎞
⎝4⎠2

⎝8⎠4

⎛
n
2

l l J
A As n T (n) = 3i T = 2T(n) i , then = i 3i ⎝ T │ 2│ i ⎛ ⎝
n
⎠
n
 N 3⎝ 2 ⎠ 32
T │
2 │i│⎞+││ ⎝+ ⎠│ 2 2. -│ -1 1 ⎛ ⎝ │││⎠ │ ⎠ │ ⎞ i=

U 3
k . n k . n - 2 k n log2n and by definition as T (1) = 3i k + 2. i. kn-2kn

= 3i
(3k) - 2 k n
= 3 k . 3log 2n - 2 k n
= 3 k nlog2 3 - 2 k n

Wo
n-1r
n
-1 as ∑ r i=
i=0r -1

(6) = k

rld
Example 2.13.5. Towers of Hanoi
46
The Towers of Hanoi is a game played with a set of donut shaped disks stacked on one of three posts. The
disks are graduated in size with the largest on the bottom. The objective of the game is to transfer all the
disks from post B to post A moving one disk at a time without placing a larger disk on top of a smaller one.
What is the minimum number of moves required when there are n disks?
In order to visualize the most efficient procedure for winning the game consider the following:
1. Move the first n-1 disks, as per the rules in the most efficient manner
possible, from post B to post C.
2. Move the remaining, largest disk from post B to post A.
3. Move the n - 1 disks, as per the rules in the most efficient manner possible,
from post C to post A.
Let mn be the number of moves required to transfer n disks as per the rules in the most efficient manner
possible from one post to another. Step 1 requires moving n - 1 disks or mn-1 moves. Step 2 requires 1
move. Step 3 requires m n-1 moves again. We have then,
Mn = mn-1 + 1 + mn-1, for n > 2 = 2mn-1 + 1
Because only one move is required to win the game with only one disk, the initial condition for this sequence
is m1 = 1. Now we can determine the number of moves required to win the game, in the most efficient
manner possible, for any number of disks.
m1 = 1
m2 = 2(1) + 1 = 3
m3 = 2(3) + 1 = 7
m4 = 2(7) + 1 = 15
m5 = 2(15) + 1 = 31
m6 = 2(31) + 1 = 63 m7 = 2(63) + 1 = 127
Unfortunately, the recursive method of determining the number of moves required is exhausting if we
wanted to solve say a game with 69 disks. We have to calculate each previous term before we can
determine the next.
Lets look for a pattern that may be useful in determining an explicit formula for m n. In looking for patterns it
is often useful to forego simplification of terms. m n = 2mn- 1 + 1, m1 = 1
m1 = 1
 m2 = 2(1) + 1 = 2+1 m3 = 2(2+1) + 1 = 22+2+1 m4 = 2(22+2+1) + 1 = 23+22+2+1 m5 = 2(23+22+2+1) + 1 =

All JNTU
24+23+22+2+1

World
47
m6 = 2(24+23+22+2+1) + 1 = 25+24+23+22+2+1
m7 = 2(25+24+23+22+2+1) + 1 = 26+25+24+23+22+2+1
So we guess an explicit formula:
Mk = 2k-1 + 2k-2 + . . . . . . + 22 + 2 + 1
By sum of a Geometric Sequence, for any real number r except 1, and any integer n > 0.

n ∑r ir n+1
- 1 = - i=0
r1

our formula is then

k 2n
n-1 mk = ∑ 2 -1 = = 2n -1
k=02 -1
This is nothing but O (2n)
Thus providing an explicit formula for the number of steps required to win the Towers of Hanoi Game.
Example 2.13.6 Solve the recurrence relation T (n) = 5 T (n/5) + n/log n
Using iteration method,
T (n) = 25 T (n/25) + n/log (n/5) + n/log n
=125 T (n/125) + n/ log (n/25) + n/log (n/5) + n/log n
= 5k T (n/5k) + n/log (n/5k-1) + n/log (n/5k-2) . . . . + n/log n
When n = 5k
= n * T(1) + n/log 5 + n/log 25 + n/log 125 . . . . . + n/log n
= c n + n (1/log 5 + 1/log 25 + 1/log 125 + . . . . . 1/log n)
= c n + n log5 2 (1/1 + 1/2 + 1/3 + . .. . 1/k)
= c n + log5 2 n log (k)
= c n + log5 2 n log (log n)
= θ (n log (log n))
Example 2.13.7. Solve the recurrence relation T (n) = 3 T (n/3 + 5) + n/2 Use the substitution method,

All
guess that the solution is T (n) = O (n log n)

JNTU
 World
48
We need to prove T (n) < = c n log n for some c and all n > no.
Substitute c (n/3 + 5) log (n/3 + 5) for T (n/3 + 5) in the recurrence
T (n) <= 3 * c (n/3 + 5) * (log (n/3 + 5)) + n/2
If we take n0 as 15, then for all n > n0, n/3+5 <= 2n/3, so
T (n) <= (c n + 15 c) * (log n/3 + log 2) + n/2
<= c n log n – c n log 3 + 15 c log n – 15 c log 3 + c n + 15 c + n/2
<= c n log n - (c n (log 3-1) + n/2 – 15 c log n - 15 c (log 3 - 1)
<= c n log n, by choosing an appropriately large constant c
Which implies T (n) = O (n log n)
Similarly, by guessing T (n) >= c1 n log n for some c1 and all n > n0 and substituting in the recurrence, we
get.
T (n) >= c1 n log n - c1 n log 3 + 15 c1 log n – 15 c1 log3 + c1 n + 15 c1 + n/2
>= c1 n log n + n/2 + c1 n (1 - log 3) + 15 c1 + 15 c1 (log n – log 3)
>= c1 n log n + n (0.5 + c1 (1 - log 3) + 15 c1 + 15 c1 (log n - log 3)
by choosing an appropriately small constant c1 (say c1 = 0.01) and for all n > 3
T (n) >= c1 n log n
Which implies T (n) = Ω (n log n)
Thus, T (n) = θ (n log n)
Example 2.13.8. Solve the recurrence relation T (n) = 2 T (n/2) + n/log n.
Using iteration method,
T (n) = 4 T (n/4) + n/log (n/2) + n/log n
= 8 T (n/8) + n/log (n/4) + n/log (n/2) + n/log n
= 2k T (n/2k) + n/log (n/2k-1) + n/log (n/2k-2) . . . . . + n/log n
When n = 2k
= n * T(1) + n/log 2 + n/log 4 + n/log 8 . . . . . + n/log n
= c n + n (1/log 2 + 1/log 4 + 1/log 8 + . . . . 1/log n)
= c n + n (1/1 + 1/2 + 1/3 + . . . . . . 1/k)
 All
= c n + n log (k) = c n + n log (log n)

JNTU

World
49
= θ (n log (log n))
Example 2.13.9: Solve the recurrence relation:
T (n) = T (n/2) + T (n/4) + T (n/8) + n
Use the substitution method, guess that the solution is T (n) = O (n log n).
We need to prove T (n) <= c n log n for some c and all n > no.
Substituting the guess in the given recurrence, we get
T (n) <= c (n/2) log (n/2) + c (n/4) log (n/4) + c (n/8) log (n/8) + n
<= c (n/2) (log n - 1) + c (n/4) (log n - 2) + c (n/8) (log n - 3) + n
<= c n log n (1/2 + 1/4 + 1/8) – c n/2 – c n/2 – 3 c n/8 + n
<= c n log n (7/8) - (11 c n – 8 n) / 8
<= c n log n (7/8) (if c = 1)
<= c n log n
From the recurrence equation, it can be inferred that T (n) > = n. So, T (n) = Ω (n)

A l l
Example Let According It f (n) can us for be consider n c these log

=
J c n
N
written nto 3 3 28 2.13.10: the from : law as: n log
3-log the of h a b Solve (n) ⇒

28 3 indices: table = n= f the log c (n) O can n(nrecurrence b r = cn3 nlog 3

a
, we ay
r)
where be taken T(n) = 3 =
3
TU
relation: T (n) = 28 T (n/3) + cn3

a2

can write ax-y

where r = 3 - log 28 < 0

r<0
O (1)

World
log 3 │⎠ ⎞ x

nlog28
[T(1) + h(n)]
nlog28
[T (1) + 0(1)] = θ │⎛ n ⎝
28

Example 2.13.12: Solve the recurrence relation T (n) = T ( √n) + c. n > 4

50
T (n) = T (n1/2)+ C
T (n)1/2 = T (n1/2)1/2 + C
T (n1/2) = T (n1/4)+ C + C
T (n) = T (n1/4)+ 2c T (n) = T (n1/2)+ 3c
= T (n1/2i)+ i c
= T (n1 / n ) + C log n T ( n ) + C log
n
2n = θ (log n)
Example 2.13.13: Solve the recurrence relation
⎧1 │ n < 4 + logn n > 4
T(n) = ⎨ │⎩ 2 T ( n)

[
T (n) = 2 2 T (n1 / 2 )1 / 2 + log ] + logn
= 4 T (n1/4) + 2 log n1/2 + log n
=4⌈
T (n1/4) = 2 T (n1/2)1/4 + log n1/4 T (n) (
2T n1 / 2 )1 / 4 +
⌊

ll J
n A 2T = T(n) (n2i 1/8T (n) = = = 1/ = [
8 22= 16 i 2 ) i 2T + 8
T
T2

│ ⎛
N
T T i-1 ⎝ (n (n(nn(n logn 1 1/2 1 1/61/8/ / 2)16 ) i ) 1/8 1/
⎠ │+
⎞
) 2+ + i-1 + + 8 + 4 k ∑ lognlog
 T U W i-2 1 2 / 4 log ⌉ +2
log n =1 2log nn1/8 logn1/4 + ⌋

o
logn1 / 4
+ logn n1/2 + log n 1/8 ] + 4 logn 1/4+ 2 logn1 / 2 + logn

n1/8 + 4 log n1/4 + 2 log n1/2 + log n

i i
logn 1/ 2i-2 + 2 -3 logn 1/ 2i-3 + 2 -4 logn1/ 2i-4

rld
2i - k logn1 / 2 i -k
 (
= n T n1 / n ) + ∑ n 2k = θ (log n)
lognk / n
51

All JNTU
52

World
Chapter 3
Divide and Conquer
General Method
Divide and conquer is a design strategy which is well known to breaking down efficiency barriers. When the
method applies, it often leads to a large improvement in time complexity. For example, from O (n 2) to O (n
log n) to sort the elements.
Divide and conquer strategy is as follows: divide the problem instance into two or more smaller instances
of the same problem, solve the smaller instances recursively, and assemble the solutions to form a solution
of the original instance. The recursion stops when an instance is reached which is too small to divide. When
dividing the instance, one can either use whatever division comes most easily to hand or invest time in
making the division carefully so that the assembly is simplified.
Divide and conquer algorithm consists of two parts:
Divide : Divide the problem into a number of sub problems. The sub problems
are solved recursively. Conquer : The solution to the original problem is then formed from the solutions
to the sub problems (patching together the answers).
Traditionally, routines in which the text contains at least two recursive calls are called divide and conquer
algorithms, while routines whose text contains only one recursive call are not. Divide–and–conquer is a
very powerful use of recursion.
Control Abstraction of Divide and Conquer
A control abstraction is a procedure whose flow of control is clear but whose primary operations are
specified by other procedures whose precise meanings are left undefined. The control abstraction for divide
and conquer technique is DANDC(P), where P is the problem to be solved.
DANDC (P) {
if SMALL (P) then return S (p); else {
divide p into smaller instances p1, p2, .... Pk, k ≥ 1; apply DANDC to each of these sub problems; return
(COMBINE (DANDC (p1) , DANDC (p2),...., DANDC (pk)); } } SMALL (P) is a Boolean valued function

which determines whether the input size is small enough so that the answer can be computed without
splitting. If this is so function ‘S’ is invoked otherwise, the problem ‘p’ into smaller sub problems. These

All
sub problems p1, p2, . . . , pk are solved by recursive application of DANDC.

JNTU
World
53
If the sizes of the two sub problems are approximately equal then the computing time of DANDC is:
⎧ g (n)
T (n) = ⎨ 2 T (n/2)+ f (n)
n small
⎩
otherwise
Where, T (n) is the time for DANDC on ‘n’ inputs
g (n) is the time to complete the answer directly for small f (n) is the time for Divide and Combine
Binary Search
If we have ‘n’ records which have been ordered by keys so that When we are given a element ‘x’, binary
search is used to find element from the list. In case ‘x’ is present, we have to determine that a[j] = x
(successful search). If ‘x’ is not in the list then j successful search).

A
In compare If at the a[mid] ‘x’ Since a[mid], reaching BINSRCH // // // { low } Binary x < all.
 l l
file with Algorithm a[mid] the determine Similarly, low while { array else of which a[mid]

and a ‘x’ (a, search the array trivial :=1 return since (low with n, a(1 then Algorithm follows mid if else un-

J
searched will ; x) (x if high : length, size we whether < j a[mid]. an :=|(low if n) < ‘x’ eliminate

N
a[mid] high) (x jump a a[mid]. of array must :=n is [mid]) > elements the roughly a do ‘x’ ; >

into + If be portion [mid]) of roughly worst is high)/2| x, x If then length in present, the then = we
 T
that halved in then a[mid] case high:=mid middle of use increasing half further ‘n’ portion a

U
low:= recursive complexity and file, can the often then of if be search then un-

searched of the – mid so, order, each the halved the 1; procedure file, set every + of file is desired 1

W
comparison j n Binary where only such only that ≥ un-successful 0, portion necessary

that of precedes about record we search finding is find x inputs from between x1 = log2n the to is has key
 o
a(j) < a set a[mid], the about in consideration. comparison corresponding value x 2 and times

r
been that a[mid], to middle < log2n zero ‘x’ ... if ‘j’ past found. before < there such

and

l
and key (un xn of of .

else return mid; } return 0;

and high are integer variables such that each time through the loop either ‘x’ is
d
found or low is increased by at least one or high is decreased by at least one. Thus we have two sequences
of integers approaching each other and eventually low will become greater than high causing termination
in a finite number of steps if ‘x’ is not present.
54
Example for Binary Search
Let us illustrate binary search on the following 9 elements:
Index 1 2 3 4 5 6 7 8 9 Elements -15 -6 0 7 9 23 54 82 101
The number of comparisons required for searching different elements is as follows:
1. Searching for x = 101
low 1
Number of comparisons = 4
2. Searching for x = 82

ll JNT U
A 1 1 mid 4

1 high 9
World
5 2 1 mid
5697898999
low 1
found
Number of comparisons = 3
3. Searching for x = 42
7 6 not found
Number of comparisons = 4
4. Searching for x = -14
2 1 not found Number of comparisons = 3
Continuing in this manner the number of element comparisons needed to find each of nine elements is:
Index 1 2 3 4 5 6 7 8 9 Elements -15 -6 0 7 9 23 54 82 101 Comparisons 3 2 3 4 1 3 2 3 4
No element requires more than 4 comparisons to be found. Summing the comparisons needed to find all
nine items and dividing by 9, yielding 25/9 or approximately 2.77 comparisons per successful search on the
average.
There are ten possible ways that an un-successful search may terminate depending upon the value of x.
high
9
mid
5697898
found
low 1
high 9
mid 5 6 9 7 6 6 6

low 1
high
9
55
If x < a[1], a[1] < x < a[2], a[2] < x < a[3], a[5] < x < a[6], a[6] < x < a[7] or a[7] < x < a[8] the algorithm
requires 3 element comparisons to determine that ‘x’ is not present. For all of the remaining possibilities
BINSRCH requires 4 element comparisons. Thus the average number of element comparisons for an
unsuccessful search is:
(3 + 3 + 3 + 4 + 4 + 3 + 3 + 3 + 4 + 4) / 10 = 34/10 = 3.4
The time complexity for a successful search is O(log n) and for an unsuccessful search is Θ(log n).
Successful searches un-successful searches Θ(1), Θ(log n), Θ(log n) Θ(log n) Best average worst
best, average and worst
Analysis for worst case
Let T (n) be the time complexity of Binary search
The algorithm sets mid to [n+1 / 2]
Therefore,
T(0) = 0
T(n) = 1 if x = a [mid]
= 1 + T([(n + 1) / 2] – 1) if x < a [mid]
= 1 + T(n – [(n + 1)/2]) if x > a [mid]
Let us restrict ‘n’ to values of the form n = 2K – 1, where ‘k’ is a non-negative integer. The array always
breaks symmetrically into two equal pieces plus middle element.
2K – 1 - 1 2K – 1 - 1
2K 1
⌈ n + 1⌉ =⌈ - 1 + 1⌉
Algebraically this is │ │ │ 2K │ = 2K – 1 for K > 1

A l l JN
Giving, In the worst T(0) T(2w(0) k
⌊ 2=–

1) = = = case the test = 0

TU 2
⌋ │⌊ │⌋
0
1 1 + T(2K - 1 – 1) 1 + T(2k - 1 – 1) x = a[mid] always fails, so

World
if x = a [mid]
if x < a [mid]
if x > a [mid]
w(2k – 1) = 1 + w(2k - 1 – 1)
56
This is now solved by repeated substitution:
w(2k – 1) = 1 + w(2k - 1 – 1)
= 1 + [1 + w(2k - 2 –1)]
= 1 + [1 + [1 + w(2k - 3 –1)]]
=........
=........
= i + w(2k - i – 1)
For i < k, letting i = k gives w(2k –1) = K + w(0) = k
But as 2K – 1 = n, so K = log2(n + 1), so
w(n) = log2(n + 1) = O(log n)
for n = 2K–1, concludes this analysis of binary search.
Although it might seem that the restriction of values of ‘n’ of the form 2 K–1 weakens the result. In practice
this does not matter very much, w(n) is a monotonic increasing function of ‘n’, and hence the formula given
is a good approximation even when ‘n’ is not of the form 2K–1.
External and Internal path length:
The lines connecting nodes to their non-empty sub trees are called edges. A non- empty binary tree with n
nodes has n–1 edges. The size of the tree is the number of nodes it contains.
When drawing binary trees, it is often convenient to represent the empty sub trees explicitly, so that they
can be seen. For example:
The tree given above in which the empty sub trees appear as square nodes is as follows:
The square nodes are called as external nodes E(T). The square node version is sometimes called an
extended binary tree. The round nodes are called internal nodes I(T). A binary tree with n internal nodes
has n+1 external nodes.
The height h(x) of node ‘x’ is the number of edges on the longest path leading down from ‘x’ in the
extended tree. For example, the following tree has heights written inside its nodes:
 ll JN
A ba

T
c

U World
57
3
12
0010
00
The depth d(x) of node ‘x’ is the number of edges on path from the number of internal nodes on this path,
excluding ‘x’ itself. following tree has depths written inside its nodes:
0
11
2222

l
33 The internal path length I(T) is the sum of the depths of the internal I(T)
Al
= ∑

JNTU W
x ∈ I(T ) 10 11 12 the root to

o r ld
For example, nodes of ‘x’. It the is
‘T’:
d(x)
The external path length E(T) is the sum of the depths of the external nodes:

E(T) = ∑
x ∈ E(T )
13 14 15 16 d(x) For example, the tree above has I(T) = 4 and E(T) = 12.

A binary tree T with ‘n’ internal nodes, will have I(T) + 2n = E(T) external nodes.
A binary tree corresponding to binary search when n = 16 is
8
4 12
2 6 10 14

1 3 5 7 9 11 13 15
16
0123456789
Represents internal nodes which lead for successful search
External square nodes, which lead for unsuccessful search.
Let CN be the average number of comparisons in a successful search.
C 'N be the average number of comparison in an un successful search.
58
Then we have,
internal pathlengthof
CN = 1 + N
tree

External path length

C' N = N+ 1
of tree

⎛ 1⎞
CN = │1 ⎝ + N⎠ │ C'N - 1
External path length is always 2N more than the internal path length.
Merge Sort
Merge sort algorithm is a classic example of divide and conquer. To sort an array, recursively, sort its left
and right halves separately and then merge them. The time complexity of merge mort in the best case,
worst case and average case is O(n log n) and the number of comparisons used is nearly optimal.
This strategy is so simple, and so efficient but the problem here is that there seems to be no easy way to
merge two adjacent sorted arrays together in place (The result must be build up in a separate array).
The fundamental operation in this algorithm is merging two sorted lists. Because the lists are sorted, this
can be done in one pass through the input, if the output is put in a third list.
The basic merging algorithm takes two input arrays ‘a’ and ’b’, an output array ‘c’, and three counters, a
ptr, b ptr and c ptr, which are initially set to the beginning of their respective arrays. The smaller of a[a ptr]
and b[b ptr] is copied to the next entry in ‘c’, and the appropriate counters are advanced. When either
input list is exhausted, the remainder of the other list is copied to ‘c’.
A To containing

between and ptr

l J
11h l illustrate then 13 2 1 2 24 1, and 3 and how 13, 26 2. 13 4 24, 1

merge are is 26 copied compared. and process to ‘b’ ‘c’. containing 2 works. Increment is added For 2, to
example, a 15, ‘c’. ptr 1 27, Increment and 38. c let ptr.
First us b consider a ptr comparison and c the ptr.

NTU
array is done ‘a’
 World
5 6 7 8 2 15 27 28 j ptr

2 3 4 5 6 7 8 1 2 i ptr 1 2 3 4 5 6 7 8 1 i ptr
1 2 3 4 1 13 24 26
h ptr

5 6 7 8 2 15 27 28 j ptr
59
then 13 and 15 are compared. 13 is added to ‘c’. Increment a ptr and c ptr.
24 and 15 are compared. 15 is added to ‘c’. Increment b ptr and c ptr.
24 and 27 are compared. 24 is added to ‘c’. Increment a ptr and c ptr.
26 and 27 are compared. 26 is added to ‘c’. Increment a ptr and c ptr.
As one of the lists is exhausted. The remainder of the b array is then copied to ‘c’.
h ptr

i ptr
Algorithm
Algorithm MERGESORT (low, high) // a (low : high) is a global array to be sorted. {
if (low < high) {
mid := │(low + high)/2│ //finds where to split the set MERGESORT(low, mid) //sort one subset
123
MERGESORT(mid+1, high) //sort the other subset MERGE(low, mid, high) // combine the results } }
4
1 13 24 26

h ptr
5 6 7 8 2 15 27 28
j ptr

1 2 3 4 5 6 7 8 1 2 13 i ptr
1 2 3 4 1 13 24 26
 ll
h ptr 1 A 5 6 7 8 2 15 27 28

j ptr
1 2 3 4 1 13 24 26

JNTU
h ptr 1 5 6 7 8 2 15 27 28

j ptr

1 2 3 4 1 2 13 15 i ptr

W or
56 7 8
ld
2 3 4 5 6 7 8 2 13 15 24 26 27 28 1 2 3 4 5 6 7 8 1 2 13 15 24 i ptr

1 2 3 4 1 13 24 26 h ptr
5 6 7 8 2 15 27 28
j ptr

1 2 3 4 5 6 7 8 1 2 13 15 24 26 i ptr
1 2 3 4 1 13 24 26
5 6 7 8 2 15 27 28
j ptr
60
Algorithm MERGE (low, mid, high) // a (low : high) is a global array containing two sorted subsets // in a
(low : mid) and in a (mid + 1 : high). // The objective is to merge these sorted sets into single sorted // set
residing in a (low : high). An auxiliary array B is used. {
h :=low; i := low; j:= mid + 1; while ((h < mid) and (J < high)) do {
if (a[h] < a[j]) then { b[i] := a[h]; h := h + 1; } else { b[i] :=a[j]; j := j + 1; } i := i + 1; } if (h > mid) then

for k := j to high do { b[i] := a[k]; i := } else

l l
for k := h to mid do { b[i] := a[K]; i A } For merge Example example sort for
 J N
k algorithm: := let } a[k] low us 7, := select to 2, b[k]; high 9, the 4 do | following

T
3, 8, 6, i + 1;

U W
:= i + l; 8 entries 7, 2, 9, 4, 3, 8, 6, 1 1 → 1, 2, 3, 4, 6, 7, 8, 9
orld
to illustrate
7, 2 | 9, 4 → 2, 4, 7, 9 3, 8 | 6, 1 → 1, 3, 6, 8
7 | 2 → 2, 7 9 | 4 → 4, 9 3 | 8 → 3, 8 6 | 1 → 1, 6
7→72→29→94→43→38→86→61→1
61
Tree Calls of MERGESORT(1, 8)
The following figure represents the sequence of recursive calls that are produced by MERGESORT when
it is applied to 8 elements. The values in each node are the values of the parameters low and high.
1, Tree Calls of MERGE()
The tree representation of the calls to procedure MERGE by MERGESORT is as follows:
Analysis of Merge Sort
We will assume that ‘n’ is a power of 2, so that we always split into even halves, so we solve for the case
n = 2k.
For n = 1, the time to merge sort is constant, which we will be denote by 1. Otherwise, the time to merge
sort ‘n’ numbers is equal to the time to do two recursive merge sorts of size n/2, plus the time to merge,
which is linear. The equation says this exactly:
T(1) = 1 T(n) = 2 T(n/2) + n
This is a standard recurrence relation, which can be solved several ways. We will solve by substituting
recurrence relation continually on the right–hand side.
We have, T(n) = 2T(n/2) + n
1, 8
4 5, 8
 ll J N
1, 2 3, 4 1, 1 2, 2 3, 3 4, 4 A 1, 1, 2 1, 2, 4

TU
3, 3, 4 5, 6 5, 5 5, 7, 8

Wo
6, 6 7, 7 8, 8

5, 6 7, 7, 8
5, 6, 8 1, 4, 8
rld
62
Since we can substitute n/2 into this main equation
2 T(n/2)
We have,
==
2 (2 (T(n/4)) + n/2) 4 T(n/4) + n

A
Again, So Continuing As Representing We is Although memory extra back, The The of

l
The follows for Strassen’s not divide n i we have := Best matrix usual = throughout a memory have,
 l
by 1 T(n/2) T(n) 4T T(n/4) T(n) T(n) T(n) 2T : power kto and (n) , sorts. and assumed merge

substituting (n/4) way K n in multiplication = = = do conquer this worst this = = Matrix O(n 2of log2n,

J
and to log The n n sort’s 2. the in manner, 2T(1) log multiply n log that case The the O main = =

N ⎛
= = = = = T substituting algorithm, Multiplication: n technique n/4 notation: │ ⎝ n) +

running n + 22 answer │ additional time k k of n = n into │ problem │ ⎠ ⎞ we two log algorithm 2 4 4 8 2 8

22k + kcomplexity T(n/4) T(n/4) (2T(n/8)) T(n/8) T(n/8) T(n/8) obtain: . T(n/2the n n The log turns (1969).
 T
time has this x main 2 work n is kanalysis n the ) + + + + + is out in matrices due . + that n 2n

U
n 3n n equation, + the O(n effect K. spent of to to n/4 n Merge merging be above

can log Strassens of almost A copying n), be slowing and we sort equation refined it two see B, identical.

W
is is is to yielding O(n down that the sorted hardly the to most log handle the
 o
temporary n). ever lists result dramatic sort requires cases used considerably. matrix array

r
for when example ‘C’ linear main

and ‘n’

ld
as

for j :=1 to n do
c[i, j] := 0; for K: = 1 to n do
c[i, j] := c[i, j] + a[i, k] * b[k, j];
63
This algorithm requires n3 scalar multiplication’s (i.e. multiplication of single numbers) and n3 scalar
additions. So we naturally cannot improve upon.
We apply divide and conquer to this problem. For example let us considers three multiplication like this:
A A B B
⎛A │ 11 A 12 ⎞ │ ⎛B │ 11 B 12 ⎞ │ = ⎛C 11 │ ⎝│ 21 22 ⎠ ││⎝ 21 22 ⎠ │
C C
│⎝ 21 C 12 22 ⎞ ││⎠
Then cij can be found by the usual matrix multiplication algorithm,
C11 = A11 . B11 + A12 . B21

C12 = A11 . B12 + A12 . B22

C21 = A21 . B11 + A22 . B21

C22 = A21 . B12 + A22 . B22

This leads to a divide–and–conquer algorithm, multiplication by partitioning the matrices into (n/2)x(n/2)
matrix multiplications and four (n/2)x(n/2) T(1) = 1
T(n) = 8 T(n/2)

A l l
Which Strassens seven additions leads P Q R S T U V C11 C12 C21 C22

(n/2) = = and = = = = = = = = = insight (A12 to R (A11 P Q P (A21 subtractions: A11 x (A11 (A21 A22 + T + +

J N
+ – (n/2) (n) T S R + + was (B12 (B21 + – S A22) – - A22) A22) A11) A12) = Q T – to

- O matrix + + (B21 B11) B22) (B11 (B11 (nB11 B22 find V

 T U
U. 3), + + + an where multiplications B12) B22) B22) alternative n is the

method power and of quarters 2.

W o
which matrix performs and additions. performing nxn for calculating

rld
eighteen (n/2)

matrix eight
the Cij, requiring x (n/2) matrix
This method is used recursively to perform the seven (n/2) x (n/2) matrix multiplications, then the recurrence
equation for the number of scalar multiplications performed is:
64
T(1) = 1 T(n) = 7 T(n/2)
Solving this for the case of n = 2k is easy:
T(2k) = =
7 T(2k–1)
72 T(2k-2)
==
------------
= 7i T(2k–i)
Put i = k
= 7k T(1)
= 7k
That is, T(n) = 7 log2 n = n log2 7
= O(n log2 7 ) = O(n 2.81
)

A
that is So, standard does The exchanges adjacent place In into chosen greater The in to

l
rearranged, The O(n Hoare this essence, each improves concluding chosen function main two not
 l
log to way Quick than value work his n). pay pairs. groups. algorithm. of reason with between

J
value the devised the the or partition() the off Sort its taken In equal that original respect until

N
quick way O(ntwo for The is this known In a 2Strassen’s from the keys into ‘n’ to ) way

very first subsets. sort practice, behavior makes array to the revolves slowness its the in the it as group
efficient algorithm chosen proper takes a is the pivot, use set, sorted. When algorithm sequence
 T
the of the pivot contains of a of SIS value. and way position overhead relatively the

U
hundreds. Algorithms two partitions all element. algorithm of the very pointers w1, is

the implementing those in second asymptotically same long of the w2, subsets the Once like managing

W
elements with sorted ‘i’ . time partitioning original group and SIS . more efficient
 o r ld
than the

the many small matrices

is that all comparisons and . . , wn take place between for a key that is badly out of
sequence.
this idea in the early 1960’s an expected performance that
array by rearranging it less than some arbitrary contains those elements
the array has been rearranged is recursively applied have been partitioned and
‘j’ which are moved toward each other in the following fashion:
• Repeatedly increase the pointer ‘i’ until a[i] >= pivot.
• Repeatedly decrease the pointer ‘j’ until a[j] <= pivot.
65
• If j > i, interchange a[j] with a[i]
• Repeat the steps 1, 2 and 3 till the ‘i’ pointer crosses the ‘j’ pointer. If ‘i’ pointer crosses ‘j’ pointer, the
position for pivot is found and place pivot element in ‘j’ pointer position.
The program uses a recursive function quicksort(). The algorithm of quick sort function sorts all elements
in an array ‘a’ between positions ‘low’ and ‘high’.
• It terminates when the condition low >= high is satisfied. This condition will be satisfied only when the
array is completely sorted.
• Here we choose the first element as the ‘pivot’. So, pivot = x[low]. Now it calls the partition function to find
the proper position j of the element x[low] i.e. pivot. Then we will have two sub-arrays x[low], x[low+1], . . .
. . . . x[j-1] and x[j+1], x[j+2], . . .x[high].
• It calls itself recursively to sort the left sub-array x[low], x[low+1], . . . . . . . x[j-1] between positions low
and j-1 (where j is returned by the partition function).
• It calls itself recursively to sort the right sub-array x[j+1], x[j+2], . . . . . . . . . x[high] between positions j+1
and high.
Algorithm Algorithm
QUICKSORT(low, high)
/* sorts the elements a(low), . . . . . , a(high) which reside in the global array A(1 : n) into ascending order
a (n + 1) is considered to be defined and must be greater than all elements in a(1 : n); A(n + 1) = + ∝ */ {
if low < high then {
j := PARTITION(a, low, high+1);
// J is the position of the partitioning element QUICKSORT(low, j – 1); QUICKSORT(j + 1 , high); } }

Algorithm PARTITION(a, m, p) {
V ← a(m); i ← m; j ← p; // A (m) is the partition element do {
loop i := i + 1 until a(i) > v // i moves left to right loop j := j – 1 until a(j) < v // p moves right to left if (i < j)
then INTERCHANGE(a, i, j) } while (i > j); a[m] :=a[j]; a[j] :=V; // the partition element belongs at position P

ll JNTU
return j; } A
World
66
Algorithm INTERCHANGE(a, i, j) {
P:=a[i]; a[i] := a[j]; a[j] := p; }
Example
Select first element as the pivot element. Move ‘i’ pointer from left to right in search of an element larger
than pivot. Move the ‘j’ pointer from right to left in search of an element smaller than pivot. If such elements
are found, the elements are swapped. This process continues till the ‘i’ pointer crosses the ‘j’ pointer. If ‘i’
pointer crosses ‘j’ pointer, the position for pivot is found and interchange pivot and element at ‘j’ position.
Let us consider the following example with 13 elements to analyze quick sort:
1 2 3 4 5 6 7 8 9 10 11 12 13 Remarks
38 08 16 06 79 57 24 56 02 58 04 70 45
pivot i j swap i & j
04 79
i j swap i & j
02 57
swap pivot
j i (24 08 16 06 04 02) 38 (56 57 58 79 70 45)
swap pivot
& j pivot j, i
& j (02 08 16 06 04) 24 pivot,
i swap pivot
j
& j 02 (08 16 06 04)
pivot i j swap i & j
04 16
swap pivot
j i (06 04) 08 (16)
& j pivot,
i swap pivot
j (04) 06
& j 04 pivot, j, i

All
16 pivot, j, i (02 04 06 08 16 24) 38 (56 57 58 79 70 45)

JNTU
 World
67
pivot i j swap i & j
68 45 57
swap pivot
j i (45) 56 (58 79 70 57)
& j 45 pivot, j, i
(58 79 pivot

i 70 57 j i
(57) 58 (70 57 pivot, j, i

A
02 04 Like merge recurrence (and no We will take The plus time). Where, The which
 l l
Worst running pivot the is This T i cut (n) linear formula. gives off sort, = time 06 T

J
Analysis T for time (0) (i) the of quick small 08 = + We quick basic spent T T

N
(1) of will (n sort files). 16 sort quick Quick – = in do i is 1, – the 24 the is recursive, 1) as
 T U
sort Sort: equal partition + analysis in relation: 38 merge C to n the (45

W
and 45 for (The sort. running a hence quick 56 56 pivot its sort, time 57 57 selection -

= |S1| is the number of elements in S1.

Case Analysis
is the smallest element, all the time. Then i=0 insignificant, the recurrence is:

57)
o
(70 pivot, j 70 58 70 58 70 analysis requires assuming of the two takes and if we j 79 79) 79)
 i swap
r l
swap pivot i &j
swap & & pivot j j

swap pivot
&j
79 pivot, j, i 79)
79
solving a a random pivot
recursive calls only constant
(1)
ignore T(0)=1,

d
T (n) = T (n – 1) + C n n > 1 - (2)
Using equation – (1) repeatedly, thus
T (n – 1) = T (n – 2) + C (n – 1)
T (n – 2) = T (n – 3) + C (n – 2)
-- - - - - - -
T (2) = T (1) + C (2)
Adding up all these equations yields
T (n) = T (1) +
In the best two sub-files slight over answer.
T (n) = Divide both T(n) n Substitute T(n / 2) n / 2 Substitute T(n / 4)
 A l l
Continuing We T(2) T(n) 2 n n / add 4 -- --
=
all
=
ni ∑ =2i

= O (n2) - Best Case Analysis

case, the pivot is in the middle. To simply the are each exactly estimate, this is acceptable 2 T (n/2) + C n
sides by n

= T(n / 2) ==
J T(n T(n //
n/2 + C n/2 for ‘n’ in equation - - n/4 - - - - for - - -- -- n n ‘n’ / / 4 8

N 4) 8)
T
in equation + + C C half (5)
 U W
the because size of the we original are only - - - (6) - math, and

o r l
interested although we assume (3) in this a Big that gives – the oh a

(4)
(5)
(6)
(7)
in this manner, we obtain:
T(1)
1 + C - (8)
the equations from 4 to 8 and note that there are log n of them:
T(1)

+ C log n - (9)
1
d
Which yields, T (n) = C n log n + n = O(n log n) - (10)
This is exactly the same analysis as merge sort, hence we get the same answer.
69
Average Case Analysis
The number of comparisons for first call on partition: Assume left_to_right moves over k smaller element
and thus k comparisons. So when right_to_left crosses left_to_right it has made n-k+1 comparisons. So,

A
first call on partition makes n+1 comparisons. T(n) ----- nT(n) (n-1)T(n-1) Subtracting

nT(n) nT(n) T(n) The T(n)/(n+1) Using T(n)/(n+1) T(n-1)/n T(n-2)/(n-1) T(n-3)/(n-2) . . T(3)/4 T(2)/3 Adding
 l
T(n)/(n+1) = Cancelling T(n)/(n+1) + {Σ [2/(n+1) [T(n-1)/n 1<=nleft,nright<=n recurrence = = + –(n-

l
1)T(n-1) the = = T(n) comparisons 2 T(n-1)]/n both T(n)= n(n+1) 2n + method + the (n+1)T(n-

1)/n =2(n+1) + = + = =2n =2(n+1)[log = both = + sides: 2/n The (n-1)T(n-1) 2/(n+1) 2[1/2 [T(n-1)/n = =

J
(n+1)2[ (n-1)n common T(n-2)/(n-1) = = = = O(n relation + log + sides: average of 2 2/(n-1) =
 N
log (n+1) +1/3 for [T(0) subsititution: [ 2/(n+1) 2/n 2/(n-1) 2/(n-2) . . + 2/4 2/3 [ ∑+ n(n+1)

terms:
n) first obtained 2 + [T(nleft) T(n-1)/n (n+1) + +1/4+--------------+1/n+1/(n+1)] 2≤k [T(0) + + + +T(1) T(n-
2)/(n-1) case + + T(n-2)/(n-1) + T(2)/3 call T(1)/2 2T(n-1) ≤n+1 log + + ---------- ] + ------------- – T(n-3)/(n-2)

T
T(n-4)/(n-3) +T(1) – complexity T(n-1)/n on 1 (n+1)-2n + is: (n-1)n] + log / k T(2) quicksort

U
T(1)/2 T(nright)]}n 2] = + +2/4 + 2n + T(2) + ------------- = + ----- log + of 2T(n-1) +
 2/2 T(2)/3 (n+1)T(n-1) + quicksort 2 2/3] + ----- –log + = T(n-2) T(0) (n+1) + = + 2 T(1)/2] + 2n T(n-2)] is

W o
T(2)/3 + + + T(n-1)] 2T(n-1) 2 + [T(0) \ + T(0) T(1)/2] +T(1) + + T(2)

r
+

ld
3.8. Straight insertion sort:
Straight insertion sort is used to create a sorted list (initially list is empty) and at each iteration the top
number on the sorted list is removed and put into its proper
70
place in the sorted list. This is done by moving along the sorted list, from the smallest to the largest number,
until the correct place for the new number is located i.e. until all sorted numbers with smaller values comes
before it and all those with larger values comes after it. For example, let us consider the following 8 elements
for sorting:
Index 1 2 3 4 5 6 7 8 Elements 27 412 71 81 59 14 273 87
Solution:
Iteration 0: unsorted 412 71 81 59 14 273 87
Sorted 27
Iteration 1: unsorted 412 71 81 59 14 273 87
Sorted 27 412
Iteration 2: unsorted 71 81 59 14 273 87
Sorted 27 71 412
Iteration 3: unsorted 81 39 14 273 87
Sorted 27 71 81 412
Iteration 4: unsorted 59 14 273 87
Sorted 274 59 71 81 412
Iteration 5: unsorted 14 273 87
Sorted 14 27 59 71 81 412
Iteration 6: unsorted 273 87
Sorted 14 27 59 71 81 273 412
Iteration 7: unsorted 87

ll
Sorted 14 27 59 71 81 87 273 412 A
 JNTU

World
71

Chapter
4
Greedy Method
GENERAL METHOD
Greedy is the most straight forward design technique. Most of the problems have n inputs and require us
to obtain a subset that satisfies some constraints. Any subset that satisfies these constraints is called a
feasible solution. We need to find a feasible solution that either maximizes or minimizes the objective
function. A feasible solution that does this is called an optimal solution.
The greedy method is a simple strategy of progressively building up a solution, one element at a time, by
choosing the best possible element at each stage. At each stage, a decision is made regarding whether or
not a particular input is in an optimal solution. This is done by considering the inputs in an order determined
by some selection procedure. If the inclusion of the next input, into the partially constructed optimal solution
will result in an infeasible solution then this input is not added to the partial solution. The selection procedure
itself is based on some optimization measure. Several optimization measures are plausible for a given
problem. Most of them, however, will result in algorithms that generate sub-optimal solutions. This version
of greedy technique is called subset paradigm. Some problems like Knapsack, Job sequencing with
deadlines and minimum cost spanning trees are based on subset paradigm.
For the problems that make decisions by considering the inputs in some order, each decision is made using
an optimization criterion that can be computed using decisions already made. This version of greedy
method is ordering paradigm. Some problems like optimal storage on tapes, optimal merge patterns and
single source shortest path are based on ordering paradigm.
CONTROL ABSTRACTION
Algorithm Greedy (a, n) // a(1 : n) contains the ‘n’ inputs {
solution := φ; // initialize the solution to empty for i:=1 to n do {
x := select (a); if feasible (solution, x) then
solution := Union (Solution, x); } return solution; } Procedure Greedy describes the essential way that a

greedy based algorithm will look, once a particular problem is chosen and the functions select, feasible
and union are properly implemented.
The function select selects an input from ‘a’, removes it and assigns its value to ‘x’. Feasible is a Boolean
valued function, which determines if ‘x’ can be included into the solution vector. The function Union

All
combines ‘x’ with solution and updates the objective function.

JNTU
 World
72
KNAPSACK PROBLEM
Let us apply the greedy method to solve the knapsack problem. We are given ‘n’ objects and a knapsack.
The object ‘i’ has a weight wi and the knapsack has a capacity ‘m’. If a fraction x i, 0 < xi < 1 of object i is
placed into the knapsack then a profit of pi xi is earned. The objective is to fill the knapsack that maximizes
the total profit earned.
Since the knapsack capacity is ‘m’, we require the total weight of all chosen objects to be at most ‘m’. The
problem is stated as:
maximize
subject to

∑ n pi xi i = 1 ∑ n ai xi < M where, 0 < xi < 1 and 1 < i = 1

The profits and weights are positive numbers.

A If algorithm Algorithm // // // { } Running The disregard Consider (25, the Algorithm P[1
 l l
Objects m Example: objects 24, is objects : for U for { } if the n] 15) (i := the time: the

given i i ordered and < knapsack GreedyKnapsack := := are m; and following n) time are if x 1 1 w[1 to [i]

J
below then (w(i) to to (w1, already be to so := n n : size n] do x[i] do initially sorted that w2,

N
1.0; > obtains instance contain x[i] U) := and w3) been p[i] U then into U := sort := = x[1:
 T
solutions (m, / / sorted of the (18, U w[i] w[i]; break; 0.0 non-decreasing the the – n] n)

profits w[i]; > 15, is objects, knapsack into p[i the corresponding 10). + non-increasing and

U
solution 1] the weights / problem: w[i order // algorithm vector. initialize + to 1]. of

W
respectively this pi n order = / requires x wi strategy. 3, ratio. of m i < n
 orld
p[i] / w[i] then the
of
But if we only O(n) time.
= 20, (p1, p2, p3) =
73
1. First, we try to fill the knapsack by selecting the objects in some order:
x1 x2 x3 ∑ wi xi ∑ pi xi
1/2 1/3 1/4 18 x 1/2 + 15 x 1/3 + 10 x 1/4
= 16.5
25 x 1/2 + 24 x 1/3 + 15 x 1/4 = 24.25
2. Select the object with the maximum profit first (p = 25). So, x 1 = 1 and profit earned is 25. Now, only 2
units of space is left, select the object with next largest profit (p = 24). So, x2 = 2/15
x1 x2 x3 ∑ 1 2/15 0 18 x 1 + 3. Considering the objects in the x1 x2 x3 ∑ 0 2/3 1 15 x 2/3 4. Considered the

objects in the p1/w1 25/18 1.4 Sort object units profit x1 0 This A 4.4. There program only We
 l l
initially solution the of earned x2 1 are with OPTIMAL space objects ‘i’ ‘n’ the is 1/2 x3 is

J
is of programs is the maximum 7.5. in left, length STORAGE order optimal select 15 li,

N
that of x pi 1 1 solution. the the / ≤ ON + are xi ∑ i if the sum of the lengths of shall assume

that whenever positioned at the front. wi xi 15 x 2/15 order wi xi + 10 x 1 order of p2/w2 non-increasing
 T
object ratio, 24/15 1.6 so, with wi xi 10 x 1/2 TAPES to be ≤ n. All the programs a program If

U
the programs of = the = non-decreasing 20 20 ratio pi p3/w3 15/10 / 1.5 wi . 25

W o
24 weights x x 1 2/3 + 24 ∑ wi. ∑ + pi pi x 15 2/15 xi xi x 1 = = 28.2 31
 r
order of the ratio pi / xi. Select the x2 = 1 and profit earned is 24. Now, only 5

next largest pi / xi ratio, so x3 = 1⁄2 and the

∑ pi xi
= 20 24 x 1 + 15 x 1/2 = 31.5
stored on a computer tape of length ‘L’. Each the programs can be stored on the tape if and

l
is at most ‘L’. is to be retrieved from this tape, the tape are stored in the order i = i 1, i2, . . . . is .

d
, in, the time tJ needed to retrieve program iJ is proportional to

∑ 1≤k≤jl i
k

74
If all the programs are retrieved equally often then the expected or mean retrieval time (MRT) is: 1 n . t 1 ≤
J≤n
For the optimal storage on tape problem, we are required to find the permutation for the ‘n’ programs so
that when they are stored on the tape in this order the MRT is minimized.
d (I) =

∑
j

∑ ∑ n Jl ikJ = 1

K=1

A Let Solution: There From (increasing algorithm sort The T o, The Example . n tape total

l l
algorithm. . = . the are Ordering . 3, 1, 1, 2, 2, 3, 3, . storage retrieval n! (Heap . (l1,
 J
above, 2, 3, 1, 3, 1, 2, order) ,Tm = 3 2 3 1 2 1 l2, 6 – I 1, l3) sort). possible problem time it of

N
then = simply their (5, This (RT) the 10, orderings. can lengths. programs ordering is

T
requires 3). be Then extended 5 5 10 10 3 3 This They can + + + + to are + + find (5 (5 (3 (3

can (10 (10 store be are:

 U
to +10) + + + the to be 3) 5) 10) carried + + be several optimal 5) 3) the + +

distributed +(5 carried + (5 (3 + + (3 programs (10 (10 out + + + tapes. + 10 ordering? 3 5 out 10 in + + +

W
+ d(I) + 5 3 over O 10) 10) + by 3) If + + (n 5) in there 3) 5) using these log non-

decreasing n) are a tapes.

o rld
= = = = = = time efficient m 38

31
43
41
29
34
order sorting using heap
> 1 tapes,

m ∑ -1J=0

d (IJ )
The objective is to store the programs in such a way as to minimize RT.
The programs are to be sorted in non decreasing order of their lengths li’s, l1 < l2 < .. . .. . . ln. The first ‘m’
programs will be assigned to tapes T o, . . . . ,Tm-1 respectively. The next ‘m’ programs will be assigned to
T0, . . . . ,Tm-1 respectively. The general rule is that program i is stored on tape T i mod m.
75
Algorithm:
The algorithm for assigning programs to tapes is as follows:
Algorithm Store (n, m) // n is the number of programs and m the number of tapes {
j := 0; // next tape to store on for i :=1 to n do {
Print (‘append program’, i, ‘to permutation for tape’, j); j := (j + 1) mod m; } } On any given tape, the

programs are stored in non-decreasing order of their lengths.

JOB SEQUENCING WITH DEADLINES
When we are given a set of ‘n’ jobs. Associated with each Job i, deadline d i > 0 and profit Pi > 0. For any
job ‘i’ the profit pi is earned iff the job is completed by its deadline. Only one machine is available for
processing jobs. An optimal solution is the feasible solution with maximum profit.
Sort the jobs in ‘j’ ordered by their deadlines. The array d [1 : n] is used to store the deadlines of the order
of their p-values. The set of jobs j [1 : k] such that j [r], 1 ≤ r ≤ k are the jobs in ‘j’ and d (j [1]) ≤ d (j[2]) ≤ . . .
≤ d (j[k]). To test whether J U {i} is feasible, we have just to insert i into J preserving the deadline ordering
and then verify that d [J[r]] ≤ r, 1 ≤ r ≤ k+1.
Example:
Let n = 4, (P1, P2, P3, P4,) = (100, 10, 15, 27) and (d1 d2 d3 d4) = (2, 1, 2, 1). The feasible solutions and their
values are:

ll
S. No Feasible Solution Procuring sequence A 123456789
 J NT U
1,2 1,3 1,4 2,3 3,4 1 2 3 4 2,1 1,3 or 3,1

World
4,1 2,3 4,3 1 2 3 4

Value Remarks
110
115
127 OPTIMAL
25
42
100
10
15
27
76
Algorithm:
The algorithm constructs an optimal set J of jobs that can be processed by their deadlines.
Algorithm GreedyJob (d, J, n)
// J is a set of jobs that can be completed by their deadlines.
{
J := {1}; for i := 2 to n do {
if (all jobs in J U {i} can be completed by their dead lines) then J := J U {i}; } }
OPTIMAL MERGE PATERNS
Given ‘n’ sorted files, there are many ways to pair wise merge them into a single sorted file. As, different
pairings require different amounts of computing time, we want to determine an optimal (i.e., one requiring
the fewest comparisons) way to pair wise merge ‘n’ sorted files together. This type of merging is called as
2-way merge patterns. To merge an n-record file and an m-record file requires possibly n + m record moves,
the obvious choice choice is, at each step merge the two smallest files together. The two-way merge
patterns can be represented by binary merge trees.
Algorithm to Generate Two-way Merge Tree:
struct treenode {
treenode * lchild; treenode * rchild; }; Algorithm TREE (n) // list is a global of n single node binary trees {
for i := 1 to n – 1 do {
pt ← new treenode (pt → lchild) ← least (list); // merge two trees with smallest lengths (pt → rchild) ←

least (list);
(pt → weight) ← ((pt → lchild) → weight) + ((pt → rchild) → weight);
insert (list, pt);

l J
Al tree } } return least (list);

NTU W o
// The tree left in list
 rld
is the

merge
77