Design and Analysis of

Algorithms- 21AML144

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 1

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 2
 Course outcomes:

The students will able to

1. Describe computational solution to engineering
problems. (Understand)
2. Estimate the computational complexity of different
algorithms. (Apply)
3. Develop an algorithm using appropriate design
strategies for problem solving. (Apply)
4. Analyze computational complexity of an algorithm to
increase efficiency. (Analyze)

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 3

 Pre-requisites

- Idea of programming
- Basic mathematics
- Concept of data structure
- Graph theory

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 4

• Muhammad ibn Musa al-Khwarizmi
• The Father of Algebra
• Persian Mathematician

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 5

 How to prepare tea?

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 6

 Applications
• Google Search
• Facebook
• Online Shopping

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 7

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 8
Shortest Distance between two cities

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 9

 Module 1
1. Introduction
2. Performance Analysis
3. Asymptotic Notations
4. Mathematical analysis
i. Recursive Algorithms
ii. Non-recursive Algorithms
5. Important Problem Types

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 10

 What is an Algorithm?
• An algorithm is a finite sequence of unambiguous
instructions to solve a particular problem.
• In addition, all algorithms must satisfy the following criteria:
– Input: Zero or more quantities are externally supplied.
– Output: At least one quantity is produced.
– Definiteness: Each instruction is clear and unambiguous.
– Finiteness: algorithm terminates after a finite number of steps.
– Effectiveness : Every instruction must be very basic so that it
can be carried out, in principle, by a person using only
pencil and paper. it also must be feasible.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 11

 What is an Algorithm?

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 12

 Algorithm specification
• An algorithm can be specified in
1. Simple English
2. Graphical representation like flow chart
3. Programming language like c++ / java
4. Combination of above methods.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 13

 Algorithm specification-
PSEUDO-CODE CONVENTIONS
1. Comments begin with // and continue until the end of line.

2. Blocks are indicated with matching braces { and }.

3. An identifier begins with a letter. The data types of

variables are not explicitly declared.
4. There are two Boolean values TRUE and FALSE.
Logical Operators
AND, OR, NOT
Relational Operators
<, <=,>,>=, =, !=
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 14
 Algorithm specification
PSEUDO-CODE CONVENTIONS

5. Assignment of values to variables is done using the

assignment statement.
<Variable>:= <expression>;

6. Array indices start at zero & elements are accessed

using [] . Eg: A[i ,j]

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 15

 Algorithm specification-
Contd…
7. The following looping statements are employed. For, while and repeat-until While Loop:
While < condition > do
{
<statement-1>
..
..
<statement-n>
}
For Loop:
For variable: = value-1 to value-2 step step do
{
<statement-1>
.
. <statement-n>
}
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 16
 Algorithm specification-
repeat-until:
repeat
<statement-1>
.
.
<statement-n>
until<condition>
8. A conditional statement has the following forms.

 If <condition> then <statement>

 If <condition> then <statement-1>

else <statement-1>
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 17
 Algorithm specification-
Case statement:
Case
{
: <condition-1> : <statement-1>
.
.
: <condition-n> : <statement-n>
: else : <statement-n+1>
}

9. Input and output are done using the instructions read & write.
No format is used to specify the size of input or output quantities

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 18

 Algorithm specification-
10. There is only one type of procedure: Algorithm, the
heading takes the form,

Algorithm Name (Parameter lists)

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 19

Algorithm to find max of n given numbers return
the results?

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 20

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 21
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 22
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 23
 Issue in the study of algorithm

1. How to create an algorithm.

2. How to validate an algorithm.
3. How to analyses an algorithm
4. How to test a program.

1 .How to create an algorithm: To create an algorithm we have following design

technique
a) Divide & Conquer
b) Greedy method
c) Dynamic Programming
d) Branch & Bound
e) Backtracking

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 24

2.How to validate an algorithm: Once an algorithm is created it is necessary to
show that it computes the correct output for all possible legal input , this
process is called algorithm validation.
3.How to analyses an algorithm: Analysis of an algorithm or performance
analysis refers to task of determining how much computing Time & storage
algorithms required.
Computing time-Time complexity: Frequency or Step count method Storage
space- To calculate space complexity we have to use number of input used in
algorithms.
4.How to test the program: Program is nothing but an expression for the
algorithm using any programming language. To test a program we need
following
Debugging: It is processes of executing programs on sample data sets
to determine whether faulty results occur & if so correct them.
Profiling or performance measurement is the process of executing a correct
program on data set and measuring the time & space it takes to compute the
result.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 25

 What is an Algorithm?
• An algorithm is a finite sequence of unambiguous
instructions to solve a particular problem.
• In addition, all algorithms must satisfy the following criteria:
– Input: Zero or more quantities are externally supplied.
– Output: At least one quantity is produced.
– Definiteness: Each instruction is clear and unambiguous.
– Finiteness: algorithm terminates after a finite number of steps.
– Effectiveness : Every instruction must be very basic so that it
can be carried out, in principle, by a person using only
pencil and paper. it also must be feasible

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 26

 Types of Algorithm
1) Iterative Algorithm
2) Recursive Algorithm

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 27

 Recursive Algorithm
1)Direct Algorithm: A recursive function is
function that is defined in terms of itself.

2)Indirect Algorithm: Algorithm A is said to be

indirect recursive if it calls another algorithm
which in turns call A

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 28

 Tower of Hanoi Algorithm

• Tower of Hanoi is a mathematical puzzle where

we have three rods (A, B, and C) and N disks.
Initially, all the disks are stacked in decreasing
value of diameter i.e., the smallest disk is placed
on the top and they are on rod A.

• The objective of the puzzle is to move the entire

stack to another rod (here considered C), obeying
the following simple rules:
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 29
• Only one disk can be moved at a time.
• Each move consists of taking the upper disk
from one of the stacks and placing it on top of
another stack i.e. a disk can only be moved if
it is the uppermost disk on a stack.
• No disk may be placed on top of a smaller
disk.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 30

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 31
• The idea is to use the helper node to reach the
destination using recursion. Below is the
pattern for this problem:

• Shift ‘N-1’ disks from ‘A’ to ‘B’, using C.

• Shift last disk from ‘A’ to ‘C’.
• Shift ‘N-1’ disks from ‘B’ to ‘C’, using A.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 32

public class TowerOfHanoi
{
static void towerOfHanoi(int n, char source, char temp, char dest)
{
if (n == 0) return;

towerOfHanoi(n - 1, source, dest, temp);

System.out.println("Move disk "+ n + " from tower " + source +" to tower " + dest );
towerOfHanoi(n - 1, temp, source, dest);
}

public static void main(String args[])

{
int n = 3; // Number of disks
towerOfHanoi(n, ‘S', ‘T', ‘D'); // S, T and D are name of towers
}
}

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 33

• Move disk 1 from rod S to rod D
• Move disk 2 from rod S to rod T
• Move disk 1 from rod D to rod T
• Move disk 3 from rod S to rod D
• Move disk 1 from rod T to rod S
• Move disk 2 from rod T to rod D
• Move disk 1 from rod S to rod D

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 34

• Tower of Hanoi mathematical puzzle that has
n disks with 3 towers can be solved in
minimum 2^n−1 steps.
• For an example
• A puzzle with 3 disks has taken
2^3 – 1 = 7 steps.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 35

 Analysis Framework
• Measuring an Input’s Size
– run longer on larger inputs
• Units for Measuring Running lime
– Basic operation
• Order of Growth

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 36

• Worst-Case
• Best-Case
• Average-Case Efficiencies

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 37

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 38
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 39
 PERFORMANCE ANALYSIS
• Performance Analysis: An algorithm is said to be efficient and fast
if it take less time to execute and consumes less memory space at run
time is called Performance Analysis.
1. SPACE COMPLEXITY:
• The space complexity of an algorithm is the amount of Memory
Space required by an algorithm during course of execution is called
space complexity .There are three types of space
a) Instruction space :executable program
b) Data space: Required to store all the constant and variable data
space.
c) Environment: It is required to store environment information
needed to resume the suspended space.
2. TIME COMPLEXITY:
• The time complexity of an algorithm is the total amount of time
required by an algorithm to complete its execution
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 40
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 41
 DAA

n+3

6/23/2023
Algorithm-1 Pankaja R Dept. of AI&ML,BNMIT
Algorithm-2 Algorithm-3:recursive procedure 42
 DAA

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 43

 Compile time
• The compile time does not depend on the
instance characteristics. Also, we may assume
that a compiled program will be run several
times without recompilation.
• we concern ourselves with just the run time
of a program. This run time is denoted by tp
(instance characteristics)

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 44

 C

DAA

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 45

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 46
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 47
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 48
 Analysis Framework
• Measuring an Input’s Size
• All algorithms run longer on larger inputs. For example, it
takes longer to sort larger arrays, multiply larger matrices,
and so on. Therefore, it is logical to investigate an
algorithm's efficiency as a function of some parameter n
indicating the algorithm's input size.
– run longer on larger inputs
• Units for Measuring Running time
• dependence on the speed of a particular computer, dependence on
the quality of a program implementing the algorithm and of the
compiler used in generating the machine code, and the difficulty of
clocking the actual running time of the program. [extraneous factors. ]
• Identify the most important operation of the algorithm, called the
basic operation, the operation contributing the most to the total
running time, and compute the number of times the basic operation is
executed.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 49

• - the execution time of an algorithm's basic
operation on a particular computer, and
• C(n)- be the number of times this operation needs to
be executed for this algorithm.
• Then we can estimate the running time T (n)
of a program implementing this algorithm on
that computer

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 50

 Orders of Growth
• Order of growth of an algorithm is a way of
stating how execution time or memory
occupied by it changes with the input size.
• Why emphasis on large input sizes?
• Because for large values of n, it is the
function's order of growth that counts.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 51

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 52
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 53
 Asymptotic Notations
• To compare orders of growth, computer scientists use
three notations:
– O(big oh),
– Ω(big omega),
– Θ (big theta) and

– o(little oh)

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 54

 1.Big oh (O)notation
1.Big oh (O)notation : This notation mainly represent upper
bound of algorithm run time. Big oh (O)notation is useful to
calculate maximum amount of time of execution.
By using Big-oh notation we have to calculate worst case time
complexity.

Formula : f(n)<=c g(n) n>=n0 , c>0 ,n0 >=1

Definition: f(n) is in O(g(n)) if order of growth of f(n) ≤ order of growth

of g(n) (within constant multiple), that is, there exist positive constant c
and non-negative integer n0 such that f(n) ≤ c g(n) for every n ≥ n0

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 55

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 56
 Examples
Example : f(n)=2n +3 & g(n)= n
Formula : f(n)<=c g(n) n>=n0 , c>0 ,n0 >=1
f(n)=2n+3 & g(n)=n
Now 3n+2<=c.n
3n+2<=4.n
Put the value of n =1
5<=4 false
N=2 8<=8 true now n0>2 For all value of n>2 & c=4
now f(n)<= c.g(n)
3n+2<=4n for all value of n>2
Above condition is satisfied this notation takes maximum
amount of time to execute .so that it is called worst case
complexity.
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 57
 Ω-Omega notation
 For a given function, we denote by the set of
functions
We use Ω-notation to give an asymptotic
lower bound on a function, to within a
constant factor.
 means that there exists some constant c s.t.
• is always for large enough n.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 58

 Definition: Let f(n) ,g(n) be two non negative
(positive) function now the f(n)= (g(n)) if
there exist two positive constant c,n0 such
that f(n)>=c.g(n) for all value of n, n>=n0

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 59

Example : f(n)=3n +2
Formula : f(n)>=c g(n) n>=n0 , c>0 ,n0 >=1
f(n)=3n+2
3n+2>=1*n, c=1 put the value of n=1
n=1 5>=1 true n0>=1 for all value
of n
It means that f(n)= Ω g(n).

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 60

 3.  -Theta notation
Theta (Θ) notation : It represent average bond
of algorithm running time. By using theta
notation we can calculate average amount of
time.
Formula : c1 g(n)<=f(n)<=c2 g(n)

where c is constant, n is function

Average bound

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 61

• A function f(n) is said to be in Θ(g(n)), denoted
f(n) =Θ(g(n)), if t (n) is bounded both above and
below by some positive constant multiples of
g(n) for all large n,i.e., if there exist some
positive constants c1 and c2 and some
nonnegative integer n0 such that
• c2 g(n) ≤ f(n) ≤ c1g(n) for all n ≥ n0.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 62

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 63
Example : f(n)=3n+2
Formula : c1 g(n)<=f(n)<=c2 g(n)
f(n)=2n+3
1*n<=3n+2<=4*n now put the
value of n=1 we get 1<=5<=4 false
n=2 we get 2<=8<=8 true
n=3 we get 3<=11<=12 true
Now all value of n>=2 it is true above condition is satisfied.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 64

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 65
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 66
Theorem: If t1(n) ∈ O(g1(n)) and t2(n) ∈ O(g2(n)), then
t1(n) + t2(n) ∈ O(max{g1(n), g2(n)}).
(The analogous assertions are true for the Ω and Ө notations as well.)

Proof:
The proof extends to orders of growth the following simple fact
about four arbitrary real numbers a1, b1, a2, b2:
if a1 ≤ b1 and a2 ≤ b2, then a1 + a2 ≤ 2 max{b1, b2}.
Since t1(n) ∈ O(g1(n)), there exist some positive constant c1 and
some nonnegative integer n1 such that
t1(n) ≤ c1g1(n) for all n ≥ n1.

Similarly, since t2(n) ∈ O(g2(n)), t2(n) ≤ c2g2(n) for all n ≥ n2.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 67
(The analogous assertions are true for the Ω and Ө notations as well.)

Proof: (continued):
Let us denote c3 = max{c1, c2} and consider n ≥ max{n1, n2} so
that we can use both inequalities.
Adding them yields the following:
t1(n) + t2(n) ≤ c1g1(n) + c2g2(n)
≤ c3 g1(n) + c3g2(n) = c3[g1(n) + g2(n)]
≤ c32 max{g1(n), g2(n)}.
Hence, t1(n) + t2(n) ∈ O(max{g1(n), g2(n)}), with the constants c
and n0 required by the O definition being 2c3 = 2 max{c1, c2}
and max{n1, n2}, respectively.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 68

 Using limits for Comparing Orders of
Growth

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 69

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 70
 Mathematical Analysis of Non-
recursive Algorithms
1. Decide on a parameters indicating an input’s size.
2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic operation
is executed depends only on the size of an input.
• If it also depends on some additional property, the
worst- case, average-case, and, if necessary, best-case
efficiencies have to be investigated separately.
4. Set up a sum expressing the number of times the
algorithm’s basic operation is executed.
5. Using standard formulas and rules of sum
manipulation, either find a closed form formula for
the count or, at the very least, establish its order of
growth.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 71

sum manipulation

• summation formulas

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 72

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 73
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 74
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 75
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 76
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 77
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 78
 Mathematical analysis of Recursive
algorithms
1. Decide on a parameter indicating an input’s size.
2. Identify the algorithm’s basic operation.
3. Check whether the number of times the basic
operation is executed can vary on different inputs of
the same size;
• if it can, the worst-case, average-case, and best-
case efficiencies must be investigated separately.
4. Set up a recurrence relation, with an appropriate
initial condition, for the number of times the basic
operation is executed.
• Solve the recurrence or, at least, ascertain the order
of growth of its solution

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 79

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 80
• We can use backward substitutions method to
solve this

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 81

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 82
• The idea is to use the helper node to reach the
destination using recursion. Below is the
pattern for this problem:

• Shift ‘N-1’ disks from ‘A’ to ‘B’, using C.

• Shift last disk from ‘A’ to ‘C’.
• Shift ‘N-1’ disks from ‘B’ to ‘C’, using A.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 83

towerOfHanoi(int n, char source, char temp, char dest)
{
if (n == 0) return;

towerOfHanoi(n - 1, source, dest, temp)

("Move disk "+ n + " from tower " + source +" to tower " + dest );
towerOfHanoi(n - 1, temp, source, dest);
}

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 84

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 85
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 86
 Important Problem Types
Most important problem types:
• Sorting
• Searching
• String processing
• Graph problems
• Combinatorial problems

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 87

 Sorting
• The sorting problem - rearrange the items of a
given list in ascending order.
• sort lists of numbers, characters from an
alphabet, character strings, and, most
important, records, libraries about their
holdings, and companies about their
employees.
• Key- choose a piece of information to guide
sorting.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 88

Two properties of sorting algorithms
- algorithm is called stable if it preserves the
relative order of any two equal elements in its
input.
- The amount of extra memory the algorithm
requires. An algorithm is said to be in place if
it does not require extra memory, except,
possibly, for a few memory units.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 89

 Why sorting?
• Sorting makes many questions about the list
easier to answer.
• The most important of them is searching: it is
why dictionaries, telephone books, class lists,
and so on are sorted.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 90

 Searching
• The searching problem deals with finding a given value, called a
search key, in a given set.

• Some algorithms work faster than others but require more memory;
some are very fast but applicable only to sorted arrays; and so on.

• Data may change frequently relative to the number of searches,

searching has to be considered in conjunction with two other
operations: addition to and deletion from the data set of an item.

• Also, organizing very large data sets for efficient searching poses
special challenges with important implications for real-life
applications.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 91

 String processing
• string matching
• string-processing algorithms have been
important for computer science

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 92

 Graph Problems
• Graph - a collection of points called vertices,
some of which are connected by line
segments called edges.
• Graphs can be used for modeling a wide
variety of real-life applications, including
transportation and communication networks,
project scheduling, and games.

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 93

• Basic graph algorithms include graph traversal
algorithms (How can one visit all the points in a
network?),
• shortest-path algorithms (What is the best route
between two cities?), and
topological sorting for graphs with directed edges
- traveling salesman problem(TSP) and the graph-
coloring problem
- The graph-coloring problem asks us to assign the
smallest number of colors to vertices of a graph so that
no two adjacent vertices are the same color
6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 94
 Combinatorial Problems
• The traveling salesman problem and the graph-
coloring problem are examples of combinatorial
problems.
• These are problems that ask (explicitly or
implicitly) to find a combinatorial object-such as a
permutation, a combination, or a subset-that
satisfies certain constraints and has some desired
property (e.g., maximizes a value or minimizes a
cost).

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 95

6/23/2023 Pankaja R Dept. of AI&ML,BNMIT 96