Design and Analysis of Algorithms

UNIT-I
By

Dr.M.Rajababu

Associate Professor&HoD
Dept of Information Technology
Aditya University
Surampalem.
 Algorithm
• An algorithm is a finite set of instructions that, if followed, accomplishes a particular task.
• All algorithms must satisfy the following criteria:
• Input
-An algorithm has zero or more inputs, taken from a specified set of objects.
• Output
-An algorithm has one or more outputs, which have a specified relation to the inputs.
• Definiteness
-Each step must be precisely defined; Each instruction is clear and unambiguous.
• Finiteness
-The algorithm must always terminate after a finite number of steps.
• Effectiveness
-All operations to be performed must be sufficiently basic that they can be done exactly and in finite
length.
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Algorithm specification
• Algorithms can be specified in the following three ways:
• 1.using a natural language like English
• 2.using pseudo code
• 3.flow chart
• An algorithm is a step-by-step procedure or formula for solving a
problem.
• Pseudocode uses a combination of natural language and programming
language-like syntax to describe the steps of an algorithm.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • Algorithm for sum of N natural numbers
1.Start
2.Set the value of sum to 0
3.For each number from 1 to N:
1. Add the number to sum
4.Output sum
5.End

Design&analysisof Algorithms Dr.M.Raja Babu Thursday, December 5, 2024

 Pseudocode for sum of N natural numbers
BEGIN
SET sum TO 0
FOR each number FROM 1 TO N
DO sum = sum + number
END FOR
PRINT sum
END

Design&analysisof Algorithms Dr.M.Raja Babu Thursday, December 5, 2024

 Pseudo code Conventions
• 1.Commentsbegin with // and continue until the end of line.
• 2. Blocks are indicated with matching braces: { and } and Statements
are delimited by ;
• 3. The data types of variables are not explicitly declared.
• 4. Assignment of values to variables is done using the assignment
statement
variable := expression

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Pseudo code Conventions
• 5.There are two boolean values true and false. In order to produce
these values, the logical operators and , or, and not and the relational
operators<, ≤ , >, ≥,=, ≠ are provided.
• 6. Elements of multidimensional arrays are accessed using [ and ].
• 7. The following looping statements are employed:
For
while
repeat until

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Pseudo code Conventions

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Pseudo code Conventions
• 8.A conditional statement has the following forms:
if {condition)then{statement)
if {condition)then{statement1)else(statement2)
• 9. Input and output are done using the instructions read and write.
• 10. An algorithm consists of a heading and a body. The heading takes
the following general form
Algorithm Name(parameterlist)

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • As an example, the following algorithm finds and returns the maximum
of n given numbers:
1 Algorithm Max(A, n)
2 // A is an array of size n.
3 {
4 Result:=A[l];
5 for i :=2 to n do
6 if A[i] >Result then Result:=A[i];
7 Return Result;
8 }
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Algorithm Analysis
• Analysis of algorithms is important for two reasons :
• 1. To estimate the efficiency of the given algorithm
• 2. for comparing the algorithms for the given problem.
• Efficiency of an algorithm can be analyzed at two different stages,
before implementation and after implementation.
1. priori analysis − This is theoretical analysis of an algorithm.
2. posterior analysis − This is empirical analysis of an algorithm.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 priori analysis Posterior analysis

Priori analysis is an absolute analysis. Posteriori analysis is a relative analysis.

It is independent of language of compiler and types of It is dependent on language of compiler and type of
hardware. hardware.

It will give approximate answer. It will give exact answer.

It uses the asymptotic notations to represent how much time It doesn’t use asymptotic notations to represent the time
the algorithm will take in order to complete its execution. complexity of an algorithm.

It is same for every system. posteriori analysis differ from system to system.

It is done before execution of an algorithm. It is done after execution of an algorithm.

It is costlier than priori analysis because of requirement of

It is less costly than Posteriori Analysis.
software and hardware for execution.

Design&analysisof Algorithms Dr.M.Raja Babu Thursday, December 5, 2024

 Algorithm Analysis
• Algorithm analysis deals with the execution or running time of various
operations involved.
• Running time of an operation can be defined as no. of computer
instructions executed per operation.
• Suppose X is an algorithm and n is the size of input data, the time and
space used by the Algorithm X are the two main factors which decide
the efficiency of X.
• Time Factor − The time is measured by counting the number of key
operations such as comparisons in sorting algorithm
• Space Factor − The space is measured by counting the maximum
memory space required by the algorithm.
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Algorithm Analysis
• The complexity of an algorithm f(n) gives the running time and / or
storage space required by the algorithm in terms of n as the size of
input data.
• Space Complexity: Space complexity of an algorithm represents the
amount of memory space required by the algorithm in its life cycle.
Space required by an algorithm is equal to the sum of the following
two components :
• A fixed part
• A variable part
• Space complexity S(P) of any algorithm P is S(P) = C + SP(I)
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 • Algorithm: SUM(A, B)
• Step 1 - START
• Step 2 - C ← A + B + 10
• Step 3 – Stop
S(P) = 1+3.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Time complexity
• Time Complexity of an algorithm represents the amount of time
required by the algorithm to run to completion.
• Time requirements can be defined as a numerical function T(n).
• Methods for finding time complexity of an algorithm :
• 1.step count
• 2.operation count
• 3.asymptotic analysis
• 4.recurrence relations
• 5.amortized analysis
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Step count
• In this method, we count number of times one instruction is executing.
From that we will try to find the complexity of the algorithm.
• Algorithm sum(a[], n)
•{
• sum = 0; // 1 step * 1
• for ( i = 0; i < n; i++) // 1 step * (n+1)
• sum += a[i]; // 1 step * N
• return sum; // 1 step * 1
•}
• T(n)=1+n+1+n+1=2n+3=O(n)
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
Design&analysisof Algorithms Dr.M.Raja Babu Thursday, December 5, 2024
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Operation count
• In this method, time complexity is computed based on the count of basic
operations.
• Time complexity is expressed as the number of times the basic operation is
executed.
• Algorithm sum(a[], n)
•{
• sum = 0;
• for ( i = 0; i < n; i++)
• sum += a[i];
• return sum;
•}
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Best ,worst and Average case Time complexity
• Based on the distribution of input data time complexity can be
measured in three cases:
1. Best Case
2. Average Case
3. Worst Case

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Asymptotic analysis
• Asymptotic analysis is the process of calculating the running time of an
algorithm by considering the rate of growth of an algorithm.
• In Asymptotic Analysis, we evaluate the performance of an algorithm in
terms of input size (we don’t measure the actual running time).
• We calculate, how the time taken by an algorithm increases with the
input size.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Rate of Growth of Algorithm
• The rate at which running time increases as a function of input is
called Rate of Growth.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Asymptotic analysis
• Using asymptotic analysis, we can express the best case, average case
and worst case time complexity of an algorithm.
• We can have three cases to analyze an algorithm
1. Best Case − we calculate lower bound on running time of an
algorithm(Minimum time required for program execution)
2. Average Case − Average time required for program execution.
3. Worst Case − we calculate upper bound on running time of an
algorithm.(Maximum time required for program execution)

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Asymptotic Notations
• Asymptotic notations are the mathematical notations used to
describe the running time of an algorithm.
• Following are commonly used asymptotic notations used in
calculating running time complexity of an algorithm.
• Big Oh Notation(Ο)
• Omega Notation(Ω)
• Theta Notation (θ)

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Big Oh Notation(Ο)
• The Ο(n) is the way to express the upper bound of an algorithm's running time.
• It measures the worst case time complexity or longest amount of time an
algorithm can possibly take to complete.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Big Oh Notation(Ο)
• If f(n) and g(n) are two functions then
• f(n) = O(g(n)) if there exists c > 0 and n0 such that f(n) ≤ c.g(n) for all
n > n0.
Example:
• 1. 3n+2=O(n)
as 3n+2≤4n for all n≥2
• 2. 3n+3=O(n)
as 3n+3≤4n for all n≥3

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • Prove that running time T(n) = n 3 + 20n + 1 is O(n 3 )
• Prove that running time T(n) = n 3 + 20n + 1 is O(n 4 )
• Show that f(n) = (n + 1)3 is O(n 3 ).

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 What is the time complexity of following code:
int a = 0;
for (i = 0; i < n; i++)
{
for (j = 0; j <n; j++)
{
a = a + i + j;
}
}
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Omega Notation(Ω)
• The Ω(n) is the way to express the lower bound of an algorithm's
running time.
• It measures the best case time complexity or best amount of time an
algorithm can possibly take to complete.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Omega Notation(Ω)
• If f(n) and g(n) are two functions then
• f(n) = Ω(g(n)) if there exists c > 0 and n0 such that f(n) >=c.g(n) for all
n > n0.
Example:
f(n) = 3n + 2
g(n) = n
3n + 2 >= C n for all values of C = 1 and n >= 1.
3n + 2 = Ω(n)

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • Prove that T(n)=3n 2 − 100n + 6 is Ω(n 2 )

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Theta Notation (θ)
• The θ(n) is the way to express both the lower bound and upper bound
of an algorithm's running time.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Theta Notation (θ)
• If f(n) and g(n) are two functions then f(n)= θ(g(n))
if there exists c1,c2 > 0 and n0 such that
c1* g(n) ≤f(n) ≤ c2*g(n) for all n > n0.
Example:
f(n) = 3n + 2
g(n) = n
C1 n <= 3n + 2 <= C2 n for all values of C1 = 1, C2 = 4 and n >= 2
3n + 2 = Θ(n)

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • Prove that T(n) = 10n 4 − 50n 3 + 200n 2 − 1000n is Θ(n 4 )

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Recursive algorithms
• A recursive algorithm is an algorithm which calls itself with smaller input
values.
• An iterative algorithm uses a looping structure while a recursive algorithm,
a function calls itself again and again until the base condition is satisfied.
Algorithm Sum(n ) Algorithm Sum(n)
{ {
if (n == 1) Sum=0;
for(i=1;i<n; i++)
return 1
sum= sum+i;
return n + Sum(n-1) }
}
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Algorithm for sum of digits of a number
Algorithm sum_digits(num) Algorithm sum_digits(num)
{ {
int sum = 0; if (num <= 0)
while (num > 0) {
{ return 0;
sum += num % 10; }
num = num / 10; return (num % 10) + sum_digits(num / 10);
} }
return sum;
}

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Recurrence relations
• This method is used to analyze recursive algorithms.
• When we analyze a recursive algorithm two steps are required:
• 1.forming a recurrence equation
• 2.solving the recurrence equation
• There are mainly two ways for solving recurrences.
1. Substitution Method
2. Master Theorem Method

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Substitution method
• Here the solution can be obtained by repeated substitutions of the
RHS of the RE till a pattern is obtained.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • Algorithm Sum(n )
•{
• if (n == 1)
return 1
return n + Sum(n-1)
}
• recurrence relation is
• T(1) = 1
• T(n) = 1 + T(n-1), when n > 1.

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Substitution method
• T (n) = T (n-1) +1 and T (1) = 1.
• Solution:
• T (n) = T (n-1) +1
= (T (n-2) +1) +1
= (T (n-3) +1) +1+1
= T (n-4) +4
= T (n-5) +1+4
= T (n-5) +5
= T (n-k) + k
Where k = n-1
T (n-k) = T (1) = 1
T (n) = 1+ (n-1) = 1+n-1=n= 0 (n).
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 What is the time complexity of

T(n)=T(n-1)+T(n-2)-T(n-3) ,if n>3

=n , otherwise

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Algorithm to find factorial of a number
Algorithm factorial (num) Algorithm factorial(num)
{ {
fact=1; if (n==0 )
for (i=0;i<=n;i++) return 1
{ return n * factorial(n-1)
fact=fact*i; }
}
return fact
} The recurrence relation is defined as:
T(n) = 1 for n = 0
T(n) = n * T(n-1) for all n >0

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Algorithm to find factorial of a number
• From the above analysis we can write:
T(n) = T(n —1) + c
T(0) = 1
T(n) = T(n-1) + c
= T(n-2) + 2c
= T(n-3) + 3c
= T(n-4) + 4c
= ...
= T(n-k) + kc we know T(0) = 1
we need to find the value of k for which n - k = 0, k = n
T(n) = T(0) + cn , k = n
= 1 + cn that gives us a time complexity of O(n)
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 1.solve the equation by using Substitution Method. O(n2)
T (n) = 1 if n=0
= T (n-1)+n if n>0

2.solve the equation by using Substitution Method. O(nlogn)

T (n) = 1 if n=0
= T (n-1)+logn if n>0

2.solve the equation by using Substitution Method.O(2n)

T (n) = 1 if n=0
= 2T (n-1) if n>0
Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024
 Master theorem Method
• Master Method is a direct way to get the solution.
• The master method works only for following type of recurrences or for
recurrences that can be transformed to following type.
T(n) = aT(n/b) + f(n) where a >= 1 and b > 1
• If T(n) =a T(n/b)+cnk is the equation then the solution is given as follows:
Let d=a/bk
1. if d<1 then T(n) = O(nk )
2. if d=1 then T(n) = O(nk logb n)
3.if d!=1 then T(n)=O(n logb a )

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • time complexity of binary search
• Recurrence relation-> T(n)=T(n/2)+1
• a=1 b=2 k=0
• d=a/bk=1/1=1
• T(n)=O(logn)

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 • Solve the following recurrence equation using master theorem
• T(n)=8T(n/2)+n2 O(n3)
• T(n)=3T(n/3)+n2 O(nlogn)

Design&analysisof Algorithms M.Raja Babu Thursday, December 5, 2024

 Randomized algorithms
• Randomized algorithms are algorithms that use randomness in their
computations to achieve a desired outcome.
• They use random numbers to make decisions or solve problems.
• They are particularly useful in situations where exact solutions are difficult
to find or when a probabilistic approach is acceptable.
• They can be useful in a variety of contexts because they often provide
simpler, more efficient, or more practical solutions compared to
deterministic algorithms.
• They are widely used in optimization, computational geometry and
number theory.
Example: Randomized Quick Sort
Design&analysisof Algorithms Dr.M.Raja Babu Thursday, December 5, 2024
 Randomized algorithms
• There are two types of randomized algorithms:
1.Las Vegas algorithms
2.Monte-Carlo algorithms
Las Vegas algorithms always return correct results or fail to give one;
however, its runtime may vary. An upper bound can be defined for its
runtime.
Example: Quicksort
The Monte-Carlo algorithms work in a fixed running time; however, it does
not guarantee correct results. One way to control its runtime is by limiting the
number of iterations.
Example: Freivalds’ algorithm
Design&analysisof Algorithms Dr.M.Raja Babu Thursday, December 5, 2024
 • Given three matrices A, B and C, find if C is a product of A and B.

Input : A = Input : A =
11 111
11 111
B= 111
11 B=
11 111
C= 111
22 111
22 C=
Output : Yes 333
C=AxB 312
333
Output : No
Design&analysisof Algorithms Dr.M.Raja Babu Thursday, December 5, 2024