Lecture 1: Introduction

Design & Analysis of Algorithms

Rajesh Kumar Tripathi

Associate Professor, GLA University, Mathura
Introduction
 Algorithm
 Algorithm characteristics
 General steps to develop an algorithm
 Analyzing parameters
 Algorithm

 An algorithm is a sequence of computational steps that

transform input to desired output.
(Thomas H. Coreman)

 An algorithm is a finite set of instructions that, if followed,

accomplishes a particular task
(Sartaj Sahni)
 Criteria of algorithm
 All algorithms must satisfy the following five criteria:
 Input
 Output
 Definitiveness
 Finiteness
 Effectiveness
 Algorithm…
 Algorithm_name <(parameter list)>
1. Algorithm_Max(A, n) //A is an array of size n
{
2. Result=A[1];
3. for i=2 to n do
4. if(A[i] > Result) then Result=A[i];
5. return Result;
}
General steps to develop an Algorithm
 There are four steps:
 To devise an algorithm
 Validate an algorithm
 Analyze an algorithm
 Test an algorithm
 Debugging
 Profiling
 Lecture 2: Analyzing Parameters

Design & Analysis of Algorithms

Rajesh Kumar Tripathi

Assistant Professor, GLA University, Mathura
 Analyzing parameters
 Space Complexity
 Time Complexity
 Space Complexity
 This is an amount of memory requirement for an algorithm to
completion of its execution.
 Space Complexity…
 Space Complexity:
 Fixed part:
 It is independent of characteristics of inputs & outputs.
 It includes:
 instruction space,
 space for simple variables and
 fixed size component variables.

Algorithm abc(a, b, c)
{
Return (a+b+b*c+(a+b-c)/(a+b)+4.0);
}

Space is computed for a+b+b*c+(a+b-c)/(a+b)+4.0

 Space Complexity…
 Space Complexity:
 Variable part:
 A variable part that consists of space needed by
component variables whose size is dependent on the
particular problem instance.
// n is the length of array a[ ]
sum(a[], n)
{
x=0
for i=1 to n
x = x + a[i]
return(x);
}
Time Complexity
 Time complexity of an algorithm is the amount of computer
time it needs to run the completion.
Compute complexity
A()
{ int i;
for i=1 to n
pf(“Hello”);
}
Complexity: O(n)
A()
{int i, j;
for i=1 to n
for j=1 to n
pf(“Hello”);
}
Complexity: O(n2)
 Running Time of Algorithm
1. Get a positive integer from input Time Occurrence

2. if n>10 t1 1

3. print “This might take a while…” t2 1

4. for i=1 to n do t3 1

5. for j=1 to i do t4 n+1

6. print i*j; t5 2t5+3t5+4t5+…+(n+1)t5

7. Print “done”; t6 t6+2t6+3t6+……+nt6

t7 1
 Running Time of Algorithm
In this program, line 1,2,3,7 will take t1,t2,t3,t7 time with one
occurrence.
 Running Time of Algorithm
Statement 4 will take t4 time with (n+1) occurrence.

Statement 5 will take:

=2t5+3t5+4t5+…+nt5+(n+1)t5+t5-t5
=t5[(1/2)(n2+3n+2)]-t5
=(1/2)(n2+3n)t5
 Running Time of Algorithm
Statement 6 will take
=t6+2t6+3t6+……+nt6=t6[(1/2)(n2+n)]
Total time
=t1+t2+t3+t7+(n+1).t4+(1/2).(n2+3n).t5+t6.(1/2).(n2+n)
=t1+t2+t3+t7+t4+(t4+3/2.t5+t6).n +……
(1/2.n2+1/2.n2)(t5+t6)
=t1+t2+t3+t4+t7+n.[t4+3/2.t5+t6]+n2.[t5+t6]

It is in the form of quadratic equation.

 Conclusion
 Learned to compute the running time of an algorithm
 Analyses of the algorithm can be performed by computing
running time of algorithm
Asymptotic Notation
 Notation is used to describe the asymptotic running time of
an algorithm are defined in terms of functions whose domains
are set of natural numbers N={0,1,2,3,….}.
 Such notations are convenient for describing the worst case
running time function T(n) which is usually defined only on
integer input size.
Asymptotic Notation…
 Practical Significance
 Big Omega :
 Best case
 Never achieve better than this
 Big Theta:
 Average case
 Big O:
 Worst case
 Upper bound
 Time must not exceed
Asymptotic Notation…
10 12 35 11 40 52 22 21 25 50

0……………………………………………………………….n-1
If element=10 , then best case Ω(1)

If element=40, then average case Θ(n/2)

If element=50, then worst case O(n)

Asymptotic Notation…
 The Big-Oh (O) Notation
 Asymptotic upper bound

 f(n)=O(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) ≤ c g(n) for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.

 Used for worst case analysis
Input size
Asymptotic Notation…
 The Big-Oh (O) Notation
 Asymptotic upper bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n)= c < ∞
Then
f(n) € O(g(n))

Input size
Asymptotic Notation…
 For example: f(n) = 3n+2 g(n)=n
 f(n)= O(g(n))
f(n) ≤ c. g(n)
3n+2 ≤ c . n
c=4 , n0 = 2
Asymptotic Notation…
 For example: f(n) = 3n+2 g(n)=n
 f(n)= O(g(n))
limn->∞f(n)/g(n)= c < ∞
= limn->∞(3n+2)/n
= limn->∞(3+2/n)
= limn->∞(3+2/ꝏ)
= 3+ 2/(1/0)
= 3+ 2*0/1
= 3+0
=3<ꝏ
Hence, we can say f(n)=O(g(n))
f(n)=O(n)
Asymptotic Notation…
 Learned about the asymptotic notation
 Categories worst, average and best cases
 Learned worst case complexity representation using Big Oh
notation
Asymptotic Notation…
 The Big-Omega (Ω) Notation
 Asymptotic lower bound

 f(n)= Ω(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) ≥ c g(n) ≥ 0 for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.

 Used for best case running time
or lower bound of algorithmic problem. Input size
Asymptotic Notation
 The Big-Omega (Ω) Notation
 Asymptotic lower bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n) > 0
Then
f(n) € Ω(g(n))

Input size
Asymptotic Notation
 For example: f(n) = 3n+2 g(n)=n
 f(n)= Ω(g(n))
f(n) ≥ c. g(n)
3n+2 ≥ c . n
c=1, n0 = 1 n ≥ 1
Asymptotic Notation
 The Big-Theta (Θ) Notation
 Asymptotically tight bound

 f(n)= Θ(g(n)) , if there exists

Running time
constants c1 & c2 and n0 , s.t.
c1. g(n) ≤ f(n) ≤ c2. g(n)
for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.

Input size
 Used for average case running time
Asymptotic Notation
 The Big-Theta (Θ) Notation
 Asymptotically tight bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n) < ∞
Then,
f(n) € Θ(g(n))

Input size
Asymptotic Notation
 For example: f(n) = 3n+2 g(n)=n
 f(n) = Θ (g(n))
 c1.g(n) ≤ f(n) ≤ c2. g(n)
f(n) ≤ c2 . g(n)
3n+2 ≤ c2 . n
c2=4 , n0 ≥ 1

f(n) ≥ c1. g(n)

3n+2 ≥ c1. n
c1=1, n0 ≥ 1
Asymptotic Notation
 The Little-oh (o) Notation
 Asymptotic upper bound

 f(n)=o(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) < c g(n) for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.

Input size
Asymptotic Notation
 The Little-oh (o) Notation
 Asymptotic upper bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n)= 0
Then
f(n) € o(g(n))

Input size
Asymptotic Notation
 The Little-omega (Ω) Notation
 Asymptotic lower bound

 f(n)= Ω(g(n)) , if there exists

Running time
constants c > 0 and n0 ≥ 1 ,
s.t. f(n) > c g(n) ≥ 0 for n ≥ n0
 f(n) and g(n) are functions

over non-negative integers.

Input size
Asymptotic Notation
 The Little-omega (Ω) Notation
 Asymptotic lower bound
 if f & g be the functions then

Running time
if limn->∞f(n)/g(n) = ∞
Then
f(n) € Ω(g(n))

Input size
Introduction
 Recurrence
 Solving Methods
 Substitution Method
 Master Method
 Recursion Tree
 Iterative Method
Introduction
 Recurrence:
 A recurrence is an equation or inequality that describes a
function in terms of its value on smaller inputs.
 An algorithm contains a recursive call to itself, we can often
describe its running time by a recurrence equation or
recurrence.
 Different forms:
 T(n)= 2 T(n/2)+f(n)
 T(n)=T(n-1)+1
 T(n)=T(2n/3)+T(n/3)+f(n)
Recurrence
 Substitution Method
 Guess a bound and then use mathematical induction to
prove our guess correct.
Recurrence
 Substitution Method
 The substitution method for solving recurrences comprises
two steps:
 Guess the form of the solution.
 Use mathematical induction to find the constants and show
that the solution works.
Recurrence
 Substitution Method
T(n)=2T(n/2)+n
Guess the solution is: O(n log n)
Prove that T(n)<= c. (n log n) for c>0
T(n)=2T(n/2)+n………………………………..(1)
Compute for T(n/2)<= c. (n/2).log(n/2) and put into (1)
T(n)<= 2. c. (n/2).log(n/2) +n
T(n)<= c. n. log(n/2) +n
T(n)<= c.n. log n –c.n.log2 +n
T(n)<= c.n.log n –c.n + n for c>=1
Recurrence
 Substitution Method
T(n)<= c.n.log n –c.n + n for c=1
T(n)<= 1.n.log n -1.n +n= n.log n
T(n)<= 2. n. log n -2. n +n = n.logn + n.logn –n
.
.
.
T(n)=O(n log n)
Recurrence
 Substitution Method
T(n)=T(n/3)+T(2n/3)+d. n
Guess the solution is: O(n log n)
Prove that T(n)<= c. (n log n) for c>0
T(n)=T(n/3)+T(2n/3)+n………………………………..(1)
Compute for T(n/3)<= c. (n/3).log(n/3) and
T(2n/3)<=c. (2n/3). log(2n/3) put into (1)
T(n)<= c. n/3. log(n/3)+ c. 2n/3. log(2n/3) + d. n
T(n)<= c. n/3. log n – c. n/3. log 3 + c. 2n/3.log 2n – c. 2n/3
log 3 + d. n
T(n)<= c. n/3. log n – c. n/3. log 3 + c. 2n/3.log 2 + c. 2n/3 log n
– c. 2n/3 log 3 + d. n
Recurrence
 Substitution Method
T(n)<= c. n/3. log n – c. n/3. log 3 + c. 2n/3.log 2 + c. 2n/3 logn
– c. 2n/3 log 3 +d. n
<= c.n.log n –c.n log 3 + c. 2n/3 + d. n
<= c. n. log n – c.n.(log 3 -2/3) + d.n
for c>= d/(log3 – 2/3)


Thank You
Keep Learning…
 Design & Analysis of Algorithms

Recurrence-Master Method

Rajesh Kumar Tripathi

Associate Professor

Dept. of Computer Engineering & Applications, IET,

GLA University, Mathura
Introduction
 Recurrence
 Solving Methods
 Substitution Method
 Master Method
 Recursion Tree
 Iterative Method
Recurrence

Recurrence
 Master Method
 1. T(n) = 9 T(n/3) + n
Compare with standard formula T(n) = a T(n/b) + f(n)
a=9, b=3 and f(n)=n
Compute n(logb(a)) = n(log3 9) = n2 = n2-E

It is the form of I Case then T(n)=ѳ(n2)

Recurrence
 Master Method
 1. T(n) = T(2n/3) + 1
Compare with standard formula T(n) = a T(n/b) + f(n)
a=1, b=3/2 and f(n)=n0
Compute n(logb(a)) = n(log3/2 1) = n0

It is the form of II Case then T(n)=ѳ(lgn)

Recurrence
 Master Method
 1. T(n) = 3T(n/4) + n log n
Compare with standard formula T(n) = a T(n/b) + f(n)
a=3, b=4 and f(n)=n log n
Compute n(logb(a)) = n(log4 3) = n0.793+E
Check a f(n/b)<=c.f(n)

3. n/4 . log n/4 <= c. n log n

(3/4) n. log n/4 <= c. n.logn for c=3/4

It is the form of III Case then T(n)=ѳ(f(n))= ѳ(nlogn)

Recurrence



Thank You
Keep Learning…
 Design & Analysis of Algorithms

Recurrence-Recursion Tree

Presented by:
Rajesh Kumar Tripathi
Associate Professor
Dept. of Computer Engineering & Applications, IET,
GLA University, Mathura
Introduction
 Recurrence…
 Solving Methods…
 Substitution
 Master Method
 Recursion Tree
 Iterative Method
 Recurrence

Recurrence
 Recursion Tree…
 T(n)= 3 T(n/4) + c n2
Recurrence
 Recursion Tree…
 T(n)= 3 T(n/4) + c n2

Total=O(n2)
 Recurrence


1
 n log 4
2 3

3 cn
1
16


Thank You
Keep Learning…
 Design & Analysis of Algorithms
(BCSC0012)

Iterative Method

Presented by:
Rajesh Kumar Tripathi
AssociateProfessor
Dept. of Computer Engineering & Applications, IET,
GLA University, Mathura
Introduction
 Recurrence…
 Solving Methods…
 Substitution Method
 Master Method
 Recursion Tree
 Iterative Method
Recurrence
 Iterative Method…T(n)= 2 T(n/2)+ n s.t T(n/2)=2. T(n/2/2)+n/2
T(n)= 2 T(n/2)+ n
T(n)= 2 T(n/2)+ n ………………………………………………….(1)
T(n/2)= 2 T(n/2/2)+ n/2 = 2. T(n/22)+n/2
T(n)= 2 (2. T(n/22)+n/2 )+ n
T(n)= 22 . T(n/22) +n + n
Recurrence
 Iterative Method…T(n)= 2 T(n/2)+ n T(n/22)=2. T(n/22/2)+n/22
T(n)= 22 . T(n/22) +n + n ………………………………………..(2)
T(n/22) = 2 .T(n/22/2)+n/22
T(n/22)=2. T(n/23)+n/22 ……………………………………....(3)
T(n)= 22. (2.T(n/23)+n/22) + n + n
T(n)= 23. T(n/23) + 22. n/22 +n + n
T(n)= 23. T(n/23) + n +n + n
T(n)= 23. T(n/23) +3.n
.
.
T(n)= 2i. T(n/2i) +…+ i.n …………………………………….........(4)
 Recurrence

 Recurrence

Complexity of Algorithm
A()
{
int i=1,s=1;
while(s<=n)
{ i++;
s=s+i;
pf(“Hello”);
}
}
s: 1 3 6 10 15 21 ……….n
i: 1 2 3 4 5 6 ………..k
k*(k+1)/2>n
k=O(n1/2 )