B.

TECH V SEM CSE

ACADEMIC YEAR: 2023-2024

Course Name: Design and Analysis of Algorithm

Topic: Time & Space Complexity – Hands-on (Analysis of Linear and Binary Search)

Course code : CS 3102

Credits : 4
Mode of delivery : Hybrid (Power point presentation)
Faculty : Mr. Sunil Kumar Patel
Email-id : [email protected]
CS3102 (DAA), Dept. of CSE 1
Assignment
quiz Assessment
Mid term examination criteria’s
End term Examination

CS3102 (DAA), Dept. of CSE 2

Time & Space Complexity – Hands-on
(Analysis of Linear and Binary Search)

CS3102 (DAA), Dept. of CSE 3

 Course Information
• Course Handout
• Communicate through eMail
• Office hours
• To be communicated
• Grading policy
• Will be communicated as per university guidelines

CS3102 (DAA), Dept. of CSE

4
 Syllabus
• Introduction: Fundamentals of Algorithms, Important Problem Types, Analysis
of algorithm efficiency. Analysis Framework: Asymptotic Notations and Basic
Efficiency Classes. Mathematical Analysis of Nonrecursive and Recursive
Algorithms: Brute force Techniques, Divide and Conquer. Decrease and
Conquer: Insertion Sort, Depth First Search, Breadth First Search, Topological
Sorting. Transform and Conquer: Presorting, BST, Heapsort. Space and Time
tradeoffs: Input Enhancement in String Matching. Dynamic Programming:
Warshall's and Floyd's Algorithms, The Knapsack Problem. Greedy Techniques:
Prim's, Kruskal's and Dijkstra's Algorithm, Huffman Trees. Coping with
limitations of algorithmic power. Backtracking: nQueens problem, Hamiltonian
Circuit Problem, Subset Sum Problem. Branch and Bound: Assignment
Problem, Knapsack Problem, TSP. P, NP, and NP-complete Problems.

CS3102 (DAA), Dept. of CSE

5
 More Information

• Textbook
•Introduction to Algorithms 3rd ,Cormen,
Leiserson, Rivest and Stein, The MIT Press,
• Fundamentals of Computer Algorithms,
2nd, Sartaj Sahni, Ellis Horowitz,
Sanguthevar Rajasekaran

• Others
• Introduction to Design & Analysis Computer Algorithm 3rd,
Sara Baase, Allen Van Gelder, Adison-Wesley, 2000.
• Algorithms, Richard Johnsonbaugh, Marcus Schaefer, Prentice
Hall, 2004.
• Introduction to The Design and Analysis of Algorithms 2nd
Edition, Anany Levitin, Adison-Wesley, 2007.

CS3102 (DAA), Dept. of CSE

6
 Course Objectives
• CS1501.1 Analyse the running times of algorithms using asymptotic analysis.
• CS1501.2 Demonstrate and Design algorithms using divide-and-conquer
paradigm to solve business problems hence enhance skills.

• CS1501.3 Illustrate the concept of greedy and dynamic-programming approach

to solve real life problems to enhance entrepreneurship capabilities.
• CS1501.4 Demonstrate the concept of backtracking and branch & bound
algorithms.
• CS1501.5 Synthesize and analyse various advanced algorithms concept such as
graphs, string matching, approximation algorithms and complexity classes to
enhance employability.

CS3102 (DAA), Dept. of CSE

7
 Analysis of Algorithms
• Analysis is performed with respect to a
computational model
• We will usually use a generic uni-
processor random-access machine (RAM)
• All memory equally expensive to access
• No concurrent operations
• All reasonable instructions take unit time
• Except, of course, function calls
• Constant word size
• Unless we are explicitly manipulating bits
CS3102 (DAA), Dept. of CSE
8
 Analysis of Algorithms
➢ Analysis of efficiency of an algorithm can be performed at two different
stages, before implementation and after implementation, as

▪ A priori analysis − This is defined as theoretical analysis of an

algorithm. Efficiency of algorithm is measured by assuming that all
other factors e.g. speed of processor, are constant and have no
effect on implementation.

• A posterior analysis − This is defined as empirical analysis of an

algorithm. The chosen algorithm is implemented using
programming language. Next the chosen algorithm is executed on
target computer machine. In this analysis, actual statistics like
running time and space needed are collected.

➢ Algorithm analysis is dealt with the execution or running time of various

operations involved. Running time of an operation can be defined as
number of computer instructions executed per operation.

CS3102 (DAA), Dept. of CSE

9
 Input Size
• Time and space complexity
• This is generally a function of the input size
• E.g., sorting
• How we characterize input size depends:
• Sorting: number of input items
• Graph algorithms: number of nodes & edges
• Etc

CS3102 (DAA), Dept. of CSE

10
 Running Time
• Number of primitive steps that are
executed
• Except for time of executing a function call
most statements roughly require the same
amount of time
• y=m*x+b
• c = 5 / 9 * (t - 32 )
• z = f(x) + g(y)
• We can be more exact if need be
CS3102 (DAA), Dept. of CSE
11
 Analysis
• Worst case
• Provides an upper bound on running time
• An absolute guarantee
• Average case
• Provides the expected running time
• Very useful, but treat with care: what is
“average”?
• Random (equally likely) inputs
• Real-life inputs

CS3102 (DAA), Dept. of CSE

12
 Analysis of Algorithms
• How good is the algorithm?
• Correctness
• Time efficiency
• Space efficiency

CS3102 (DAA), Dept. of CSE

13
 What is an algorithm?
• Recipe, process, method, technique, procedure,
routine,… with following requirements:
1. Finiteness
terminates after a finite number of steps
2. Definiteness
rigorously and unambiguously specified
3. Input
valid inputs are clearly specified
4. Output
can be proved to produce the correct output given a valid input
5. Effectiveness
steps are sufficiently simple and basic

CS3102 (DAA), Dept. of CSE

14
Algorithm Design and Analysis
Process
Understand the problem

Decide on : algorithm
design techniques etc.

Design an algorithm

CS3102 (DAA), Dept. of CSE

15
Algorithm Design and Analysis
Process
Understand the problem

Decide on : algorithm
design techniques etc.

Design an algorithm

Prove correctness

Analyze efficiency etc.

CS3102 (DAA), Dept. of CSE

16
Algorithm Design and Analysis
Process
Understand the problem

Decide on : algorithm
design techniques etc.

Design an algorithm

Prove correctness

Analyze efficiency etc

Code the algorithm correctness

CS3102 (DAA), Dept. of CSE
efficiency 17
 Complexity
➢ Suppose X is treated as an algorithm and N is treated as the size of input
data, the time and space implemented by the Algorithm X are the two main
factors which determine the efficiency of X.

▪ Time Factor − The time is calculated or measured by counting the number

of key operations such as comparisons in sorting algorithm.
▪ Space Factor − The space is calculated or measured by counting the
maximum memory space required by the algorithm.

➢ The complexity of an algorithm f(N) provides the running time and / or storage
space needed by the algorithm with respect of N as the size of input data.

CS3102 (DAA), Dept. of CSE

18
 Complexity

Time complexity:
– How much time it takes to compute
– Measured by a function T(N)

Space complexity:
– How much memory it takes to compute
– Measured by a function S(N)

CS3102 (DAA), Dept. of CSE

19
 Time complexity
• N = Size of the input
• T(N) = Time complexity function
• Order of magnitude:
– How rapidly T(N) grows when N grows
– For example: O(N) O(logN) O(N²) O(2N)

CS3102 (DAA), Dept. of CSE

20
Asymptotic analysis

CS3102 (DAA), Dept. of CSE

21
 Space Complexity
• Space complexity = The amount of memory required
by an algorithm to run to completion
• [Core dumps = the most often encountered cause is
“memory leaks” – the amount of memory required larger
than the memory available on a given system]
• Some algorithms may be more efficient if data
completely loaded into memory
• Need to look also at system limitations
• E.g. Classify 2GB of text in various categories [politics,
tourism, sport, natural disasters, etc.] – can I afford to load
the entire collection?

CS3102 (DAA), Dept. of CSE

22
 Space Complexity (cont’d)
• S(P) = c + S(instance characteristics)
• c = constant
• Example:
SUM(P, Q)
Step 1 - START
Step 2 - R ← P + Q + 10
Step 3 – Stop

Here we have three variables P, Q and R and one constant.

Hence S(p) = 1+3. Now space is dependent on data types of
given constant types and variables and it will be multiplied
accordingly.

CS3102 (DAA), Dept. of CSE

23
 Analysis of Algorithms
• Efficiency of an algorithm can be
measured in terms of:
• Execution time (time complexity)
• The amount of memory required (space
complexity)
• Which measure is more important?
• Answer often depends on the limitations of
the technology available at time of analysis
CS3102 (DAA), Dept. of CSE
24
 Time Complexity
• For most of the algorithms associated
with this course, time complexity
comparisons are more interesting than
space complexity comparisons
• Time complexity: A measure of the
amount of time required to execute an
algorithm

CS3102 (DAA), Dept. of CSE

25
 Time Complexity
• Factors that should not affect time
complexity analysis:
• The programming language chosen to
implement the algorithm
• The quality of the compiler
• The speed of the computer on which the
algorithm is to be executed

CS3102 (DAA), Dept. of CSE

26
 Time Complexity
• Time complexity analysis for an
algorithm is independent of
programming language,machine used
• Objectives of time complexity analysis:
• To determine the feasibility of an algorithm
by estimating an upper bound on the
amount of work performed
• To compare different algorithms before
deciding on which one to implement
CS3102 (DAA), Dept. of CSE
27
 Time Complexity
• Analysis is based on the amount of
work done by the algorithm
• Time complexity expresses the
relationship between the size of the
input and the run time for the algorithm
• Usually expressed as a proportionality,
rather than an exact function

CS3102 (DAA), Dept. of CSE

28
 Time Complexity
• To simplify analysis, we sometimes
ignore work that takes a constant
amount of time, independent of the
problem input size
• When comparing two algorithms that
perform the same task, we often just
concentrate on the differences between
algorithms
CS3102 (DAA), Dept. of CSE
29
 Time Complexity
• Simplified analysis can be based on:
• Number of arithmetic operations performed
• Number of comparisons made
• Number of times through a critical loop
• Number of array elements accessed
• etc

CS3102 (DAA), Dept. of CSE

30
Worst Case vs. Average Case
• Worst case analysis is used to find an
upper bound on algorithm performance
for large problems (large n)
• Average case analysis determines the
average (or expected) performance
• Worst case time complexity is usually
simpler to work out

CS3102 (DAA), Dept. of CSE

31
 Big-O Analysis in General
• With independent nested loops: The
number of iterations of the inner loop is
independent of the number of iterations
of the outer loop
• Example:
Outer loop executes n/2 times.
int x = 0;
For each of those times, inner
for ( int j = 1; j <= n/2; j++ ) loop executes n2 times, so the
body of the inner loop is
for ( int k = 1; k <= n*n; k++ )
executed (n/2)*n2 = n3/2 times.
x = x + j + k; The algorithm is O(n3) .
CS3102 (DAA), Dept. of CSE
32
 Big-O Analysis in General
• With independent nested loops: The
number of iterations of the inner loop is
independent of the number of iterations
of the outer loop If you notice, j keeps doubling till it is
less than or equal to n. Number of
• Example: times, we can double a number till it is
less than n would be log(n).
int i, j, k = 0; Let’s take the examples here.
for (i = n / 2; i <= n; i++) { for n = 16, j = 2, 4, 8, 16
for (j = 2; j <= n; j = j * 2) { for n = 32, j = 2, 4, 8, 16, 32
So, j would run for O(log n) steps.
k = k + n / 2;
i runs for n/2 steps.
} So, total steps = O(n/ 2 * log (n)) =
} O(n*logn)
CS3102 (DAA), Dept. of CSE
33
 Big-O Analysis in General
• Example:
int a = 0, i = N;
while (i > 0) {
a += i;
i /= 2;
}

Complexity = O(logN)

CS3102 (DAA), Dept. of CSE

34
 Big-O Analysis in General
• With dependent nested loops: Number
of iterations of the inner loop depends
on a value from the outer loop
• Example:
When j is 1, inner loop executes 3
int x = 0; times; when j is 2, inner loop executes
for ( int j = 1; j <= n; j++ ) 3*2 times; … when j is n, inner loop
executes 3*n times. In all the inner loop
for ( int k = 1; k < 3*j; k++ ) executes 3+6+9+…+3n =
x = x + j; 3(1+2+3+…+n) = 3n2/2 + 3n/2 times.
The algorithm is O(n2).
CS3102 (DAA), Dept. of CSE
35
 Big-O Analysis in General
• With dependent nested loops: Number
of iterations of the inner loop depends
on a value from the outer loop
• Example:
int a = 0; The above code runs total no of times
for (i = 0; i < N; i++) { = N + (N – 1) + (N – 2) + … 1 + 0
for (j = N; j > i; j--) { = N * (N + 1) / 2
a = a + i + j; = 1/2 * N^2 + 1/2 * N
} O(N^2) times.
}

CS3102 (DAA), Dept. of CSE

36
 Big-O Analysis in General
Assume that a computer executes a million instructions a second.
This chart summarizes the amount of time required to execute f(n)
instructions on this machine for various values of n.

f(n) n=103 n=105 n=106

log2(n) 10-5 sec 1.7 * 10-5 sec 2 * 10-5 sec
n 10-3 sec 0.1 sec 1 sec
n*log2(n) 0.01 sec 1.7 sec 20 sec
n2 1 sec 3 hr 12 days
n3 17 min 32 yr 317 centuries
2n 10285 centuries 1010000 years 10100000 years

CS3102 (DAA), Dept. of CSE

37
 Big-O Analysis in General
• Examples of dominant terms:
n dominates log2(n)
n*log2(n) dominates n
n2 dominates n*log2(n)
nm dominates nk when m > k
an dominates nm for any a > 1 and m >= 0

• That is, log2(n) < n < n*log2(n) < n2 < …

< nm < an for a >= 1 and m > 2
CS3102 (DAA), Dept. of CSE
38
 Intractable problems
• A problem is said to be intractable if
solving it by computer is impractical
• Example: Algorithms with time
complexity O(2n) take too long to solve
even for moderate values of n; a
machine that executes 100 million
instructions per second can execute 260
instructions in about 365 years
CS3102 (DAA), Dept. of CSE
39
 Constant Time Complexity
• Algorithms whose solutions are independent
of the size of the problem’s inputs are said
to have constant time complexity

• Constant time complexity is denoted as O(1)

CS3102 (DAA), Dept. of CSE

40
Basic Search Algorithms and
their Complexity Analysis

CS3102 (DAA), Dept. of CSE

41
 Linear Search: Example 1
• The problem: Search an array a of size n to
determine whether the array contains the
value key; return index if found, -1 if not found

Set k to 0.
While (k < n) and (a[k] is not key)
Add 1 to k.
If k == n Return –1.
Return k.

CS3102 (DAA), Dept. of CSE

42
 Analysis of Linear Search
• Total amount of work done:
• Before loop: a constant amount a
• Each time through loop: 2 comparisons, an
and operation, and an addition: a constant
amount of work b
• After loop: a constant amount c
• In worst case, we examine all n array
locations, so T(n) = a +b*n + c = b*n + d,
where d = a+c, and time complexity is O(n)

CS3102 (DAA), Dept. of CSE

43
 Analysis of Linear Search
• Simpler (less formal) analysis:
• Note that work done before and after loop is
independent of n, and work done during a
single execution of loop is independent of n
• In worst case, loop will be executed n times,
so amount of work done is proportional to n,
and algorithm is O(n)

CS3102 (DAA), Dept. of CSE

44
 Binary Search
(on sorted arrays)

• General case: search a sorted array a

of size n looking for the value key
• Divide and conquer approach:
• Compute the middle index mid of the array
• If key is found at mid, we’re done
• Otherwise repeat the approach on the half
of the array that might still contain key
• etc…
CS3102 (DAA), Dept. of CSE
45
Example: Binary Search For Ordered List
int binarySearch(int low, int high, int key)
{
while(low<=high)
{
int mid=(low + high) / 2;
if(a[mid]<key)
{
low=mid+1;
}
else if(a[mid]>key)
{
high=mid-1;
}
else
{
return mid;
}
}
return -1; //key not found
}
CS3102 (DAA), Dept. of CSE
46
 Analysis of Binary Search
• The amount of work done before and
after the loop is a constant, and
independent of n
• The amount of work done during a
single execution of the loop is constant
• Time complexity will therefore be
proportional to number of times the loop
is executed, so that’s what we’ll analyze
CS3102 (DAA), Dept. of CSE
47
 Analysis of Binary Search
• Worst case: key is not found in the array
• Each time through the loop, at least half
of the remaining locations are rejected:
• After first time through, <= n/2 remain
• After second time through, <= n/4 remain
• After third time through, <= n/8 remain
• After kth time through, <= n/2k remain

CS3102 (DAA), Dept. of CSE

48
Analysis of Binary Search
In general, we can write the size
of the problem as n/2i after
doing i comparisons. Thus, for
the base case,

n/2i =1

 n= 2i

=> i=log2n

Also, the comparison will reach

the base case only in the worst
case and thus, binary search is a
O(logn) algorithm.
CS3102 (DAA), Dept. of CSE
49