Scribd
Upload
2 views

Algorithms Analysis - I

The document outlines the efficiency of algorithms, focusing on time and space complexity, and introduces methods for measuring running time, including experimental studies and theoretical analysis. It discusses pseudo code as a way to describe algorithms and presents concepts like Big-Oh notation, sum and product rules for analyzing complexity. The summary emphasizes the importance of understanding growth rates and the ability to empirically verify time complexity through ratio analysis.

Uploaded by

aradhyachandra5
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
2 views

Algorithms Analysis - I

The document outlines the efficiency of algorithms, focusing on time and space complexity, and introduces methods for measuring running time, including experimental studies and theoretical analysis. It discusses pseudo code as a way to describe algorithms and presents concepts like Big-Oh notation, sum and product rules for analyzing complexity. The summary emphasizes the importance of understanding growth rates and the ability to empirically verify time complexity through ratio analysis.

Uploaded by

aradhyachandra5
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 40

Outline

1. Introduction
2. Big-Oh and other Notations
3. Typical Growth Rates
4. Sum and Product Rules
5. Examples
6. Summary
What is a Good Algorithm?
The efficiency of an algorithm is measured by 2 important
factors :

T(n): Time / Steps taken by an algorithm for input of size n

S(n): Space (memory cells) required by an algorithm for


input of size n
Measuring the Running Time
How should we measure the running time of an algorithm?

Experimental Study

Write a program that implements the algorithm.


Run the program with data sets of varying size and composition.
Use command time to get an accurate measure of the actual
running time.
Limitations of Experimental Studies
It is necessary to implement and test the algorithm in order to
determine its running time.

Experiments can be done only on a limited sets of inputs, and


may not be indicative of the running time on other inputs not
included in the experiment.

In order to compare two algorithms, the same hardware and


software environments should be used.
Beyond Experimental Studies
We will develop a general methodology for analyzing running
time of algorithms. This approach

Uses a high-level description of the algorithm instead of


testing one of its implementations.

Takes into account all possible inputs

Allows one to evaluate the efficiency of any algorithm in a


way that is independent of the hardware and software
environment.
PseudoCode
A mixture of natural language and high-level programming
concepts that describes the main ideas behind a generic
implementation of a data structure and algorithm.

Eg. Algorithm arrayMax(A,n):


Input: An array A storing n integers
Output: The maximum element in A.
currentMax A[0]
for i 1 to n-1 do
if currentMax<A[i]
then currentMax A[i]
return currentMax
PseudoCode
It is more structured than usual prose but less formal than a
programming language.

Expressions:

 Use standard mathematical symbols to describe numeric


and boolean expressions.
 Use for assignment (“=” in C)
 Use = for the equality relationship (“= =” in C)
PseudoCode
Programming Constructs:

 decision Structures : if….then…..(else….)


 while-loops : while…..do
 for-loops: for…do
 array indexing : A[i], A[i, j]
A Computational Model
To summarize algorithm runtimes, we can use a
computer independent model

•instructions are executed sequentially

•count all assignments, comparisons, and increments

•every simple instruction takes one unit of time (cost


of executing instruction)
Simple Instructions
Count the simple instructions

•assignments have cost of 1


•comparisons have a cost of 1
•let's count all parts of the loop
for (int j = 0; j < n; j++)
• j=0 has a cost of 1, j<n executes n+1 times, and j++
executes n times for a total cost of 2n+2
•each statement in the repeated part of a loop have a
cost equal to number of iterations
Case Study: Linear Search Cost

for(i = 0; i < n; i++) 2n+2


n
if(item[i] == search_item) 0 or 1
return i; //index of item found 0 or 1
return -1; //item Not Found

Total Cost = 3n+3


Different Cases
• The total cost of sequential search is 3n+3
- But is it always exactly 3n+3 instructions?
- How many times will the loop actually execute?
• that depends
- If search_item is found at index 0: _____ iterations
• best case
- If search_item is found at index n-1:_____ iterations
• worst case
Big – Oh Example
•Suppose T(n) = 3n3 + 2n2
•Claim: For n0 = 1 and c = 5, 3n3 + 2n2 ≤ 5n3, Ɐn ≥ 1
•Such a claim may be proved using Mathematical
Induction
•If this claim is true, then the complexity (growth rate)
can be expressed as O(n3)
•Technically, we can also say that this T(n) is O(n4)
•But, that is a weak statement, and it is understood that
we need to find the “least upper bound”
Big 0 Complexity Chart
Rule of Sums

T1(n)

T2(n)
T(n) = T1 (n) + T2(n)

Sum Rule
Suppose T1(n) is O(f(n)) and T2(n) is O(g(n)), then T(n) is
O(max(f(n), g(n))
Rule of Products

T1(n)

T(n) = T1 (n) × T2(n)


Product Rule
Suppose T1(n) is O(f(n)) and T2(n) is O(g(n)), then T(n) is
O(f(n)g(n))
Note that O(cf (n)) is same as O(f (n))
Empirical Verification: Ratio Analysis
•Is it possible to empirically verify if the running time of
an algorithm is O(f (n))?
•Implement the algorithm and note down the running
time T(n) for different values of n.
•Now find the ratio T(n)/f(n), for those different values
on n.
•f(n) is a tight bound if this ratio converges to a positive
constant
•If the ratio converges to 0, then f(n) is an over-estimate
•f(n) is an under-estimation, if this ratio diverges.
Summary
•Time / Space complexity of an algorithm are expressed in
notations such as Big-Oh and Big-Theta
•These notations bring out the growth rate of time / space
wrt the size of the input
•These notations enable us to avoid exact calculations of
number of “basic steps” or memory space required —
overall growth rate can be estimated based on growth rates
of components
•It is possible to come up with several designs of algorithms
for a problem, and analysis is important to choose the
appropriate one
•It is also possible to perform empirical ratio analysis to
determine / verify time complexity of an algorithm
What Next?
•We will take analysis of recursive algorithms in the
next lecture
•Meanwhile, read on complexity analysis of algorithms
from standard Books
•Later in this week, we will start our discussions on
implementation of linear structures
•Topics like trees, graphs, sets, and hashing will follow
that

You might also like