Scribd
Upload
5K views

Complexity Space N Time

This document discusses algorithms and analyzing their complexity. It defines an algorithm as a set of instructions to solve a problem and notes they must be unambiguous, finite, effective, and produce an output. Algorithm complexity is a measure of performance that depends on internal and external factors. The main internal factor is time or space required. Complexity can be analyzed experimentally by implementing algorithms and measuring runtime, or theoretically by characterizing runtime as a function of input size. Common asymptotic complexities include constant, logarithmic, linear, quadratic, and exponential growth. Theoretical analysis allows evaluation independent of hardware and avoids limitations of experimentation.

Uploaded by

api-3814408
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
5K views

Complexity Space N Time

This document discusses algorithms and analyzing their complexity. It defines an algorithm as a set of instructions to solve a problem and notes they must be unambiguous, finite, effective, and produce an output. Algorithm complexity is a measure of performance that depends on internal and external factors. The main internal factor is time or space required. Complexity can be analyzed experimentally by implementing algorithms and measuring runtime, or theoretically by characterizing runtime as a function of input size. Common asymptotic complexities include constant, logarithmic, linear, quadratic, and exponential growth. Theoretical analysis allows evaluation independent of hardware and avoids limitations of experimentation.

Uploaded by

api-3814408
Copyright
© Attribution Non-Commercial (BY-NC)
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 36

Complexity of

Algorithms

MSIT
Agenda
 What is Algorithm?
 What is need for analysis?
 What is complexity?
 Types of complexities
 Methods of measuring complexity
Algorithm
 A clearly specified set of instructions to
solve a problem.
 Characteristics:
 Input: Zero or more quantities are externally supplied
 Definiteness: Each instruction is clear and unambiguous
 Finiteness: The algorithm terminates in a finite number
of steps.
 Effectiveness: Each instruction must be primitive and
feasible
 Output: At least one quantity is produced
Algorithm
Need for analysis
 To determine resource consumption
 CPU time
 Memory space
 Compare different methods for
solving the same problem before
actually implementing them and
running the programs.
 To find an efficient algorithm
Complexity

 A measure of the performance of an


algorithm
 An algorithm’s performance depends on
 internal factors
 external factors
External Factors

 Speed of the computer on which it is run


 Quality of the compiler
 Size of the input to the algorithm
Internal Factor

The algorithm’s efficiency, in terms


of:
• Time required to run
• Space (memory storage)required
to run
Note:
Complexity measures the internal factors (usually more interested in
time than space)

8
Two ways of finding
complexity
 Experimental study
 Theoretical Analysis
Experimental study
 Write a program implementing the
algorithm
 Run the program with inputs of varying
size and composition
 Get an accurate measure of the actual
running time
Use a method like
System.currentTimeMillis()
 Plot the results
Example
a. Sum=0;
for(i=0;i<N;i++)
for(j=0;j<i;j++)
Sum++;
Java Code – Simple Program
import java.io.*;
class for1
{
public static void main(String args[]) throws Exception
{
int N,Sum;
N=10000; // N value to be changed.
Sum=0;
long start=System.currentTimeMillis();
int i,j;
for(i=0;i<N;i++)
for(j=0;j<i;j++)
Sum++;
long end=System.currentTimeMillis();
long time=end-start;
System.out.println(" The start time is : "+start);
System.out.println(" The end time is : "+end);
System.out.println(" The time taken is : "+time);
}
}
Example graph

Time in millisec
Limitations of Experiments
 It is necessary to implement the
algorithm, which may be difficult
 Results may not be indicative of the
running time on other inputs not included
in the experiment.
 In order to compare two algorithms, the
same hardware and software
environments must be used
 Experimental data though important is not
sufficient
Theoretical Analysis
 Uses a high-level description of the
algorithm instead of an implementation
 Characterizes running time as a function of
the input size, n.
 Takes into account all possible inputs
 Allows us to evaluate the speed of an
algorithm independent of the
hardware/software environment
Space Complexity
 The space needed by an algorithm is the
sum of a fixed part and a variable part

 The fixed part includes space for


 Instructions
 Simple variables
 Fixed size component variables
 Space for constants
 Etc..
Cont…

 The variable part includes space for


 Component variables whose size is dependant
on the particular problem instance being
solved
 Recursion stack space
 Etc..
Time Complexity
 The time complexity of a problem is
 the number of steps that it takes to solve an
instance of the problem as a function of the
size of the input (usually measured in bits),
using the most efficient algorithm.
 The exact number of steps will depend on
exactly what machine or language is being
used.
 To avoid that problem, the Asymptotic
notation is generally used.
Asymptotic Notation

 Running time of an algorithm as a function


of input size n for large n.
 Expressed using only the highest-order
term in the expression for the exact
running time.
Example of Asymptotic
Notation

 f(n)=1+n+n2

Order of polynomial is the degree of the


highest term
 O(f(n))=O(n2)
Common growth rates
Time complexity Example
O(1) constant Adding to the front of a linked list
O(log N) log Finding an entry in a sorted array
O(N) linear Finding an entry in an unsorted array
O(N log N) n-log-n Sorting n items by ‘divide-and-conquer’
O(N2) quadratic Shortest path between two nodes in a
graph
O(N3) cubic Simultaneous linear equations
O(2N) exponential The Towers of Hanoi problem
Growth rates
O(N2)

O(Nlog N)
Time

For a short time N2 is


better than NlogN

Number of Inputs
Best, average, worst-case
complexity
 In some cases, it is important to consider
the best, worst and/or average (or typical)
performance of an algorithm:
 E.g., when sorting a list into order, if it is
already in order then the algorithm may
have very little work to do
 The worst-case analysis gives a bound for
all possible input (and may be easier to
calculate than the average case)
Comparision of two algorithms
Consider two algorithms, A and B, for solving a
given problem.
 TA(n),TB( n) is time complexity of A,B
respectively (where n is a measure of the
problem size. )
 One possibility arises if we know the problem
size a priori.
 For example, suppose the problem size is n0 and
TA(n0)<TB(n0). Then clearly algorithm A is better than
algorithm B for problem size .
 In the general case,
 we have no a priori knowledge of the problem size.
Cont..
 Limitation:
 don't know the problem size beforehand
 it is not true that one of the functions is less
than or equal the other over the entire range
of problem sizes.
 we consider the asymptotic behavior of
the two functions for very large problem
sizes.
Asymptotic Notations
 Big-Oh
 Omega
 Theta
 Small-Oh
 Small Omega
Big-Oh Notation (O)

 f(x) is O(g(x))iff there exists constants


‘c’and ‘k’ such that f(x)<=c.g(x) where
x>k

 This gives the upper bound value of a


function
Examples
 x=x+1 -- order is 1

fori 1 to n
x=x+y -- order is n

fori 1 to n
for j 1 to n
x=x+y -- order is n2
Time Complexity Vs Space
Complexity
 Achieving both is difficult and best case
 There is always trade off
 If memory available is large
 Need not compensate on Time Complexity
 If fastness of Execution is not main
concern, Memory available is less
 Can’t compensate on space complexity
Example
 Size of data = 10 MB
 Check if a word is present in the data or
not
 Two ways
 Better Space Complexity
 Better Time Complexity
Contd..

 Load the entire data into main memory


and check one by one
 Faster Process but takes a lot of space

 Load data word–by-word into main


memory and check
 Slower Process but takes less space
Run these algorithms
For loop

a. Sum=0;
for(i=0;i<N;i++)
for(j=0;j<i*i;j++)
for(k=0;k<j;k++)
Sum++;

Compare the above "for loops" for


different inputs
Example
 3. Conditional Statements
Sum=0;
for(i=1;i<N;i++)
for(j=1;j<i*i;j++)
if(j%i==0)
for(k=0;k<j;k++)
Sum++;

 Analyze the complexity of the above algorithm


for different inputs
Summary
 Analysis of algorithms
 Complexity
 Even with High Speed Processor and large
memory ,Asymptotically low algorithm is
not efficient
 Trade Off between Time Complexity and
Space Complexity
References
 Fundamentals of Computer Algorithms
Ellis Horowitz,Sartaj Sahni,Sanguthevar
Rajasekaran
 Algorithm Design

Micheal T. GoodRich,Robert Tamassia


 Analysis of Algorithms

Jeffrey J. McConnell
Thank You

You might also like