1

Algorithm analysis
BIG O
Algorithm: 2
 Is a method or a process adopted in order to solve problem

Problem :
✓ Task to be performed
✓ Input and output

Analysis of the algorithm

 Principles 3

 Algorithm
 Is a step-by-step procedure(set of instructions)
……to solve a specific problem
…….. in a finite amount of time
➢ Algorithm must be expressed in steps that the processor is capable of
performing
➢ The algorithm must eventually terminate
➢ The stated problem must be solvable(step by step)
Principles 4

 The word is derived from the last name of

Abou Jaafar Mohammad Ibn Moussa Al-Khawarizmi
 Who was a Arabic mathematician who invented a set of rules for
performing the 4 basic operation +,-,/,*
 Efficiency 5

 Given several algorithms to solve the same problem, which algorithm is the best
 Given an algorithm, is it feasible to use it at all? Is it efficient enough to be usable in
practice?
 How much time the algorithm require
 How much space (memory) does the algorithm require?
Efficiency 6
How to validate correct algorithms?
 By measuring time:
 Time:
• instructions take time
• how fast does the algorithm perform
• What effects its runtime
 Space(memory cost)
• Data structure take place
• What kind of data structures can be used
• How does the choice of data structures effect the
runtime?
➢ Quality, simplicity, compiler optimization, power consumption,
bandwidth,…..
Algorithm analysis 7

 Algorithm analysis is to determine the amount of resources that the

algorithm will require
 The performance of an algorithm is measured on the basis of the
following properties:
1. Time complexity
2. Space complexity(memory)
Time & space complexity 8

 The amount of time and space required by an algorithm depend on

the algorithm’s input(amount/size of the input)
• More data means that the program takes more time
• Any array to search in has a size of 10 elements needs time less
than of size 1000 elements

 Other factors:
1.Speed of the host machine(hardware)
2.Quality of the compiler(programming language…)
3. Qualityof the program(in some case)(methodology e.g.
procedural vs. OOP)
Hypothetical profile of 2 sorting 9

algorithms:

Algorithm B grows more slowly than A

How to compute time 10

 Measure time in seconds

1. Is useful in practice
2. Depends on language, compiler, and processor
 Count algorithm steps
1. Doesn’t depend on compiler and processor
2. Depends on granularity of steps(division of the steps)
 Count characteristics of the operations
1. Depends only on the algorithm itself
2. Measures the algorithms intrinsic efficiency(get in to the solution
directly)
Execution time of algorithm 11

 Each instruction takes a certain amount of time

Unique cost of 1 “time cycle”

Count + = 1 → take a certain amount of time, but it is cst.

 A sequence of operations:
Count + = 1 cost c1
Sum + = I cost c2
Which means : total cost = c1+c2
Execution time of algorithm 12

 If –Else
The runtime is:
The sum of the test +
The larger of the running times of If block or Else block
E.g.
Execution time of algorithm 13

 Simple loop
Loop accumulates its instructions cost(add the costs time)
Even if we may break earlier, but we always consider the worst case:
 E.g.
Execution time of algorithm 14

 Nested loop
 Analyze inside out and multiply
 E.g.
 15

i = 1, sum=0, j = 1;
while(i <= n) {
j = 0;
while(j <= m) {
operationX;
j++;
}
i++;
}
Complexity 16
 Measuring the functions Rates of Growth
For many interesting algorithms, the exact number of operation is
too difficult to analyze mathematically
 To simplify the analysis:
▪ Identify the fastest-growing term
▪ Neglect slower-growing terms
(small values of N generally are not important)

▪ neglect the constant factor in the fastest-growing term

(the exact value of the leading constant of the dominant term is not
meaningful across different machines)

 The resulting formula is the algorithm’s time complexity.

It focuses on the growth rate of the algorithm’s time requirement.
Big-O Notation 17

 Big O notation tells you the number of operation an algorithm will make
 Big O notation doesn't tell you the exact time in seconds
 Big O notation is used to compare 2 different algorithm by number of
operation he made
 Big O established the worst case run time
Big-O Notation 18

 We express complexity using big O notation.

 We use big O notation to :
▪ Capture the most dominant term in a function
▪ And to represent the growth rate
 Big-O defines the upper bound on the complexity of the algorithm
Measuring function rates of growth 19

 Small values of N generally are not important

 Avoid details, they don’t matter when input size (N) is big enough
 The difference between the best and the worst algorithm is less than a blink of
the eye

 For polynomials, use only leading term, ignore coefficients : linear, Quadratic
Complexity 20
 21

1. Nous commençons avec la série 1+2+4+...+2^n.

2. Si nous observons attentivement, chaque terme de la série est une puissance de 2. En d'autres
termes, chaque terme peut être obtenu en élevant 2 à une certaine puissance. Par exemple, le
premier terme est 2^0, le deuxième terme est 2^1, le troisième terme est 2^2, et ainsi de suite.
3. Si nous écrivons les termes de la série en utilisant les puissances de 2, la série devient :
2^0 + 2^1 + 2^2 + ... + 2^n.
4. Cette série peut être considérée comme une somme géométrique, où a = 2^0 (le premier
terme), r = 2 (le rapport entre les termes successifs), et n est le nombre de termes.
5. La formule générale pour la somme d'une série géométrique est donnée par : S = a * (r^n - 1) /
(r - 1), où S est la somme totale.
6. Appliquons maintenant cette formule à notre série :
S = 2^0 * (2^n - 1) / (2 - 1)
= 1 * (2^n - 1)
= 2^n - 1.
Classes of Algorithm
22

The following table arranges functions that commonly

describe algorithm running times in order of increasing
growth rate
GROWTH RATES OF THE COMMON 23

TIME-COMPLEXITY FUNCTIONS
Complexity analysis 24

 Consider the 2 previous version of algorithm

 Version 1:t(N)=2n2, If we let c=2,then 2n2<=2n2 for result of O(n2)
 Version 2:t(N)=n2+n,let c=2,then n2+n<=2n2 for result of O(n2)
 In this case, the choice of c comes from the observation that when
n ≥ 1, we have n ≤ n2 and n2 + n ≤ n2 + n2, which satisfies the
equation in the definition of big-O.
Complexity analysis 25

 The function f(n) = n2 is not the only choice for satisfying the
condition T(n) ≤ c.f(n)
 We could have said the algorithms had a run time of O(n3) or O(n4)
since n2 ≤ n3 and n2 ≤ n4 when n > 1.
 The objective, however, is to find a function f that provides the
lowest upper bound or limit for the run time of an algorithm
Example 26

 Recall the problem of downloading a file over te internet

 T(N)=N/160+2
Example 27

 Recall the problem of downloading a file over te internet

 T(N)=N/160+2
 T(N)<=N/160+N when N>=2
Example 28

 Recall the problem of downloading a file over te internet

 T(N)=N/160+2
 T(N)<=N/160+N when N>=2
161
 T(N)<= N
160
Example 29

 Recall the problem of downloading a file over the internet

 T(N)=N/160+2
 T(N)<=N/160+N when N>=2
161
 T(N)<= N
160
 T(N) is O(N) with c=161/160 and N0=2
 30
1. Initial time complexity equation: T(N) = N/160 + 2
The equation represents the time, T(N), required to download a file over the internet based on the file size,
N. It consists of two components: N/160, which represents the time required for the actual download, and
2, which represents some additional constant time operations.

2. Simplifying the time complexity equation: T(N) <= N/160 + N when N >= 2
Here, the inequality T(N) <= N/160 + N is derived by replacing the constant 2 with N, which is a reasonable
approximation since N is typically much larger than 2. This simplification allows for a more accurate
characterization of the time complexity.

3. Further simplification: T(N) <= 161/160 * N

By combining the terms N/160 + N, we obtain the expression 161/160 * N. This step is valid as N/160 + N =
(161/160) * N, which provides a tighter upper bound for the time complexity.

4. Establishing big O notation: T(N) is O(N) with c = 161/160 and N0 = 2

The final step involves expressing the time complexity using big O notation. Since T(N) <= 161/160 * N, we
can conclude that the time complexity of the file download problem is O(N) with the constant c = 161/160.
This means that the time required to download the file grows linearly with the file size N. The value N0 = 2
indicates that the analysis applies when N is greater than or equal to 2.
 31

ILLUSTRATION OF THE GROWTH RATES

Example 32

 Calculate the running time and then deduce the complexity in

terms of Big-O of an algorithm that calculates the average of an
array of numbers
Example 33

1
Example 34

1
1
Example 35

1
1
1
Example 36

1
1
1
n+1
Example 37

1
1
1
n+1
2*n
Example 38

1
1
1
n+1
2*n
2*n
Example 39

1
1
1
n+1
2*n
2*n
2
Example 40

1
1
1
n+1
2*n
2*n
2
1
T(n)=5n+7
Example 41

T(n) = 5n+7

T(n) = 5n+7
T(n) <= 5n+n when n >= 7

T(n) = 5n+7
T(n) <= 5n+n when n >= 7
T(n) <= 6n

T(n) = 5n+7
T(n) <= 5n+n when n >= 7
T(n) <= 6n
• T(n) is O(N) with c=6 and n0=7
Constructing T(N) 42

➢ For simplicity, we assume that each basic operation or statement, at

the abstract level, takes the same amount of time and, thus, each is
assumed to cost constant time i.e. O(1).
➢ Basic operation:
• Assigning a value to a variable
• Performing an arithmetic operation
• Comparing two numbers
• Accessing an element of an array
• Calling a function (excluding operations executed within the
function)
• Returning from a function

➢ The total number of operations required by an algorithm can be

computed as a sum of the times required to perform each step:
T(n) = f1(n) + f2(n) + … + fk(n)
Choosing the Function 43

 The function f(n) used to categorize a particular algorithm is chosen

to be the dominant term within T(n).
 That is, the term that is so large for big values of n, that we can
ignore the other terms when computing a big-O value.
 For example, in the expression n2 + log n + 3n the term n2 dominates
the other terms since for n ≥ 3,w e have
n2 + log n + 3n ≤ n2 + n2 + n2
n2 + log n + 3n ≤ 3n2
 which leads to a time-complexity of O(n2)
SOME RULES FOR CALCULATING THE 44

COMPLEXITY OF AN ALGORITHM
• The complexity of a loop is the complexity of the body multiplied by the number
of times this loop is repeated

• The complexity of if-else is the maximum of the complexity of block if and the
complexity of the block else

• The complexity of a function is determined by the complexity of its body

Simplification Rules 45
Simplification Rules 46
Constant Time Example 47

 T(n) = 6 O(1)
 T(n) = 100000 O(1)
 width = 5.5 1
 height = 2 1
 print("area is", width * height) 2
 48

LINEAR TIME EXAMPLES

 def ex1( n ):
 total = 0 O(1)
 for i in range( n ):
 total += i n*O(1)= O(n)
 return total O(1)

T(n) is O(max(1,n,1)) = O(n)

 49

LINEAR TIME EXAMPLES

def ex2( n ):
count = 0 O(1)
for i in range( n ) :
count += 1 n * O(1)= O(n)
for j in range( n ) :
count += 1 n * O(1)= O(n)
return count O(1)

T(n) is O(max(1,n,n,1)) = O(n)

 50

QUADRATIC TIME EXAMPLES

 When presented with nested loops, such as in the following, the time
required by the inner loop impacts the time of the outer loop

def ex3(n):
count = 0
for i in range(n) :
for j in range(n) :
count += 1
return count

 Both loops will be executed n, but since the inner loop is nested
inside the outer loop, the total time required by the outer loop will
be T(n) = n * n, resulting in a time of O(n2)
 51

QUADRATIC TIME EXAMPLES

 Not all nested loops result in a quadratic time. Consider the following
function:
def ex4( n ):
count = 0
for i in range( n ) :
for j in range( 25 ) :
count += 1
return count

print(ex4(1000))

 which has a time-complexity of O(n). The function contains a nested

loop, but the inner loop executes independent of the size variable n.
Since the inner loop executes a constant number of times, it is a
constant time operation. The outer loop executes n times, resulting in a
linear run time.
QUADRATIC TIME EXAMPLES 52

 The next example presents a special case of nested loops:

def ex5( n ):
count = 0
for i in range( n ) :
for j in range( i+1 ) :
count += 1
return count

 How many times does the inner loop execute?

 It depends on the current iteration of the outer loop. On the first iteration of the
outer loop, the inner loop will execute one time; on the second iteration, it
executes two times; on the third iteration, it executes three times, and so on until
the last iteration when the inner loop will execute n times
 Since the inner loop varies from 1 to n iterations by increments of 1, the total
number of times the increment statement will be executed is equal to the sum of
the first n positive integers: which results in a quadratic time of O(n2)
 53
LOGARITHMIC TIME EXAMPLES
 The next example contains a single loop, but notice the change to the modification step.
Instead of incrementing (or decrementing) by one, it cuts the loop variable in half each time
through the loop.
def ex6( n ):
count = 0
i = n
while i >= 1 :
count += 1
i = i // 2
return count

 To determine the run time of this function, we have to determine the number of loop iterations
 Since the loop variable is cut in half each time, this will be less than n
 For example, if n equals 16, variable i will contain the following five values during subsequent
iterations (16, 8, 4, 2, 1)
 When the size of the input is reduced by half in each subsequent iteration, the number of
iterations required to reach a size of one will be equal to or the largest integer less than log2,
plus 1.
 Thus, function ex6() requires O(log n) time.
LOGARITHMIC TIME EXAMPLES 54

 Consider the following definition of function ex7(), which calls ex6()

from within a loop:
def ex7(n):
count = 0
for i in range(n):
count += ex6(n)
return count
print(ex7(2))

 Since the loop is executed n times and function ex6() requires

logarithmic time, ex7() will have a run time of O(n log n)
LOGARITHMIC TIME EXAMPLES 55

 Consider the following definition of function ex8()

def ex8(n):
count = 0
k = 1
while k <= n:
for i in range(n):
count += 1
k *= 2
return count

 Since the while loop variable is multiplied by 2 each time, this will be log n
loop iteration
 Since the for loop is executed n times, ex8() will have a run time of O(n
log n)
DIFFERENT CASES 56
 Some algorithms can have run times that are different orders of magnitude for
different sets of inputs of the same size
 These algorithms can be evaluated for their best, worst, and average cases
 Algorithms that have different cases can typically be identified by the inclusion of
an event-controlled loop or a conditional statement
 Consider the following example, which traverses an array containing integer values
to find the position of the first negative value
def findNeg(intList):
n = len(intList)
for i in range( n ) :
if intList[i] < 0 :
return i
return None
 At first impression, it appears the loop will execute n times, where n is the size of the
array. But notice the return statement inside the loop, which can cause it to
terminate early.
DIFFERENT CASES 57

 If the array does not contain a negative value

 L = [ 72, 4, 90, 56, 12, 67, 43, 17, 2, 86, 33 ]

 p = findNeg( L )
 The return statement inside the loop will not be executed and the loop
will terminate in the normal fashion from having traversed all n times.
 In this case, the function requires O(n) time. This is known as the worst
case since the function must examine every value in the array requiring
the most number of steps
DIFFERENT CASES 58

 Now consider the case where the list contains a negative value in the first
element:

 L = [ -12, 50, 4, 67, 39, 22, 43, 2, 17, 28 ]

 p = findNeg( L )
 There will only be one iteration of the loop since the test of the condition
by the if statement will be true the first time through and the return
statement inside the loop will be executed.
 In this case, the findNeg() function only requires O(1) time. This is known as
the best case since the function only has to examine the first value in the
array requiring the least number of steps
DIFFERENT CASES 59

 The average case is evaluated for an expected data set or how we expect
the algorithm to perform on average
 For the findNeg() function, we would expect the search to iterate halfway
through the array before finding the first negative value, which on average
requires n/2 iterations.
 The average case is more difficult to evaluate because it's not always readily
apparent what constitutes the average case for a particular problem
 In general, we are more interested in the worst case time-complexity of an
algorithm as it provides an upper bound over all possible inputs.
 In addition, we can compare the worst case run times of different
implementations of an algorithm to determine which is the most efficient for
any input
Exercises: 60

 Calculate the running time T(n) and then deduce the complexity in
terms of Big-O of the following programs

 Redo the exercises to deduce the complexity by using the

simplification rules (without calculating T(n))
EXERCISES 61

 1. Sorting an array

def sort(a):
n = len(a)
for i in range(n-1):
imin = i
j=i+1
while j < n:
if a[j] < a[imin]:
imin = j
j += 1
a[i], a[imin] = a[imin], a[i]
 To calculate the running time T(n) of the given code, let's analyze the number of basic operations executed as a
function of the input size n.

 def sort(a): 62
 n = len(a) # 1 operation

 for i in range(n-1): # n-1 iterations

 imin = i # 1 operation

 j=i+1 # 1 operation

 while j < n: # n-1 iterations (worst case)

 if a[j] < a[imin]: # 1 comparison

 imin = j # 1 operation

 j += 1 # 1 operation

 a[i], a[imin] = a[imin], a[i] # 3 operations (swap)

 Total operations = 1 + (n-1) + 1 + 1 + (n-1) * (1 + 1 + 1) + 3

 = 5n - 3

 The running time T(n) can be expressed as a linear function of n:

 T(n) = 5n - 3
1. Outer loop: The outer loop iterates `n-1` times, where `n` is the size of the array `a`. This loop
compares each element with the subsequent elements to find the minimum element and places it in
63
the correct position.
2. Inner loop: The inner loop iterates `n-i-1` times in each iteration of the outer loop, where `i` is the
current iteration of the outer loop. This loop finds the index of the minimum element in the unsorted
portion of the array.
Based on the simplification rules, we can deduce the complexity of the algorithm as follows:
- The outer loop has a time complexity of O(n) since it iterates `n-1` times.
- The inner loop has a maximum time complexity of O(n) as well. Although it iterates `n-i-1` times in
each iteration of the outer loop, the average number of iterations decreases as the outer loop
progresses. However, we consider the worst-case scenario, in which the inner loop iterates `n-i-1`
times for each iteration of the outer loop.

The assignment statement `a[i], a[imin] = a[imin], a[i]` is a constant-time operation and does not
contribute significantly to the overall complexity.
 Considering the worst-case scenario, where the outer loop iterates `n-1` times and the inner loop
iterates a maximum of `n-i-1` times for each iteration, we can deduce the overall complexity of
the algorithm as O(n^2).
 Therefore, the complexity of the `sort` function using selection sort is O(n^2).
Exercises(Linear search) 64

def linearsearch(a, x):

for i in range(len(a)):
if a[i] == x:
return i
return -1

a = [2, 44, 55, 66, 78, 90, 87, 112]

x = eval(input("enter ur search number"))

ans = linearsearch(a, x)
print("index is : ", ans)
To calculate the running time T(n) of the given code, let's analyze the number of
basic operations executed as a function of the input size n.
65

def linearsearch(a, x):

for i in range(len(a)): # n iterations
if a[i] == x: # 1 comparison
return i # 1 operation
return -1 # 1 operation

Total operations = n * (1 + 1) + 1
= 2n + 1

The running time T(n) can be expressed as a linear function of n:

T(n) = 2n + 1

Therefore, the running time of the given code is T(n) = 2n + 1.

 66

Based on the simplification rules, we can deduce the complexity of the code as follows:
•The loop iterates n times, resulting in a complexity of O(n).
•The comparison and return statements are constant-time operations and do not impact the
overall complexity.

Therefore, we can conclude that the complexity of the given code is O(n) in terms of Big-O
notation.

Thus, the complexity of the code is linear, indicating that the running time grows linearly with the
size of the input array.
 Exercises:(Binary search) 67
def binarysearch(a, low, high, x): a = [2, 44, 55, 66, 78, 90, 87, 112]

if high >= low: x = eval(input("enter your search number"))

mid = high + low // 2 ans = binarysearch(a, 0, len(a) - 1, x)

if a[mid] == x: if ans != -1:

print("Element is present at index",
return mid
str(ans))
elif a[mid] > x: else:
print("Element is not present in array")
return binarysearch(a, low, mid - 1, x)
else:
return binarysearch(a, low, mid + 1, x)
else:
return -1
 68the
If we assume that each comparison, assignment, and arithmetic operation takes constant time, we can deduce
following:

•The initial comparison high >= low takes 1 operation.

•The arithmetic operation high + low // 2 takes 2 operations (addition and division).
•Each comparison a[mid] == x and a[mid] > x takes 1 operation.
•Each recursive call binarysearch(a, low, mid - 1, x) and binarysearch(a, low, mid + 1, x) has
a reduced problem size.

The recurrence relation for T(n) can be expressed as follows:

T(n) = 1 + 2 + 1 + T(n/2) + T(n/2) = 4 + 2T(n/2)

We can determine that the time complexity of the binary search algorithm implemented in the given code is O(log
N).

Therefore, the running time T(n) of the given code can be expressed as T(n) = O(log N).
To determine the time complexity of the given code, let's analyze the number of basic operations executed as a function of
the input size and deduce the complexity using simplification rules:
def binarysearch(a, low, high, x): 69
if high >= low: # 1 comparison
mid = high + low // 2 # 2 operations
if a[mid] == x: # 1 comparison
return mid # 1 operation
elif a[mid] > x: # 1 comparison
return binarysearch(a, low, mid - 1, x) # Recursion with reduced problem size
else:
return binarysearch(a, low, mid + 1, x) # Recursion with reduced problem size
else:
return -1 # 1 operation
 70

Using the simplification rules, we can deduce the complexity as follows:

- The code uses binary search, which divides the problem size in half at each recursive call.
- In the worst-case scenario, the algorithm continues dividing the problem size until it reaches a single
element or determines that the element is not present.
- The number of recursive calls is proportional to the logarithm of the input size, specifically log2(N),
where N is the size of the array.
- Each recursive call involves a constant number of operations (comparisons and assignments).

Therefore, the time complexity of the binary search algorithm implemented in the given code is O(log
N) in terms of Big-O notation.

This complexity indicates that the algorithm's running time grows logarithmically with the size of the
input array, making it much more efficient than linear search for large sorted arrays.