Algorithms

RUN TIME CALCULATION

ASYMPTOTIC NOTATION
REVIEW: RUNNING TIME

 Number of primitive operations that are

executed:
Except for time of executing a function call
most statements roughly require the same
amount of time

We can be more exact if need be.

AN EXAMPLE: INSERTION SORT
Input: Array A of size n
Output: Sorted array in increasing order
InsertionSort(A, n) {
for i ← 2 to n {
key ← A[i]
j ← i - 1;
while (j > 0) and (A[j] > key) {
A[j+1] ← A[j]
j←j-1
}
A[j+1] ← key
}
}
INSERTION SORT
Statement Effort
InsertionSort(A, n) {
for j = 2 to n { c1n
key = A[j] c2(n-1)
i = j - 1; c3(n-1)
while (i > 0) and (A[i] > key) { c4Tj
A[i+1] = A[i] c5(Tj-(n-1))
i=i-1 c6(Tj-(n-1))
} 0
A[i+1] = key c7(n-1)
} 0
}
Tj= t2 + t3 + … + tn where tj is number of times you shifting the
element in right side to insert new number.
 SORT
 T(n) = c1n + c2(n-1) + c3(n-1) + c4Tj + c5(Tj - (n-1)) +
c6(Tj-(n-1))+c7(n-1)

T(n) = c1n + c2 (n − 1) + 0 (n − 1) + c3 (n − 1) + c4 ∑2 ≤ j ≤ n ( Tj )

+ c5 ∑2 ≤ j ≤ n (Tj − 1) + c6 ∑2 ≤ j ≤ n (Tj − 1)
+ c7 (n − 1)                               

 What can Tj be?

 Best case –
 Worst case-
 Average case-
 SORT
T(n) = c1n + c2 (n − 1) + 0 (n − 1) + c3 (n − 1) + c4 ∑2 ≤ j ≤ n ( Tj )
+ c5 ∑2 ≤ j ≤ n (Tj − 1) + c6 ∑2 ≤ j ≤ n (Tj − 1)
+ c7 (n − 1)                               

 What can Tj be?

Best case –
 The best case occurs if the array is already sorted.
 Therefore, tj = 1 for j = 2, 3, ..., n and the best-case running time can be
computed using above equation as follows:
 This running time can be expressed as An + B for constants A
and B that depend on the statement costs ci.
 Therefore, T(n) it is a linear function of n.
 SORT
T(n) = c1n + c2 (n − 1) + 0 (n − 1) + c3 (n − 1) + c4 ∑2 ≤ j ≤ n ( Tj )
+ c5 ∑2 ≤ j ≤ n (Tj − 1) + c6 ∑2 ≤ j ≤ n (Tj − 1)
+ c7 (n − 1)                               

 What can Tj be?

Worst case–
 The worst case occurs if the array is reverse sorted.
 Therefore, tj = 1 for j = 2, 3, ..., n and the worst-case running time can be
computed using above equation.
 This running time can be expressed as An2 + Bn + C for
constants A , B and C that depend on the statement costs ci.
 Therefore, T(n) it is a quadratic function of n.
ASYMPTOTIC NOTATIONS
 ASYMPTOTIC PERFORMANCE
 Suppose we are considering two algorithms, A and B, for solving
a given problem.
 Running times of each of the algorithms are T A(n) and TB(n)
respectively, where n is a measure of the problem size.

 Which is better Algorithm:

 If we know the problem size n in advance and then if TA(n)<
TB(n) conclude that algorithm A is better than Algorithm B.
 Unfortunately, we usually don't know the problem size
beforehand.
ASYMPTOTIC PERFORMANCE
 Asymptotic performance: How does an Algorithm
behave as the problem (input) size gets very large?
Running time

Memory/storage requirements
 ALGORITHMS WITH SAME
COMPLEXITY

•Represent Complexities of both algorithms in the form of

function.

•For large value of n (input size ) Two algorithms have

same complexity, if the functions representing the
number of operations have same rate of growth.
ASYMPTOTIC NOTATION
• Fortunately, there are asymptotic notations which
allow us to characterize the main factors affecting
an algorithm’s running time without going into
detail

• A good notation for large inputs.

• The notations describe different rate-of-growth
relations between the defining function and the
 defined set of functions.
Big-Oh O(.)  Little-Oh o(.)
 Big-Omega (.)  Little-Omega (.)
 Big-Theta (.)
ASYMPTOTIC UPPER BOUND-BIG
OH

Consider a function f(n) which is non-negative for all

integers  . Consider another function g(n) also.

We say that ``f(n) is big oh g(n),'' which we

write f(n)=O(g(n)), if there exists an integer  n0  and a
constant c>0 such that for all integers 

0  f(n)  c .g(n) for all n ≥ n0

ASYMPTOTIC UPPER BOUND-BIG
OH

Let f(n) and g(n) represents complexities of two

algorithms.
f(n) is said to be in the family of g(n) iff
0  f(n)  c .g(n) for all n ≥ n0
BIG O NOTATION
Example
If f(n) = 2n2
g(n) is O(n3 )

Que: Show that f(n) is Big Oh g(n)

Ans: to prove this we have to show that 0  f(n)  c g(n)
for all n ≥ n0
If we can find positive constants c and n0 then we can
say that f(n) is big oh g(n)
 BIG O NOTATION

 Example 1
If f(n) = 2n2 , g(n) is O(n3)
f(n)  c g(n)
 2n2 ≤ cn3  2 ≤ cn  c = 1 and n0= 2
 So, for c=1 and for all n ≥ 2, f(n) is big oh g(n)

 There is no unique set of values for no and c in proving the

asymptotic bounds but for whatever value of no you take, it should
satisfy for all n > no
 Or
 For c=2 and no = 4 also, f(n) is big oh g(n)
BIG O NOTATION

 Example 2
If f(n) = 8n+128, g(n) is O(n2)
To prove f(n) is big oh g(n) we have to prove that
f(n)  c g(n) for all n ≥ n0 and constant c
Lets take c=1

 Since (n+8)>0 for all values of  n ≥ 0 , so for n0=16

 Above rule is satisfied.
 So, for c=1 and for all n ≥ 16, f(n) is big oh g(n)
BIG-O NOTATION

When determining the order of a function f(x), always

try to pick the smallest g(x) possible.

If f(x) is (g(x)) and g(x) is (f(x)), then f(x) and g(x)

are of the same order.

In practice, all big-O results are obtained for functions

that are positive for all values of x.
ASYMPTOTIC LOWER BOUND: -
NOTATION

 BigO is asymptotic upper bound

 -notation is asymptotic lower bound
-NOTATION
 A family of functions we say ``f(n) is omega g(n),'' iff

{ f(n) : there exist positive constants c and no such that

0  c g(n)  f(n) for all n≥ n0 }
-NOTATION
Example:
For any two functions f(n) and g(n) we have
f(n) = 5n2 -64n+256 and g(n) = (n2)

so, f(n) = (g(n) if and only if c. g(n)  f(n)

Sol: in order to show this we need to find an

integer  n0  and a constant c>0 such that for all
integers  n≥ n0 f(n) ≥ cn2
 -NOTATION
Example:
For any two functions f(n) and g(n) we have
f(n) = 5n2 -64n+256 and g(n) = (n2)
suppose we choose c=1, then

So here (n-8)2 Is always positive for all n ≥ 0 so n0 = 0

Hence for c=1 and n0 =0 ``f(n) is omega g(n),''
- NOTATION DEFINITION

( g(n) ) denotes a family of functions.

``f(n) is Theta g(n),'' if we can find constants c1, c2,

and n0 such that

0  c1 g(n)  f(n)  c2 g(n)

for all n ≥ n0
For any two functions f(n) and g(n) we have
f(n) = (g(n))
if and only if
f(n) = O(g(n)) and f(n) = (g(n))
 ASYMPTOTIC TIGHT BOUND
 This
is said to asymptotic tight
bound.

 For reasonably large values of

n, the function f(n) is within
the range of constant multiples
of g(n)

f(n) is bounded below as well

as above.
RELATIONS BETWEEN
NOTAIONS
- NOTATION
 Example

If f(n) = 7n2 – 54n +3

g(n) is  (n2 )

Que: Show that f(n) is Theta g(n).

Ans: If f(n) is also having the same running time as g(n)
then prove following rule: 0  c1 g(n)  f(n)  c2 g(n)
for all n ≥ n0
Now find c1, c2 and n0
 We want c1, c2 and n0 such
7n2 – 54n +3 that
c1n2 7n2 – 54n +3  c2n2
for all n ≥ n0

Select c1 < 7 and 7< c2

Say c1 = 5 and c2 = 9

To get 5n2 7n2 – 54n +3,

n must be bigger than 27.

7n2 – 54n +3  9n2 is true for all n.

Thus c1 = 5, c2 = 9 and n0 = 28
TAKE ANOTHER EXAMPLE

If f(n) = 7n3 – 54n2 + 10n + 3

Can g(n) be n2 ?

NO
c2n2 cannot be larger than n3 for large n.
Thus you cannot find c1, c2 and n0
Thus f(n)   (n2 )
LITTLE O-NOTATION

o(g(n) ={ f(n):
for any positive constant c > 0 there exists
a constant n0 > 0 such that
0 ≤ f(n) < cg(n) for all n  n0 }

Limit f/g  0
LITTLE -NOTATION

(g(n) ={ f(n):
for any positive constant c > 0 there exists
a constant n0 > 0 such that
0 ≤ cg(n) < f(n) for all n  n0 }

Limit g/f  0
NAMES OF MOST COMMON BIG
OH FUNCTIONS
 Constant O(1)
 Logarithmic O(log n)

 Linear O(n)

 Quadratic O(n2)

 Polynomial O(nk) k is a constant

 Exponential O(2n)
 Exponential O(an) a is a constant and a > 1
THEOREM
 R1: If d(n) is O(f(n)), then ad(n) is O(f(n), a > 0

 R2:If d(n) is O(f(n)) and e(n) is O(g(n)), then d(n)

+e(n) is O(f(n)+g(n))

 R3: If d(n) is O(f(n)) and e(n) is O(g(n)), then

d(n)e(n) is O(f(n)g(n))

 R4: If d(n) is O(f(n)) and f(n) is O(g(n)), then d(n)

is O(g(n))
 THEOREM

 R5: If f(n) = a0 + a1n + … + adnd, d and ak are

constants, then f(n) O(nd)

 R6: nx is O(an) for any fixed x > 0 and a > 1

 R7: log nx is O(log n) for any fixed x > 0

 R8:
log xn is O(ny) for any fixed constants x > 0 and
y>0
SOME HELPFUL MATHEMATICS
 Constant Series: For integers a and b, a  b,
b

1  b  a  1
i a

 Linear Series (Arithmetic Series): For n  0,

n
n(n  1)

i 1
i  1  2   n 
2
 Quadratic Series: For n  0,
n
n(n  1)(2n  1)
 i 2  12  22    n 2 
i 1 6
SOME HELPFUL
MATHEMATICS
 Cubic Series: For n  0,
n
n 2
( n  1) 2


i 1
i 3
 13
 2 3
   n 3

4

 Geometric Series: For real x  1,

n 1
n
x 1

k 0
x  1 x  x  x 
k 2 n

x 1
For |x| < 1,

1

k 0
x 
k

1 x
SOME HELPFUL
MATHEMATICS
 Cubic Series: For n  0,
n
n 2
( n  1) 2


i 1
i 3
 13
 2 3
   n 3

4

 Geometric Series: For real x  1,

n 1
n
x 1

k 0
x  1 x  x  x 
k 2 n

x 1
For |x| < 1,

1

k 0
x 
k

1 x