Complexity of conventional Sorting

Algorithms
Course Code: CSC2211 Course Title: Algorithms

Dept. of Computer Science

Faculty of Science and Technology

Lecturer No: 02 Week No: 02 Semester: Spring 22-23

Lecturer: Md. Faruk Abdullah Al Sohan; [email protected]
Lecture Outline

1. Sorting Algorithms
 Merge Sort
Divide and Conquer

Recursive in structure

 Divide the problem into independent sub-problems that are similar

to the original but smaller in size
 Conquer the sub-problems by solving them recursively. If they are
small enough, just solve them in a straightforward manner. This can
be done by reducing the problem until it reaches the base case,
which is the solution.
 Combine the solutions of the sub-problems to create a solution to
the original problem
Merge Sort

Sorting Problem: Sort a sequence of n elements into non-

decreasing order.
 Divide: Divide the n-element sequence to be sorted into
two subsequences of n/2 elements each
 Conquer: Sort the two subsequences recursively using
merge sort.
 Combine: Merge the two sorted subsequences to produce
the sorted answer.
 Merge Sort Algorithm
Step 1 − if it is only one element in the list it is already sorted, return.
Step 2 − divide the list recursively into two halves until it can no more be divided.
Step 3 − merge the smaller lists into new list in sorted order.
Merge Sort Algorithm
Merge(L,R,A) {
   nL = length(L)
Mergesort(A)    nR = length(R)
{    i= j = k = 0
   n = length(A)    while(i< nL && j< nR) {
   if(n<2)       if(L[i] <= R[j]) {
return          A[k] = L[i]
   mid = n/2          i= i+1
   L = array of size(mid)       }
   R = array of size(n-mid)       else {
   for(i=0; i < mid; i++)             A[k] = R[j]
      L[i] = A[i]             j= j+1
   for(i=mid; i < n; i++)       }
      R[i-mid] = A[i]       k= k+1
   Mergesort(L)    }
   Mergesort(R)    while( i<nL) {
   Merge(L , R , A)       A[k] = L[i]; i=i+1; k=k+1;
}    }
   while (j<nR) {
      A[k] = R[i]; j=j+1; k=k+1;
   }
}
Execution Example

 Partition
7 2 9 43 8 6 1

7
Execution Example (cont.)

 Recursive call, partition

7 2 9 43 8 6 1

7 29 4

8
Execution Example (cont.)

 Recursive call, partition

7 2 9 43 8 6 1

7 29 4

72

9
Execution Example (cont.)

 Recursive call, base case

7 2 9 43 8 6 1

7 29 4

72

77

10
Execution Example (cont.)

 Recursive call, base case

7 2 9 43 8 6 1

7 29 4

72

77 22

11
Execution Example (cont.)

 Merge
7 2 9 43 8 6 1

7 29 4

722 7

77 22

12
Execution Example (cont.)

 Recursive call, …, base case, merge

7 2 9 43 8 6 1

7 29 4

722 7 9 4  4 9

77 22 99 44

13
Execution Example (cont.)

 Merge
7 2 9 43 8 6 1

7 29 4 2 4 7 9

722 7 9 4  4 9

77 22 99 44

14
Execution Example (cont.)

 Recursive call, …, merge, merge

7 2 9 43 8 6 1

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

15
Execution Example (cont.)

 Merge
7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

16
 7 2 9 43 8 6 1  1 2 3 4 6 7 8 9

7 29 4 2 4 7 9 3 8 6 1  1 3 6 8

722 7 9 4  4 9 3 8  3 8 6 1  1 6

77 22 99 44 33 88 66 11

17
Merge(L,R,A) {
   nL = length(L)
   nR = length(R)
   i= j = k = 0
   while(i< nL && j< nR) {
      if(L[i] <= R[j]) {
         A[k] = L[i] n*c0+ n*c1+n*c2+ c4
         i= i+1
      }
=n*(c0+c1+c2) + c4
      else { =n*c3 + c4
            A[k] = R[j]
            j= j+1
=n (Linear time)
      } Because of having no nested loops
      k= k+1
   }
   while( i<nL) {
      A[k] = L[i]; i=i+1; k=k+1;
   }
   while (j<nR) {
      A[k] = R[i]; j=j+1; k=k+1;
   }
}
   nL = length(L)
   nR = length(R)
   i= j = k = 0

i=0 j=0

L 14 23 45 98 R 6 33 42 67
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=0 j=0

L 14 23 45 98 R 6 33 42 67

A
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=0 j=1

L 14 23 45 98 R 6 33 42 67

A 6

k=0
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=1 j=1

L 14 23 45 98 R 6 33 42 67

A 6 14

k=1
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=2 j=1

L 14 23 45 98 R 6 33 42 67

A 6 14 23

k=2
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1;
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=2 j=2

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33

k=3
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=2 j=3

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42

k=4
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=3 j=3

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45

k=5
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=3 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67

k=6
   while(i< nL && j< nR) {
      If(L[i] <= R[j]) {
         A[k] = L[i]
         i= i+1
      }
      else {
         A[k] = R[j]
         j= j+1
      }
      k= k+1
   }
i=3 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67

k=6
while( i<nL) {
      A[k] = L[i];
i=i+1;
k=k+1;
   }
while (j<nR) {
      A[k] = R[i];
j=j+1;
k=k+1;
   }
i=3 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67

k=6
while( i<nL) {
    A[k] = L[i];
i=i+1;
k=k+1;
   }
while (j<nR) {
   A[k] = R[i];
j=j+1;
k=k+1;
   }
i=4 j=4

L 14 23 45 98 R 6 33 42 67

A 6 14 23 33 42 45 67 98

k=7
 Exercise

98 23 45 14 6 67 33 42
Merge Sort Analysis

1. The divide step takes constant time, regardless of the

subarray size. After all, the divide step just computes the
midpoint q of the indices p and r. Recall that in big-O
notation, we indicate constant time by O(1).

2. The conquer step, where we recursively sort two

subarrays of approximately n/2 elements each, takes
some amount of time, but we'll account for that time
when we consider the subproblems.

3. The combine step merges a total of n elements, taking

O(n) time.
Merge Sort Analysis
Substitution Method
Mergesort(A) T(n), to sort n elements
{
   n = length(A) c1
   if(n<2) return
   mid = n/2
   L = array of size(mid)
   R = array of size(n-mid)

   for(i=0; i < mid; i++) c2* (n/2) + c2* (n/2)

      L[i] = A[i] =c2( n/2 + n/2)
   for(i=mid; i < n; i++) =c2 * n
      R[i-mid] = A[i]

   Mergesort(L) T(n/2)
   Mergesort(R) T(n/2)
   Merge(L , R , A) c3.n+c4
}

So, T(n) = O(1) when n = 1, and

2T(n/2) + (c2+c3)*n + (c1+c4) when n > 1
So

Skipping the insignificant parts,

[]
[k= ]

[= n]
 Merge Sort Analysis
Recursion Tree Method
Recursion-tree Method

 Show successive expansions of recurrences using

trees.
 Keep track of the time spent on the subproblems of a
divide and conquer algorithm.
 Help organize the algebraic bookkeeping necessary to
solve a recurrence.
Recursion-tree Method for Merge Sort

For the original problem, Each of the size n/2 problems

we have a cost of cn, plus has a cost of cn/2 plus two
two subproblems each of subproblems, each costing
size (n/2) and running time T(n/4).
T(n/2).
cn

cn
Cost of divide
and merge.
cn/2 cn/2

T(n/2) T(n/2)
T(n/4) T(n/4) T(n/4) T(n/4)
Cost of sorting
subproblems.
Recursion Tree for Merge Sort

Continue expanding until the problem size reduces to 1.

cn
•Each level has total cost cn.

•Each time we go down one level, the number of

sub-problems doubles, but the cost per sub-
cn/2 cn/2
problem halves  cost per level remains the
same.

cn/4 cn/4 cn/4 cn/4 •There are lg n + 1 levels, height is lg n.

(Assuming n is a power of 2.)

•Total cost = sum of costs at each level = (lg n +

1)cn = cnlgn + cn = O(n lgn).
c c c c c c
Recursion-tree Method for Merge Sort
Elements to sort /
Recursion Tree for Merge Sort

𝒏 𝒏 =c.n
𝟐 𝟐

𝒏 𝒏 𝒏 𝒏
𝟒 𝟒 𝟒 𝟒 =c.n

𝒏 𝒏 𝒏 𝒏 𝒏 𝒏 𝒏 𝒏
𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 =c.n
Recursion Tree for Merge Sort

n=8

𝒏 𝒏
c.n
𝟐 𝟐

𝒏 𝒏 𝒏 𝒏
=c.n
𝟒 𝟒 𝟒 𝟒

𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖
=1 =1 =1 =1 =1 =1 =1 =1
𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 𝟖 =c.n

Continue expanding until the problem size reduces to 1.

So , if n=8 we need 3cn
n=8 3 levels we need 3.cn

n=16 4 levels we need 4.cn

n=32 5 levels we need 5.cn

n=64 6 levels we need 6.cn

n=8 n= 3 levels we need 3.cn
n=16 n= 4 levels we need 4.cn
n=32 n= 5 levels we need 5.cn
n=64 n= 6 levels we need 6.cn
n=8 n= 3 levels we need 3.cn
n=16 n= 4 levels we need 4.cn
n=32 n= 5 levels we need 5.cn
n=64 n= 6 levels we need 6.cn
n n= levels . cn
Merge Sort Analysis
Master Method
The master method applies to recurrences of the form
T(n) = a T(n/b) + f (n) ,
where a ≥ 1, b > 1, and f is asymptotically positive.
Examples:
Examples:
Examples:

Þ and
Examples:
Examples:
Best Case Time Complexity: 𝛀(n*log n)
Worst Case Time Complexity: O(n*log n)
Average Time Complexity: ϴ(n*log n)

The time complexity of MergeSort is O(n*Log n) in all the 3

cases (worst, average and best) as the mergesort always
divides the array into two halves and takes linear time to
merge two halves.
Books

 Introduction to Algorithms, Thomas H. Cormen, Charle E.

Leiserson, Ronald L. Rivest, Clifford Stein (CLRS).

 Fundamental of Computer Algorithms, Ellis Horowitz, Sartaj

Sahni, Sanguthevar Rajasekaran (HSR)
References

 https://www.google.com/search?q=bubble+sort+step+by+step&sxsrf=AL
eKk01uxzgfT3Oy6k1Q3WxVnSpiIN8_4g:1587999728942&tbm=isch&sourc
e=iu&ictx=1&fir=vRwFsGwVfJ6pJM%253A%252CSzhhze6MPQr4cM%252C
_&vet=1&usg=AI4_-kSrEEXqwRL-PkHhVUtn7jNfF9dB6g&sa=X&ved=2ahU
KEwje0Pz974jpAhXRAnIKHWhMD2UQ_h0wAXoECAcQBg#imgrc=EN4Sdu7
veOWVoM&imgdii=eOqvCu85p9-eBM

 https://www.interviewcake.com/concept/java/counting-sort

 https://www.geeksforgeeks.org/counting-sort/

 https://www.hackerearth.com/practice/algorithms/sorting/quick-sort/tut
orial/