Design and Analysis of Algorithm
DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING
REF BOOK: Horowitz, Salmi &
Rajasekaran
Merge Sort Algorithm
Algorithm MergeSort(a,low, high)
{
//a[low : high] array to be sorted
// Small(P) is true if there is only one element to sort. In this
case the list is already sorted
if (low < high) // If there are more than one element
{
// Divide P into subproblems. Find where to split the set.
mid := | (low + high)/2|;
// Solve the subproblems.
MergeSort(a,low, mid);
MergeSort(a,mid + 1, high);
// Combine the solutions.
Merge(a,low, mid, high);
}
}
9/15/2022 Fundamentals of Data Strcutures 2
Algorithm Merge(a,low, mid , high)
// a[low : high] is array containing two sorted
/ / subsets in a[ low; : mid] and in a[mid + 1 : high] .
The go is to merge these two sets into a single set residing
// in a[low : high] . b[ ] is an auxiliary global array.
{
h := low ; i := low ; j := mid + 1;
while ( ( h <= mid) and (j <= high)) do
{
if ( a[ h] <= a[j])
{
b[i] := a[h] ;h := h + 1;
}
else
{
b[i ] := a[j ] ; j:= j+ l ;
}
i:= i + 1;
}
if ( h > mid) then
for k := j to high do
{
b[i] := a[ k] ; i:= i + 1;
}
else
for k := h to mid do
REF BOOK: Horowitz, Salmi &
{ Rajasekaran
b[i] := a[ k] ; i:= i+ 1 ;
}
for k := low to high do a[ k] := b[ k] ;
}
REF BOOK: THOMAS CORMEN
Pseudocode of Merge-Sort (A, p, r)
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers
MergeSort (A, p, r) // sort A[p..r] by divide & conquer
1 if p < r
2 then q ← ⎣(p+r)/2⎦
3 MergeSort (A, p, q)
4 MergeSort (A, q+1, r)
5 Merge (A, p, q, r) // merges A[p..q] with A[q+1..r]
Initial Call: MergeSort(A, 1, n)
Ref Book: Thomas Cormen
Procedure Merge REF BOOK: THOMAS CORMEN
Merge(A, p, q, r)
1 n1 ← q – p + 1
2 n2 ← r – q Input: Array containing sorted
3 for i ← 1 to n1 subarrays A[p..q] and A[q+1..r].
4 do L[i] ← A[p + i – 1] Output: Merged sorted subarray
5 for j ← 1 to n2 in A[p..r].
6 do R[j] ← A[q + j]
7 L[n1+1] ← ∞
8 R[n2+1] ← ∞
9 i←1
10 j←1
11 for k ←p to r
12 do if L[i] ≤ R[j]
13 then A[k] ← L[i] Sentinels, to avoid having to
14 i←i+1 check if either subarray is
15 else A[k] ← R[j] fully copied at each step.
16 j←j+1
Ref Book: Thomas Cormen
Idea of Quick Sort Ref Book: Internet
1) Select: pick an element
2) Divide: rearrange
elements so that x goes to
its final position E
3) Recurse and Conquer:
recursively sort
Ref Book: Thomas
Cormen
Quicksort Pseudo-code
INPUT: a sequence of n numbers stored in array A
OUTPUT: an ordered sequence of n numbers
QuickSort (A, p, r) // sort A[p..r] by divide & conquer
1 if p < r
2 then q = Partition (A, p, r)
3 QuickSort (A, p, q-1)
4 QuickSort (A, q+1, r)
Initial Call: QuickSort(A,
1, n)
Procedure Partitioning the array Ref Book: Thomas
Cormen
Input: Array containing
Partition(A, p, r) sorted subarrays A[p..q]
1 x = A[r] and A[q+1..r].
2 i=p-1 Output: sorted subarray in
A[p..r].
3 for j ← p to r-1
4 if A[j] ≤ x
5 i= i +1
6 exchange A[i] with A[j]
7 exchange A[i+1] with A[j]
8 Return i + 1
REF BOOK: THOMAS CORMEN
REF BOOK: THOMAS CORMEN
Quicksort Analysis
Assumptions:
◦ A random pivot (no median-of-three partitioning)
◦ No cutoff for small arrays
Running time
◦ pivot selection: constant time, i.e. O(1)
◦ partitioning: linear time, i.e. O(N)
◦ running time of the two recursive calls
T(N)=T(i)+T(N-i-1)+cN where c is a constant
◦ i: number of elements in S1
REF BOOK: THOMAS CORMEN
Worst-Case Analysis
worst case Partition?
◦ The pivot is the smallest element, all the time
◦ Partition is always unbalanced
REF BOOK: THOMAS CORMEN
Best-case Analysis
best case Partitioning?
◦ Partition is perfectly balanced.
◦ Pivot is always in the middle (median of the array)
REF BOOK: THOMAS CORMEN
Average-Case Analysis
Intution for Average Case : We can get an idea of average case by considering the case when
partition puts O(n/9) elements in one set and O(9n/10) elements in other set.
Following is recurrence for this case.
T(n) = T(n/9) + T(9n/10) + ϴ(n)
Solution of above recurrence is also O(nlogn)
REF BOOK: THOMAS CORMEN
Summary Analysis of Quicksort
Best case: split in the middle — Θ(n log n)
Worst case: sorted array! — Θ(n2)
Average case: random arrays — Θ(n log n)
Improvements:
◦ better pivot selection: median of three partitioning
◦ switch to insertion sort on small subfiles
Multiplication of Large Integers
Consider the problem of multiplying two (large) n-digit integers
represented by arrays of their digits such as:
A = 12345678901357986429 B = 87654321284820912836
The grade-school algorithm:
a1 a2 … an
b1 b2 … bn
(d10) d11d12 … d1n
(d20) d21d22 … d2n
…………………
(dn0) dn1dn2 … dnn
Efficiency: Θ(n2) single-digit multiplications
A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. nd
ed., Ch. 4 4-15
First Divide-and-Conquer Algorithm
A small example: A * B where A = 2135 and B = 4014
A = (21·102 + 35), B = (40 ·102 + 14)
So, A * B = (21 ·102 + 35) * (40 ·102 + 14)
= 21 * 40 ·104 + (21 * 14 + 35 * 40) ·102 + 35 * 14
In general, if A = A1A2 and B = B1B2 (where A and B are n-digit,
A1, A2, B1, B2 are n/2-digit numbers),
A * B = A1 * B1·10n + (A1 * B2 + A2 * B1) ·10n/2 + A2 * B2
Recurrence for the number of one-digit multiplications M(n):
M(n) = 4M(n/2), M(1) = 1
Solution: M(n) = n2
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 4 4-16
Second Divide-and-Conquer Algorithm
A * B = A1 * B1·10n + (A1 * B2 + A2 * B1) ·10n/2 + A2 * B2
The idea is to decrease the number of multiplications from 4 to 3:
(A1 + A2 ) * (B1 + B2 ) = A1 * B1 + (A1 * B2 + A2 * B1) + A2 * B2,
I.e., (A1 * B2 + A2 * B1) = (A1 + A2 ) * (B1 + B2 ) - A1 * B1 - A2 * B2,
which requires only 3 multiplications at the expense of (4-1) extra
add/sub.
Recurrence for the number of multiplications M(n):
What if we count
M(n) = 3M(n/2), M(1) = 1 both multiplications
log 2n log 23 A. Levitin “Introduction
and additions?
Solution: M(n) = 3
Copyright © 2007 Pearson Addison-Wesley. All rights reserved.
=n ≈n 1.585 to the Design & Analysis of Algorithms,” 2 nd
ed., Ch. 4 4-17
Example of Large-Integer Multiplication
2135 * 4014
= (21*10^2 + 35) * (40*10^2 + 14)
= (21*40)*10^4 + c1*10^2 + 35*14
where c1 = (21+35)*(40+14) - 21*40 - 35*14, and
21*40 = (2*10 + 1) * (4*10 + 0)
= (2*4)*10^2 + c2*10 + 1*0
where c2 = (2+1)*(4+0) - 2*4 - 1*0, etc.
This process requires 9 digit multiplications as opposed to 16.
Copyright © 2007 Pearson Addison-Wesley. All rights reserved. A. Levitin “Introduction to the Design & Analysis of Algorithms,” 2nd ed., Ch. 4 4-18