Divide and Conquer

Like dynamic and greedy methods, Divide and Conquer is an algorithmic paradigm. A
typical Divide and Conquer algorithm solves a problem using following three steps.

1. Divide problem into several smaller sub problems

o Normally, the sub problems are similar to the original
2. Conquer the sub problems by solving them recursively
o Base case: solve small enough problems by brute force
3. Combine the solutions to get a solution to the sub problems
o And finally a solution to the original problem
4. Divide and Conquer algorithms are normally recursive.

The following algorithms are based on divide-and-conquer algorithm design paradigm −

 Merge Sort

 Quick Sort

 Binary Search

Binary search

15 27 33 83 92 99

Search=99, low=0, high=5, mid= (low + high)/2=0+5=2

Pass 1

15 27 33 83 92 99
| Check for 99

Search=99, low=2+1=3, high=5, mid= (low + high)/2=3+5=4

Pass 2

15 27 33 83 92 99
| Check for 99

Search=99, low=5, high=5, mid= (low + high)/2=5+5=5

Pass 3
 15 27 33 83 92 99
| finds 99

Pseudo code for binary search

Binary Search (arr[],l, r, x)

{
If (r >= l) {
mid = ( l + r ) / 2;

If (arr[mid] == x)
Return mid;

If (arr[mid] > x)
Return Binary Search (arr, l, mid - 1, x);

Return binarySearch(arr, mid + 1, r, x);

}

Return -1;
}

Complexity of binary search

The recurrence relation for Binary search can be given as

T (n) = 1 for n<=2

Or

= T (n/2) +c for n>2

Let say the iteration in Binary Search terminates after k iterations. In the above example,
it terminates after 3 iterations, so here k = 3
At each iteration, the array is divided by half. So let’s say the length of array at any
iteration is n.
At Iteration 1,
Length of array = n
At Iteration 2,
Length of array = n⁄2
At Iteration 3,
Length of array = (n⁄2)⁄2 = n⁄22
Therefore, after Iteration k,
Length of array = n⁄2k
 Also, we know that after
After k divisions, the length of array becomes 1
Therefore
Length of array = n⁄2k = 1
=> n = 2k
Applying log function on both sides:
=> log2 (n) = log2 (2k)
=> log2 (n) = k log2 (2)
As (loga (a) = 1)
Therefore,
=> k = log2 (n)
Hence the time complexzity of Binary Search is
log2 (n)

Merge sort

Let's consider an array with values {14, 7, 3, 12, 9, 11, 6, and 12}

Below, we have a pictorial representation of how merge sort will sort the given array.
In merge sort we follow the following steps:

1. We take a variable p and store the starting index of our array in this. And we
take another variable r and store the last index of array in it.
2. Then we find the middle of the array using the formula (p + r)/2 and mark
the middle index as q, and break the array into two subarrays, from p to q and
from q + 1 to r index.
3. Then we divide these 2 subarrays again, just like we divided our main array
and this continues.
4. Once we have divided the main array into subarrays with single elements,
then we start merging the subarrays.

Pseudo code for merge sort

I/P: Unsorted array of size n

O/P: sorted array of size n
Mergesort (A, low, high)
{
If (low<high)
{
Mid= (low+high)/2
Mergesort (A, low, mid)
Mergesort (A, mid+1, high)
Merge (A, low, mid, high)
}
Merge (A, low, mid, high)
{
n1=mid-low+1
n2=high-mid
for (i=0; i<n1; i++)
L[i] =A[i]
for (j=0;j<n2;j++)
R[j]=A[j+mid+1]
L[i]=9999
R[j]=9999
i=0
j=0
For (k=low;k<=high;k++)
If (L[i] <=R[j])
A[k] =L[i]
I=i+1
Else
A[k] =R[j]
J=j+1
}

Complexity of merge sort

 1. First, we spend time O(1) for computing m.

 2. Then, we make two recursive calls to Merge Sort, with arrays of sizes ⌊(n − 1)/2⌋
and ⌈(n − 1)/2⌉.

3. Finally, we call Merge. Merge goes through the two subarrays with one loop, always
increasing one of i and j.

Time to merge two arrays each N/2 elements is linear, i.e. N

Thus we have:

(1) T(1) = 1

(2) T(N) = 2T(N/2) + N

Next we will solve this recurrence relation. First we divide (2) by N:

(3) T(N) / N = T(N/2) / (N/2) + 1

N is a power of two, so we can write

(4) T(N/2) / (N/2) = T(N/4) / (N/4) +1

(5) T(N/4) / (N/4) = T(N/8) / (N/8) +1

(6) T(N/8) / (N/8) = T(N/16) / (N/16) +1

(7) ……
(8) T(2) / 2 = T(1) / 1 + 1

After constituting equation (1) to (8) we get

T(N) = N + NlogN = O(NlogN)

Hence the complexity of the Merge Sort algorithm is O (NlogN).

Quik sort
Pseudo code
Complexity of quick sort:

The worst-case choice: the pivot happens to be the largest (or

smallest) item.

• Then one subarray is always empty.

• The second subarray contains n − 1 elements, i.e. all the

elements other than the pivot.

• Quicksort is recursively called only on this second group.

However, quicksort is fast on the “randomly scattered” pivots

• At each next step for n ≥ 1, the number of comparisons is

one less, so that T(n) = T(n − 1) + (n − 1); T(1) = 0.

• “Telescoping” T(n) − T(n − 1) = n − 1:

T(n)+T(n − 1)+T(n − 2)+. . .+T(3)+T(2)

−T(n − 1)−T(n − 2)−. . .−T(3)−T(2)− T(1)

= (n − 1) + (n − 2) +. . .+ 2 + 1 − 0

T(n)= (n − 1) + (n − 2) +. . .+ 2 + 1 =(n−1)n/2

This yields that T (n) ∈ Ω (n2).

For any pivot position i; i ∈ {0, . . . , n − 1}:

• Time for partitioning an array : cn

• The head and tail subarrays contain i and n − 1 − i items,

respectively: T(n) = cn + T(i) + T(n − 1 − i)

Best case
T(n) = partition(n) + 2*T(n/2) // partition(n) = n
= n + 2*T(n/2)
= 2*T(n/2) + n

T(1) = 1 // For the if-check operation

T(n) = 2*T(n/2) + n // T(n/2) = 2*T(n/4) + (n/2)

= 2*[ 2*T(n/4) + n/2 ] + n

= 22*T(n/4) + n + n
= 22*T(n/4) + 2n // T(n/4) = 2*T(n/8) + (n/4)

= 22*[ 2*T(n/8) + (n/4) ] + 2n

 = 23*T(n/8) + 22*(n/4) + 2n
= 23*T(n/8) + n + 2n
= 23*T(n/8) + 3n

= 24*T(n/16) + 4n
and so on....

= 2k*T(n/(2k)) + k*n // Keep going until: n/(2k) = 1 <==> n =

k
2

= 2k*T(1) + k*n
= 2k*1 + k*n
= 2k + k*n // n = 2k
= n + k*n
= n + (lg(n))*n
= n*( lg(n) + 1 )
~= n*lg(n))

Average case:

If the split induced by RANDOMIZED_PARTITION puts constant fraction of

elements on one side of the partition, then the recurrence tree has depth
(lgn) and (n) work is performed at (lg n) of these level. This is an intuitive
argument why the average-case running time of RANDOMIZED_QUICKSORT
is (n lg n).

Let T(n) denotes the average time required to sort an array of n elements. A call
to RANDOMIZED_QUICKSORT with a 1 element array takes a constant time, so
we have T(1) = (1).
After the split RANDOMIZED_QUICKSORT calls itself to sort two subarrays.
The average time to sort an array A[1 . . q] is T[q] and the average time to sort
an array A[q+1 . . n] is T[n-q]. We have
T(n) = 1/n (T(1) + T(n-1) + n-1∑q=1 T(q) + T(n-q))) +
(n) ----- 1
We know from worst-case analysis

T(1) = (1) and T(n -1) = O(n2)

T(n) = 1/n ( (1) + O(n2)) + 1/n n-1∑q=1 (r(q) + T(n - q)) + (n)
= 1/n n-1∑q=1(T(q) + T(n - q)) + (n) ------
-2
 = 1/n[2 n-1∑k=1(T(k)] + (n)
= 2/n n-1∑k=1(T(k) + (n) --------
-3
Solve the above recurrence using substitution method. Assume inductively
that T(n) ≤ anlgn + b for some constants a > 0 and b > 0.

If we can pick 'a' and 'b' large enough so that n lg n + b > T(1). Then for n >
1, we have
T(n) ≥ n-1∑k=1 2/n (aklgk + b) + (n)
= 2a/n n-1∑k=1 klgk - 1/8(n2) + 2b/n (n -1) + (n) -------
4
At this point we are claiming that
n-1
∑k=1 klgk ≤ 1/2 n2 lgn - 1/8(n2)
Stick this claim in the equation 4 above and we get

T(n) ≤ 2a/n [1/2 n2 lgn - 1/8(n2)] + 2/n b(n -1) + (n)

≤ anlgn - an/4 + 2b + (n) ---------
-5
In the above equation, we see that (n) + b and an/4 are polynomials and we
certainly can choose 'a' large enough so that an/4 dominates (n) +b.

We conclude that QUICKSORT's average running time is (n lg(n)).