Sorting (Divide and Conquer)
“Divide and Conquer”
• Very important strategy in computer science:
– Divide problem into smaller parts
– Independently solve the parts
– Combine these solutions to get overall solution
• Idea 1: Divide array into two halves, recursively sort
left and right halves, then merge two halves
Mergesort
• Idea 2 : Partition array into items that are “small”
and items that are “large”, then recursively sort the
two sets Quicksort
Divide and Conquer Sorting -
2/19/03 2
Lecture 14
Mergesort
8 2 9 4 5 3 1 6
• Divide it in two at the midpoint
• Conquer each side in turn (by recursively
sorting)
• Merge two halves together
Divide and Conquer Sorting -
2/19/03 3
Lecture 14
Mergesort Example
8 2 9 4 5 3 1 6
Divide
8 2 9 4 5 3 1 6
4
Mergesort Example
8 2 9 4 5 3 1 6
Divide
8 2 9 4 5 3 1 6
Divide
8 2 9 4 5 3 1 6
Divide
5
Mergesort Example
8 2 9 4 5 3 1 6
Divide
8 2 9 4 5 3 1 6
Divide
8 2 9 4 5 3 1 6
Divide
1 element 8 2 9 4 5 3 1 6
Merge
2 8 4 9 3 5 1 6
Merge
2 4 8 9 1 3 5 6
Merge
1 2 3 4 5 6 8 9
6
Auxiliary Array
• The merging requires an auxiliary array.
2 4 8 9 1 3 5 6
Auxiliary array
7
Auxiliary Array
• The merging requires an auxiliary array.
2 4 8 9 1 3 5 6
Auxiliary array
1
8
Auxiliary Array
• The merging requires an auxiliary array.
2 4 8 9 1 3 5 6
Auxiliary array
1 2 3 4 5
9
Merging
i j normal
target
Left completed
copy i j
first
target
10
Merging
first
second i j Right completed
first
target
11
Merging
Merge(A[], T[] : integer array, left, right : integer) :
{
mid, i, j, k, l, target : integer;
mid := (right + left)/2;
i := left; j := mid + 1; target := left;
while i < mid and j < right do
if A[i] < A[j] then T[target] := A[i] ; i:= i + 1;
else T[target] := A[j]; j := j + 1;
target := target + 1;
if i > mid then //left completed//
for k := left to target-1 do A[k] := T[k];
if j > right then //right completed//
k : = mid; l := right;
while k > i do A[l] := A[k]; k := k-1; l := l-1;
for k := left to target-1 do A[k] := T[k];
}
12
Recursive Mergesort
Mergesort(A[], T[] : integer array, left, right : integer) : {
if left < right then
mid := (left + right)/2;
Mergesort(A,T,left,mid);
Mergesort(A,T,mid+1,right);
Merge(A,T,left,right);
}
MainMergesort(A[1..n]: integer array, n : integer) : {
T[1..n]: integer array;
Mergesort[A,T,1,n];
}
13
Iterative Mergesort
Merge by 1
Merge by 2
Merge by 4
Merge by 8
14
Iterative Mergesort
Merge by 1
Merge by 2
Merge by 4
Merge by 8
Merge by 16
Need of a last copy
15
Mergesort Analysis
• Let T(N) be the running time for an array of N
elements
• Mergesort divides array in half and calls itself
on the two halves. After returning, it merges
both halves using a temporary array
• Each recursive call takes T(N/2) and merging
takes O(N)
16
Mergesort Recurrence Relation
• The recurrence relation for T(N) is:
– T(1) < a
• base case: 1 element array constant time
– T(N) < 2T(N/2) + cN
• Sorting N elements takes
– the time to sort the left half
– plus the time to sort the right half
– plus an O(N) time to merge the two halves
• T(N) = O(n log n)
17
Quicksort
• Quicksort uses a divide and conquer strategy, but
does not require the O(N) extra space that
MergeSort does
– Partition array into left and right sub-arrays
• Choose an element of the array, called pivot
• the elements in left sub-array are all less than pivot
• elements in right sub-array are all greater than pivot
– Recursively sort left and right sub-arrays
– Concatenate left and right sub-arrays in O(1) time
18
“Four easy steps”
• To sort an array S
1. If the number of elements in S is 0 or 1, then
return. The array is sorted.
2. Pick an element v in S. This is the pivot value.
3. Partition S-{v} into two disjoint subsets, S1 = {all
values x≤v}, and S2 = {all values x≥v}.
4. Return QuickSort(S1), v, QuickSort(S2)
19
Quick Sort
1. Pick a “pivot”
2. Divide into less-than & greater-than pivot
3. Sort each side recursively
The steps of QuickSort
S 81
43
31 57 select pivot value
13 75
92 0
65 26
S1 0
S2 partition S
31 75
43 65
13
81
26 57 92
QuickSort(S1) and
S1 S2 QuickSort(S2)
0 13 26 31 43 57 65 75 81 92
S 0 13 26 31 43 57 65 75 81 92 S is sorted
21
Quicksort Partitioning
• Need to partition the array into left and right
sub-arrays
– the elements in left sub-array are ≤ pivot
– elements in right sub-array are ≥ pivot
22
Two key steps
• How to pick a pivot?
• How to partition?
Pick a pivot
• Use the first element as pivot
– if the input is random, ok
– if the input is presorted (or in reverse order)
• all the elements go into S2 (or S1)
• this happens consistently throughout the recursive calls
• Results in O(n2) behavior (Analyze this case later)
• Choose the pivot randomly
– generally safe
– random number generation can be expensive
Partitioning in-place
– Set pointers i and j to start and end of array
– Increment i until you hit element A[i] > pivot
– Decrement j until you hit elmt A[j] < pivot
– Swap A[i] and A[j]
– Repeat until i and j cross
– Swap pivot (at A[N-2]) with A[i]
25
Partitioning
partition (arr[], low, high)
{
pivot = arr[low];
i = (low - 1) // Index of smaller element
for (j = low; j <= high- 1; j++)
{
// If current element is smaller than the pivot
if (arr[j] < pivot)
{
i++; // increment index of smaller element
swap arr[i] and arr[j]
}
}
swap arr[i + 1] and arr[low])
return (i + 1)
}
26
Partitioning:Choosing the pivot
• One implementation (there are others)
27
QuickSort Example
i j
5 1 3 9 7 0 4 2 6 8
i j
5 1 3 9 7 0 4 2 6 8
i j
5 1 3 9 7 0 4 2 6 8
i j
5 1 3 2 7 0 4 9 6 8
•Move i to the right to be larger than pivot.
•Move j to the left to be smaller than pivot.
•Swap
i j
5 1 3 2 7 0 4 9 6 8
i j
5 1 3 2 7 0 4 9 6 8
i j
5 1 3 2 7 0 4 9 6 8 Swap I and j
i j
5 1 3 2 4 0 7 9 6 8
i j
5 1 3 2 4 0 7 9 6 8
j i
5 1 3 2 4 0 7 9 6 8
j i
0 1 4 2 4 5 7 9 6 8
S1 < pivot pivot S2 > pivot
Complexity of Quicksort?
Example
We are given array of n integers to sort:
40 20 10 80 60 50 7 30 100
Pick Pivot Element
There are a number of ways to pick the pivot element. In
this example, we will use the first element in the array:
40 20 10 80 60 50 7 30 100
pivot_index = 40 20 10 80 60 50 7 30 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
pivot_index = 40 20 10 80 60 50 7 30 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
pivot_index = 40 20 10 80 60 50 7 30 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
pivot_index = 40 20 10 80 60 50 7 30 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
pivot_index = 40 20 10 80 60 50 7 30 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
pivot_index = 40 20 10 80 60 50 7 30 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
pivot_index = 40 20 10 80 60 50 7 30 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
pivot_index = 40 20 10 30 60 50 7 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 60 50 7 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 60 50 7 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 60 50 7 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 60 50 7 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 60 50 7 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 60 50 7 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 40 20 10 30 7 50 60 80 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 7 20 10 30 40 50 60 80 100
4
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
Partition Result
7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
<= data[pivot] > data[pivot]
Recursion: Quicksort Sub-arrays
7 20 10 30 40 50 60 80 100
[0] [1] [2] [3] [4] [5] [6] [7] [8]
<= data[pivot] > data[pivot]
Quicksort Analysis
• Assume that keys are random, uniformly
distributed.
• Best case running time: O(n log2n)
• Worst case running time?
– Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
– Depth of recursion tree?
Quicksort Analysis
• Assume that keys are random, uniformly
distributed.
• Best case running time: O(n log2n)
• Worst case running time?
– Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
– Depth of recursion tree? O(n)
Quicksort Analysis
• Assume that keys are random, uniformly
distributed.
• Best case running time: O(n log2n)
• Worst case running time?
Quicksort: Worst Case
• Assume first element is chosen as pivot.
• Assume we get array that is already in order:
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
too_big_index too_small_index
1. While data[too_big_index] <= data[pivot]
++too_big_index
2. While data[too_small_index] > data[pivot]
--too_small_index
3. If too_big_index < too_small_index
swap data[too_big_index] and data[too_small_index]
4. While too_small_index > too_big_index, go to 1.
5. Swap data[too_small_index] and data[pivot_index]
pivot_index = 2 4 10 12 13 50 57 63 100
0
[0] [1] [2] [3] [4] [5] [6] [7] [8]
<= data[pivot] > data[pivot]
71
Using big-Θ notation, we get the same result as for merge sort: Θ(nlogn)
72
Divide and Conquer Sorting - Lecture
73
14
Average Case:
To do average case analysis, we need to consider all possible permutation
of array and calculate time taken by every permutation which doesn’t
look easy.
We can get an idea of average case by considering the case when partition
puts O(n/9) elements in one set and O(9n/10) elements in other set. Following
is recurrence for this case.
T(n) = T(n/9) + T(9n/10) + (n)
Solution of above recurrence is also O(nLogn)
Solution of above recurrence is also O(nLogn)
75
T(n) = T(n/9) + T(9n/10) + (n)
Quicksort Analysis
• Assume that keys are random, uniformly
distributed.
• Best case running time: O(n log2n)
• Worst case running time?
– Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
– Depth of recursion tree? O(n)
– Number of accesses per partition?
Quicksort Analysis
• Assume that keys are random, uniformly
distributed.
• Best case running time: O(n log2n)
• Worst case running time?
– Recursion:
1. Partition splits array in two sub-arrays:
• one sub-array of size 0
• the other sub-array of size n-1
2. Quicksort each sub-array
– Depth of recursion tree? O(n)
– Number of accesses per partition? O(n)
Pseudo-code
Input: an array a[left, right]
QuickSort (a, left, right) {
if (left < right) {
pivot = Partition (a, left, right)
Quicksort (a, left, pivot-1)
Quicksort (a, pivot+1, right)
}
}
Compare with MergeSort:
MergeSort (a, left, right) {
if (left < right) {
mid = divide (a, left, right)
MergeSort (a, left, mid-1)
MergeSort (a, mid+1, right)
merge(a, left, mid+1, right)
}
}
