Topic 2 - Divide and Conquer Method
Topic 2 - Divide and Conquer Method
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo hi
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo mid hi
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo hi
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo mid hi
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo hi
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo mid hi
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo
hi
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo
hi
mid
Binary Search Demo
Binary search. Given value and sorted array a[], find index i
such that a[i] = value, or report that no such index exists.
6 13 14 25 33 43 51 53 64 72 84 93 95 96 97
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14
lo
hi
mid
Decision Tree for Binary Search (n = 13)
Binary Search: Time Complexity Analysis
• Theorem: If n is in the range [2k-1, 2k), then BinSearch algorithm makes at
most k element comparisons for a successful search and either k – 1 or k
comparisons for an unsuccessful search
• Proof Hints:
• Consider the binary decision tree for the scenario
• Note that all successful searches end at a circular node
• Also note that all unsuccessful searches end at a square node
• If 2k-1 ≤ n < 2k, then
• All circular nodes are at levels 1, 2, …, k
• All square nodes are at levels k and k + 1
• Corollary: The Worst-case Time for a successful search is O(log n), and the
Worst-case Time for an unsuccessful search is Θ(log n)
Binary Search: Average Case Time Complexity
• Let us assume the following terms for a binary decision tree:
Internal path length (I): Sum of distances of all internal nodes form root
External path length (E): Sum of distances of all external nodes form root
• Then, using mathematical induction, we can show that
E = I + 2n ∙ ∙ ∙ (1)
• Let us also assume the following terms:
As(n): Average number of comparisons in a successful search
Au(n): Average number of comparisons in an unsuccessful search
Binary Search: Average Case Time Complexity
• Note that, number of comparisons needed to find an element is one more
than the distance of its node in decision tree from the root
As(n) = 1 + I/n ∙ ∙ ∙ (2)
• Also note that a binary decision tree with n internal node has n + 1 external
node, which yields
Au(n) = E/(n + 1) ∙ ∙ ∙ (3)
• Combining (1), (2) and (3):
As(n) = (1 + 1/n) Au(n) – 1 ∙ ∙ ∙ (4)
• As all external nodes are at levels k and k + 1, we derive from (3):
Au(n) = Θ(log n)
• Then, we conclude further from (4): As(n) = Θ(log n)
Analysis of Divide-and-Conquer Algorithms
• let T(n) be the running time on a problem of size n
• If problem size is sufficiently small, say n < c for a constant c, then
• Suppose our target D&C algorithm divide the whole problem into a
subproblems, each of which are of 1/b size of the original problem
• So, each subproblem takes T(n/b) time
• Also, let D(n) be the time to divide the problem and C(n) be the time
combine the solutions of the subproblem
• Then, for sufficiently large n, we have
Merge Sort: The Approach
• To sort an array A[p . . r]:
• Divide
• Divide the n-element sequence to be sorted into two subsequences of n/2
elements each
• Conquer
• Sort the subsequences recursively using merge sort
• When the size of the sequences is 1 there is nothing more to do
• Combine
• Merge the two sorted subsequences
How to Merge
Compare 12 and 11
First: (12, 16, 17, 20, 21, 27)
Second: (12, 19)
New: (9, 10, 11)
Compare 12 and 12
First: (16, 17, 20, 21, 27)
Second: (12, 19)
New: (9, 10, 11, 12)
21
How to Merge
Compare 16 and 12
First: (16, 17, 20, 21, 27)
Second: (19)
New: (9, 10, 11, 12, 12)
Compare 16 and 19
First: (17, 20, 21, 27)
Second: (19)
New: (9, 10, 11, 12, 12, 16)
22
How to Merge
Compare 17 and 19
First: (20, 21, 27)
Second: (19)
New: (9, 10, 11, 12, 12, 16, 17)
Compare 20 and 19
First: (20, 21, 27)
Second: ( )
New: (9, 10, 11, 12, 12, 16, 17, 19)
23
How to Merge
24
Merge Sort Tree
An execution of merge-sort is depicted by a binary tree
– each node represents a recursive call of merge-sort and stores
unsorted sequence before the execution and its partition
sorted sequence at the end of the execution
7 29 4 2 4 7 9
72 2 7 94 4 9
25
Execution Example
Partition
7 2 9 43 8 6 1
26
Execution Example (cont.)
7 29 4
27
Execution Example (cont.)
7 29 4
72
28
Execution Example (cont.)
7 29 4
72
77
29
Execution Example (cont.)
7 29 4
72
77 22
30
Execution Example (cont.)
Merge
7 2 9 43 8 6 1
7 29 4
722 7
77 22
31
Execution Example (cont.)
Recursive call, …, base case, merge
7 2 9 43 8 6 1
7 29 4
722 7 9 4 4 9
32
Execution Example (cont.)
Merge
7 2 9 43 8 6 1
7 29 4 2 4 7 9
722 7 9 4 4 9
33
Execution Example (cont.)
7 29 4 2 4 7 9 3 8 6 1 1 3 6 8
722 7 9 4 4 9 3 8 3 8 6 1 1 6
34
Execution Example (cont.)
Merge
7 2 9 43 8 6 1 1 2 3 4 6 7 8 9
7 29 4 2 4 7 9 3 8 6 1 1 3 6 8
722 7 9 4 4 9 3 8 3 8 6 1 1 6
35
Merge - Pseudocode p q r
Alg.: MERGE(A, p, q, r)
1 2 3 4 5 6 7 8
2 4 5 7 1 2 3 6
1. Compute n1 and n2
2. Copy the first n1 elements into n1 n2
L[1 . . n1 + 1] and the next n2 elements into R[1 . . n2 + 1]
3. L[n1 + 1] ← ; R[n2 + 1] ← p q
4. i ← 1; j ← 1 L 2 4 5 7
5. for k ← p to r q+1 r
R 1 2 3 6
6. do if L[ i ] ≤ R[ j ]
7. then A[k] ← L[ i ]
8. i ←i + 1
9. else A[k] ← R[ j ]
10. j←j+1
Mergesort - Pseudocode
p q r
1 2 3 4 5 6 7 8
Alg.: MERGE-SORT(A, p, r) 5 2 4 7 1 3 2 6
MERGE-SORT(A, p, q) Conquer
MERGE-SORT(A, q + 1, r) Conquer
MERGE(A, p, q, r) Combine
(1) if n =1
T(n) =
2T(n/2) + (n) if n > 1
Solve the Recurrence
c if n = 1
T(n) =
2T(n/2) + cn if n > 1
• Advantage:
• Guaranteed to run in (nlog2n)
• Disadvantage:
• Requires extra space (n)
Quicksort
• The problem:
• Conquer
• Recursively sort A[p..q] and A[q+1..r] using Quicksort
• Combine
• Trivial: the arrays are sorted in place
• No additional work is required to combine them
• The entire array is now sorted
Quicksort: Psedocode
if p < r
then q PARTITION(A, p, r)
QUICKSORT (A, p, q)
• Hoare partition
• Select a pivot element x around which to partition
• Grows two regions A[p…i] x x A[j…r]
A[p…i] x
x A[j…r]
i j
Quicksort: Partitioning the Array
• A key step in the Quicksort algorithm is partitioning the array
• We choose some (any) number p in the array to use as a pivot
• We partition the array into three parts:
Each element is
visited once!
Running time: (n)
n = right – left + 1
Quicksort: Recurrence Relation
Initially: p=1, r=n
Alg.: QUICKSORT(A, p, r)
if p < r
then q PARTITION(A, p, r)
QUICKSORT (A, p, q)
• Maximally unbalanced
n n
• Recurrence: q=1 1 n-1 n
1 n-2 n-1
T(n) = T(1) + T(n – 1) + n, n-2
n 1 n-3
T(1) = (1) 1
2 3
T(n) = T(n – 1) + n 1 1 2
n
n k 1 ( n) ( n2 ) ( n2 ) (n2)
= k 1
When does the worst case happen?
Best Case Partitioning
• Best-case partitioning
• Partitioning produces two regions of size n/2
• Recurrence: q=n/2
T(n) = 2T(n/2) + (n)
T(n) = (nlgn) (Master theorem)
Case Between Worst and Best
• 9-to-1 proportional split
Q(n) = Q(9n/10) + Q(n/10) + n
How Does Partition Affect Quicksort Performance?
How Does Partition Affect Quicksort Performance?
Worst-Case Analysis of Quicksort: Formal Proof
• T(n) = worst-case running time
T(n) = max (T(q) + T(n-q)) + (n)
1 ≤ q ≤ n-1
58
Worst-Case Analysis of Quicksort: Formal Proof
• Proof of induction goal:
T(n) ≤ max (cq2 + c(n-q)2) + (n) =
1 ≤ q ≤ n-1
• The expression q2 + (n-q)2 achieves a maximum over the range 1 ≤ q ≤ n-1 at one of the
endpoints
59
Quicksort: Choice of Pivot
1. Choose the of the array as pivot
• But, it can be very bad
• Why?
• Remedy??
i ← RANDOM(p, r)
return PARTITION(A, p, r)
Randomized Quicksort
Alg. : RANDOMIZED-QUICKSORT(A, p, r)
if p < r
then q ← RANDOMIZED-PARTITION(A, p, r)
RANDOMIZED-QUICKSORT(A, p, q)
RANDOMIZED-QUICKSORT(A, q + 1, r)
64
Example: Median3 Method for Choosing Pivot
Choose the pivot as the median of three
0 1 2 3 4 5 6 7 8 9
8 1 4 9 0 3 5 2 7 6
Median of 0, 6, 8 is 6. Pivot is 6
0 1 4 9 7 3 5 2 6 8
i j
Place the largest at the right and the smallest at the left
0 1 4 9 7 3 5 2 6 8
i j
0 1 4 9 7 3 5 2 6 8
i j
0 1 4 9 7 3 5 2 6 8
i j
0 1 4 2 7 3 5 9 6 8
0 1 4 2 7 3 5 9 6 8
i j
0 1 4 2 7 3 5 9 6 8
i j
0 1 4 2 5 3 7 9 6 8
i j
0 1 4 2 5 3 7 9 6 8
j i
0 1 4 2 5 3 7 9 6 8 Cross-over i > j
j i
0 1 4 2 5 3 6 9 7 8
y: c d y c 10 d
2
n n
x y (a 10 b)(c 10 d )
2 2
n
ac 10 n bd 10 (ad bc)
2
a = 14
p1= P'
x = 143 b=3
P y = 256
n=3 m=1
c = 25
p2= 18 P=36608
d=6
p3= P''
a=1
p1 = 2
x = 14 b=4
P' y = 25
n=2 m=1
c=2
p2 = 20 P'=350
d=5
p3 = 35
a=1
p1 = 3
x = 17 b=7
P'' y = 31
n=2 m=1
c=3
p2 = 7 P''=527
d=1
p3 = 32
Max-Min Problem: Divide and Conquer Approach
• Broad Idea:
• The array is divided into two halves
• The maximum and the minimum numbers in each halves are found using
recursive approach
• Return the maximum of two maxima of each half and the minimum of two
minima of each half
Max-Min Problem: Divide and Conquer Algorithm
• Max−Min(x,y) will return the maximum and minimum values of an array
numbers[x...y]
A Simple Example
n Finding the maximum of a set S of n numbers
4 -77
Time Complexity
n Time complexity:
T(n): # of comparisons
2T(n/2)+1 , n>2
T(n)=
1 , n2
n Calculation of T(n):
Assume n = 2k,
T(n) = 2T(n/2)+1
= 2(2T(n/4)+1)+1
= 4T(n/4)+2+1
:
=2k-1T(2)+2k-2+…+4+2+1
=2k-1+2k-2+…+4+2+1
=2k-1 = n-1 4 -78