DivideAndConquerFin
DivideAndConquerFin
Subproblem 1 Subproblem 2
of size n/2 of size n/2
Don’t assume
always breaks up
into 2, could be > 2
subproblems
Solution to Solution to
subproblem 1 subproblem 2
Solution to the
original probelm
Divide and Conquer (contd.)
• In “politics” divide and rule (latin:
divide et impera) is a combination of
political, military, and economic
strategy of gaining and maintaining
power by breaking up larger
concentrations of power into chunks
that individually have less power than
the one who is implementing the
strategy. (read more on wiki: “Divide
and rule”)
Divide and Conquer (contd.)
• Let us add n numbers using divide and
conquer technique
a0 + a1 + …… + an-1
a0 + …… + + …… + an-1
2 10 3 5 7 1 6 10 1 3
0 1 2 3 4 0 1 2 3 4
2 10 3 5 7 1 6 10 1 3
# of additions
is same as in
brute force,
2 10 3 5 7 needs stack 1 6 10 1 3
for recursion…
12 3 5 7 7 10 1 3
Bad!
not all divide
12 and conquer 4
works!!
15 14
27 21
Could be efficient
48 for parallel processors though….
Div. & Conq. (contd.)
• Usually in div. & conq., a problem instance of size n
is divided into two instances of size n/2
• More generally, an instance of size n can be
divided into b instances of size n/b, with a of
them needing to be solved
• Assuming that n is a power of b (n = bm), we get
– T(n) = aT(n/b) + f(n) general divide-and-conquer
recurrence
– Here, f(n) accounts for the time spent in dividing an
instance of size n into subproblems of size n/b and
combining their solution
– For adding n numbers, a = b = 2 and f(n) = 1
Div & Conq. (contd.)
• T(n) = aT(n/b)+f(n), a ≥ 1, b > 1
What if a = 1?
• Master Theorem: Have we seen it?
If f(n) є Θ(nd) where d ≥ 0 then
So, A(n) є Θ()
Θ(n )
d
if a < b d
Or, A(n) є Θ(n)
Here, a = ?, b = ?, d = ? a = 2, b = 2, d = 0
Which of the 3 cases holds ? a = 2 > bd = 20, case 3
Div. & Conq. (contd.)
T(n) = aT(n/b)+f(n), a ≥ 1, b > 1
If f(n) є Θ(nd) where d ≥ 0, then Θ(nd) if a < bd
T(n) = 2T(n/2)+6n-1?
T(n) є Θ(ndlgn) if a = bd
Θ() if a > bd
T(n) = 3 T(n/2) + n a = 3, b = 2, f(n) є Θ(n1), so d = 1
A[0……] A[……n-1]
sort sort
A[0……] A[……n-1]
merge
A[0……n-1]
Go on dividing recursively…
Div. & Conq.: Mergesort(contd.)
ALGORITHM Mergesort(A[0..n-1])
//sorts array A[0..n-1] by recursive mergesort
//Input: A[0..n-1] to be sorted
//Output: Sorted A[0..n-1]
if n > 1
copy A[0..-1] to B[0.. -1]
copy A[..n-1] to C[0..-1]
Mergesort(B[0..-1])
Mergesort(C[0..-1])
Merge(B, C, A)
B: 2 3 8 9 C: 1 4 5 7
A: 1 2 3 4 5 7 8 9
Div. & Conq.: Mergesort(contd.)
ALGORITHM Merge(B[0..p-1], C[0..q-1], A[0..p+q-1])
//Merges two sorted arrays into one sorted array
//Input: Arrays B[0..p-1] and C[0..q-1] both sorted
//Output: Sorted array A[0..p+q-1] of elements of B and C
i <- 0; j <- 0; k <- 0;
while i < p and j < q do
if B[i] ≤ C[j]
A[k] <- B[i]; i <- i+1
else
A[k] <- C[j]; j <- j+1
k <- k+1
if i = p
copy C[j..q-1] to A[k..p+q-1]
else
copy B[i..p-1] to A[k..p+q-1]
Div. & Conq.: Mergesort(contd.)
Divide:
8 3 2 9 7 1 5 4
Merge:
8 3 2 9 7 1 5 4
8 3 2 9 7 1 5 4
8 3 2 9 7 1 5 4
3 8 2 9 1 7 4 5
2 3 8 9 1 4 5 7
1 2 3 4 5 7 8 9
Div. & Conq.: Mergesort(contd.)
ALGORITHM Mergesort(A[0..n-1]) ALGORITHM Merge(B[0..p-1], C[0..q-1], A[0..p+q-1])
//sorts array A[0..n-1] by recursive //Merges two sorted arrays into one sorted array
mergesort //Input: Arrays B[0..p-1] and C[0..q-1] both sorted
//Output: Sorted array A[0..p+q-1] of elements of B
//Input: A[0..n-1] to be sorted //and C
//Output: Sorted A[0..n-1] i <- 0; j <- 0; k <- 0;
if n > a while i < p and j < q do
copy A[0..-1] to B[0.. -1] if B[i] ≤ C[j]
copy A[..n-1] to C[0..-1] A[k] <- B[i]; i <- i+1
Mergesort(B[0..-1]) else
Mergesort(C[0..-1]) A[k] <- C[j]; j <- j+1
k <- k+1
Merge(B, C, A) if i = p
copy C[j..q-1] to A[k..p+q-1]
What is the time-efficiency else
of Meresort? copy B[i..p-1] to A[k..p+q-1]
i j
p
i j
p all ≤ p ≥p ≤p all ≥ p
j i
p all ≤ p ≤p ≥p all ≥ p
j=i
p all ≤ p =p all ≥ p
Div. & Conqr.: Quicksort (contd.)
ALGORITHM HoarePartition(A[l..r])
//Output: the split position
p <- A[l] i could go out of array’s
i <- l; j <- r+1 bound, we could check
or we could put a “sentinel”
repeat at the end…
Do you see any repeat i <- i+1 until A[i] ≥ p
possible problem
with this pseudocode ? repeat j <- j-1 until A[j] ≤ p
swap( A[i], A[j] )
until i ≥ j
swap( A[i], A[j] ) // undo last swap when i ≥ j
swap( A[l], A[j] )
return j
More sophisticated pivot selection
that we shall see briefly makes this
“sentinel” unnecessary…
Div. & Conqr.: Quicksort (contd.)
i j i j
5 3 1 9 8 2 4 7 2 3 1 4 5 8 9 7
i j i j
5 3 1 9 8 2 4 7 2 3 1 4 5 8 9 7
1 2 3 4 5 8 9 7
i j i j
i j
5 3 1 4 8 2 9 7 2 1 3 4 5 8 9 7
1 2 3 4 5 8 9 7
i j j i
i j
5 3 1 4 8 2 9 7 2 1 3 4 5 8 9 7
1 2 3 4 5 8 7 9
i j
j i
5 3 1 4 2 8 9 7 1 2 3 4 5 8 9 7
1 2 3 4 5 8 7 9
j i
5 3 1 4 2 8 9 7 1 2 3 4 5 8 9 7
1 2 3 4 5 7 8 9
i=j
2 3 1 4 5 8 9 7 1 2 3 4 5 8 9 7
1 2 3 4 5 7 8 9
j i
1 2 3 4 5 8 9 7
1 2 3 4 5 7 8 9
Div. & Conqr.: Quicksort (contd.)
0 1 2 3 4 5 6 7
5 3 1 9 8 2 4 7 0 1 2 3 4 5 6 7
5 3 1 9 8 2 4 7
l=0,r=7 0 1 2 3 4 5 6 7
s=4 2 3 1 4 5 8 9 7
l=0,r=3 l=5,r=7
0 1 2 3 5 6 7
s=1 s=6
1 2 3 4 7 8 9
l=0,r=0 l=2,r=3 l=5,r=5 l=7,r=7
s=2 0 5
l=2,r=1 l=3,r=3 1 7
2 3 7
3 4 9
4
Div. & Conqr.: Quicksort (contd.)
• Let us analyze Quicksort
ALGORITHM Quicksort(A[l..r]) Time-complexity of this line ?
if l < r i j
s <- HoarePartition ( A[l..r] ) So, n+1 5 3 1 4 8 2 9 7
Quicksort( A[l..s-1] ) comparisons
when cross-over i j
Quicksort( A[s+1]..r )
5 3 1 4 2 8 9 7
If all splits ALGORITHM HoarePartition(A[l..r]) j i
happen in the //Output: the split position 5 3 1 4 2 8 9 7
So, n
middle, it is p <- A[l]
comparisons
the best-case! i <- l; j <- r+1 when coincide What if,
repeat i,j
repeat i <- i+1 until A[i] ≥ p
repeat j <- j-1 until A[j] ≤ p
5 3 1 4 5 8 9 7
swap( A[i], A[j] )
until i ≥ j
swap( A[i], A[j] ) // undo last swap when i ≥ j
swap( A[l], A[j] )
return j C (n) = 2C (n/2)+n for n > 1
best best
Cbest(1) = 0
Div. & Conqr.: Quicksort (contd.)
T(n) = aT(n/b)+f(n), a ≥ 1, b > 1
ALGORITHM Quicksort(A[l..r])
If f(n) є nd with d ≥ 0, then
if l < r
Θ(nd) if a < bd
s <- Partition( A[l..r] )
Quicksort( A[l..s-1] )
T(n) є Θ(ndlgn) if a = bd Quicksort( A[s+1]..r )
Θ() if a > bd j ij j j j
2 5 6 8 9 5+1=6
Cbest(n) = 2Cbest(n/2)+n for n > 1 i j
Cbest(1) = 0 5 6 8 9 4+1=5
i j
Using Master Thm, Cbest(n) є Θ(nlgn)
6 8 9 3+1=4
What is the worst-case ? ij
8 9 2+1=3
Cworst(n) = (n+1) + (n-1+1) + … + (2+1) = (n+1) + … + 3
= (n+1) + … + 3 + 2 + 1 – (2 + 1) = - 3
= - 3 є Θ(n2) !
So, Quicksort’s fate
depends on its average-case!
Div. & Conqr.: Quicksort (contd.)
• Let us sketch the outline of average-
case analysis… ALGORITHM Quicksort(A[l..r])
if l < r
s <- Partition( A[l..r] )
Cavg(n) is the average number of key-comparisons Quicksort( A[l..s-1] )
made by the Quicksort on a randomly ordered array Quicksort( A[s+1]..r )
of size n
After n+1 comparisons, a partition can Let us assume that
happen in any position s (0 ≤ s ≤ n-1) partition split can
happen in each position s
After the partition, left part has s elements, with equal probability 1/n
Right part has n-1-s elements
0 s n-1 Cavg(n) = Expected[ Cavg(s) + Cavg(n-1-s) + (n+1) ]
p
Average over all possibilities
s elements n-1-s elements Cavg(n) =
Cavg(0) = 0, Cavg(1) = 0
Cavg(n) ≈ 1.39nlgn
Div. & Conqr.: Quicksort (contd.)
• Recall that for Quicksort, Cbest(n) ≈ nlgn
• So, Cavg(n) ≈ 1.39nlgn is not far from Cbest(n)
• Quicksort is usually faster than Mergesort or
Heapsort on randomly ordered arrays of nontrivial
sizes
• Some possible improvements
• Randomized quicksort: selects a random element as pivot
• Median-of-three: selects median of left-most, middle, and
right-most elements as pivot
• Switching to insertion sort on very small subarrays, or not
sorting small subarrays at all and finish the algorithm with
insertion sort applied to the entire nearly sorted array
• Modify partitioning: three-way partition
c2 = a1*b1 c0 = a0*b0
c1 = (a1+a0)*(b1+b0)-(c2+c0)
Div. & Conq. : Multiplication of Large
Integers
If we have a pair of 2-digits numbers a and b
a = a1a0 and b = b1b0
we can write c = a*b = c2102+c1101+c0100
c2 = a1*b1 , c0 = a0*b0
a = 1234 = 1*103+2*102+3*101+4*100
= (12)102+(34)
c1 = (a1+a0)*(b1+b0)-(c2+c0)
(12)102+(34) = (1*101+2*100)102+3*101+4*100
If we have two n-digits numbers,
a and b (assume n is a power of 2) Apply the same idea
a: a1 a0 recursively to get c2, c1, c0
n/2 digits until n is so small that you can
b: b1 b0 you can directly multiply
c = a*b =
=
We can write, =
a = a110n/2 + a0
Why?
b = b110 n/2
+ b0
c2 = a1*b1 c0 = a0*b0
c1 = (a1+a0)*(b1+b0)-(c2+c0)
Div. & Conq. : Multiplication of Large
Integers
c = a*b =
Notice: a1, a0, b1, b0 all are n/2 5 additions
digits numbers 1 subtraction
c2 = a1*b1 c0 = a0*b0
So, computing a*b requires
three n/2-digits multiplications c1 = (a1+a0)*(b1+b0)-(c2+c0)
dmin
d d
p
Now comes the crucial observation, x=m
everything hinges on this one…
How many points can there d d One of these 8
being p, we need
be in the dmin-by-2d rectangle? d to check 7 pairs
to find if any pair
has distance < dmin
My claim is “this”
is the most you can
put in the rectangle…
Div. & Conq.: Closest pair (contd.)
ALGORITHM EfficientClosestPair(P, Q)
//Solves closest-pair problem by divide and conquer
//Input: An array P of n ≥ 2 points sorted by x-coordinates and another array Q of
same //points sorted by y-coordinates The algorithm spends linear time
//Output: Distance between the closest pair in dividing and merging, so assuming
if n ≤ 3 n = 2m, we have the following
return minimal distance by brute-force recurrence for runnning-time,
else
T(n) = 2T(n/2)+f(n) where f(n) є Θ(n)
copy first points of P to array Pl
copy the same points from Q to array Q l
copy the remaining points of P to array P r Applying Master Theorem,
copy the same points from Q to array Q r T(n) є Θ(nlgn)
dl <- EfficientClosestPair( Pl, Ql )
dr <- EfficientClosestPair( Pr, Qr )
Dividing line d <- min{ dl, dr } Brute froce
m <- P[-1].x Inside the
Points in 2*d
copy all points of Q for which |x-m| < d into array S[0..num-1] 2*d width strip
width strip
dminsq <- d2
for i <- 0 to num-2 do
k <- i+1
while k ≤ num-1 and ( S[k].y – S[i].y )2 < dminsq
dminsq <- min( (S[k].x-S[i].x)2+(S[k].y-S[i].y)2 , dminsq)
k <- k+1
return sqrt( dminsq ) // could easily keep track of the pair of points
Thank you…