Divide-and-Conquer

7 29 4  2 4 7 9

72  2 7 94  4 9

77 22 99 44

Divide-and-Conquer 1
Outline and Reading
Divide-and-conquer paradigm (§5.2)
Review Merge-sort (§4.1.1)
Recurrence Equations (§5.2.1)
 Iterative substitution
 Recursion trees
 Guess-and-test
 The master method
Integer Multiplication (§5.2.2)

Divide-and-Conquer 2
Divide-and-Conquer
Divide-and conquer is a
general algorithm design
paradigm:
 Divide: divide the input data S in
two or more disjoint subsets S1,
S2 , …
 Recur: solve the subproblems
recursively
 Conquer: combine the solutions
for S1, S2, …, into a solution for S
The base case for the
recursion are subproblems of
constant size
Analysis can be done using
recurrence equations
Divide-and-Conquer 3
Merge-Sort Review
Merge-sort on an input
sequence S with n Algorithm mergeSort(S, C)
elements consists of Input sequence S with n
three steps: elements, comparator C
 Divide: partition S into Output sequence S sorted
two sequences S1 and S2 according to C
of about n/2 elements if S.size() > 1
each
(S1, S2)  partition(S, n/2)
 Recur: recursively sort S1
and S2 mergeSort(S1, C)
 Conquer: merge S1 and mergeSort(S2, C)
S2 into a unique sorted S  merge(S1, S2)
sequence

Divide-and-Conquer 4
Recurrence Equation
Analysis
The conquer step of merge-sort consists of merging two sorted
sequences, each with n/2 elements and implemented by means of
a doubly linked list, takes at most bn steps, for some constant b.
Likewise, the basis case (n < 2) will take at b most steps.
Therefore, if we let T(n) denote the running time of merge-sort:

 b if n  2
T (n)  
2T (n / 2)  bn if n  2
We can therefore analyze the running time of merge-sort by
finding a closed form solution to the above equation.
 That is, a solution that has T(n) only on the left-hand side.

Divide-and-Conquer 5
Iterative Substitution
In the iterative substitution, or “plug-and-chug,” technique, we
iteratively apply the recurrence equation to itself and see if we can
find a pattern: T ( n )  2T ( n / 2)  bn
 2( 2T ( n / 2 2 ))  b( n / 2))  bn
 2 2 T ( n / 22 )  2bn
 23 T ( n / 23 )  3bn
 2 4 T ( n / 24 )  4bn
 ...
 2i T ( n / 2i )  ibn
Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n.
So, T (n)  bn  bn log n
Thus, T(n) is O(n log n).
Divide-and-Conquer 6
The Recursion Tree
Draw the recursion tree for the recurrence relation and look for a
pattern:
 b if n  2
T (n)  
2T (n / 2)  bn if n  2
time
depth T’s size
0 1 n bn

1 2 n/2 bn

i 2i n/2i bn

… … … …

Total time = bn + bn log n

(last level plus all previous levels)
Divide-and-Conquer 7
Guess-and-Test Method
In the guess-and-test method, we guess a closed form solution
and then try to prove it is true by induction:
 b if n  2
T (n)  
2T (n / 2)  bn log n if n  2
Guess: T(n) < cn log n.
T ( n )  2T ( n / 2)  bn log n
 2( c( n / 2) log( n / 2))  bn log n
 cn (log n  log 2)  bn log n
 cn log n  cn  bn log n

Wrong: we cannot make this last line be less than cn log n

Divide-and-Conquer 8
Guess-and-Test Method,
Part 2
Recall the recurrence equation:
 b if n  2
T (n)  
2T (n / 2)  bn log n if n  2
Guess #2: T(n) < cn log2 n.
T (n)  2T (n / 2)  bn log n
 2(c(n / 2) log 2 (n / 2))  bn log n
 cn(log n  log 2) 2  bn log n
 cn log 2 n  2cn log n  cn  bn log n
 cn log 2 n
 if c > b.
So, T(n) is O(n log2 n).
In general, to use this method, you need to have a good guess
and you need to be good at induction proofs.
Divide-and-Conquer 9
Master Method
Many divide-and-conquer recurrence equations have
the form:
 c if n  d
T (n)  
aT (n / b)  f (n ) if n  d

The Master Theorem:

1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.

Divide-and-Conquer 10
Master Method, Example 1
The form: T (n)   c if n  d
aT (n / b)  f (n ) if n  d
The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
Example:
T (n)  4T (n / 2)  n
Solution: logba=2, so case 1 says T(n) is O(n2).

Divide-and-Conquer 11
Master Method, Example 2
The form: T (n)   c if n  d
aT (n / b)  f (n ) if n  d
The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
Example:
T (n)  2T (n / 2)  n log n
Solution: logba=1, so case 2 says T(n) is O(n log2 n).

Divide-and-Conquer 12
Master Method, Example 3
The form: T (n)   c if n  d
aT (n / b)  f (n ) if n  d
The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
Example:
T (n)  T (n / 3)  n log n
Solution: logba=0, so case 3 says T(n) is O(n log n).

Divide-and-Conquer 13
Master Method, Example 4
The form: T (n)   c if n  d
aT (n / b)  f (n ) if n  d
The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
Example:
T (n)  8T (n / 2)  n 2

Solution: logba=3, so case 1 says T(n) is O(n3).

Divide-and-Conquer 14
Master Method, Example 5
The form: T (n)   c if n  d
aT (n / b)  f (n ) if n  d
The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
Example:
T (n)  9T (n / 3)  n 3

Solution: logba=2, so case 3 says T(n) is O(n3).

Divide-and-Conquer 15
Master Method, Example 6
The form: T (n)   c if n  d
aT (n / b)  f (n ) if n  d
The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
Example:
T (n)  T (n / 2)  1 (binary search)
Solution: logba=0, so case 2 says T(n) is O(log n).

Divide-and-Conquer 16
Master Method, Example 7
The form: T (n)   c if n  d
aT (n / b)  f (n ) if n  d
The Master Theorem:
1. if f (n) is O(n logb a  ), then T (n) is (n logb a )
2. if f (n) is (n logb a log k n), then T (n) is (n logb a log k 1 n)
3. if f (n) is (n logb a  ), then T (n) is ( f (n)),
provided af (n / b)  f (n) for some   1.
Example:
T (n)  2T (n / 2)  log n (heap construction)
Solution: logba=1, so case 1 says T(n) is O(n).

Divide-and-Conquer 17
Iterative “Proof” of the
Master Theorem
Using iterative substitution, let us see if we can find a pattern:
T (n)  aT (n / b)  f (n)
 a(aT (n / b 2 ))  f (n / b))  bn
 a 2T (n / b 2 )  af (n / b)  f (n)
 a 3T (n / b 3 )  a 2 f (n / b 2 )  af (n / b)  f (n)
 ...
(logb n ) 1
a logb n
T (1)  a
i 0
i
f (n / b i )
(logb n ) 1
n logb a
T (1)   a
i 0
f ( ni
/ b i
)
We then distinguish the three cases as
 The first term is dominant
 Each part of the summation is equally dominant
 The summation is a geometric series
Divide-and-Conquer 18
Integer Multiplication
Algorithm: Multiply two n-bit integers I and J.
 Divide step: Split I and J into high-order and low-order bits
I  I h 2n / 2  I l
J  J h 2n / 2  J l
 We can then define I*J by multiplying the parts and adding:
I * J  ( I h 2n / 2  I l ) * ( J h 2n / 2  J l )
 I h J h 2n  I h J l 2n / 2  I l J h 2n / 2  I l J l
 So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2).
 But that is no better than the algorithm we learned in grade
school.

Divide-and-Conquer 19
An Improved Integer
Multiplication Algorithm
Algorithm: Multiply two n-bit integers I and J.
 Divide step: Split I and J into high-order and low-order bits
I  I h 2n / 2  I l
J  J h 2n / 2  J l
 Observe that there is a different way to multiply parts:
I * J  I h J h 2 n  [( I h  I l )( J l  J h )  I h J h  I l J l ]2 n / 2  I l J l
 I h J h 2 n  [( I h J l  I l J l  I h J h  I l J h )  I h J h  I l J l ]2 n / 2  I l J l
 I h J h 2 n  ( I h J l  I l J h )2 n / 2  I l J l
 So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by
the Master Theorem.
 Thus, T(n) is O(n1.585).
Divide-and-Conquer 20