Outline and Reading Divide-and-Conquer

7 29 4 2 4 7 9 72 2 7 77 22 94 4 9 99 44

Divide-and-conquer paradigm (5.2) Review Merge-sort (4.1.1) Recurrence Equations (5.2.1)

Iterative substitution Recursion trees Guess-and-test The master method

Integer Multiplication (5.2.2)

Divide-and-Conquer

Divide-and-Conquer

Divide-and-Conquer
Divide-and conquer is a general algorithm design paradigm:

Merge-Sort Review
Merge-sort on an input sequence S with n elements consists of three steps:

The base case for the recursion are subproblems of constant size Analysis can be done using recurrence equations

Divide: divide the input data S in two or more disjoint subsets S1, S2, Recur: solve the subproblems recursively Conquer: combine the solutions for S1, S2, , into a solution for S

Divide: partition S into two sequences S1 and S2 of about n/2 elements each Recur: recursively sort S1 and S2 Conquer: merge S1 and S2 into a unique sorted sequence

Algorithm mergeSort(S, C) Input sequence S with n elements, comparator C Output sequence S sorted according to C if S.size() > 1 (S1, S2) partition(S, n/2) mergeSort(S1, C) mergeSort(S2, C) S merge(S1, S2)

Divide-and-Conquer

Divide-and-Conquer

Recurrence Equation Analysis

The conquer step of merge-sort consists of merging two sorted sequences, each with n/2 elements and implemented by means of a doubly linked list, takes at most bn steps, for some constant b. Likewise, the basis case (n < 2) will take at b most steps. Therefore, if we let T(n) denote the running time of merge-sort:

Iterative Substitution
In the iterative substitution, or plug-and-chug, technique, we iteratively apply the recurrence equation to itself and see if we can find a pattern: T (n ) = 2T (n / 2) + bn

= 2( 2T (n / 2 2 )) + b( n / 2)) + bn = 22 T (n / 22 ) + 2bn = 23 T (n / 23 ) + 3bn = 24 T (n / 24 ) + 4bn = ... = 2i T (n / 2i ) + ibn

Note that base, T(n)=b, case occurs when 2i=n. That is, i = log n. So, T (n ) = bn + bn log n Thus, T(n) is O(n log n).

b T (n) = 2T ( n / 2) + bn

if n < 2 if n 2

We can therefore analyze the running time of merge-sort by finding a closed form solution to the above equation.
That is, a solution that has T(n) only on the left-hand side.

Divide-and-Conquer

Divide-and-Conquer

The Recursion Tree

Draw the recursion tree for the recurrence relation and look for a pattern: b if n < 2 T (n ) = 2T ( n / 2) + bn if n 2
depth 0 1 i Ts 1 2 2i size n n/2 n/2i time bn bn bn

Guess-and-Test Method
In the guess-and-test method, we guess a closed form solution and then try to prove it is true by induction: b if n < 2 T (n ) = 2T ( n / 2) + bn log n if n 2 Guess: T(n) < cn log n.

T (n ) = 2T (n / 2) + bn log n = 2( c(n / 2) log(n / 2)) + bn log n = cn (log n log 2) + bn log n = cn log n cn + bn log n
Wrong: we cannot make this last line be less than cn log n

Total time = bn + bn log n

(last level plus all previous levels)
Divide-and-Conquer 7

Divide-and-Conquer

Guess-and-Test Method, Part 2

Recall the recurrence equation: b T (n ) = 2T ( n / 2) + bn log n Guess #2: T(n) < cn log2 n.
if n < 2 if n 2

Master Method
Many divide-and-conquer recurrence equations have the form:

T (n) = 2T (n / 2) + bn log n = 2(c (n / 2) log 2 (n / 2)) + bn log n = cn(log n log 2) 2 + bn log n = cn log 2 n 2cn log n + cn + bn log n

c T (n) = aT n b ( / ) + f (n)

if n < d if n d

The Master Theorem:

1. if f (n) is O (n logb a ), then T (n) is ( n logb a ) 2. if f (n) is ( n logb a log k n), then T (n) is (n logb a log k +1 n) 3. if f ( n) is ( n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.
9 Divide-and-Conquer 10

if c > b.

cn log 2 n

So, T(n) is O(n log2 n). In general, to use this method, you need to have a good guess and you need to be good at induction proofs.
Divide-and-Conquer

Master Method, Example 1

The form:
c T (n) = aT ( n / b) + f (n ) if n < d if n d

Master Method, Example 2

The form: T (n ) =
c aT ( n / b) + f (n ) if n < d if n d

The Master Theorem:

1. if f (n) is O( n logb a ), then T (n) is ( n logb a ) 2. if f (n) is (n logb a log k n), then T ( n) is (n logb a log k +1 n) 3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.

The Master Theorem:

1. if f (n) is O( n logb a ), then T (n) is ( n logb a ) 2. if f (n) is (n logb a log k n), then T ( n) is (n logb a log k +1 n) 3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.

Example:

T (n) = 4T (n / 2) + n
O(n2).
11

Example:

T (n) = 2T (n / 2) + n log n

Solution: logba=2, so case 1 says T(n) is

Divide-and-Conquer

Solution: logba=1, so case 2 says T(n) is O(n log2 n).

Divide-and-Conquer 12

Master Method, Example 3

The form:
c T (n) = aT ( n / b) + f (n ) if n < d if n d

Master Method, Example 4

The form: T (n ) =
c aT ( n / b) + f (n ) if n < d if n d

The Master Theorem:

1. if f (n) is O( n logb a ), then T (n) is ( n logb a ) 2. if f (n) is (n
log b a

The Master Theorem:

1. if f (n) is O( n logb a ), then T (n) is ( n logb a )
log b a k

log n), then T ( n) is (n

log

k +1

n)

2. if f (n) is (n logb a log k n), then T ( n) is (n logb a log k +1 n) 3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.

3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.

Example:

T (n) = T (n / 3) + n log n

Example:

T (n) = 8T (n / 2) + n 2

Solution: logba=0, so case 3 says T(n) is O(n log n).

Divide-and-Conquer 13

Solution: logba=3, so case 1 says T(n) is O(n3).

Divide-and-Conquer 14

Master Method, Example 5

The form:
c T (n) = aT ( n / b) + f (n ) if n < d if n d

Master Method, Example 6

The form: T (n ) =
c aT ( n / b) + f (n ) if n < d if n d

The Master Theorem:

1. if f (n) is O( n logb a ), then T (n) is ( n logb a ) 2. if f (n) is (n logb a log k n), then T ( n) is (n logb a log k +1 n) 3. if f (n) is (n
log b a +

The Master Theorem:

1. if f (n) is O( n logb a ), then T (n) is ( n logb a ) 2. if f (n) is (n logb a log k n), then T ( n) is (n logb a log k +1 n) 3. if f (n) is (n logb a + ), then T (n) is ( f (n)), provided af (n / b) f (n) for some < 1.

), then T (n) is ( f (n)),

provided af (n / b) f (n) for some < 1.

Example:

T (n) = 9T (n / 3) + n 3

Example:

T ( n ) = T ( n / 2) + 1

(binary search)

Solution: logba=2, so case 3 says T(n) is O(n3).

Divide-and-Conquer 15

Solution: logba=0, so case 2 says T(n) is O(log n).

Divide-and-Conquer 16

Master Method, Example 7

The form: T (n ) =
c aT ( n / b) + f (n ) if n < d if n d

Iterative Proof of the Master Theorem

Using iterative substitution, let us see if we can find a pattern: T (n) = aT (n / b) + f (n)
= a (aT (n / b 2 )) + f (n / b)) + bn = a 2T (n / b 2 ) + af (n / b) + f (n) = a 3T (n / b 3 ) + a 2 f (n / b 2 ) + af (n / b) + f (n) = ... = a logb nT (1) +
(log b n ) 1 i

The Master Theorem:

1. if f (n) is O( n logb a ), then T (n) is ( n logb a ) 2. if f (n) is (n logb a log k n), then T ( n) is (n logb a log k +1 n) 3. if f (n) is (n
log b a +

), then T (n) is ( f (n)),

provided af (n / b) f (n) for some < 1.

a
i =0 i =0

f (n / bi ) f (n / bi )

Example:

T (n) = 2T (n / 2) + log n

(heap construction)

= n logb aT (1) +

(log b n ) 1 i

Solution: logba=1, so case 1 says T(n) is O(n).

Divide-and-Conquer 17

We then distinguish the three cases as

The first term is dominant Each part of the summation is equally dominant The summation is a geometric series
Divide-and-Conquer

18

Integer Multiplication
Algorithm: Multiply two n-bit integers I and J.

An Improved Integer Multiplication Algorithm

Algorithm: Multiply two n-bit integers I and J.

Divide step: Split I and J into high-order and low-order bits

Divide step: Split I and J into high-order and low-order bits

I = I h 2n / 2 + Il J = J h 2n / 2 + J l

I = I h 2n / 2 + Il

Observe that there is a different way to multiply parts:

J = J h 2n / 2 + J l

We can then define I*J by multiplying the parts and adding:

I * J = ( I h 2n / 2 + Il ) * ( J h 2n / 2 + J l ) = I h J h 2n + I h J l 2n / 2 + I l J h 2n / 2 + I l J l

I * J = I h J h 2 n + [( I h I l )( J l J h ) + I h J h + I l J l ]2 n / 2 + I l J l = I h J h 2 n + [( I h J l I l J l I h J h + I l J h ) + I h J h + I l J l ]2 n / 2 + I l J l = I h J h 2 n + ( I h J l + I l J h )2 n / 2 + I l J l

So, T(n) = 4T(n/2) + n, which implies T(n) is O(n2). But that is no better than the algorithm we learned in grade school.
Divide-and-Conquer 19

So, T(n) = 3T(n/2) + n, which implies T(n) is O(nlog23), by the Master Theorem. Thus, T(n) is O(n1.585).
Divide-and-Conquer 20