Scribd
Upload
28 views

Divide and Conquer: Andreas Klappenecker (Based On Slides by Prof. Welch)

The document discusses the divide and conquer paradigm and provides examples such as mergesort and matrix multiplication. It introduces the master theorem for solving divide and conquer recurrences. It then describes Strassen's improved matrix multiplication algorithm which uses divide and conquer to reduce the number of recursive subproblems from 8 to 7, providing a faster running time of O(n2.81) compared to the naive O(n3) algorithm. However, Strassen's method has limitations such as larger constants and numerical instability issues.

Uploaded by

samar
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
28 views

Divide and Conquer: Andreas Klappenecker (Based On Slides by Prof. Welch)

The document discusses the divide and conquer paradigm and provides examples such as mergesort and matrix multiplication. It introduces the master theorem for solving divide and conquer recurrences. It then describes Strassen's improved matrix multiplication algorithm which uses divide and conquer to reduce the number of recursive subproblems from 8 to 7, providing a faster running time of O(n2.81) compared to the naive O(n3) algorithm. However, Strassen's method has limitations such as larger constants and numerical instability issues.

Uploaded by

samar
Copyright
© © All Rights Reserved
Available Formats
Download as PPT, PDF, TXT or read online on Scribd
You are on page 1/ 27

Divide and Conquer

Andreas Klappenecker
[based on slides by Prof. Welch]
Divide and Conquer Paradigm

• An important general technique for


designing algorithms:
• divide problem into subproblems
• recursively solve subproblems
• combine solutions to subproblems to get
solution to original problem
• Use recurrences to analyze the
running time of such algorithms
Mergesort
Example: Mergesort

• DIVIDE the input sequence in half


• RECURSIVELY sort the two halves
• basis of the recursion is sequence with 1
key
• COMBINE the two sorted subsequences
by merging them
Mergesort Example
5 2 4 6 1 3 2 6

5 2 4 6 1 3 2 6

5 2 4 6 1 3 2 6

5 2 4 6 1 3 2 6

2 5 4 6 1 3 2 6

2 4 5 6 1 2 3 6

1 2 2 3 4 5 6 6
Mergesort Animation

• http://ccl.northwestern.edu/netlogo/m
odels/run.cgi?MergeSort.862.378
Recurrence Relation for
Mergesort

• Let T(n) be worst case time on a


sequence of n keys
• If n = 1, then T(n) = (1) (constant)
• If n > 1, then T(n) = 2 T(n/2) + (n)
• two subproblems of size n/2 each that are
solved recursively
 (n) time to do the merge
Recurrence Relations
How To Solve Recurrences

• Ad hoc method:
• expand several times
• guess the pattern
• can verify with proof by induction
• Master theorem
• general formula that works if recurrence has the form
T(n) = aT(n/b) + f(n)
• a is number of subproblems
• n/b is size of each subproblem
• f(n) is cost of non-recursive part
Master Theorem

Consider a recurrence of the form


T(n) = a T(n/b) + f(n)
with a>=1, b>1, and f(n) eventually positive.
a)If f(n) = O(nlogb(a)-), then T(n)=(nlogb(a)).
b)If f(n) = (nlogb(a) ), then T(n)=(nlogb(a) log(n)).
c) If f(n) = (nlogb(a)+) and f(n) is regular, then
T(n)=(f(n))
[f(n) regular iff eventually af(n/b)<= cf(n) for some
constant c<1]
Excuse me, what did it say???

Essentially, the Master theorem compares the


function f(n)
with the function g(n)=nlogb(a).
Roughly, the theorem says:
a)If f(n) << g(n) then T(n)=(g(n)).
b)If f(n)  g(n) then T(n)=(g(n)log(n)).
c)If f(n) >> g(n) then T(n)=(f(n)).
Now go back and memorize the theorem!
Déjà vu: Master Theorem

Consider a recurrence of the form


T(n) = a T(n/b) + f(n)
with a>=1, b>1, and f(n) eventually positive.
a)If f(n) = O(nlogb(a)-), then T(n)=(nlogb(a)).
b)If f(n) = (nlogb(a) ), then T(n)=(nlogb(a) log(n)).
c) If f(n) = (nlogb(a)+) and f(n) is regular, then
T(n)=(f(n))
[f(n) regular iff eventually af(n/b)<= cf(n) for some
constant c<1]
Nothing is perfect…

The Master theorem does not cover all


possible cases. For example, if
f(n) = (nlogb(a) log n),
then we lie between cases 2) and 3), but
the theorem does not apply.
There exist better versions of the Master
theorem that cover more cases, but these
are even harder to memorize.
Idea of the Proof
Let us iteratively substitute the recurrence:
T (n)  aT (n / b)  f (n)
 a (aT (n / b 2 ))  f (n / b))  bn
 a 2T (n / b 2 )  af (n / b)  f (n)
 a 3T (n / b 3 )  a 2 f (n / b 2 )  af (n / b)  f (n)
 ...
(logb n ) 1
 a logb nT (1)   a
i 0
f ( ni
/ b i
)
(logb n ) 1
 n logb aT (1)   a
i 0
f ( ni
/ b i
)
Idea of the Proof
Thus, we obtained
T(n) = nlogb(a) T(1) + a i f(n/bi)
The proof proceeds by distinguishing three cases:
1) The first term in dominant: f(n) = O(nlogb(a)-)
2) Each part of the summation is equally dominant: f(n)
= (nlogb(a) )
3) The summation can be bounded by a geometric series:
f(n) = (nlogb(a)+) and the regularity of f is key to make
the argument work.
Further Divide and Conquer
Examples
Additional D&C Algorithms
• binary search
• divide sequence into two halves by comparing search
key to midpoint
• recursively search in one of the two halves
• combine step is empty
• quicksort
• divide sequence into two parts by comparing pivot to
each key
• recursively sort the two parts
• combine step is empty
Additional D&C applications

• computational geometry
• finding closest pair of points
• finding convex hull
• mathematical calculations
• converting binary to decimal
• integer multiplication
• matrix multiplication
• matrix inversion
• Fast Fourier Transform
Strassen’s Matrix Multiplication
Matrix Multiplication
• Consider two n by n matrices A and B
• Definition of AxB is n by n matrix C whose (i,j)-
th entry is computed like this:
• consider row i of A and column j of B
• multiply together the first entries of the rown and
column, the second entries, etc.
• then add up all the products
• Number of scalar operations (multiplies and
adds) in straightforward algorithm is O(n3).
• Can we do it faster?
Divide-and-Conquer

A  B = C
A0 A1 B0 B1 A0B0+A1B2 A0B1+A1B3
 =
A2 A3 B2 B3 A2B0+A3B2 A2B1+A3B3

• Divide matrices A and B into four submatrices each


• We have 8 smaller matrix multiplications and 4
additions. Is it faster?
Divide-and-Conquer

Let us investigate this recursive version of the


matrix multiplication.

Since we divide A, B and C into 4 submatrices


each, we can compute the resulting matrix C by
• 8 matrix multiplications on the submatrices
of A and B,
• plus (n2) scalar operations
Divide-and-Conquer

• Running time of recursive version of


straightfoward algorithm is
• T(n) = 8T(n/2) + (n2)
• T(2) = (1)
where T(n) is running time on an n x n matrix
• Master theorem gives us:
T(n) = (n3)
• Can we do fewer recursive calls (fewer
multiplications of the n/2 x n/2 submatrices)?
Strassen’s Matrix Multiplication

A  B = C
A0 A1 B0 B1 C11 C12
 =
A2 A3 B2 B3 C21 C22

P1 = (A11+ A22)(B11+B22)
C11 = P1 + P4 - P5 + P7
P2 = (A21 + A22) * B11
C12 = P3 + P5
P3 = A11 * (B12 - B22)
C21 = P2 + P4
P4 = A22 * (B21 - B11)
C22 = P1 + P3 - P2 + P6
P5 = (A11 + A12) * B22
P6 = (A21 - A11) * (B11 + B12)
Strassen's Matrix Multiplication

• Strassen found a way to get all the


required information with only 7 matrix
multiplications, instead of 8.

• Recurrence for new algorithm is


• T(n) = 7T(n/2) + (n2)
Solving the Recurrence Relation

Applying the Master Theorem to


T(n) = a T(n/b) + f(n)
with a=7, b=2, and f(n)=(n2).

Since f(n) = O(nlogb(a)-) = O(nlog2(7)-),


case a) applies and we get
T(n)= (nlogb(a)) = (nlog2(7)) = O(n2.81).
Discussion of Strassen's
Algorithm

• Not always practical


• constant factor is larger than for naïve method
• specially designed methods are better on sparse
matrices
• issues of numerical (in)stability
• recursion uses lots of space
• Not the fastest known method
• Fastest known is O(n2.376)
• Best known lower bound is (n2)

You might also like