Chapter A4: More Divide and Conquer

Here are some more divide-and-conquer algorithms.

A4.1 Multiplying Long Integers

Another example of divide-and-conquer is the problem of multiplying long integers. The

following is based on the discussion by Aho, Hopcroft, and Ullman.

Consider the problem of multiplying two n-bit integers X and Y . At elementary school
we learnt long multiplication: if we take two n-digit numbers then long multiplication
takes O(n2 ) (quadratic) time. (Why?)

We can apply divide and conquer. For example, suppose we split X and Y into two
halves:
X := A B Y := C D

Then X = A2n/2 + B and Y = C2n/2 + D. The product of X and Y can now be written
as:
XY = AC2n + (AD + BC)2n/2 + BD
This product requires four multiplications and three additions. To add two integers
takes linear time. (Why?) So we obtain the recurrence:

T (n) = 4T (n/2) + O(n)

Alas, the solution to this is again quadratic, and in practice this algorithm is worse than
normal long multiplication.

However, consider the following formula:


n
XY = AC 2 + (A B)(D C) + AC + BD 2n/2 + BD

At first glance this looks more complicated. But we only need three multiplications to
do this: AC, BD and (A B)(D C). Therefore we obtain the recurrence:

T (n) = 3T (n/2) + cn

whose solution is T (n) = O(nlog2 3 ) = O(n1.59 ).

There is some more work needed to write this down in a program. We need to consider
shifts and also deal with negative integers. Left as an exercise.
A4.2 Matrix Multiplication

Consider the problem of multiplying two n ⇥ n matrices, that is with n rows and n
columns. Recall that the result is itself an n ⇥ n matrix, where the entry in row i
column j of the result is the “dot product” of row i in the first matrix with column
j in the second matrix. Each dot product multiplies the corresponding entries in the
two arrays and sums this up. Thus, computing each entry in the product takes n
multiplications and there are n2 entries for a total of O(n3 ) work. Can one do better?
Strassen devised a better method which has the same basic flavor as the multiplication
of long integers. The key idea is to save one multiplication on a small problem and then
use recursion.
We look first at multiplying two 2 ⇥ 2 matrices. This would normally take 8 multiplica-
tions.
! ! !
a11 a12 b11 b12 a11 b11 + a12 b21 a11 b12 + a12 b22
A= B= AB =
a21 a22 b21 b22 a21 b11 + a22 b21 a21 b12 + a22 b22
But it turns out we can do it with 7 multiplications. You can check the following. If
and one calculates the following products:
m1 = (a21 + a22 a11 )(b22 b12 + b11 )
m2 = a11 b11
m3 = a12 b21
m4 = (a11 a21 )(b22 b12 )
m5 = (a21 + a22 )(b12 b11 )
m6 = (a12 a21 + a11 a22 )b22
m7 = a22 (b12 + b21 b11 b22 )
then the product of A and B is given by
!
m2 + m3 m1 + m2 + m5 + m6
AB =
m1 + m2 + m4 + m7 m1 + m2 + m4 + m5
Wow! But what use is this?
What we do is to use this idea recursively. If we have a 2n ⇥ 2n matrix, we can split it
into four n ⇥ n sub-matrices:
! !
A11 A12 B11 B12
A= and B=
A21 A22 B21 B22
and form seven products M1 up to M7 , and the overall product AB can thus be calculated
with seven multiplications of n⇥n matrices. Since adding matrices is clearly proportional
to their size, we obtain a recurrence relation:
f (n) = 7f (n/2) + O(n2 )
whose solution is O(nlog2 7 ), an improvement on O(n3 ).

14
A4.3 Closest Pair

The problem is to find, given n points in the plane, the pair of points that are the closest.
Quadratic time is trivial. But one can do better.

We again use a divide and conquer algorithm. The presentation of the algorithm assumes
no two points are coincident.

Presort the points from left to right. Then:

Closest Pair
1. Divide the n points into left and right halves, called L and R.
2. Find the closest pair inside L and inside R.
3. Determine the overall closest pair.

Again it is the merge Step (3) that does all the work. The overall closest pair is either
inside L, inside R, or has one point in L and one in R.

Let be the smaller of the smallest distance in L and the smallest distance in R. Consider
a strip either side of the dividing line. If in fact there are two points that are closer
than apart, then they must both lie inside the strip.

L R

Of course there could be many points in the strip. But on each side they are at least
apart. This means—and this is the key observation—that the points in the strip are
spread out vertically. (If one draws the circle of radius /2 around each point in say L,
none of the circles intersect.) In particular, if one considers a rectangle on one side with
width and any fixed height, there is a limit to the number of points inside the rectangle.

So the idea is to presort all the points from top to bottom. When the merge step is
reached, the points in L are examined in order from top to bottom. For each point,
there is a small window of points in R that one needs to worry about. In particular, it
can be shown that:

15
 for each given point v` in L, if there is a point vr in R that is closer than to
it, then vr must be one of the 3 points either side of v` in the overallvertical
ordering.

So, as we process the points v` in L, we advance the window of points in R as needed.

Thus the total work in Step (3) is at most O(n). By the Master Theorem, this means
that the overall algorithm runs in O(n log n) time.

Exercises

1. Illustrate the multiplication of 1234 and 5678 using the algorithm described above
(converted to decimal).

2. Create a class for long integers. Then provide the operations shift, addition and
multiplication. Then calculate the square of the 50th Fibonacci number.

3. Use Strassen’s algorithm to multiply the matrices

! !
1 2 5 6
and
3 4 7 8

4. Implement the closest-point algorithm.

5. Give a good algorithm to determine if, given n points in the plane, any 3 points
form a line.

16