APPLIED ALGORITHMS

DIVIDE-AND-CONQUER

1
CONTENT

• Basis of Divide-And-Conquer
• Karatsuba algorithm
• Closest pair points
• Decrease and Conquer
• Inversion

2
Basis of Divide and Conquer

• Generic schema
• Divide the original problem into smaller independent subproblems
• Solve subproblems (recursion)
• Combine solutions of subproblems

3
Basis of Divide and Conquer

• Complexity analysis
• T(n): running time of input size n

• Consider: 𝑇 𝑛 = 𝑎𝑇 𝑛/𝑏 + 𝑛𝑘 với 𝑎, 𝑏, 𝑐 , 𝑘 are positive constants and 𝑎 ≥ 1, 𝑏 ≥ 2:

→ T(n) = O 𝑛log𝑏𝑎 , nếu 𝑎 > 𝑏 𝑘
O 𝑛𝑘 log𝑛 , nếu 𝑎 = 𝑏 𝑘
O 𝑛𝑘 , nếu 𝑎 < 𝑏 𝑘

4
Multiplication of 2 big numbers: Karatsuba algorithm

• Multiplication of 2 big numbers A and B (containing n digits)

• A = A1x10n/2 + A2
• B = B1x10n/2 + B2
• A x B = (A1x10n/2 + A2) x (B1x10n/2 + B2) = A1 x B1x10n + (A1 x B2 + A2 x B1)x10n/2 + A2 x B2
• A1 x B2 + A2 x B1 = (A1+A2) x (B1 + B2) – A1 x B1 – A2 x B2
• A x B = A1 x B1x10n + ((A1+A2) x (B1 + B2) – A1 x B1 – A2 x B2)x10n/2 + A2 x B2

5
Multiplication of 2 big numbers: Karatsuba algorithm

• Multiplication of 2 big numbers A and B (containing n digits)

• A = A1x10n/2 + A2
• B = B1x10n/2 + B2
• A x B = (A1x10n/2 + A2) x (B1x10n/2 + B2) = A1 x B1x10n + (A1 x B2 + A2 x B1)x10n/2 + A2 x B2
• A1 x B2 + A2 x B1 = (A1 + A2) x (B1 + B2) – A1 x B1 – A2 x B2
• A x B = A1 x B1x10n + ((A1+A2) x (B1 + B2) – A1 x B1 – A2 x B2)x10n/2 + A2 x B2
• Complexity:
• T(n) = 3T(n/2) + O(n)
• T(n) = O(𝑛𝑙𝑜𝑔23 )

6
Closest pair of Points

• Given n points P = 0, 1, . . ., n-1 on the plane, 12 9

find the pair of 2 points such that the distance
between these points is the smallest. 11
• Denote A.x and A.y the x-coordinate and y- 10 3
coordinate of point A. 9
• Denote dist(A, B): the distance between points 8 8 7
A and B 7 11
• Denote d(P): the smallest distance among the 6
distances between 2 points of P.
5 4
4 1 10
3
2 0 2
1 5
0 6
0 1 2 3 4 5 6 7 8 9 10 11 12
7
Closest pair of Points

• X_SORT(P): return the list of points of P sorted 12 9

in a non-decreasing order of x-coordinates (two
points with the same x-coordinate, the point 11
having smaller y-coordinate will be located 10 3
before the other)
9
• Y_SORT(P): return the list of points of P sorted 8 8 7
in a non-decreasing order of y-coordinates (two
points with the same y-coordinate, the point 7 11
having smaller x-coordinate will be located 6
before the other) 4
5
• Example 4 1 10
• X_SORT(P) = 3, 11, 0, 4, 5, 1, 8, 6, 10, 9, 2, 7
3
• Y_SORT(P) = 6, 5, 0, 2, 1, 10, 4, 11, 8, 7, 3, 9
2 0 2
1 5
0 6
0 1 2 3 4 5 6 7 8 9 10 11 12
8
 Closest pair of Points

• Let Px = X_SORT(P) 12 9
• Let O the point in the middle of Px. 11
• Let LEFT(Px, O) be the sub-list of points of Px 10 3
before O (O inclusive)
9
• Let RIGHT(Px, O) be the sub-list of points of Px 8 7
8
after O
7 11
• Example
6
• Px = 3, 11, 0, 4, 5, 1, 8, 6, 10, 9, 2, 7
5 4
• O = point 1
• LEFT(Px, O) = 3, 11, 0, 4, 5, 1 4 1 10
• RIGHT(Px, O) = 8, 6, 10, 9, 2, 7 3
2 0 2
1 5
0 6
0 1 2 3 4 5 6 7 8 9 10 11 12
9
 Closest pair of Points

• Divide and Conquer 12 9

• Let Px = X_SORT(P)
11
• Let O the point in the middle of Px.
10 3
• PL = LEFT(Px, O)
• PR = RIGHT(Px, O) 9
• Let  = min(d(PL), d(PR)) 8 8 7
• Let S be the set of points A of Px such that 7 11
|O.x – A.x| <  (S is called Strip). 6
 
• Combination: find the closest points of S 5 4
4 1 10
3
2 0 2
1 5
0 6
0 1 2 3 4 5 6 7 8 9 10 11 12
10
 Closest pair of Points

ClosestPair(P) {
Px = X_SORT(P); 12 9
n = length(P); O = middle point of Px; 11
PL = LEFT(Px, O); PR = RIGHT(Px, O); 10 3
dL = ClosestPair(PL); dR = ClosestPair(PR); 9
 = min(DL, dR); 8 8 7
S = {A  Px |  > |O.x – A.x|}; 7 11
dm = ClosestPairStrip(S, length(S), ); 6
return dm; 5 4
} 4 1 10
3
2 0 2
1 5
0 6
0 1 2 3 4 5 6 7 8 9 10 11 12
11
 Closest pair of Points
0
• Find the closest points in the strip 12 11
11 9
10 3 2
1. ClosestPairStrip(S, n, ) { 9
2. S = Y_SORT(S); 8 8
3. dm = ;
7
4. for i = 0 to n-1 do { 6 4 7
5. for j = i+1 to n-1 do {
5
6. if S[j].y – S[i].y ≥  then break;
4 1 10
7. dm = min(dm, dist(S[i], S[j]);
3 6
8. }
2
9. }
1 5
10. return dm;
11.} 0
0 1 2 3 4 5 6 7 8 9 10 11 12
12
 Closest pair of Points
• Lines 5—8 run at most 8 iterations as each square
ABCD and BCEF contains at most 4 points 12
(distance between 2 points within each square is
greater or equal to ) 11
10
1. ClosestPairStrip(S, n, ) { 9
2. S = Y_SORT(S); 8
3. dm = ;
7
4. for i = 0 to n-1 do { 6
5. for j = i+1 to n-1 do {
5 
6. if S[j].y – S[i].y ≥  then break; A B F
4
7. dm = min(dm, dist(S[i], S[j]);
3
8. } 
2
9. } C E
1
D S[i]
10. return dm;
11.} 0
0 1 2 3 4 5 6 7 8 9 10 11 12
13
 Closest pair of Points – O(nlogn) implementation

• Px = X_SORT(P) and Py = Y_SORT(P) 12 9

• O is the middle point of Px 11
• PxL = LEFT(Px, O) and PyL = RIGHT(Px, O) 10 3
• PyL = Y_SORT(PxL) and PyR = Y_SORT(PyL) 9
• Example: 8 8 7
• Px = 3, 11, 0, 4, 5, 1, 8, 6, 10, 9, 2, 7 7 11
• Py = 6, 5, 0, 2, 1, 10, 4, 11, 8, 7, 3, 9 6
• Left part 5 4
• PxL = 3, 11, 0, 4, 5, 1 4 1 10
• PyL = 5, 0, 1, 4, 11, 3 3
• Right part 2 0 2
• PxR = 8, 6, 10, 9, 2, 7 5
1
• PyR = 6, 2, 10, 8, 7, 9
0 6
0 1 2 3 4 5 6 7 8 9 10 11 12
14
 Closest pair of Points – O(nlogn) implementation

ClosestPair(P, n){// points of the list P are indexed 0, 1, . . ., n-1

Px = X_SORT(P);
Py = Y_SORT(P);
dm = ClosestPair(Px, Py, n);
return dm;
}

15
 Closest pair of Points – O(nlogn) implementation

ClosestPair(Px, Py, n) {
if n <= 3 then return BruteforceClosestPair(Px,n);
PxL, PyL, PxR, PyR = []; mid = n/2; O = Px[mid];
for i = 0 to mid – 1 do PxL.push(Px[i]);
for i = mid to n-1 do PxR.push(Px[i]);
for i = 0 to n-1 do {
if ((Py[i].x < O.x) or Py[i].x = O.x and Py[i].y < O.y) and length(PyL) < mid then
PyL.push(Py[i]);
else PyR.push(Py[i]);
}
dL = ClosestPair(PxL, PyL, mid); dR = ClosestPair(PxR, PyR, n-mid);  = min(DL, dR);
S = []; for i = 0 to n-1 do if |Py[i].x – O.x| <  then S.push(Py[i]);
dm = ClosestPairStrip(S, length(S), );
return dm;

}
16
 Closest pair of Points – O(nlogn) implementation

ClosestPairStrip(S, n, ) { BruteforceClosestPair(P, n){

dm = ; dm = INF;
for i = 0 to n-1 do { for i = 0 to n-2 do
for j = i+1 to n-1 do { for j = i + 1 to n-1 do
if S[j].y – S[i].y ≥  then break; if dist(P[i], P[j]) < dm then
dm = min(dm, dist(S[i], S[j]); dm = dist(P[i], P[j]);
} return dm;
} }
return dm;
}

17
Inversion

• Given a sequence a[1], a[2], . . ., a[n]. Count the number of pair (i, j) such that 1 i < j  n and a[i] >
a[j]
• Example: 5, 2, 7, 9, 4, 1
• Inversions: (1, 2), (1, 5), (1, 6), (2, 6), (3, 5), (3, 6), (4, 5), (4, 6), (5, 6)

18
 Inversion

• Divide and conquer: Apply merge sort algorithms countInversions(L, R) {

• Divide the given sequence into 2 equal size parts if L >= R then return 0;
• Count the number of inversions of the left sub- M = (L+R)/2;
sequence (after counting, the left sub-sequence is cntL = countInversions(L, M);
sorted in a non-decreasing order)
cntR = countInversions(M+1, R);
• Count the number of inversions of the right sub-
sequence (after counting, the right sub-sequence is cnt = countMerge(L, M, R);
sorted in a non-decreasing order) return cntL + cntR + cnt;
• Count the number of pair (i, j) in which a[i] > a[j] (i is }
an index of the left sub-sequence and j is an index
of the left sub-sequence)

19
 Inversion

• Divide and conquer: Apply merge sort countMerge(L, M, R) {

algorithms i = L; j = M+1; cnt = 0;
• Merge operation: for k = L to R do {
• Run an index i on the left sub-sequence if(i > M) then { ta[k] = a[j]; j++; }
and an index j on the right sub-sequence else if(j > R) then { ta[k] = a[i]; i++; }
• If a[i] > a[j] then a[q] > a[j] for all q = i ,…, M: else{
number of inversions is augmented by M –
i+1 if(a[i] <= a[j]) then { ta[k] = a[i]; i++; }
else{
ta[k] = a[j]; j++; cnt = cnt + M - i + 1;
}
L i M M+1 j R
}
}
2 3 8 10 15 20 25 4 5 14 19 21 29
for(int k = L; k <= R; k++) a[k] = ta[k];
return cnt;
}

20
 Inversion

• Divide and conquer: Apply merge sort countMerge(L, M, R) {

algorithms i = L; j = M+1; cnt = 0;
• Merge operation: for k = L to R do {
• Run an index i on the left sub-sequence if(i > M) then { ta[k] = a[j]; j++; }
and an index j on the right sub-sequence else if(j > R) then { ta[k] = a[i]; i++; }
• If a[i] > a[j] then a[q] > a[j] for all q = i ,…, M: else{
number of inversions is augmented by M –
i+1 if(a[i] <= a[j]) then { ta[k] = a[i]; i++; }
else{
• Time complexity: O(nlogn)
ta[k] = a[j]; j++; cnt = cnt + M - i + 1;
}
L i M M+1 j R
}
}
2 3 8 10 15 20 25 4 5 14 19 21 29
for(int k = L; k <= R; k++) a[k] = ta[k];
return cnt;
}

21
Decrease and Conquer

• Given a binary sequence X of length n which can be divided into 2 parts: the prefix contains only 0
and the suffix contains only 1.
• Example: 0000000011111111111111111
• Goal: Find the index of the first 1-bit (from left to right)

22
Decrease and Conquer

• Given a binary sequence X of length n which can be divided into 2 parts: the prefix contains only 0
and the suffix contains only 1.
• Example: 0000000011111111111111111
• Goal: Find the index of the first 1-bit (from left to right)
• Example 0 0 0 0 0 0 0 0 0 1 1 1 1 1

23
Decrease and Conquer

• Given a binary sequence X of length n which can be divided into 2 parts: the prefix contains only 0
and the suffix contains only 1.
• Example: 0000000011111111111111111
• Goal: Find the index of the first 1-bit (from left to right)
• Decrease and conquer
• Let m the middle position of X
• Consider the bit X[m] in the middle of X
• If X[m] = 0 then find in the result in the right sub-sequence
• If X[m] = 1
• If X[m-1] = 0 then return m m
• Otherwise, find the result in the left sub-sequence

left sub-sequence right sub-sequence

24
Decrease and Conquer

• Given a binary sequence X of length n which can be divided into 2 parts: the prefix contains only 0
and the suffix contains only 1.
• Example: 0000000011111111111111111
• Goal: Find the index of the first 1-bit (from left to right)
• Decrease and conquer
• Let m the middle position of X
• Consider the bit X[m] in the middle of X
• If X[m] = 0 then find in the result in the right sub-sequence
• If X[m] = 1
• If X[m-1] = 0 then return m m
• Otherwise, find the result in the left sub-sequence
• Time complexity: O(logn)

left sub-sequence right sub-sequence

25
THANK YOU !

26