CSE-101 Lecture 14

Divide and Conquer

Dr. Aparajita Khan

Computer Science & Engineering
INDIAN INSTITUTE OF TECHNOLOGY ROORKEE
28 September 2024

1
The divide-and-conquer
design paradigm

1. Divide the problem (instance)

into subproblems.
2. Conquer the subproblems by
solving them recursively.
3. Combine subproblem solutions.
 The divide-and-conquer
design paradigm
1. Divide the problem (instance)
into subproblems.
2. Conquer the subproblems by
solving them recursively.

3. Combine subproblem solutions.

Merge Sort
Merge Sort

MERGE-SORT A[1 . . n]
1. If n = 1, done.
2. Recursively sort A[ 1 . . n/2]
and A[ n/2+1. . n ] .
3. “Merge” the 2 sorted lists.

Key subroutine: MERGE

Merge Sort
MERGE Procedure
• The key operation of the merge sort
– “Combine/Merge” Step: Merging of two sorted sequences into one
– Subarrays 𝐴[𝑝. . 𝑞] and 𝐴[𝑞 + 1. . 𝑟] are in sorted order
– MERGE(𝑃, 𝑞, 𝑟) merges these two sorted array into one

• MERGE procedure takes time Θ(𝑛) where 𝑛 = 𝑟 − 𝑝 + 1

 Merging two sorted arrays
L R
20 12
13 11
7 9
i 2 1 j
 Merging two sorted arrays
L R
20 12
13 11
7 9
i 2 1 j

1
 Merging two sorted arrays
L R L R
20 12 20 12
13 11 13 11
7 9 7 9 j

2 1 i 2

1
 Merging two sorted arrays
L R
20 12 20 12
13 11 13 11
7 9 7 9 j

2 1 i 2

1 2
 Merging two sorted arrays
L R
20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 i 7 9 j

2 1 2

1 2
 Merging two sorted arrays

20 12 20 12 20 12
13 11 13 11 13 11
7 9 7 9 7 9
2 1 2

1 2 7
 Merging two sorted arrays

20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7
 Merging two sorted arrays

20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9
 Merging two sorted arrays

20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9
 Merging two sorted arrays

20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11
 Merging two sorted arrays

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11
 Merging two sorted arrays

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11 12
 Merging two sorted arrays

20 12 20 12 20 12 20 12 20 12 20 12
13 11 13 11 13 11 13 11 13 11 13
7 9 7 9 7 9 9
2 1 2

1 2 7 9 11 12

Time = Θ(n) to merge a total

of n elements (linear time).
MERGE Algorithm
• Sentinel element ∞: Avoid additional checks and simplify the code
Merge sort complexity analysis
 Merge sort complexity analysis

1. Divide: Trivial.
2. Conquer: Recursively sort 2 subarrays.
3. Combine: Linear-time merge.
T(n) = 2 T(n/2) + Θ(n)

# subproblems work dividing

and combining
subproblem size
 Recurrence for merge sort

Θ(1) if n = 1;
T(n) =
2T(n/2) + Θ(n) if n > 1.

Using Master Theorem

𝑛log𝑏 𝑎 = 𝑛log2 2 = 𝑛
𝑓 𝑛 = 𝑐𝑛

𝑇 𝑛 = Θ(𝑛 log 𝑛)
 Using Recursion tree

Solve T(n) = 2T(n/2) + cn, where c > 0 is constant.

cn cn
cn/2 cn/2 cn
h = lg n cn/4 cn/4 cn/4 cn/4 cn

…
Θ(1) #leaves = n Θ(n)
Total = Θ(n lg n)
 Divide and conquer
Quicksort an n-element array:
1. Divide: Partition the array into two subarrays
around a pivot x such that elements in lower
subarray ≤ x ≤ elements in upper subarray.
≤x x ≥x
2. Conquer: Recursively sort the two subarrays.
3. Combine: Trivial.
Key: Linear-time partitioning subroutine.
Quicksort
• Quicksort Pseudocode

Initial call: QUICKSORT(A, 1, n)

• PARTITION Procedure Key Procedure
– PARTITION Procedure selects a pivot element
• Usually pivot is the first/last element of the subarray
𝐴 𝑝. . 𝑟
– It partitions the array into regions
PARTITION Example
Example: 2, 8, 7, 1, 3, 5, 6, 4
− 𝑝 and 𝑟 are starting and end index of
array 𝐴
− Pivot 𝑥 = 𝐴 𝑟 = 4
− Initially 𝑖 = 𝑝 − 1 = 0 and 𝑗 = 𝑝 = 1
− Loop:

− The pivot element is swapped so that it

lies between the two partitions
PARTITION Pseudocode

The running time of PARTITION on

the subarray 𝐴[𝑝. . 𝑟] is Θ(𝑛),
where 𝑛 = 𝑟 − 𝑝 + 1
 Worst-case of quicksort

• Input sorted or reverse sorted.

• Partition around min or max element.
• One side of partition always has no elements.
T (n) = T (0) +T (n −1) + Θ(n)
= Θ(1) +T (n −1) + Θ(n)
= T (n −1) + Θ(n)
= Θ(n2 ) (by unfolding recurrence)
Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
T(n)
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn
T(0) T(n–1)
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn
T(0) c(n–1)
T(0) T(n–2)
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn
T(0) c(n–1)
T(0) c(n–2)
T(0)

Θ(1)
 Worst-case recursion tree
T(n) = T(0) + T(n–1) + cn
cn n
Θ(1) c(n–1) Θ∑ k = Θ(n2 )
k =1
Θ(1) c(n–2)
h=n T(n) = Θ(n) + Θ(n2)
Θ(1)
= Θ(n2)
Θ(1)
 Best-case analysis

If we’re lucky, PARTITION splits the array evenly:

T(n) = 2T(n/2) + Θ(n)
= Θ(n lg n) (same as merge sort)
Analysis of “almost-worst” case
1 9
What if the split is always 10 : 10 ?
T (n) = T (101 n)+ T (109 n)+ Θ(n)
What is the solution to this recurrence?
Analysis of “almost-worst” case
T(n)
Analysis of “almost-worst” case

cn
T (101 n ) T (109 n )
 Analysis of “almost-worst” case

cn
1
10
cn 9 cn
10

T (100
1
n )T (100
9
n ) T (100
9
n )T (100
81
n)
 Analysis of “almost-worst” case

cn cn
1
10
cn 9 cn cn
10
log10/9n
1 cn 9 cn 9cn 81 cn
cn
100 100 100 100

…
Θ(1) O(n) leaves
Θ(1)
 Analysis of “almost-worst” case

cn cn
1
10
cn 9 cn cn
10
log10 log10/9n
n 1 cn 9 cn 9cn 81 cn
cn
100 100 100 100

…
Θ(1) O(n) leaves

Θ(n lgn) Θ(1)

Lucky! cn log10n ≤ T(n) ≤ cn log10/9n + Ο(n)
Take Aways

• Recurrence Relations
• Solving Recurrences
• Divide & Conquer
• Sorting using Divide & Conquer Multiplication
– Merge sort Algorithms
– Quick sort

43