CS651-Algorithms

Divide-and-Conquer

Computer Science Dept.

Instructor: Ameera Jaradat
 Divide-and-Conquer
• Divide the problem into a number of sub-problems
– Similar sub-problems of smaller size

• Conquer the sub-problems

– Solve the sub-problems recursively

– Sub-problem size small enough  solve the problems in

straightforward manner

• Combine the solutions of the sub-problems

– Obtain the solution for the original problem
 Merge Sort Approach
• To sort an array A[p . . r]:
• Divide
– Divide the n-element sequence to be sorted into two
subsequences of n/2 elements each
• Conquer
– Sort the subsequences recursively using merge sort
– Keep dividing the resulting sup sequences until the
size of the sequences is 1
• Combine
– Merge the two sorted subsequences

3
 Merge Sort
lo q hi
MERGE-SORT(A, p, r) 1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6
if lo < hi

then q ← (lo + hi)/2

MERGE-SORT(A, lo, q)
MERGE-SORT(A, q + 1, hi)
MERGE(A, lo, q, hi)

• Initial call: MERGE-SORT(A, 1, n)

4
Example –
1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6 q=6
Divide
1 2 3 4 5 6 7 8 9 10 11

q=3 4 7 2 6 1 4 7 3 5 2 6 q=9

1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6

1 2 3 4 5 6 7 8 9 10 11

4 7 2 6 1 4 7 3 5 2 6

1 2 4 5 7 8

4 7 6 1 7 3

5
Example – cont
1 2 3 4 5 6 7 8 9 10 11

Conquer 1 2 2 3 4 4 5 6 6 7 7
and
Merge
1 2 3 4 5 6 7 8 9 10 11

1 2 4 4 6 7 2 3 5 6 7

1 2 3 4 5 6 7 8 9 10 11

2 4 7 1 4 6 3 5 7 2 6

1 2 3 4 5 6 7 8 9 10 11

4 7 2 1 6 4 3 7 5 2 6

1 2 4 5 7 8

4 7 6 1 7 3

6
Analysis for Merge Sort
• Divide:
– compute q as the average of lo and hi: T(n) = (1)
• Conquer:
– recursively solve 2 sublists, each of size n/2  2T
(n/2)
• Combine:
– MERGE on an n-element subarray takes (n) time
 C(n) = (n)
(1) if n =1
T(n) = 2T(n/2) + (n) if n > 1

7
Analysis for Merge Sort
Merge Sort Recurrence
T(n) = c if n = 1
2T(n/2) + cn if n > 1

Use Master’s Theorem: a=1 , b=1

Compare n with f(n) = cn

Case 2: T(n) = Θ(nlgn)

• Running time unaffected by the input

• Guaranteed to run in (nlgn)
• Requires extra space N
9
 Quicksort
A[p…q] A[q+1…r]
To Sort an array A[p…r]
• Divide
– Partition the array A into 2 subarrays A[p..q-1] and A[q+1..r]
around pivot element x , such that A[p..q-1] ≤ x ≤ A[q+1..r]
– Need to find index q to partition the array

• Conquer
– Recursively sort A[p..q] and A[q+1..r] using Quicksort
• Combine
– Trivial: the arrays are sorted in place
– No additional work is required to combine them
10
 QUICKSORT
Quicksort(A, p, r)
if p < r then Initially: p=1, r=n
– q  Partition(A, p, r)
– Quicksort(A, p, q − 1)
– Quicksort(A, q +1, r)

Total time = partitioning time + the time need to repeat

qicksort for the left part {A[p..q] } and the right {A[q+1..r] }

Recurrence:
T(n) = T(q) + T(n – q) + f(n) F(n) depends on PARTITION())

11
 Partitioning

Partition a list A[] into two non- X

empty parts.
Pick any value in the array, pivot.
left = the elements in A[] ≤ pivot
X
right = the elements in A[] > pivot
Place The pivot in the slot X X
between them.  

X X X

An illustration of one step partitioning

 Partitioning the Array
Partition( A[ ], first, last)
//Define the pivot value as the contents of A[First]
P = A[first] ; Q=first;
//Initialize Up to First and Down to Last
Up = first ; down = last+1;
Repeat
Repeat up++ until (A[up] > P)
repeat down- - until (A[down] ≤ P)
if (up < Down)
exchange (A[up] , A[down] )
until (up ≥ Down )
Exchange (A[First] , A[Down])
Q= Down
Return Q

Running time: (n)

The effects of the linear time partitioning step:

1. The pivot element ends up in the position it retains in the final

sorted order.
2. After a partitioning, no element flops to the other side of the
pivot in the final sorted order.

Thus we can sort the elements to the left of the pivot

and the right of the pivot independently!

44 75 23 43 55 12 64 77 33

12 33 23 43 44 55 64 77 75
 Example

q=5

q=3 q=6
 Quicksort– worst case
• Worst-case partitioning
– One region has one element and the other has n – 1 elements

• Recurrence:
n n
T(n) = T(1) + T(n – 1) + n, 1 n-1 n
1 n-2 n-1
T(1) = (1)
n 1 n-3 n-2
T(n) = T(n – 1) + n 1
2 3
1 1 2

When does the (n2)

worst case happen?
Quicksort– best case
• The best case comes when we split the input as evenly as possible. Thus,
each subproblem is of size n/2.
• The partition step on each subproblem is linear in its size. Thus the total
effort in partitioning the problems of size is O(n).
• The total partitioning on each level is O(n), and it take logn levels of
partitions to get to single element subproblems.

The recursion tree for

the best case looks like
this

Recurrence:
T(n) = 2T(n/2) + (n)
Average Performance of Quicksort
• Average case
– On a random input array, we will have a mix of well balanced
and unbalanced splits
– Good and bad splits are randomly distributed across throughout
the tree

partitioning cost:
n partitioning cost: n n = (n)
1 n-1 2n-1 = (n)
(n – 1)/2 + 1 (n – 1)/2
(n – 1)/2 (n – 1)/2

Alternate of a good Nearly well

and a bad split balanced split

• Running time of Quicksort when levels alternate

between good and bad splits is O(nlogn) 18
Randomized version of quicksort
• In exploring the average‐case behavior of quicksort, we
have assumed that all input permutations are equally
likely.
 This is not always true.
 We use random sampling, or picking one element at random.
 Don’t always use A[first] as the pivot.
 Instead, randomly pick an element from the subarray that is
being sorted.
1. RANDOMIZED‐PARTITION(A, p, r)
2. i  RANDOM(p, r)
3. exchange (A[p] , A[i])
4. return PARTITION(A, p, r)

19
Randomized version of quicksort
• Randomly selecting the pivot element will, on
average, cause the split of the input array to be
reasonably well balanced.

1. RANDOMIZED‐QUICKSORT(A, p, r)
2. if p < r
3. then q RANDOMIZED‐PARTITION(A, p, r)
4. RANDOMIZED‐QUICKSORT(A, p, q -1)
5. RANDOMIZED‐QUICKSORT(A, q + 1, r)

20