Improvement on the Quick Sort Algorithm

Prerequisite: QuickSort Algorithm

The quicksort algorithm discussed in this article can take O(N²) time in the worst case. Hence, certain variations are needed which can efficiently partition the array and rearrange the elements around the pivot.

Single Pivot Partitioning: In single pivot partitioning the array A[] can be divided into {A[p], A[p + 1], A[p + 2], …, A[r]} into two parts A[p…q] and A[q+1…r] such that:

• All elements in the first partition, A[p…q] are lesser than or equal to the pivot value A[q].
• All elements in the second partition, A[q+1…r] are greater than or equal to the pivot value A[q].
• After that, the two partitions are treated as independent input arrays and fed themselves to the Quick Sort Algorithm.

Lomuto's Partitioning: This is the simplest partitioning procedure to implement. Starting from the leftmost element and keeping track of the index of smaller (or equal to) elements as i. While traversing, if a smaller element is found, swap the current element with A[i]. Otherwise, ignore the current element. 

It is discussed in detail in this article.

Hoare’s Partitioning: It works by initializing two indexes that start at two ends, the two indexes move toward each other until an inversion is (A smaller value on the left side and a greater value on the right side) found. When an inversion is found, two values are swapped and the process is repeated. Hoare’s Partition Scheme is more efficient than Lomuto’s Partition Scheme because it does three times fewer swaps on average, and it creates efficient partitions even when all values are equal.

The comparison of Lomuto's and Hoare's Partitioning is discussed in detail here.

Random Pivoting: In the above two approaches, the idea is to take the first or the last element of the unexplored array and find the correct position of the element in the array considering it as the pivot. If the array is already sorted in either increasing or decreasing order, and the first element is always chosen as a pivot, then it will be always the maximum or minimum element among the unexplored array elements. It will reduce the size of the unexplored array by 1 only and thus can cause the quick sort algorithm to perform the worst-case time complexity of O(N²).

Instead of picking the first element as pivot every time, if we pick a random element from the unexplored array and swap it with the first element and then perform the partitioning procedure (any one of the above two), then it will improve the expected or average time complexity to O(N*log N). The Worst-Case complexity is still O(N²). Though, when the array is already sorted, the worst case is avoided, but the worst case is hit when the array contains many repeated elements. 

To know how a randomized algorithm improves the average case time complexity, please refer to this article.

Performance with Repeated Elements: Consider an array arr[] = [3, 3, 3, 3, 3, 3], i.e. array has all (or many) equal elements. On partitioning this array with the single-pivot partitioning scheme, we'll get two partitions. The first partition will be empty, while the second partition will have (N - 1) elements. Further, each subsequent invocation of the partition procedure will reduce the input size by only one. Since the partition procedure has O(N) complexity, the overall time complexity will be O(N²). This is the worst-case scenario when the input array has many repeated elements.

To reduce the worst-case scenarios when repeated elements are in large numbers, instead of a single-pivot partitioning scheme, we can implement a three-way partitioning scheme.

Three-way Partitioning: To efficiently sort an array having a high number of repeated keys, we can place keys in the right position when we first encounter them. So three-way partitioning of the array consists of the following three partitions:

• The left-most partition contains elements that are strictly less than the pivot.
• The middle partition contains all elements which are equal to the pivot.
• The right-most partition contains all elements which are strictly greater than the pivot.

Dutch National Flag (DNF) Sorting Algorithm: It can categorize all the numbers of an array into three groups with respect to a given pivot:

• The Lesser group contains all elements that are strictly lesser than the pivot.
• The Equal group contains all elements that are equal to the pivot.
• The Greater group contains all elements that strictly greater than the pivot.

To solve the DNF problem, pick the first element as the pivot, and scan the array from left to right. As we check each element, we move it to its correct group, namely Lesser, Equal, and Greater.

To learn the implementation of the DNF Sorting algorithm, please refer to this article.

Bentley-McIlroy's Optimized 3-Way Partitioning: This algorithm is an iteration-based partitioning scheme. All the elements of the array are unexplored initially. Start exploring the elements of the array from the left and right directions. After each iteration, it can be visualized that the array as a composition of five regions:

• Extreme two ends are regions that have elements equal to the pivot value.
• The unexplored region stays in the center and its size keeps on shrinking with each iteration.
• On the left side of the unexplored region are elements lesser than the pivot value.
• On the right side of the unexplored region are elements greater than the pivot value.

As the iteration procedure continues, the unexplored region of the array keeps on decreasing, and finally, the array will have no unexplored element remaining. 

Now, since the elements equal to the pivot element is greater than elements in the Less region and less than elements in the greater region, they need to be moved to the center between the lesser and greater region. They are brought to the center by swapping with elements from lesser and greater regions.

Finally, the problem reduces to sort the elements in a lesser and greater region using the same approach again until the entire array becomes sorted.

Analysis of Three-Way Partitioning: In general, the Quick Sort Algorithm has an average-case time complexity of O(N*log(N)) and worst-case time complexity of O(N²). With a high number of duplicate keys, we always get the worst-case performance with the trivial implementation of Quick Sort.

However, using three-way partitioning reduces the impact of numerous duplicate keys. In fact, an array with all elements sorts in O(N²) time in single-pivot partitioning while it sorts in O(N) (linear) time complexity in three-way partitioning. Thus, what used to be worst-case turns out to be the best-case scenario for three-way partitioning.