Sorting

Stage II Sem III

 Objectives

• To learn how to use the standard sorting methods in the

Java API
• To learn how to implement the following sorting
algorithms: selection sort, bubble sort, insertion sort,
Shell sort, merge sort, heapsort, and quicksort
• To understand the difference in performance of these
algorithms, and which to use for small arrays, which to
use for medium arrays, and which to use for large arrays

2
 Using Java Sorting Methods

• Java API provides a class Arrays with several

overloaded sort methods for different array types
• The Collections class provides similar sorting methods
• Sorting methods for arrays of primitive types are based
on quicksort algorithm
• Method of sorting for arrays of objects and Lists based
on mergesort

3
Using Java Sorting Methods (continued)

4
 Selection Sort

• Selection sort is a relatively easy to understand

algorithm
• Sorts an array by making several passes through the
array, selecting the next smallest item in the array each
time and placing it where it belongs in the array
• Efficiency is O(n*n)

5
 Selection Sort (continued)

• Selection sort is called a quadratic sort

• Number of comparisons is O(n*n)
• Number of exchanges is O(n)

6
 Bubble Sort

• Compares adjacent array elements and exchanges their

values if they are out of order
• Smaller values bubble up to the top of the array and
larger values sink to the bottom

7
 Analysis of Bubble Sort

• Provides excellent performance in some cases and very

poor performances in other cases
• Works best when array is nearly sorted to begin with
• Worst case number of comparisons is O(n*n)
• Worst case number of exchanges is O(n*n)
• Best case occurs when the array is already sorted
• O(n) comparisons
• O(1) exchanges

8
 Insertion Sort

• Based on the technique used by card players to arrange

a hand of cards
• Player keeps the cards that have been picked up so
far in sorted order
• When the player picks up a new card, he makes room
for the new card and then inserts it in its proper place

9
 Insertion Sort Algorithm

• For each array element from the second to the last

(nextPos = 1)
• Insert the element at nextPos where it belongs in the
array, increasing the length of the sorted subarray by
1

10
 Analysis of Insertion Sort

• Maximum number of comparisons is O(n*n)

• In the best case, number of comparisons is O(n)
• The number of shifts performed during an insertion is
one less than the number of comparisons or, when the
new value is the smallest so far, the same as the number
of comparisons
• A shift in an insertion sort requires the movement of only
one item whereas in a bubble or selection sort an
exchange involves a temporary item and requires the
movement of three items

11
 Comparison of Quadratic Sorts

• None of the algorithms are particularly good for large

arrays

12
 Shell Sort: A Better Insertion Sort
• Shell sort is a type of insertion sort but with O(n^(3/2)) or
better performance
• Named after its discoverer, Donald Shell
• Divide and conquer approach to insertion sort
• Instead of sorting the entire array, sort many smaller
subarrays using insertion sort before sorting the entire
array

13
 Analysis of Shell Sort

• A general analysis of Shell sort is an open research

problem in computer science
• Performance depends on how the decreasing sequence
of values for gap is chosen
• If successive powers of two are used for gap,
performance is O(n*n)
• If Hibbard’s sequence is used, performance is O(n^(3/2))

14
 Merge Sort

• A merge is a common data processing operation that is

performed on two sequences of data with the following
characteristics
• Both sequences contain items with a common
compareTo method
• The objects in both sequences are ordered in
accordance with this compareTo method

15
 Merge Algorithm

• Merge Algorithm
• Access the first item from both sequences
• While not finished with either sequence
• Compare the current items from the two sequences, copy the
smaller current item to the output sequence, and access the
next item from the input sequence whose item was copied
• Copy any remaining items from the first sequence to
the output sequence
• Copy any remaining items from the second sequence
to the output sequence

16
 Analysis of Merge

• For two input sequences that contain a total of n

elements, we need to move each element’s input
sequence to its output sequence
• Merge time is O(n)
• We need to be able to store both initial sequences and
the output sequence
• The array cannot be merged in place
• Additional space usage is O(n)

17
Algorithm and Trace of Merge Sort

18
mergeSort example

19
Algorithm and Trace of Merge Sort
(continued)

20
 Simple MergeSort Algorithm
function merge_sort(m)
var list left, right, result
if length(m) ≤ 1
return m

var middle = length(m) / 2 - 1

for each x in m up to middle
add x to left
for each x in m after middle
add x to right
left = merge_sort(left)
right = merge_sort(right)
if left.last_item > right.first_item
result = merge(left, right)
else
result = append(left, right)
return result

21
 Merge function
• function merge(left,right)
• var list result
• while length(left) > 0 and length(right) > 0
• if first(left) ≤ first(right)
• append first(left) to result
• left = rest(left)
• else
• append first(right) to result
• right = rest(right)
• end while
• if length(left) > 0
• append left to result
• else
• append right to result
• return result

22
• In sorting n objects, merge sort has an average and worst-case
performance of O(n log n). If the running time of merge sort for a list
of length n is T(n), then the recurrence T(n) = 2T(n/2) + n follows
from the definition of the algorithm (apply the algorithm to two lists of
half the size of the original list, and add the n steps taken to merge
the resulting two lists). The closed form follows from the master
theorem.
• In the worst case, merge sort does an amount of comparisons equal
to or slightly smaller than (n ⌈lg n⌉ - 2⌈lg n⌉ + 1), which is between
(n lg n - n + 1) and (n lg n + n + O(lg n)). [1]
• For large n and a randomly ordered input list, merge sort's expected
(average) number of comparisons approaches α·n fewer than the
worst case where
• In the worst case, merge sort does about 39% fewer comparisons
than quicksort does in the average case; merge sort always makes
fewer comparisons than quicksort, except in extremely rare cases,
when they tie, where merge sort's worst case is found
simultaneously with quicksort's best case. In terms of moves, merge
sort's worst case complexity is O(n log n)—the same complexity as
quicksort's best case, and merge sort's best case takes about half
as many iterations as the worst case.

23
 Heapsort

• Merge sort time is O(n log n) but still requires,

temporarily, n extra storage items
• Heapsort does not require any additional storage

24
 Algorithm for In-Place Heapsort

• Build a heap by arranging the elements in an unsorted

array
• While the heap is not empty
• Remove the first item from the heap by swapping it
with the last item and restoring the heap property

25
 Quicksort

• Developed in 1962
• Quicksort rearranges an array into two parts so that all
the elements in the left subarray are less than or equal to
a specified value, called the pivot
• Quicksort ensures that the elements in the right subarray
are larger than the pivot
• Average case for Quicksort is O(n log n)

26
Quicksort (continued)

27
28
Algorithm for Partitioning

29
 Revised Partition Algorithm

• Quicksort is O(n*n) when each split yields one empty

subarray, which is the case when the array is presorted
• Best solution is to pick the pivot value in a way that is
less likely to lead to a bad split
• Requires three markers
• First, middle, last
• Select the median of the these items as the pivot

30
 Testing the Sort Algorithms

• Need to use a variety of test cases

• Small and large arrays
• Arrays in random order
• Arrays that are already sorted
• Arrays with duplicate values
• Compare performance on each type of array

31
 The Dutch National Flag Problem

• A variety of partitioning algorithms for quicksort have

been published
• A partitioning algorithm for partitioning an array into three
segments was introduced by Edsger W. Dijkstra
• Problem is to partition a disordered three-color flag into
the appropriate three segments

32
The Dutch National Flag Problem

33
 Chapter Review
• Comparison of several sorting algorithms were made
• Three quadratic sorting algorithms are selection sort,
bubble sort, and insertion sort
• Shell sort gives satisfactory performance for arrays up to
5000 elements
• Quicksort has an average-case performance of O(n log
n), but if the pivot is picked poorly, the worst case
performance is O(n*n)
• Merge sort and heapsort have O(n log n) performance

34
 Chapter Review (continued)

• The Java API contains “industrial strength” sort

algorithms in the classes java.util.Arrays and
java.util.Collections

35
 Bubble sort
• Compare each element (except the last one) with its
neighbor to the right
• If they are out of order, swap them
• This puts the largest element at the very end
• The last element is now in the correct and final place
• Compare each element (except the last two) with its
neighbor to the right
• If they are out of order, swap them
• This puts the second largest element next to last
• The last two elements are now in their correct and final places
• Compare each element (except the last three) with its
neighbor to the right
• Continue as above until you have no unsorted elements on the
left
36
 Example of bubble sort

7 2 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8

2 7 8 5 4 2 7 5 4 8 2 5 4 7 8 2 4 5 7 8

2 7 8 5 4 2 5 7 4 8 2 4 5 7 8 (done)

2 7 5 8 4 2 5 4 7 8

2 7 5 4 8

37
 Code for bubble sort
• public static void bubbleSort(int[] a) {
int outer, inner;
for (outer = a.length - 1; outer > 0; outer--) { //
counting down
for (inner = 0; inner < outer; inner++) { //
bubbling up
if (a[inner] > a[inner + 1]) { // if out of order...
int temp = a[inner]; // ...then swap
a[inner] = a[inner + 1];
a[inner + 1] = temp;
}
}
}
}

38
 Analysis of bubble sort
• for (outer = a.length - 1; outer > 0; outer--) {
for (inner = 0; inner < outer; inner++) {
if (a[inner] > a[inner + 1]) {
// code for swap omitted
}
}
}
• Let n = a.length = size of the array
• The outer loop is executed n-1 times (call it n, that’s close enough)
• Each time the outer loop is executed, the inner loop is executed
• Inner loop executes n-1 times at first, linearly dropping to just
once
• On average, inner loop executes about n/2 times for each
execution of the outer loop
• In the inner loop, the comparison is always done (constant
time), the swap might be done (also constant time)
• Result is n * n/2 * k, that is, O(n2/2 + k) = O(n2) 39
 Loop invariants

• You run a loop in order to change things

• Oddly enough, what is usually most important in
understanding a loop is finding an invariant: that is, a
condition that doesn’t change
• In bubble sort, we put the largest elements at the end,
and once we put them there, we don’t move them again
• The variable outer starts at the last index in the array and
decreases to 0
• Our invariant is: Every element to the right of outer is in the
correct place
• That is, for all j > outer, if i < j, then a[i] <= a[j]
• When this is combined with outer == 0, we know that all
elements of the array are in the correct place

40
 Selection sort

• Given an array of length n,

• Search elements 0 through n-1 and select the smallest
• Swap it with the element in location 0
• Search elements 1 through n-1 and select the smallest
• Swap it with the element in location 1
• Search elements 2 through n-1 and select the smallest
• Swap it with the element in location 2
• Search elements 3 through n-1 and select the smallest
• Swap it with the element in location 3
• Continue in this fashion until there’s nothing left to search

41
Example and analysis of selection sort

• The selection sort might swap an array

7 2 8 5 4
element with itself--this is harmless, and
not worth checking for
2 7 8 5 4 • Analysis:
• The outer loop executes n-1 times
2 4 8 5 7 • The inner loop executes about n/2
times on average (from n to 2 times)
2 4 5 8 7 • Work done in the inner loop is
constant (swap two array elements)
2 4 5 7 8 • Time required is roughly (n-1)*(n/2)
• You should recognize this as O(n2)

42
 Code for selection sort
public static void selectionSort(int[] a) {
int outer, inner, min;
for (outer = 0; outer < a.length - 1; outer++) { // outer counts
down
min = outer;
for (inner = outer + 1; inner < a.length; inner++) {
if (a[inner] < a[min]) {
min = inner;
}
// Invariant: for all i, if outer <= i <= inner, then a[min]
<= a[i]
}
// a[min] is least among a[outer]..a[a.length - 1]
int temp = a[outer];
a[outer] = a[min];
a[min] = temp;
// Invariant: for all i <= outer, if i < j then a[i] <= a[j]
}
}
43
 Invariants for selection sort

• For the inner loop:

• This loop searches through the array, incrementing inner
from its initial value of outer+1 up to a.length-1
• As the loop proceeds, min is set to the index of the
smallest number found so far
• Our invariant is:
for all i such that outer <= i <= inner, a[min] <= a[i]
• For the outer (enclosing) loop:
• The loop counts up from outer = 0
• Each time through the loop, the minimum remaining value
is put in a[outer]
• Our invariant is:
for all i <= outer, if i < j then a[i] <= a[j]

44
 Insertion sort

• The outer loop of insertion sort is:

for (outer = 1; outer < a.length; outer++) {...}
• The invariant is that all the elements to the left of outer
are sorted with respect to one another
• For all i < outer, j < outer, if i < j then a[i] <= a[j]
• This does not mean they are all in their final correct place; the
remaining array elements may need to be inserted
• When we increase outer, a[outer-1] becomes to its left; we
must keep the invariant true by inserting a[outer-1] into its
proper place
• This means:
• Finding the element’s proper place
• Making room for the inserted element (by shifting over other
elements)
• Inserting the element 45
 One step of insertion sort

sorted next to be inserted

3 4 7 12 14 14 20 21 33 38 10 55 9 23 28 16
temp
less than
10
10
3 4 7 10 12 14 14 20 21 33 38 55 9 23 28 16

sorted

46
 Analysis of insertion sort

• We run once through the outer loop, inserting each of n

elements; this is a factor of n
• On average, there are n/2 elements already sorted
• The inner loop looks at (and moves) half of these
• This gives a second factor of n/4
• Hence, the time required for an insertion sort of an array
of n elements is proportional to n2/4
• Discarding constants, we find that insertion sort is O(n2)

47