Faculty of Science

Lecture 5. Sorting Algorithms

Commonwealth of Australia Copyright Act 1968
Notice for paragraph 135ZXA (a) of the Copyright Act 1968
Warning
This Material has been reproduced and communicated to you by or on behalf of Federation University
Australia under Part VB of the Copyright Act 1968 (the Act).
The Material in this communication may be subject to copyright under the Act. Any further reproduction
or communication of this material by you may be the subject of copyright protection under the Act.
Do not remove this notice.
Lecture 5.Sorting Algorithms

Lecture 5.Sorting Algorithms

Lecture 5.Sorting Algorithms
Outline

Selection Sort

Bubble Sort

Insertion Sort

Mergesort

Quicksort

A comparison of Sorting Algorithms

Lecture 5.Sorting Algorithms
Outline

Sorting is a process that organizes a collection of data into

either ascending or descending order.
Lecture 5.Sorting Algorithms
Outline

Sorting is a process that organizes a collection of data into

either ascending or descending order.

Often you must perform a sort as an initialization step for

certain algorithm. For example, the binary search algorithm
requires that the data be sorted.
Lecture 5.Sorting Algorithms
Outline

Sorting is a process that organizes a collection of data into

either ascending or descending order.

Often you must perform a sort as an initialization step for

certain algorithm. For example, the binary search algorithm
requires that the data be sorted.

The data items to be sorted might be integers, character

strings, or even structures.
Lecture 5.Sorting Algorithms
Outline

Sorting is a process that organizes a collection of data into

either ascending or descending order.

Often you must perform a sort as an initialization step for

certain algorithm. For example, the binary search algorithm
requires that the data be sorted.

The data items to be sorted might be integers, character

strings, or even structures.
Lecture 5.Sorting Algorithms
Selection Sort

To sort a list of items the selection sort selects the largest item
and puts it in its place, then selects the next largest item and
puts it in its place, and so on.
Lecture 5.Sorting Algorithms
Selection Sort

To sort a list of items the selection sort selects the largest item
and puts it in its place, then selects the next largest item and
puts it in its place, and so on.
For example, to sort an array into ascending order, you first
search it for the largest item, and swap it with the the last item,
even if these items happen to be identical.
Lecture 5.Sorting Algorithms
Selection Sort

To sort a list of items the selection sort selects the largest item
and puts it in its place, then selects the next largest item and
puts it in its place, and so on.
For example, to sort an array into ascending order, you first
search it for the largest item, and swap it with the the last item,
even if these items happen to be identical.
Now, ignoring the last - and largest - item of the array, you
search the rest of the array for its largest item and swap it with
next-to-last item in the original array.
Lecture 5.Sorting Algorithms
Selection Sort

To sort a list of items the selection sort selects the largest item
and puts it in its place, then selects the next largest item and
puts it in its place, and so on.
For example, to sort an array into ascending order, you first
search it for the largest item, and swap it with the the last item,
even if these items happen to be identical.
Now, ignoring the last - and largest - item of the array, you
search the rest of the array for its largest item and swap it with
next-to-last item in the original array.
You continue until you have selected and swapped n 1 of the
n items in the array.
Lecture 5.Sorting Algorithms
Selection Sort

To sort a list of items the selection sort selects the largest item
and puts it in its place, then selects the next largest item and
puts it in its place, and so on.
For example, to sort an array into ascending order, you first
search it for the largest item, and swap it with the the last item,
even if these items happen to be identical.
Now, ignoring the last - and largest - item of the array, you
search the rest of the array for its largest item and swap it with
next-to-last item in the original array.
You continue until you have selected and swapped n 1 of the
n items in the array.
The remaining item, which is now in the first position in the
array, is in its proper order.
Lecture 5.Sorting Algorithms
Selection Sort

Example of the selection sort algorithm

5 2 7 3 4 1 6
Lecture 5.Sorting Algorithms
Selection Sort

Example of the selection sort algorithm

5 2 6 3 4 1 7
Lecture 5.Sorting Algorithms
Selection Sort

Example of the selection sort algorithm

5 2 1 3 4 6 7
Lecture 5.Sorting Algorithms
Selection Sort

Example of the selection sort algorithm

4 2 1 3 5 6 7
Lecture 5.Sorting Algorithms
Selection Sort

Example of the selection sort algorithm

3 2 1 4 5 6 7
Lecture 5.Sorting Algorithms
Selection Sort

Example of the selection sort algorithm

1 2 3 4 5 6 7
Lecture 5.Sorting Algorithms
Selection Sort

Example of the selection sort algorithm

1 2 3 4 5 6 7
Lecture 5.Sorting Algorithms
Selection Sort

C implementation of the selection sort algorithm

void swap(int*, int, int);

int indOfMax(int*, int, int);

void selectionSort(int *v, int size){

int last;
for(last = size - 1; last >= 1; last--){
int largest = indOfMax(v, 0, last + 1);
swap(v, largest, last);
}
}
Lecture 5.Sorting Algorithms
Selection Sort

int indOfMax(int *v, int from,int to){

int ind = from;
int max = v[from];

for(int i = from; i < to; i++){

if(max < v[i]){
max = v[i];
ind = i;
}
}
return ind;
}
Lecture 5.Sorting Algorithms
Selection Sort

void swap(int *v, int first, int second){

int temp;
temp = v[first];
v[first] = v[second];
v[second] = temp;
}
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of sorting algorithms

As you can see from the previous algorithm, sorting in general

compares, exchanges, or moves items.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of sorting algorithms

As you can see from the previous algorithm, sorting in general

compares, exchanges, or moves items.
Depending on the programming language and implementation
used for storing the data, the cost associated with each of
these operations varies.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of sorting algorithms

As you can see from the previous algorithm, sorting in general

compares, exchanges, or moves items.
Depending on the programming language and implementation
used for storing the data, the cost associated with each of
these operations varies.
Moving or exchanging data in C is not expensive, since you
always can move or exchange pointers (addresses) of the
manipulated objects.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of sorting algorithms

As you can see from the previous algorithm, sorting in general

compares, exchanges, or moves items.
Depending on the programming language and implementation
used for storing the data, the cost associated with each of
these operations varies.
Moving or exchanging data in C is not expensive, since you
always can move or exchange pointers (addresses) of the
manipulated objects.
The comparison operation is typically more involved, since a
comparison of two structures may be quite sophisticated.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of sorting algorithms

As a first step in analyzing sorting algorithms, we should count
the move, exchange, and compare operations.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of sorting algorithms

As a first step in analyzing sorting algorithms, we should count
the move, exchange, and compare operations.
When a sorting algorithm is designed to work with the arrays of
references, the comparison operation is usually the most
expensive and is thus often the only operation analyzed.
However, since other implementations may incur more expense
in moving or exchanging data, we provide some analysis of
these operations here.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of sorting algorithms

As a first step in analyzing sorting algorithms, we should count
the move, exchange, and compare operations.
When a sorting algorithm is designed to work with the arrays of
references, the comparison operation is usually the most
expensive and is thus often the only operation analyzed.
However, since other implementations may incur more expense
in moving or exchanging data, we provide some analysis of
these operations here.
Generally, move, exchange, and compare operations are more
expensive than the ones that control loops or manipulate array
indexes, particularly when the data to be sorted is more
complex than integers or characters. Thus, we will ignore these
less expensive operations.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of the selection sort algorithm

I Clearly, the for loop in the method selectionSort executes
n 1 times. Thus, selectionSort calls the methods
indOfMax and swap n 1 times.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of the selection sort algorithm

I Clearly, the for loop in the method selectionSort executes
n 1 times. Thus, selectionSort calls the methods
indOfMax and swap n 1 times.
I Total n 1 calls to indOfMax for values of last that range
from n 1 down to 1, cause the loop in indOfMax to
execute a total of
n (n 1)
(n 1) + (n 2) + ... + 1 =
2
times.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of the selection sort algorithm

I Clearly, the for loop in the method selectionSort executes
n 1 times. Thus, selectionSort calls the methods
indOfMax and swap n 1 times.
I Total n 1 calls to indOfMax for values of last that range
from n 1 down to 1, cause the loop in indOfMax to
execute a total of
n (n 1)
(n 1) + (n 2) + ... + 1 =
2
times.
I Because each execution of indOfMaxs loop performs one
comparison, the calls to indOfMax require n(n1)
2
comparisons.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of the selection sort algorithm

I Total n 1 calls to swap requires 3(n 1) assignments, or
data moves.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of the selection sort algorithm

I Total n 1 calls to swap requires 3(n 1) assignments, or
data moves.
I Together, a selection sort of n items requires

n (n 1) n2 5n
+ 3(n 1) = + 3
2 2 2
major operations. Thus, selection sort is O(n2 ).
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of the selection sort algorithm

I Total n 1 calls to swap requires 3(n 1) assignments, or
data moves.
I Together, a selection sort of n items requires

n (n 1) n2 5n
+ 3(n 1) = + 3
2 2 2
major operations. Thus, selection sort is O(n2 ).
I A selection sort is appropriate only for small n because
O(n2 ) grows rapidly.
Lecture 5.Sorting Algorithms
Selection Sort

Analysis of the selection sort algorithm

I Total n 1 calls to swap requires 3(n 1) assignments, or
data moves.
I Together, a selection sort of n items requires

n (n 1) n2 5n
+ 3(n 1) = + 3
2 2 2
major operations. Thus, selection sort is O(n2 ).
I A selection sort is appropriate only for small n because
O(n2 ) grows rapidly.
I While the algorithm requires O(n2 ) comparisons, it
requires only O(n) data moves. Thus, a selection sort
could be a good choice over other methods when data
moves are costly but comparisons are not.
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

5 2 7 3 6 1 4
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

2 5 7 3 6 1 4
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

2 5 7 3 6 1 4
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

2 5 3 7 6 1 4
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

2 5 3 6 7 1 4
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

2 5 3 6 1 7 4
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

2 5 3 6 1 4 7
Lecture 5.Sorting Algorithms
Bubble Sort

The bubble sort algorithm compares adjacent items and

exchanges them if they are out of order. This sort usually
requires several passes over the data.
During the first pass, you compare the first two items in the
array. If they are out of order, you exchange them.
You then compare the items in the next pair - that is , in
positions 2 and 3 of the array. If they are out of order, you
exchange them.
You proceed, comparing and exchanging items two at a time
until you reach the end of the array.

2 5 3 6 1 4 7
Although the array is not sorted after the first pass, the largest
item has "bubbled" to its proper position in the end of the array.
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 5 3 6 1 4 7
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 5 3 6 1 4 7
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 3 5 6 1 4 7
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 3 5 6 1 4 7
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 3 5 1 6 4 7
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 3 5 1 4 6 7
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 3 5 1 4 6 7
After the second pass, the second largest item in the array will
be in its proper place.
Lecture 5.Sorting Algorithms
Bubble Sort

During the second pass of the bubble sort, you return to the
beginning of the array and consider pairs of items in exactly the
same manner as the first pass.
You do not, however, include the last - and largest - item of the
array. That is, the second pass considers the first n 1 items of
the array.

2 3 5 1 4 6 7
After the second pass, the second largest item in the array will
be in its proper place.
Now, ignoring the last two items, which are in order, you
continue with subsequent passes until the array is sorted.
Lecture 5.Sorting Algorithms
Bubble Sort

Although a bubble sort requires at most n 1 passes to sort the

array, fewer passes might be possible to sort a particular array.
Lecture 5.Sorting Algorithms
Bubble Sort

Although a bubble sort requires at most n 1 passes to sort the

array, fewer passes might be possible to sort a particular array.

Thus, you could terminate the process if no exchanges occur

during any pass.
Lecture 5.Sorting Algorithms
Bubble Sort

Although a bubble sort requires at most n 1 passes to sort the

array, fewer passes might be possible to sort a particular array.

Thus, you could terminate the process if no exchanges occur

during any pass.

The following C function bubbleSort uses a flag to signal when

an exchange occurs during a particular pass.
Lecture 5.Sorting Algorithms
Bubble Sort

C implementation of the bubble sort algorithm

void bubbleSort(int* v, int size) {

int sorted = 0, pass;
for(pass = 1; pass < size && !sorted; pass++) {
sorted = 1;
int i;
for(i = 0; i < size - pass; i++){
if(v[i + 1] < v[i]){
swap(v, i, i + 1);
sorted = 0;
}
}
}
}
Lecture 5.Sorting Algorithms
Bubble Sort

Analysis of the bubble sort algorithm

The bubble sort requires at most n 1 passes through the
array. In general, pass i requires n i comparisons and at most
n i exchanges.
Lecture 5.Sorting Algorithms
Bubble Sort

Analysis of the bubble sort algorithm

The bubble sort requires at most n 1 passes through the
array. In general, pass i requires n i comparisons and at most
n i exchanges.
Therefore, in the worst case, a bubble sort will require a total of

n (n 1)
(n 1) + (n 2) + ... + 1 =
2
comparisons and the same number of exchanges.
Lecture 5.Sorting Algorithms
Bubble Sort

Analysis of the bubble sort algorithm

The bubble sort requires at most n 1 passes through the
array. In general, pass i requires n i comparisons and at most
n i exchanges.
Therefore, in the worst case, a bubble sort will require a total of

n (n 1)
(n 1) + (n 2) + ... + 1 =
2
comparisons and the same number of exchanges.
Therefore, the bubble sort algorithm is O(n2 ) in the worst case.
Lecture 5.Sorting Algorithms
Bubble Sort

Analysis of the bubble sort algorithm

The bubble sort requires at most n 1 passes through the
array. In general, pass i requires n i comparisons and at most
n i exchanges.
Therefore, in the worst case, a bubble sort will require a total of

n (n 1)
(n 1) + (n 2) + ... + 1 =
2
comparisons and the same number of exchanges.
Therefore, the bubble sort algorithm is O(n2 ) in the worst case.
The best case occures when the original data is already sorted:
bubbleSort uses one pass, during which no comparisons and
no exchanges occur.Thus, the bubble sort is O(n) in the best
case.
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

5 2 7 3 6 1 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

5 2 7 3 6 1 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 5 7 3 6 1 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 5 7 3 6 1 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 5 3 7 6 1 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 3 5 7 6 1 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 3 5 6 7 1 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 3 5 6 1 7 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 3 5 1 6 7 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 3 1 5 6 7 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

2 1 3 5 6 7 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

1 2 3 5 6 7 4
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

1 2 3 5 6 4 7
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

1 2 3 5 4 6 7
Lecture 5.Sorting Algorithms
Insertion Sort

Insertion sort partitions an array into two regions: sorted and

unsorted. Initially, the entire array is the unsorted region.
At each step, the insertion sort takes the first item of the
unsorted region and places it into its correct position in the
sorted region.
Below is the example of how the insertion sort works.

1 2 3 4 5 6 7
Lecture 5.Sorting Algorithms
Insertion Sort

C implementation of the insertion sort algorithm

void insertionSort(int *v, size) {

int i;
for(i =1; i < size; i++) {
int t = v[i];
int j;
for( j = i-1; j >= 0 && t < v[j]; j--)
v[j+1] = v[j];
v[j+1] = t;
}
}
Lecture 5.Sorting Algorithms
Insertion Sort

Analysis of the insertion sort algorithm

The insertion sort algorithm requires

n (n 1)
1 + 2 + ... + (n 1) =
2
comparisons in the worst case. Which occurs if the initial data
is sorted in the descending order.
Lecture 5.Sorting Algorithms
Insertion Sort

Analysis of the insertion sort algorithm

The insertion sort algorithm requires

n (n 1)
1 + 2 + ... + (n 1) =
2
comparisons in the worst case. Which occurs if the initial data
is sorted in the descending order.

For small arrays the simplicity of the insertion sort makes it an

appropriate choice. For large arrays, however, an insertion sort
can be prohibitively inefficient.
Lecture 5.Sorting Algorithms
Insertion Sort

A Lower Bound for Simple Sorting Algorithms

All sorting methods considered so far, compared and swapped

only adjacent elements of the array. For such sorting methods it
is easy to prove the following theorem.

Theorem. Any algorithm that sorts by exchanging adjacent

elements requires (n2 ) time on average.
Lecture 5.Sorting Algorithms
Mergesort

Mergesort is a recursive sorting algorithm that always gives the

same performance, regardless of the initial order of the array
items.
Lecture 5.Sorting Algorithms
Mergesort

Mergesort is a recursive sorting algorithm that always gives the

same performance, regardless of the initial order of the array
items.
Mergesort halves the array, recursively sorts its halves, and
then merges the halves. Obviously, most effort in the mergesort
algorithm is in the merge step. So, let us discuss it.
Lecture 5.Sorting Algorithms
Mergesort

Mergesort is a recursive sorting algorithm that always gives the

same performance, regardless of the initial order of the array
items.
Mergesort halves the array, recursively sorts its halves, and
then merges the halves. Obviously, most effort in the mergesort
algorithm is in the merge step. So, let us discuss it.
Suppose that you are going to merge two sorted arrays <1,4,8>
and <2,3>.
Lecture 5.Sorting Algorithms
Mergesort

Mergesort is a recursive sorting algorithm that always gives the

same performance, regardless of the initial order of the array
items.
Mergesort halves the array, recursively sorts its halves, and
then merges the halves. Obviously, most effort in the mergesort
algorithm is in the merge step. So, let us discuss it.
Suppose that you are going to merge two sorted arrays <1,4,8>
and <2,3>.
This merge compares an item in one array with an item in the
other array and moves the smaller item to a temporary array.
Lecture 5.Sorting Algorithms
Mergesort

Mergesort is a recursive sorting algorithm that always gives the

same performance, regardless of the initial order of the array
items.
Mergesort halves the array, recursively sorts its halves, and
then merges the halves. Obviously, most effort in the mergesort
algorithm is in the merge step. So, let us discuss it.
Suppose that you are going to merge two sorted arrays <1,4,8>
and <2,3>.
This merge compares an item in one array with an item in the
other array and moves the smaller item to a temporary array.
This process continues until there are no more items to
consider in one array. At that time, you simply move the
remaining items to the temporary array.
Lecture 5.Sorting Algorithms
Mergesort

Illustration of merging of two arrays

1 4 8
2 3
Lecture 5.Sorting Algorithms
Mergesort

Illustration of merging of two arrays

1 4 8
2 3
Lecture 5.Sorting Algorithms
Mergesort

Illustration of merging of two arrays

1 4 8
2 3
1
Lecture 5.Sorting Algorithms
Mergesort

Illustration of merging of two arrays

1 4 8
2 3
1 2
Lecture 5.Sorting Algorithms
Mergesort

Illustration of merging of two arrays

1 4 8
2 3
1 2 3
Lecture 5.Sorting Algorithms
Mergesort

Illustration of merging of two arrays

1 4 8
2 3
1 2 3 4 8
Lecture 5.Sorting Algorithms
Mergesort

C implementation of the mergesort algorithm

void mergeSort(int *v, int size) {

mergeSortPart(v, 0, size - 1);
}

void mergeSortPart(int *v,int first,int last){

if(first < last){
int mid = (first + last)/2;
mergeSortPart(v, first, mid);
mergeSortPart(v, mid + 1, last);
merge(v,first, mid, last);
}
}
Lecture 5.Sorting Algorithms
Mergesort

C implementation of the mergesort algorithm

void merge(int *v, int first, int mid,int last){

int temp[last + 1];
int first1 = first; int last1 = mid; int first2 = mid + 1;
int last2 = last; int index = first1;
while((first1 <= last1) && (first2 <= last2)){
if(v[first1 ]< v[first2])
temp[index++] = v[first1++];
else
temp[index++] = v[first2++];
}
while(first1 <= last1)
temp[index++] = v[first1++];
while(first2 <= last2)
temp[index++] = v[first2++];
for(index = first; index <= last; index++)
v[index] = temp[index];
}
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

Because the merge step of the algorithm requires the most

effort, lets begin the analysis there.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

Because the merge step of the algorithm requires the most

effort, lets begin the analysis there.
If the total number of items in the two array segments to be
merged is n, then merging the segments requires at most n 1
comparisons.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

Because the merge step of the algorithm requires the most

effort, lets begin the analysis there.
If the total number of items in the two array segments to be
merged is n, then merging the segments requires at most n 1
comparisons.
In addition, there are n moves from the original array to the
temporary array, and n moves from the temporary array back to
the original array.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

Because the merge step of the algorithm requires the most

effort, lets begin the analysis there.
If the total number of items in the two array segments to be
merged is n, then merging the segments requires at most n 1
comparisons.
In addition, there are n moves from the original array to the
temporary array, and n moves from the temporary array back to
the original array.
Thus, each merge step requires 3n 1 major operations.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

Because the merge step of the algorithm requires the most

effort, lets begin the analysis there.
If the total number of items in the two array segments to be
merged is n, then merging the segments requires at most n 1
comparisons.
In addition, there are n moves from the original array to the
temporary array, and n moves from the temporary array back to
the original array.
Thus, each merge step requires 3n 1 major operations.
Each call of mergesort recursively calls itself twice. And each
call halves the array. So, there are 1 + log2 n(rounded down)
levels of recursive calls to mergesort.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

At level m of the recursion 2m calls to merge occur; each of

these calls merges n/2m items and so requires 3n/2m 1
operations.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

At level m of the recursion 2m calls to merge occur; each of

these calls merges n/2m items and so requires 3n/2m 1
operations.
Together the 2m calls to merge require 3n 2m operations.
Thus each level of the recursion requires O(n) operations.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

At level m of the recursion 2m calls to merge occur; each of

these calls merges n/2m items and so requires 3n/2m 1
operations.
Together the 2m calls to merge require 3n 2m operations.
Thus each level of the recursion requires O(n) operations.
Because there are no more then 1 + log2 n levels, mergesort is
O(n log2 n) in both the worst and average case.
Lecture 5.Sorting Algorithms
Mergesort

Analysis of the mergesort algorithm

At level m of the recursion 2m calls to merge occur; each of

these calls merges n/2m items and so requires 3n/2m 1
operations.
Together the 2m calls to merge require 3n 2m operations.
Thus each level of the recursion requires O(n) operations.
Because there are no more then 1 + log2 n levels, mergesort is
O(n log2 n) in both the worst and average case.
Although mergesort is an extremely efficient algorithm with
respect to time, it does have one drawback: To perform the step
the algorithm requires an auxiliary array whose size equals the
size of the original array. In our example, this array is simply a
vector of references and hence has little impact.
Lecture 5.Sorting Algorithms
Quicksort

The Quicksort algorithm requires the following steps:

Lecture 5.Sorting Algorithms
Quicksort

The Quicksort algorithm requires the following steps:

I Choose an element of the array to act as a "pivot"
Lecture 5.Sorting Algorithms
Quicksort

The Quicksort algorithm requires the following steps:

I Choose an element of the array to act as a "pivot"
I Place the pivot in position so that it has the following
property
Lecture 5.Sorting Algorithms
Quicksort

The Quicksort algorithm requires the following steps:

I Choose an element of the array to act as a "pivot"
I Place the pivot in position so that it has the following
property
I All of the elements that are smaller than it appear before
the pivot
Lecture 5.Sorting Algorithms
Quicksort

The Quicksort algorithm requires the following steps:

I Choose an element of the array to act as a "pivot"
I Place the pivot in position so that it has the following
property
I All of the elements that are smaller than it appear before
the pivot
I All of the elements that are greater than it appear after the
pivot
Lecture 5.Sorting Algorithms
Quicksort

The Quicksort algorithm requires the following steps:

I Choose an element of the array to act as a "pivot"
I Place the pivot in position so that it has the following
property
I All of the elements that are smaller than it appear before
the pivot
I All of the elements that are greater than it appear after the
pivot
I Use Quicksort to sort the subsections of the array
(provided they consist of more than one element -
otherwise they do not need sorting).
Lecture 5.Sorting Algorithms
Quicksort

The first task is to find a pivot. We will use the first element of
the collection.
Lecture 5.Sorting Algorithms
Quicksort

The first task is to find a pivot. We will use the first element of
the collection.
The hard work of Quicksort involves the second step - ie
placing the pivot in the correct position.
Lecture 5.Sorting Algorithms
Quicksort

The first task is to find a pivot. We will use the first element of
the collection.
The hard work of Quicksort involves the second step - ie
placing the pivot in the correct position.
One way to do this is through the use of two indicators. One
indicator is placed at the lower end of the collection, the other is
placed at the upper end.
Lecture 5.Sorting Algorithms
Quicksort

The first task is to find a pivot. We will use the first element of
the collection.
The hard work of Quicksort involves the second step - ie
placing the pivot in the correct position.
One way to do this is through the use of two indicators. One
indicator is placed at the lower end of the collection, the other is
placed at the upper end.
The left indicator moves up the collection until it either finds an
element that is greater than the pivot or it crosses the other
indicator.
Lecture 5.Sorting Algorithms
Quicksort

The first task is to find a pivot. We will use the first element of
the collection.
The hard work of Quicksort involves the second step - ie
placing the pivot in the correct position.
One way to do this is through the use of two indicators. One
indicator is placed at the lower end of the collection, the other is
placed at the upper end.
The left indicator moves up the collection until it either finds an
element that is greater than the pivot or it crosses the other
indicator.
If the former then the right indicator moves down the collection
until it finds an element less than or equal to the pivot. The two
elements thus indicated are swapped.
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)

12 3 20 19 5 25 7
L R
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)

7 3 20 19 5 25 12
L R
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)

7 3 20 19 5 25 12
L R
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)

7 3 12 19 5 25 20
L R
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)

7 3 12 19 5 25 20
L R
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)

7 3 5 19 12 25 20
L R
Lecture 5.Sorting Algorithms
Quicksort

The process continues until the two indicator cross or when the
pivot is swapped with the element marked by the right indicator
moving down the list.
Lets look at how it works. Assume we wish to sort the following
collection (we will choose 12 to be the pivot, being the first
element of the collection)

7 3 5 12 19 25 20
L/R
Lecture 5.Sorting Algorithms
Quicksort

C implementation of the quicksort algorithm

void quickSort(int *v, int size){

quickSortPart(v, 0, size-1);
}

void quickSortPart(int *v,int left, int right){

int pivot = partition(v,left,right);
if (left < pivot-1)
quickSortPart(v, left, pivot-1);
if(pivot+1<right)
quickSortPart(v, pivot+1,right);
return;
}
Lecture 5.Sorting Algorithms
Quicksort

C implementation of the quicksort algorithm

int partition(int *v,int left,int right){

while(true){
while(left < right && v[left]< v[right])
right--;
if(left < right)
swap(v, left++, right);
else
return left;
while(left < right && v[left]<v[right])
left++;
if(left < right)
swap(v,left,right--);
else
return right;
}
}
Lecture 5.Sorting Algorithms
Quicksort

Analysis of the quicksort algorithm

The major effort in the quicksort method occurs during the

partitioning step.
Lecture 5.Sorting Algorithms
Quicksort

Analysis of the quicksort algorithm

The major effort in the quicksort method occurs during the

partitioning step.
If each partition splits the collection into equal parts then the
efficiency of quicksort is O(n log2 n).
Lecture 5.Sorting Algorithms
Quicksort

Analysis of the quicksort algorithm

The major effort in the quicksort method occurs during the

partitioning step.
If each partition splits the collection into equal parts then the
efficiency of quicksort is O(n log2 n).
However, if the array is already sorted into ascending order,
and you choose the first array item as the pivot, the number of
comparisons required is n(n 1)/2 - this is the worst case!
Lecture 5.Sorting Algorithms
Quicksort

Analysis of the quicksort algorithm

The major effort in the quicksort method occurs during the

partitioning step.
If each partition splits the collection into equal parts then the
efficiency of quicksort is O(n log2 n).
However, if the array is already sorted into ascending order,
and you choose the first array item as the pivot, the number of
comparisons required is n(n 1)/2 - this is the worst case!
A formal analysis of quicksorts average-case behavior would
show that it is O(n log2 n).
Lecture 5.Sorting Algorithms
A comparison of Sorting Algorithms

A comparison of Sorting Algorithms

Worst Case Average Case

Selection sort O(n2 ) O(n2 )
Bubble sort O(n2 ) O(n2 )
Insertion sort O(n2 ) O(n2 )
Mergesort O(n log n) O(n log n)
Quicksort O(n2 ) O(n log n)