Mekelle University

School of Computing, EiT-M, MU

Computer Science Department
Data Structures and Algorithms
Chapter III – Sorting and Searching

3.1. Sorting

Introduction

We sort many things in our everyday lives: A handful of cards when playing Bridge; bills and other
piles of paper; jars of spices; and so on. And we have many intuitive strategies that we can use to do the
sorting, depending on how many objects we have to sort and how hard they are to move around.
Sorting is also one of the most frequently performed computing tasks. We might sort the records in a
database so that we can search the collection efficiently.

Because sorting is so important, naturally it has been studied intensively and many algorithms have
been devised. Some of these algorithms are straightforward adaptations of schemes we use in everyday
life. Others are totally alien to how humans do things, having been invented to sort thousands or even
millions of records stored on the computer. After years of study, there are still unsolved problems
related to sorting. New algorithms are still being developed and refined for special purpose
applications.

While introducing this central problem in computer science, this chapter has a secondary purpose of
illustrating many important issues in algorithm design and analysis. The collection of sorting
algorithms presented will illustrate the different classification of sorting algorithms.

Sorting algorithms will be used to illustrate a wide variety of analysis techniques in this chapter. We’ll
see how it is possible to speed up sorting algorithms by taking advantage of the best case behavior of
another algorithm. We’ll see several examples of how we can tune an algorithm for better performance.

Sorting Terminology and Notation

Input to the sorting algorithms presented in this chapter is a collection of records stored in an array.
Records are compared to one another by means of a comparator.

The Sorting Problem allows input with two or more records that have the same key value. Certain
applications require that input not contain duplicate key values. The sorting algorithms presented in this
chapter and in Chapter 8 can handle duplicate key values unless noted otherwise.

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 1|Page

Defining Sorting

Sorting is the process of taking a list of n objects

a0, a1, ..., an – 1

and rearranging these into a list

a'0, a'1, ..., a'n – 1

such that

a'0 ≤ a'1 ≤ ··· ≤ a'n – 1.

This is essentially the process of converting an abstract list into an abstract sorted list.

Classification of Sorting Algorithms

The classification of sorting operations is based on the action performed:

1. Inserting a new object into a sorted list,

2. Selecting an item from an unsorted list,

3. Exchanging two items in a list

4. Merging two sorted lists

Types of Sorting Algorithm

Under this chapter we are going to cover four types of basic sorting algorithm:

1. Bubble Sort
2. Insertion sort
3. Selection Sort

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 2|Page

3.1.1. Bubble Sort

The first sorting algorithm we are going to look into is called Bubble Sort- the way it operates is
supposedly reminiscent of bubbles floating up in a glass of spa rood.

Strategy (Sorting activities for Bubble):

• Go through multiple passes over the array.
• In every pass:
– Compare adjacent elements in the list.
– Exchange the elements if they are out of order.
– Each pass moves the largest (or smallest) elements to the end of the array
• Repeating this process in several passes eventually sorts the array into ascending (or descending)
order.
• Bubble sort complexity is O(n2) and only suitable to sort array with small size of data.

Implementation of Bubble Sort

// Sorts items in an array into ascending order.

void BubbleSort(dataType data[], int n)
{
int temp;
for ( i =1; i < n; i++ )
{
// moves the largest element to the end of the array
for (int j = 0; j < n ‐ i; j++)
{ //compare adjacent elements
if ( data[j] > data[j+1] )
{ // swap elements
temp = data[j];
data[j] = data[j+1];
data[j+1] = temp;
}
}
}
}

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 3|Page

Illustration of bubble sort:
Consider the following array: 4 75 74 2 54

Pass
i=1 j=0 j=1 j=2 j=3

i=2 j=0 j=1 j=2

i=3 j=0 j=1

i=4 j=0

Bubble Sort Analysis

• The number of comparison between elements and the number of exchange between elements
determine the efficiency of Bubble Sort algorithm.
• Generally, the number of comparisons between elements in Bubble Sort can be stated as follows:

(n-1)+(n-2)+…….+2+1= n(n-1)/2 = O(n2)

In any cases, (worse case, best case or average case) to sort the list in ascending order the number of
comparisons between elements is the same.
–Worse Case 75 74 54 4 2
– Average Case 74 75 4 2 54
– Best Case 2 4 54 74 75
All lists with 5 elements need 10 comparisons to sort all the data.

In the example given, it can be seen that the number of comparison for worse case and best case is the
same ‐ with 10 comparisons.
• The difference can be seen in the number of swapping elements. Worse case has maximum number of
swapping: 10, while best case has no swapping since all data is already in the right position.
Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 4|Page
• For best case, starting with pass one, there is no exchange of data occur.
• From the example, it can be concluded that in any pass, if there is no exchange of data occur, the list
is already sorted. The next pass shouldn't be continued and the sorting process should stop.

Bubble Sort Improvement

Bubble sort is not very fast. Various suggestions have been made to improve it.

For example, a Boolean variable can be set to false at the beginning of each pass through the list and set to
true whenever a swap is made. If the flag is false when the pass is completed, then no swaps were done and
the array is sorted, so the algorithm can halt. This gives exactly the same worst case complexity, but a best
case complexity of only n. The average case complexity is still in O(n2), however, so this is not much of an
improvement.

// Improved Bubble Sort

// Sorts items in an array into ascending order.
void bubbleSort(DataType data[], int n)
{ int temp;
bool sorted = false; // false when swaps occur
for (int i = 1; (i < n) && !sorted; ++i)
{ sorted = true; // assume sorted
for (int j = 0; j < n-i; ++j)
{
if (data[j] > data[j+1])
{ // exchange items
temp = data[j];
data[j] = data[j+1];
data[j+1] = temp;
sorted = false; // signal exchange
} // end if
} // end for
} // end for
} // end bubbleSort

To improve the efficiency of Bubble Sort, a condition that check whether the list is sorted should be
add at the external loop. A Boolean variable, sorted is added in the algorithm to signal whether there is
any exchange of elements occur in certain pass.

In external loop, sorted is set to true. If there is exchange of data inside the inner loop, sorted is set to
false. Another pass will continue, if sorted is false and will stop if sorted is true.

Consider the Best case for the improved bubble sort with array 2 4 54 74 75
Pass
i=1 j=0 j=1 j=2 j=3
2 4 54 74 75 2 4 54 74 75 2 4 54 74 75 2 4 54 74 75

No swaps made on an already sorted array, (n-1) comparisons, zero (0) exchange (swap)
Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 5|Page
In pass 1 (when i=1), there is no exchange of data occur and variable sorted is always True. Therefore,
condition statement in external loop will become false and the loop will stop execution. In this
example, pass 2 will not be continued.

pass = 1
Analysis - For best case, the number of comparison between elements is 4, (n-1) which is O(n).

Worst Case: Sorted in reverse order. Ex

75 74 54 4 2

→ n(n-1)/2 comparisons, n(n-1)/2 exchanges (swap)

Average case: Somewhat less than n-1 passes, only about half the exchanges.
→ n(n-x) comparisons, n(n-x)/4 exchanges (swap)
3.1.2. Insertion Sort

We continue with insertion sort, which is an efficient algorithm for sorting a small number of elements.
Insertion sort works the way many people sort a hand of playing cards. We start with an empty left
hand and the cards face down on the table. We then remove one card at a time from the table and insert
it into the correct position in the left hand. To find the correct position for a card, we compare it with
each of the cards already in the hand, from right to left, as illustrated in Figure 3.1. At all times, the
cards held in the left hand are sorted, and these cards were originally the top cards of the pile on the
table.

Figure 3.1. Sorting a hand of cards using insertion sort.

It is a simple sorting algorithm which sorts the array by shifting elements one by one. Following are
some of the important characteristics of Insertion sort.

· It has one of the simplest implementation

· It is efficient for smaller data sets, but very inefficient for larger lists.
· Insertion sort is adaptive, that means it reduces its total number of steps if given a partially
sorted list, hence it increases its efficiency.
· It is better than selection sort and Bubble algorithms.
· Appropriate for small arrays due to its simplicity

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 6|Page

Strategy
Insertion sort works by repeatedly taking an element (key) from the unsorted portion of a list and
inserting it into the sorted portion of the list until every element has been inserted. This algorithm is the
one usually used by people when sorting piles of papers.

Insertion sort Implementation

void insertionSort(dataType data[ ], int n)

{ dataType key;
for(int i=1; i<n; i++)
{
key = data[i];
j = i;
while((j >0) && (key < data[j] ))
{
//insert the right item
data[j]= data[j -1];
j --;
}
data[j] = key;
//insert item at the right place
}
}

A list with only one element is already sorted, so the elements inserted begin with the second element
in the array. The inserted element is held in the key variable and values in the sorted portion of the array
are moved up to make room for the inserted element in the same loop where the search is done to find
the right place to make the insertion. Once it is found, the loop ends and the inserted element is placed
into the sorted portion of the array.

Illustration of Insertion sort: initial array data = <5, 2, 4, 6, 1, 3>

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 7|Page

Figure 3.2 The operation of INSERTION-SORT on the array data = <5, 2, 4, 6, 1, 3>. Array indices appear above the rectangles, and values
stored in the array positions appear within the rectangles. (a) – (e) The iterations (passes) of the outer for loop. In each iteration, the black
rectangle holds the key taken from A[j], which is compared with the values in shaded rectangles to its left in the test of while loop
condition. Shaded arrows show array values moved (shifted) one position to the right, and black arrows indicate where the key moves. (f)
The final sorted array.

Though Insertion sort works in the way depicted above, each pass contains several steps to shift
elements. For example:

I=1(or a): 5, 2, 4, 6, 1, 3 i=2(or b): 2, 5, 4, 6, 1, 3 I=3(or c): 2, 4, 5, 6, 1, 3

5, 5, 4, 6, 1, 3 2, 5, 5, 6, 1, 3
2, 5, 4, 6, 1, 3 2, 4, 5, 6, 1, 3

I=4(or d): 2, 4, 5, 6, 1, 3 I=5(or e): 1, 2, 4, 5, 6, 3

2, 4, 5, 6, 6, 3 1, 2, 4, 5, 6, 6
2, 4, 5, 5, 6, 3 1, 2, 4, 5, 5, 6
2, 4, 4, 5, 6, 3 1, 2, 4, 4, 5, 6
2, 2, 4, 5, 6, 3 1, 2, 3, 4, 5, 6
1, 2, 4, 5, 6, 3

Insertion sort Analysis

The time taken by the INSERTION-SORT procedure depends on the input: sorting a thousand numbers
takes longer than sorting three numbers. Moreover, INSERTIONSORT can take different amounts of
time to sort two input sequences of the same size depending on how nearly sorted they already are. In
general, the time taken by an algorithm grows with the size of the input, so it is traditional to describe
the running time of a program as a function of the size of its input. To do so, we need to define the
terms “running time” and “size of input” more carefully.
Insertion sort does different things depending on the contents of the list, so we must consider its worst, best,
and average case behavior. If the list is already sorted, one comparison is made for each of n-1 elements as
they are “inserted” into their current locations. So the best case behavior of Insertion sort is
BC(n) = n-1= O (n)
The worst case occurs when every inserted element must be placed at the beginning of the already sorted
portion of the list; this happens when the list is in reverse order. In this case, the first element inserted
requires one comparison, the second two, the third three, and so forth, and n-1 elements must be inserted.
Hence
WC(n) = Σi=1 to n-1i = n(n-1)/2 = O (n2)

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 8|Page

3.1.3. Selection Sort

The idea behind Selection sort is to make repeated passes through the list, each time finding the largest
(or smallest) value in the unsorted portion of the list, and placing it at the end (or beginning) of the
unsorted portion, thus shrinking the unsorted portion and growing the sorted portion. Thus, the
algorithm works by repeatedly “selecting” the item that goes at the end of the unsorted portion of the
list.

Selection Sort is essentially a Bubble Sort, except that rather than repeatedly swapping adjacent values
to get the next smallest record into place, we instead remember the position of the element to be
selected and do one swap at the end.

Implementation of Selection Sort

This algorithm finds the minimum value in the unsorted portion of the list n-1 times and puts it where it
belongs. Like Bubble sort, it does exactly the same thing no matter what the contents of the array, so we
need only consider its basic complexity.

void Selection_Sort(datatype data[], int n)

{
int min, temp;
for (int i=0; i<n-1; i++)
{
min = i;
for (int j=i+1; j<n; j++)
{
if (data[j] < data[min])
{
min = j;
}
}
temp = data [i];
data[i] = data[min];
data[min] = temp;
}
}

Although the number of comparisons that Selection sort makes is identical to the number that Bubble sort
makes, Selection sort usually runs considerable faster. This is because Bubble sort typically makes many
swaps on every pass through the list, while Selection sort makes only one. Nevertheless, neither of these
sorts is particularly fast.

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 9|Page

Illustration of Selection sort:
Suppose you have the array: 42 20 17 13 28 14 23 15
[0] [1] [2] [3] [4] [5] [6] [7]

[0]
[1]
[2]
[3]
[4]
[5]
[6]
[7]

An example of Selection Sort. Each column shows the array after the iteration with the indicated value of i in the outer for
loop. Numbers above the line in each column have been sorted and are in their final positions.

Selection Sort Analysis

On the first pass through the list, Selection sort makes n-1 comparison; on the next pass, it makes n-2
comparisons; on the third, it makes n-3 comparisons, and so forth. It makes n-1 passes altogether, so its
complexity is O(n2)

Summary

Sort Algorithm Best Case Average Case Worst Case

Bubble Sort O(n2) O(n2) O(n2)
Improved Bubble Sort O(n) O(n2) O(n2)
Insertion Sort O(n) O(n2) O(n2)
Selection Sort O(n2) O(n2) O(n2)

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 10 | P a g e

3.2. Searching

Organizing and retrieving information is at the heart of most computer applications, and searching is
surely the most frequently performed of all computing tasks. Search can be viewed abstractly as a
process to determine if an element with a particular value is a member of a particular set. The more
common view of searching is an attempt to find the record within a collection of records that has a
particular key value or those records in a collection whose key values meet some criterion such as
falling within a range of values.

Objective:
This section allows students to grasp the following:
· Understand the searching technique concept and the purpose of searching operation.
· Understand the implementation of basic searching algorithm:
1. Sequential search
§ Sequential search on unsorted data.
§ Sequential search on sorted data.
2. Binary search
· Able to analyze the efficiency of the searching technique.
· Able to implement searching technique in problem solving.

Searching Definition
1. Clifford A. Shaffer [1997] defines searching as a process to determine whether an element is a
member of a certain data set.
2. The process of finding the location of an element with a specific value (key) within a collection
of elements.
3. The process can also be seen as an attempt to search for a certain record in a life.
a. Each record contains data field and key field
b. Key field is a group of character or numbers used as an identifier for each record.
c. Searching can be done based on the key field.

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 11 | P a g e

3.2.1. Sequential Search

The sequential search (also called linear search) on array based always starts at the first element in the
list and continues until either the item is found in the list or the entire list is searched.

If the search item is found, its index (that is, its location in the array) is returned. If the search is
unsuccessful, -1 is returned.

· Basic sequential search usually is implemented to search item from unsorted list/ array.

· The technique can be implemented on a small size of list. This is because the efficiency of
sequential search is low compared to other searching techniques.

· In a sequential search:
1. Every element in the array will be examined sequentially, starting from the first element.
2. The process will be repeated until the last element of the array or until the searched data is
found.

· Used for searching that involves records stored in the main memory (RAM)

· The simplest search algorithm, but is also the slowest

· Searching strategy:
1. Examines each element in the array one by one (sequentially) and compares its value with
the one being looked for – the search key
2. Search is successful if the search key matches with the value being compared in the array.
Searching process is terminated.
3. else, if no matches is found, the search process is continued to the last element of the array.
Search is failed array if there is no match found from the array.

Implementation of Sequential Search

int Sequential_Search(int data[ ], int n, int key)

{
int index = -1; // -1 means key not found
for (int i = 0; i < n; i++)
{
if (key = = data [i] )
{
index = i;
break;
}
}
return index;
}

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 12 | P a g e

Example: Consider the following array and key

Suppose that you want to determine whether key = 22 is in the list. The sequential search works as
follows:

First, you compare 22 with data [0] – that is, compare 22 with 11. Because data [0] ≠ 22, you then
compare with data [1] (that is 33, the second element in the array). Because data [1] ≠ 22, you
compare 22 with the next element in the array list – that is, compare 22 with data [2]. Because data [2]
= 22, return index of data [2] which is 2 and stop searching. This is a successful search.

Let us now search for key = 10. As before, the search starts with the first element in the array – that is,
at data [0]. This time the search key, which is 10, is compared with every element in the array.
Eventually, no more data is left in the array to compare with the search key. This is an unsuccessful
search. Return index = -1, to indicate that the search key is not found.

Sequential Search analysis

This section analyzes the performance of the sequential search algorithm in both the worst case and the
average case.

The statements before and after the loop are executed only once and, hence, require very little computer
time. The statements in the for loop are the ones that are repeated several times. For each iteration of the
loop, the search item is compared with an element in the list, and a few other statements are executed,
including some other comparisons. Clearly, the loop terminates as soon as the search item is found in the
list. Therefore, the execution of the other statements in the loop is directly related to the outcome of the key
comparison. Also, different programmers might implement the same algorithm differently, although the
number of key comparisons would typically be the same. The speed of a computer can also easily affect the
time an algorithm takes to perform, but not the number of key comparisons.

Therefore, when analyzing a search algorithm, we count the number of key comparisons because this
number gives us the most useful information. Furthermore, the criteria for counting the number of key
comparisons can be applied equally well to other search algorithms.

· Searching time for sequential search is O(n).

· If the searched key is located at the end of the list or the key is not found, then the loop will be
repeated based on the number of element in the list, O(n). (it’s worst case)

· If the list can be found at index 0, then searching time is, O(1). (It’s best case)

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 13 | P a g e

Improvement of Sequential search (using ordered list)

Problem:
Search key is compared with all elements in the list, O(n) time consuming for large datasets.
Solution:
The efficiency of basic search technique can be improved by searching on a sorted list.
For searching on ascending list, the search key will be compared one by one until:
1. The searched key is found.
2. Or until the searched key value is smaller than the item compared in the list.
=> This will minimize the searching process.

Implementation of Improved Sequential search

int Improved_Sequential_Search(int data[ ], int n, int key)

{
int index = -1; // -1 means key not found
for (int i = 0; i < n; i++)
{
if (key < data[i])
break;
else if (key = = data [i] )
{
index = i;
break;
}
}
return index;
}

Example:

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 14 | P a g e

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 15 | P a g e
3.2.2. Binary Search

As you can see, the sequential search is not efficient for large list because, on average, the sequential search
algorithm is having to traverse the entire list, O(n). We therefore describe another search algorithm, called
the binary search, which is very fast. However, a binary search can be performed only on ordered list. We,
therefore, assume that the list is ordered. Sorting the list (array) does minimize the cost of traversing the
whole data set, but we can improve the searching efficiency by using the Binary search algorithm.

When people search for something in an ordered list (like a dictionary or a phone book) they do not start at
the first element and march through the list one element at a time. They jump into the middle of the list, see
where they are relative to what they are looking for, and then jump either forward or backward and look
again, continuing in this way until they find what they are looking for, or determine that it is not in the list.

The binary search algorithm uses the divide-and-conquer technique to search the list. Binary search
takes the same tack in searching for a key value in a sorted list: the key is compared with the middle
element in the list. If it is the key, the search is done; if the key is less than the middle element, then the
process is repeated for the first half of the list; if the key is greater than the middle element, then the process
is repeated for the second half of the list. Eventually, either the key is found in the list, or the list is reduced
to nothing (the empty list), at which point we know that the key is not present in the list.

Strategy

Left Middle Right

0 1 2 3 4 5 6 7 8 9

Binary search Starts by comparing the search key with the element at the middle.

i) If the value matches, return index to the calling function and stop searching (index = Middle)
ii) If the search key < the middle element, search will be focused on the elements between the first element
to the element before the middle element (middle-1) (change value of right to right = middle-1 and
calculate new middle=(left + right)/2)

Left Middle

0 1 2 3 4 5 6 7 8 9
Middle (new) Right

iii) If the search key > the middle element, search will only be focused on the elements between the
element next to middle and the last element (right) (change value of left to left= middle +1 and calculate
new value of middle, middle=(left + right )/2)

Middle Right

0 1 2 3 4 5 6 7 8 9
Left Middle (new)
Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 16 | P a g e
iv) Search is repeated until the searched key is found or the last element in the subset is traversed
(left>right)

Implementation of Binary search

int Binary_Search(int data[ ], int n, int key)

{
bool found = false;
int index = -1; // -1 means key not found
int middle, left = 0, right = n-1;
while ((left <= right) && (! found))
{
middle = (left + right)/2; // Get middle index
if (data [middle] = = key)
{
index= middle;
found = true;
}
else if (data [ middle] > key)
right = middle - 1; // Search is focused on the left side of list
else
left = middle + 1; // Search is focused on the right side of the list
}
return index;
}

Example: consider the following array and key

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 17 | P a g e

Steps to execute Binary Search

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 18 | P a g e

A match is found, return index and stop searching. (key = = data [middle])

Binary Search Analysis

· Binary Search starts searching by comparing element in the middle. Thus the searching process
start at n/2 for a list size = n.

· If the middle value does not match with the search key, then the searching area will be reduced
to the left or right sub-list only. This will reduce the searching area to ½ n.

· From half of the list, the second middle value will be identified. Again, if the middle value does
not match with the search key, the searching area will be reduced to the left or right sub list
only. The searching area will reduce ½ ( ½ n).
· The process of looking for middle point and reducing the searching area to the left or right sub-
list will be repeated until the middle value is equal to the middle value (search key is found) or
the last value in sub-list has been traverse.

· If the repetition occur k times, then at iteration k , the searching area is reduced to ( ½ )kn.

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 19 | P a g e

Figure above shows the reducing size for binary search area. At iteration k for array size = n , searching
area will be reduced from n to ( ½ )kn.

Best case for binary search happen if the search key is found in the middle array. O(1).

Worse case for binary search happens if the search key is not in the list or the searching size equal to 1.

The searching time for worse case is O(logn2).

Search Conclusion

· Searching is a process to allocate an element in a list and return the index of the searched
element.
· Basic searching techniques: sequential search and binary search.
· Sequential search can be implemented on sorted and unsorted list, while binary search can be
implemented only on sorted list.
· Sequential search on sorted data is more efficient than sequential search on unsorted data.
· Binary search is more efficient than sequential search.
· Basic searching techniques explained in this class are only suitable for small sets of data.
Hashing and indexing are suitable for searching large sets of data.

Ins: Fikrezgy Yohannes CoSC2083-Data Structures and Algorithms 20 | P a g e