Searching and Sorting

CS10003 PROGRAMMING AND DATA STRUCTURES

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

1
 Search

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

2
Searching
Check if a given element (called key) occurs in the array.
• Example: array of student records; rollno can be the key.

Two methods to be discussed:

• If the array elements are unsorted.
• Linear search
• If the array elements are sorted.
• Binary search

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

3
Basic Concept of Linear Search
Basic idea:
• Start at the beginning of the array.
• Inspect elements one by one to see if it matches the key.
Time complexity:
• A measure of how long an algorithm runs before terminating.
• If there are n elements in the array:
• Best case:
match found in first element (1 search operation)
• Worst case:
no match found, or match found in the last element (n search operations)
• Average case: (n + 1) / 2 search operations

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

4
Linear Search
Function linear_search returns the array index where a match is found.
It returns -1 if there is no match.

int linear_search (int a[], int size, int key)

{
int pos = 0;
while ((pos < size) && (a[pos] != key)) pos++;
if (pos < size)
return pos; /* Return the position of match */
return -1; /* No match found */
}

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

5
 Binary Search

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

6
Basic Concept

Binary search is applicable if the array is sorted.

BASIC IDEA
• Look for the target in the middle.
• If you don’t find it, you can ignore half of the array, and repeat the process with the other half.

In every step, we reduce the number of elements to search in by half.

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

7
The Basic Strategy
What do we want?
• Find split between values larger and smaller than key:
0 n-1
x: <=key >key
L R

• Situation while searching:

• Initially L and R contains the indices of first and last elements.

• Look at the element at index [(L+R)/2].

• Move L or R to the middle depending on the outcome of test.

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

8
Binary Search
/* If key appears in x[0..size-1], return its location, pos such that x[pos]==key.
If not found, return -1 */

int bin_search (int x[ ], int size, int key)

{
int L, R, mid;
_________________ ;
while ( ____________ )
{
__________________;
}
_________________ ;
}

INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR

9
The basic search iteration
int bin_search (int x[ ], int size, int key)
{
int L, R, mid;
_________________;
while ( ____________ )
{
mid = (L + R) / 2;
if (x[mid] <= key) L = mid;
else R = mid;
}
_________________ ;
}

10
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Loop termination criterion
int bin_search (int x[ ], int size, int key)
{
int L, R, mid;
_________________;
while ( L+1 != R )
{
mid = (L + R) / 2;
if (x[mid] <= key) L = mid;
else R = mid;
}
_________________ ;
}

11
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Initialization and Return Value

int bin_search (int x[ ], int size, int key)

{
int L, R, mid;
L = −1; R = size;
while ( L+1 != R )
{
mid = (L + R) / 2;
if (x[mid] <= key) L = mid;
else R = mid;
}
if (L >= 0 && x[L] == key) return L;
else return −1;
}

12
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Binary Search Examples

Sorted array

-17 -5 3 6 12 21 45 63 50

Trace : L= –1; R=9; x[4]=12;

bin_search (x, 9, 3); L= –1; R=4; x[1]= –5;
L= 1; R=4; x[2]=3;
bin_search (x, 9, 145); L=2; R=4; x[3]=6;
bin_search (x, 9, 45); L=2; R=3; return L;

We may modify the algorithm by checking equality with x[mid].

13
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Is it worth the trouble ?
Suppose that the array x has 1000 elements.

Ordinary search
– If key is a member of x, it would require 500 comparisons on the average.

Binary search
• after 1st compare, left with 500 elements.
• after 2nd compare, left with 250 elements.
• After at most 10 steps, you are done.

14
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Time Complexity
If there are n elements in the array.
• Number of iterations required: log2n

For n = 64 (say).
• Initially, list size = 64. 2k= n, where k is the
• After first compare, list size = 32. number of steps.
• After second compare, list size = 16.
• After third compare, list size = 8.
• …….
log264 = 6
• After sixth compare, list size = 1.
log21024 = 10

15
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Are exactly log2n steps required for all cases?

-17 -5 3 6 12 21 45 63 50

Trace of binsearch(x,9,12):
L= –1; R=9; x[4]=12;
L= 4; R=9; x[6]= 45;
L= 4; R=6; x[5]=21;
L=4; R=5; return L;

We know in first iteration that x[4] = 12. Why not stop then?

16
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Are exactly log2n steps required for all cases?

int bin_search (int x[ ], int size, int key) int bin_search_1 (int x[ ], int size, int key)
{ {
int L, R, mid; int L, R, mid;
L = −1; R = size; L = 0; R = size-1;
while ( L+1 != R ) while ( L <= R )
{ {
mid = (L + R) / 2; mid = (L + R) / 2;
if (x[mid] <= key) L = mid; if (x[mid] == key) return mid;
else R = mid; if (x[mid] < key) L = mid+1;
} else R = mid-1;
if (L >= 0 && x[L] == key) return L; }
else return −1; return -1;
} }

17
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
 Sorting

18
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
The Basic Problem
Given an array: x[0], x[1], ... , x[size-1] reorder entries so that
x[0] <= x[1] <= . . . <= x[size-1]

• List is in non-decreasing order.

We can also sort a list of elements in non-increasing order.

19
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Example
Original list:
10, 30, 20, 80, 70, 10, 60, 40, 70

Sorted in non-decreasing (ascending) order:

10, 10, 20, 30, 40, 60, 70, 70, 80

Sorted in non-increasing (descending) order:

80, 70, 70, 60, 40, 30, 20, 10, 10

20
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
BUBBLESORT

We had studied this earlier. Repeatedly scan the array from left to right and exchange successive elements
if they are out of order.

void bubbleSort( int arr[ ], int n)

{
int i, j;

for (i = 0; i < n – 1; i++)

for (j = 0; j < n – i – 1; j++) // Last i elements are already in place
if (arr[j] > arr[j + 1]) { temp = arr[j]; arr[j] = arr[j+1]; arr[j+1] = temp; }
}

21
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
SELECTION SORT
General situation :

0 k size-1
x: smallest elements, sorted remainder, unsorted

Step :
• Find smallest element, mval, in x[k..size-1]
• Swap smallest element with x[k], then increase k.

0 k mval size-1

swap

22
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Subproblem

/* Find index of smallest element in x[k..size-1] */

int min_loc (int x[ ], int k, int size)

{
int j, pos;

pos = k;
for (j=k+1; j<size; j++)
if ( x[ j ] < x[ pos ] ) pos = j;
return pos;
}

23
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Selection Sort Function

/* Sort x[0..size-1] in non-decreasing order */ /* Find index of smallest element in x[k..size-1] */

int sel_sort (int x[ ], int size) int min_loc (int x[ ], int k, int size)
{ {
int k, m; int j, pos;

for (k=0; k<size-1; k++) pos = k;

{ for (j=k+1; j<size; j++)
m = min_loc (x, k, size); if ( x[ j ] < x[ pos ] ) pos = j;
temp = a[ k ]; a[ k ] = a[ m ]; a[ m ] = temp; return pos;
} }
}

24
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Example

x: 3 12 -5 6 142 21 -17 45
x: -17 -5 3 6 12 21 142 45
x: -17 12 -5 6 142 21 3 45
x: -17 -5 3 6 12 21 142 45
x: -17 -5 12 6 142 21 3 45
x: -17 -5 3 6 12 21 45 142
x: -17 -5 3 6 142 21 12 45
x: -17 -5 3 6 12 21 45 142
x: -17 -5 3 6 142 21 12 45

25
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Analysis
How many steps are needed to sort n items ?
• Total number of steps proportional to n2.
• What is the number of comparisons?

(n-1)+(n-2)+……+1= n(n-1)/2

• Worst Case? Best Case? Average Case? Of the order of n2

26
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
INSERTION SORT
General situation :

0 i size-1
x: sorted remainder, unsorted

i Compare and
shift till item = x[i]
is larger.

i
<k >k
0 j size-1

27
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Insertion Sort

void insert_sort ( int x[ ], int size )

{
int i, j, item;

for (i=1; i<size; i++)

{
item = x[i] ;
for (j=i-1; (j >= 0) && (x[j] > item); j - -) x[j+1] = x[j];
x[j+1] = item ;
}
}

28
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Time Complexity
Number of comparisons and shifting:

• Worst case?
1 + 2 + 3 + …… + (n-1) = n(n-1)/2

• Best case?
1 + 1 + …… to (n-1) terms = (n-1)

29
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
 Some Efficient Sorting Methods

30
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Two of the most popular sorting algorithms are based on divide-and-conquer approach.
• Quick sort
• Merge sort

Basic idea (divide-and-conquer method):

sort (list)
{
if the list has length greater than 1
{
Partition the list into lowlist and highlist;
sort (lowlist);
sort (highlist);
combine (lowlist, highlist);
}
}

31
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
QUICKSORT
At every step, we select a pivot element in the list (usually the first element).
• We put the pivot element in the final position of the sorted list.
• All the elements less than or equal to the pivot element are to the left.
• All the elements greater than the pivot element are to the right.

32
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Partitioning

0 size-1
x:
pivot

Values smaller Values greater

Perform Perform
partitioning partitioning

33
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Preparation

void print (int x[ ], int low, int high)

{
int i;
for(i=low; i<=high; i++) printf(" %d ", x[i]);
printf("\n");
}

void swap (int *a, int *b)

{
int tmp=*a; *a=*b; *b=tmp;
}

35
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Quicksort

void partition ( int x[ ], int low, int high ) if (j==high) {

{ swap (&x[j], &x[low]);
int i = low+1, j = high; partition (x, low, high-1);
int pivot = x[low]; }
else if (i==low+1)
if (low >= high) return; partition (x, low+1, high);
while (i<j) { else {
while ((x[i]<=pivot) && (i<high)) i++; swap (&x[j], &x[low]);
while ((x[j]>pivot) && (j>low)) j--; partition (x, low, j-1);
if (i<j) swap (&x[i], &x[j]); partition (x, j+1, high);
} }
}

36
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Time Complexity

Worst case:
n2 ==> list is already sorted

Average case:
n log2n

Statistically, quick sort has been found to be one of the fastest algorithms.

37
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Merge Sort

Input Array

Part-I Part-II

Part-I Part-II Part-I Part-II

Merge Split
Sorted Arrays

38
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Merging two sorted arrays

pa pb

Array a Sorted Array Array b Sorted Array

0 m 0 n

Array c Merged sorted array

0 m+n-1

Move and copy elements pointed by pa if its value is smaller than the element pointed by pb in (m+n-1)
operations; otherwise, copy elements pointed by pb .

39
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Example
Initial array A contains 14 elements:
• 66, 33, 40, 22, 55, 88, 60, 11, 80, 20, 50, 44, 77, 30
Pass 1 :: Merge each pair of elements
• (33, 66) (22, 40) (55, 88) (11, 60) (20, 80) (44, 50) (30, 70)
Pass 2 :: Merge each pair of pairs
• (22, 33, 40, 66) (11, 55, 60, 88) (20, 44, 50, 80) (30, 77)
Pass 3 :: Merge each pair of sorted quadruplets
• (11, 22, 33, 40, 55, 60, 66, 88) (20, 30, 44, 50, 77, 80)
Pass 4 :: Merge the two sorted subarrays to get the final list
• (11, 20, 22, 30, 33, 40, 44, 50, 55, 60, 66, 77, 80, 88)

40
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
void merge_sort ( int *a, int n ) void merge (int *a, int *b, int *c, int m, int n)
{ {
int i, j, k, m; int i=0, j=0, k=0, p;
int *b, *c;
do {
if (n>1) { if (a[i] < b[j]) { c[k]=a[i]; i++; }
k = n/2; m = n−k; else { c[k]=b[j]; j++; }
b = (int *) calloc(k,sizeof(int)); k++;
c = (int *) calloc(m,sizeof(int)); } while ((i<m) && (j<n));
for (i=0; i<k; i++) b[i] = a[i];
for (j=k; j<n; j++) c[j− i] = a[j]; if (i == m) {
for (p=j; p<n; p++) { c[k]=b[p]; k++; }
merge_sort (b, k); }
merge_sort (c, m); else {
merge (b, c, a, k, m); for (p=i; p<m; p++) { c[k]=a[p]; k++; }
free(b); free(c); }
} }
}

41
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR
Practice Problems

1. Write a recursive function for binary search.

2. Write iterative version of merge sort.
3. Write merge sort without using additional storage (i.e., extra arrays).
4. You are given a sorted array with entries rotated clockwise by k positions. That is, if the sorted order is a_0,
a_1, … , a_{n-1} then the given array to you has the form a_k, a_{k+1}, … , a_{n-1}, a_0, a_1, … , a_{k-1}. Write
a variant of binary search on such an array. Assume that k is known.
5. In the previous problem, suppose that k is not given. Write a function that takes the array as input and finds k.

42
INDIAN INSTITUTE OF TECHNOLOGY KHARAGPUR