Linear Search, Binary Search, Bubble Sort

Problem: Search
 We are given a list of records.
 Each record has an associated key.
 Give efficient algorithm for searching for a
record containing a particular key.
 Efficiency is quantified in terms of average
time analysis (number of comparisons) to
retrieve an item.
Search

[0] [1] [2] [3] [4] [ 700 ]

…
Number 281942902 Number 233667136 Number 580625685
Number 701466868 Number 506643548 Number 155778322

Each record in list has an associated key. Number 580625685

In this example, the keys are ID numbers.

Given a particular key, how can we efficiently

retrieve the record from the list?
Serial Search (Linear Search)
 Step through array of records, one at a time.
 Look for record with matching key.
 Search stops when
 record with matching key is found
 or when search has examined all records without
success.
Worst Case Time for Serial Search

 For an array of n elements, the worst

case time for serial search requires n
array accesses: O(n).
 Consider cases where we must loop
over all n records:
 desired record appears in the last position of
the array
 desired record does not appear in the array at
all
Average Case for Serial Search

Assumptions:
1. All keys are equally likely in a search
2. We always search for a key that is in the array
Example:
 We have an array of 10 records.
 If search for the first record, then it
requires 1 array access; if the second,
then 2 array accesses. etc.
The average of all these searches is:
(1+2+3+4+5+6+7+8+9+10)/10 = 5.5
Average Case Time for Serial Search

Generalize for array size n.

Expression for average-case running time:

(1+2+…+n)/n = n(n+1)/2n = (n+1)/2

Therefore, average case time complexity for

serial search is O(n).
 Implementation

Linear Search ( Array A, Value x)

 Step 1: Set i to 1
 Step 2: if i > n then go to step 7
 Step 3: if A[i] = x then go to step 6
 Step 4: Set i to i + 1
 Step 5: Go to Step 2
 Step 6: Print Element x Found at index i and go
to step 8
 Step 7: Print element not found
 Step 8: Exit
8
Binary Search
 Perhaps we can do better than O(n) in the
average case?
 Assume that we are give an array of records
that is sorted. For instance:
 an array of records with integer keys sorted from
smallest to largest (e.g., ID numbers), or
 an array of records with string keys sorted in
alphabetical order (e.g., names).
Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53
Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Find approximate midpoint

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Is 7 = midpoint key? NO.

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Is 7 < midpoint key? YES.

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Search for the target in the area before midpoint.

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Find approximate midpoint

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Target = key of midpoint? NO.

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Target < key of midpoint? NO.

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Target > key of midpoint? YES.

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Search for the target in the area after midpoint.

Binary Search

Example: sorted array of integer keys. Target=7.

[0] [1] [2] [3] [4] [5] [6]

3 6 7 11 32 33 53

Find approximate midpoint.

Is target = midpoint key? YES.
 Binary Search Implementation
Step 1: Start
Step 2: Input Sorted array in "a[]" and element to be searched in "x" and size
of array in "size"
Step 3: Initialize low=0, high=size-1
Step 4: Repeat until low>=high
Step 4.1: mid=(low+high)/2
Step 4.2: If a[mid] is equal to x,
then, print index value of mid and Goto step 6
Else
If a[mid]<x
low=mid+1
else
high=mid-1
Step 5: Print x not found in the list
Stop 6: Stop
Binary Search: Analysis
 Worst case complexity?
 Each level in the recursion, we split the array
in half (divide by two).
 Therefore maximum recursion depth is
floor(log2n) and worst case = O(log2n).
 Average case is also = O(log2n).
Analysis of input size at each iteration of Binary Search:
 At Iteration 1:

Length of array = n
 At Iteration 2:

Length of array = n/2

 At Iteration 3:

Length of array = (n/2)/2 = n/22

 Therefore, after Iteration k:

Length of array = n/2k

Also, we know that after After k iterations, the length of the array becomes 1 Therefore, the
Length of the array
n/2k = 1
=> n = 2k
Applying log function on both sides:
 => log n = log 2k
2 2

 => log2n = k * log22

 As (loga (a) = 1) Therefore, k = log2(n)

23
Why Sorting?
 Practical application
 People by last name
 Countries by population
 Search engine results by relevance

 Fundamental to other algorithms

24
Problem statement
 There are n comparable elements in an array and we
want to rearrange them to be in increasing order

 Pre:
 An array A of data records
 A value in each data record
 A comparison function
 <, =, >, compareTo

 Post:
 For each distinct position i and j of A, if i<j then A[i] 
A[j]
 A has all the same data it started with

25
Bubble sort
 bubble sort: orders a list of values by
repetitively comparing neighboring elements
and swapping their positions if necessary

 more specifically:
 scan the list, exchanging adjacent elements if
they are not in relative order; this bubbles the
highest value to the top
 scan the list again, bubbling up the second highest
value
 repeat until all elements have been placed in their
proper order

26
"Bubbling" largest element
 Traverse a collection of elements
 Move from the front to the end
 "Bubble" the largest value to the end using pair-
wise comparisons and swapping

0 1 2 3 4 5
42Swap77 12 101
77 42 35 5

27
"Bubbling" largest element
 Traverse a collection of elements
 Move from the front to the end
 "Bubble" the largest value to the end using pair-
wise comparisons and swapping

0 1 2 3 4 5

42 35Swap35
77 77 12 101 5

28
"Bubbling" largest element
 Traverse a collection of elements
 Move from the front to the end
 "Bubble" the largest value to the end using pair-
wise comparisons and swapping

0 1 2 3 4 5

42 35 12Swap12
77 77 101 5

29
"Bubbling" largest element
 Traverse a collection of elements
 Move from the front to the end
 "Bubble" the largest value to the end using pair-
wise comparisons and swapping

0 1 2 3 4 5

42 35 12 77 101 5

No need to swap

30
"Bubbling" largest element
 Traverse a collection of elements
 Move from the front to the end
 "Bubble" the largest value to the end using pair-
wise comparisons and swapping

0 1 2 3 4 5

42 35 12 77 5 Swap101
101 5

31
"Bubbling" largest element
 Traverse a collection of elements
 Move from the front to the end
 "Bubble" the largest value to the end using pair-
wise comparisons and swapping

0 1 2 3 4 5

42 35 12 77 5 101

Largest value correctly placed

32
 Bubble Sort

1. algorithm Bubble_Sort(list)
2. Pre: list != fi
3. Post: list is sorted in ascending order for
all values
4. for i <- 0 to list:Count - 1
5. for j <- 0 to list:Count - 1
6. if list[i] < list[j]
7. Swap(list[i]; list[j])
8. end if
9. end for
10. end for
11. return list
12. end Bubble_Sort
Another Example
 Implementation

void bubbleSort(int[] a) {
for (int i = 0; i < a.length; i++) {
for (int j = 1; j < a.length - i; j++) {
// swap adjacent out-of-order elements
if (a[j-1] > a[j]) {
swap(a, j-1, j);
}
}
}
}

35
Bubble sort runtime
 Running time (# comparisons) for input size n:
n  1 n  1 i n1

 1  (n  1  i)
i0 j 1 i0
n1 n1 n1

n 1  1   i
i0 i0 i0

2 (n  1)n
n  n 
2
(n 2 )
 number of actual swaps performed depends on the
data; out-of-order data performs many swaps

36
Selection sort
 selection sort: orders a list of values by
repetitively putting a particular value into its
final position

 more specifically:
 find the smallest value in the list
 switch it with the value in the first position
 find the next smallest value in the list
 switch it with the value in the second position
 repeat until all values are in their proper places

37
Selection sort example

38
Selection sort example 2
Index
0 1 2 3 4 5 6 7

Value
27 63 1 72 64 58 14 9

1st pass
1 63 27 72 64 58 14 9

2nd pass
1 9 27 72 64 58 14 63

3rd pass
1 9 14 72 64 58 27 63

39
Selection sort code
void selectionSort(int[] a) {
for (int i = 0; i < a.length; i++) {
// find index of smallest element
int minIndex = i;
for (int j = i + 1; j < a.length; j++) {
if (a[j] < a[minIndex]) {
minIndex = j;
}
}

// swap smallest element with a[i]

swap(a, i, minIndex);
}
}

40
Selection sort runtime
 Running time for input size n:

n1 n1 n1

 1  (n  1  (i 1) 1)
i0 j i1 i0
n1

 (n  i  1)
i0
n1 n1 n1

n 1   i  1
i0 i0 i0

2 (n  1)n
n   n
2
(n 2 )
41
Insertion sort
 insertion sort: orders a list of values by
repetitively inserting a particular value into a
sorted subset of the list

 more specifically:
 consider the first item to be a sorted sublist of
length 1
 insert the second item into the sorted sublist,
shifting the first item if needed
 insert the third item into the sorted sublist, shifting
the other items as needed
 repeat until all values have been inserted into their
proper positions
42
Insertion Sort
 Idea: like sorting a hand of playing cards
 Start with an empty left hand and the cards facing
down on the table.
 Remove one card at a time from the table, and
insert it into the correct position in the left hand
 compare it with each of the cards already in the hand,
from right to left
 The cards held in the left hand are sorted
 these cards were originally the top cards of the pile on
the table

43
Insertion Sort

To insert 12, we need to

make room for it by moving
first 36 and then 24.
6 10 24 36

12

44
Insertion Sort

6 10 24 36

12

45
Insertion Sort

6 10 24 3
6

12

46
Insertion Sort example 1
input array

5 2 4 6 1
3
at each iteration, the array is divided in two sub-arrays:

left sub-array right sub-array

sorted unsorted

47
Insertion Sort example 1 (Cont.)

48
INSERTION-SORT
Alg.: INSERTION-SORT(A) 1 2 3 4 5 6 7 8

for j ← 2 to n a1 a2 a3 a4 a5 a6 a7 a8
do key ← A[ j ]
key
Insert A[ j ] into the sorted sequence A[1 . . j -1]
i←j-1
while i > 0 and A[i] > key
do A[i + 1] ← A[i]
i←i–1
A[i + 1] ← key
 Insertion sort – sorts the elements in place

49
Insertion sort example 2
 Simple sorting algorithm.
 n-1 passes over the array
 At the end of pass i, the elements that occupied
A[0]…A[i] originally are still in those spots and in
sorted order.
2 15 8 1 17 10 12 5

0 1 2 3 4 5 6 7
after
2 8 15 1 17 10 12 5
pass 2
0 1 2 3 4 5 6 7
after
1 2 8 15 17 10 12 5
pass 3
0 1 2 3 4 5 6 7
50
Insertion sort example 3

51
Insertion sort code
void insertionSort(int[] a) {
for (int i = 1; i < a.length; i++) {
int temp = a[i];

// slide elements down to make room for a[i]

int j = i;
while (j > 0 && a[j - 1] > temp) {
a[j] = a[j - 1];
j--;
}

a[j] = temp;
}
}

52
 Insertion sort runtime
 worst case: reverse-ordered elements in array.
n1
(n  1)n
 i 1 2  3  ... (n  1)  2
i1

(n 2 )
 best case: array is in sorted ascending order.
n1


1 n  1 (n)
i1
 average case: each element is about halfway in
order.
n1
i 1 (n  1)n

 2 2 (1 2  3... (n  1))  4
i1

(n 2 )
53
Comparing sorts
 We've seen "simple" sorting algorithms so far,
such as selection sort and insertion sort.
 They all use nested loops and perform
approximately n2 comparisons
 They are relatively inefficient

54
Insertion Sort - Summary
 Advantages
 Good running time for “almost sorted” arrays (n)
 Disadvantages
 (n2) running time in worst and average case
  n2/2 comparisons and exchanges

55