Data Structures

<Lecture 11: Sorting Algorithm>

Sungmin Cha
New York University

10.09.2024
 Outline
• Notice

• Sorting Algorithm
– Why sorting is necessary?
– Basic sorting algorithms
 Bubble
 Selection
 Insertion sort

2
 Outline
• Notice

• Sorting Algorithm
– Why sorting is necessary?
– Basic sorting algorithms
 Bubble
 Selection
 Insertion sort

3
 Notice

• HW1
– Grades will be published!
– The answer code of HW1 Part3 is in Ed Workspace

4
 Notice

• HW1 Part3 – exceptions

– Some students implemented a code to create a large-sized
array when the ArrayList reaches its full capacity
– It was not considered in the current Autotrader
 Therefore, it makes a problem in ArrayList exception case 1 of
add() and append()
– If you implemented it this way, please appeal it in Gradescope!
 TA will manually check your code and provide a make-up score for
those cases

5
 Notice

• HW2 is posted!
– Deadline: 11:59PM 10/27 Sunday
• HW2 Part 1
– Implementing Queue and Stack using an array and a linked list
– The template codes are in Ed Workspace

6
 Notice

• HW2 Part 1
– ArrayStack / LinkedStack
 Implement a stack using an array and linked list
– ArrayQueue / LinkedQueue
 Implement a queue using a circular array and singly circular linked
list
– The template code has a similar form with HW1 Part 3
– Students should submit their code to Gradescope

7
 Notice

• HW2 Part 2
– It is about Sorting Algorithms

• HW2 Part 3
– It is about implementing Heap

8
 Outline
• Notice

• Sorting Algorithm
– Why sorting is necessary?
– Basic sorting algorithms
 Bubble
 Selection
 Insertion sort

9
 Sorting
• Sorting
– Arranging given data in a predefined order
– In the field of computer science,
 It is common practice to sort data in either numerical or
lexicographical order.

10
 Sorting
• Why is sorting necessary?

11
 Sorting
• Why is sorting necessary? – for search!
– Computers typically handle data on the scale of millions or more
– Moreover, searching for data is constantly carried out
 Log in to a website, searching an item on Amazon, etc.

12
 Sorting
• Why is sorting necessary? – for search!
– Computers typically handle data on the scale of millions or more
– Moreover, searching for data is constantly carried out
 Log in to a website, searching an item in Amazon, etc.

– How can the process of searching be conducted?

 Example:
Dataset
123 23 1 87 15 48 74

Query
15

13
 Sorting
• How can the process of searching be conducted?
– Sequential search

Dataset
123 23 1 87 15 48 74

Comparing

15
Query

14
 Sorting
• How can the process of searching be conducted?
– Sequential search

Dataset
123 23 1 87 15 48 74

Comparing

15
Query

15
 Sorting
• How can the process of searching be conducted?
– Sequential search

Dataset
123 23 1 87 15 48 74

Comparing => matched!

15
Query

– What is the time complexity of it?

16
 Sorting
• How can the process of searching be conducted?
– Sequential search

Dataset
123 23 1 87 15 48 74

Comparing => matched!

15
Query

– What is the time complexity of it?

 O(n), n is the number of data in the dataset
– Is there a better way? 17
 Sorting
• How can the process of searching be conducted?
– Sequential search

Dataset
123 23 1 87 15 48 74

Comparing => matched!

15
Query

– What is the time complexity of it?

 O(n), n is the number of data in the dataset
– Is there a better way? => binary search 18
 Sorting
• When can we use Binary Search?
– The dataset needs to be sorted

Dataset
123 23 1 87 15 48 74

sorting

1 15 23 48 74 87 123

19
 Sorting
• Binary search with the sorted data
Dataset (n-1) // 2
1 15 23 48 74 87 123

15 < 48

15
Query

20
 Sorting
• Binary search with the sorted data
Dataset
1 15 23 48 74 87 123

15 == 15: matched!

15
Query

– What is the time complexity of binary search?

21
 Sorting
• Binary search with the sorted data
Dataset
1 15 23 48 74 87 123

15 == 15: matched!

15
Query

– What is the time complexity of binary search?

 O(log n)

22
 Sorting
• Which one is more effective?
– To use binary search, the dataset is sorted whenever data is
added and removed
 In most cases, querying (q) data overwhelmingly outweighs
insertion(i)/deletions(d) ( q >> i or d)
– (approximately) The total number of operations of each case
 Sequential search: q * O(n)
 Sorting + binary search: (i + d) * time complexity of sorting + q * O(log n)

23
 Sorting
• Which one is more effective?
– (approximately) The total number of operations of each case
 Sequential search: q * O(n)
 Sorting + binary search: (i + d) * time complexity of sorting + q * O(log n)
 q >> i or d

n =10 n = 100 n = 1000 n = 10000 n = 100000 n = 10000000

O(n) 10 100 1000 10,000 100,000 10,000,000
O(log n) 3 6 9 13 16 23
O(n^2) 10^2 100^2 1000^2 10,000^2 100,000^2 10,000,000^2
O(n*logn) 33 664 9965 13,287 166,096 23,253,496

– If we use an effective sorting algorithm, we can dramatically

reduce the total number of operations in the program!
24
 Sorting
• Sorting algorithms
– There are many sorting algorithms but we will focus on learning
the below basic algorithms
 Basic sort (O(n^2))
o Bubble/Selection/Insertion sort
 Advanced sort (O(nlogn))
o Merge/Quick sort

25
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

23 12 1 74 15 48 128

26
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

Starting from the left and progressively comparing adjacent pairs

23 12 1 74 15 48 128

23 > 12

27
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

Swap them if they are not in order

12 23 1 74 15 48 128

23 > 12

28
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

comparing adjacent pairs

12 23 1 74 15 48 128

23 > 1

29
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

Swap them if they are not in order

12 1 23 74 15 48 128

23 > 1

30
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

comparing adjacent pairs

12 1 23 74 15 48 128

23 < 74

31
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

comparing adjacent pairs

12 1 23 74 15 48 128

74 > 15

32
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

Swap them if they are not in order

12 1 23 15 74 48 128

74 > 15

33
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

comparing adjacent pairs

12 1 23 15 74 48 128

74 > 48

34
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

Swap them if they are not in order

12 1 23 15 48 74 128

74 > 48

35
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

comparing adjacent pairs

12 1 23 15 48 74 128

74 < 128

36
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

comparing adjacent pairs

12 1 23 15 48 74 128

74 < 128

37
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the first iteration

12 1 23 15 48 74 128

As a result of the first iteration, the largest value among

the given data ends up positioned at the far right

Therefore, we don’t need to compare this value anymore

38
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the second iteration

Starting from the left and progressively comparing adjacent pairs

12 1 23 15 48 74 128

12 > 1

39
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the second iteration

Swap them if they are not in order

1 12 23 15 48 74 128

1 > 12

40
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the second iteration

Starting from the left and progressively comparing adjacent pairs

1 12 23 15 48 74 128

12 < 23

41
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the second iteration

1 12 15 23 48 74 128

The second-largest value among the given data

42
 Bubble Sort
• Process of bubble sort (ascending order)
– Example: the third iteration

Starting from the left and progressively comparing adjacent pairs

1 12 15 23 48 74 128

1 < 12

– Do it until the (n-1)-th iteration, as a result we can get

1 12 15 23 48 74 128

43
 Bubble Sort
• Pseudo algorithm of bubble sort n-1 iterations

(n-1) + (n-2) + …. + 1 = n(n-1)/2

O(1)

• What is the time complexity (Big-O, worst case)?

44
 Bubble Sort
• Pseudo algorithm of bubble sort n-1 iterations

(n-1) + (n-2) + …. + 1 = n(n-1)/2

O(1)

• What is the time complexity (Big-O, worst case)?

– O(n(n-1)/2) = O(n^2)

45
 Bubble Sort
• Time complexity of Bubble sort in the best case
– Example: the first iteration

1 12 15 23 48 74 128

1 < 12

46
 Bubble Sort
• Time complexity of Bubble sort in the best case
– Example: the first iteration

1 12 15 23 48 74 128

74 < 128
In the case where no element swaps
occur during the first iteration

It means the elements are sorted

47
 Bubble Sort
• Time complexity of Bubble sort in the best case
– Example: the first iteration

1 12 15 23 48 74 128

74 < 128

– If we add a code to check whether any element swaps occur

during each iteration,
 It is possible to reduce the runtime when the array is already sorted

48
 Bubble Sort
• Time complexity of Bubble sort in the best case
– Example: the first iteration

1 12 15 23 48 74 128

74 < 128

– If we add a code to check whether any element swaps occur

during each iteration,
 It is possible to reduce the runtime when the array is already sorted

– Time complexity in the best case?

 n, n is the number of elements
49
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-1]

23 12 1 74 15 128 48

Find the largest element in A[0, …, n-1]

50
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-1] and
swap it with the last element A[n-1] of the array

Swap!

23 12 1 74 15 128 48

51
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-1] and
swap it with the last element A[n-1] of the array

Swap!

23 12 1 74 15 48 128

The largest element

in the array

52
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-2]

23 12 1 74 15 48 128

Find the largest element in A[0, …, n-2]

53
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-2] and
swap it with the last element A[n-2] of the array

Swap!

23 12 1 74 15 48 128

54
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-2] and
swap it with the last element A[n-2] of the array

Swap!

23 12 1 48 15 74 128

55
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-3]

23 12 1 48 15 74 128

Find the largest element in A[0, …, n-3]

56
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-3] and
swap it with the last element A[n-3] of the array

Swap!

23 12 1 48 15 74 128

57
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-3] and
swap it with the last element A[n-3] of the array

Swap!

23 12 1 15 48 74 128

58
 Selection Sort
• Process of selection sort (ascending order)
– Locate the largest element in the array A[0, …, n-4]

23 12 1 15 48 74 128

Find the largest element in A[0, …, n-4]

– And repeat the same process..

59
 Selection Sort
• Process of selection sort (ascending order)
– The final array after sorting is complete

1 12 15 23 48 74 128

60
 Selection Sort
• Pseudo algorithm of selection sort
n-1 iterations Number of comparison
: (n-1) + (n-2) + … + 1 = n(n-1)/2

O(1)

• What is the time complexity (Big-O, worst case)?

61
 Selection Sort
• Pseudo algorithm of selection sort
n-1 iterations Number of comparison
: (n-1) + (n-2) + … + 1 = n(n-1)/2

O(1)

• What is the time complexity (Big-O, worst case)?

– O(n(n-1)/2) = O(n^2)

62
 Selection Sort
• Is there the best case in using selection sort?

1 12 15 23 48 74 128

63
 Selection Sort
• Is there the best case in using selection sort?

1 12 15 23 48 74 128

Find the largest element in A[0, …, n-1]

64
 Selection Sort
• Is there the best case in using selection sort?
Swap!

1 12 15 23 48 74 128

– No!
 It will find the largest element for n-1 times
 Namely, time complexity of selection sort is always n^2

65
 Thank you!

E-mail: [email protected]

66