Data Structures & Algorithms

Sorting
Lecturer: Duc Q. Nguyen
October 3rd, 2023
 Motivations
Outline Taxonomies of sorting algorithms

“Failure is the opportunity to begin again Insertion

more intelligently.”
Henry Ford Selection

Exchange

Divide-and-conquer
Learning outcomes
● L.O.6.1 - Depict the working steps of sorting algorithms step-by-steps.
● L.O.6.2 - Describe sorting algorithms by using pseudocode.
● L.O.6.3 - Implement sorting algorithms using C/C++ .
● L.O.6.4 - Analyze the complexity and develop experiment (program) to evaluate sorting
algorithms.
● L.O.6.5 - Use sorting algorithms for problems in real-life.
● L.O.8.4 - Develop recursive implementations for methods supplied for the following
structures: list, tree, heap, searching, and graphs.
● L.O.1.2 - Analyze algorithms and use Big-O notation to characterize the computational
complexity of algorithms composed by using the following control structures: sequence,
branching, and iteration (not recursion).
Motivations

● Goal: Given some data points, arrange those data points into
ascending/descending order by some quantity
○ E.g. sort cards by face value or suit
Motivations

● Advantages:
○ Sorted data is often easier to work with
○ Sorted data can allow for faster insert/retrieval/deletion
Motivations

● Questions:
○ What are the different ways we can sort data?
○ What is the “best” strategy?
 Taxonomies of sorting algorithms
Sorting

Internal External

Insertion Selection Exchange Divide-and-

conquer
- Insertion sort - Selection sort - Bubble sort
- Shell sort - Heap sort - Quick sort - Quick sort
- Merge sort
Insertion

● Insertion sort idea:

○ Split the array into 2 parts: sorted and unsorted
○ Insert the first element of the unsorted into the right position in the sorted
Insert

Sorted Unsorted
Insertion

● Insertion sort
23 78 45 8 32 56 12 64

23 78 45 8 32 56 12 64

23 78 45 8 32 56 12 64

23 45 78 8 32 56 12 64

8 23 45 78 32 56 12 64
Insertion

● Insertion sort
8 23 32 45 78 56 12 64

8 23 32 45 56 78 12 64

8 12 23 32 45 56 78 64

8 12 23 32 45 56 64 78
Insertion

● Shell sort idea:

○ For K reducing to 1:
■ Split the array into K segments.
■ Sort each segment using Insertion sort
Insertion

● Shell sort
23 78 45 8 32 56 12 64

8 32 45 12 64 56 23 78

23 78 45 8 32 56 12 64

12 8 23 56 32 64 45 78
Insertion

● Shell sort
12 8 23 56 32 64 45 78

8 12 23 32 45 56 64 78
Insertion

● Complexity:
○ Insertion sort:
■ f (n) = n(n+1)/2 = O(n2)
○ Shell sort
■ O(n1.25)
Selection

● Selection sort idea:

○ Split the array into 2 parts: sorted and unsorted
○ Select the candidate and swap it with the first element in the unsorted

Swap

Sorted Unsorted
Selection

● Selection sort
23 78 45 8 32 56 12 64

8 78 45 23 32 56 12 64

8 12 45 23 32 56 78 64

8 12 23 45 32 56 78 64

8 12 23 32 45 56 78 64
Selection

● Selection sort
8 12 23 32 45 56 78 64

8 12 23 32 45 56 78 64

8 12 23 32 45 56 64 78

8 12 23 32 45 56 64 78
Selection

● Heap sort idea:

○ The unsorted part is organized as a heap (the minimal value is on the root)
○ Select the root and exchange it with the beginning of the unsorted

Unsorted Sorted
Selection

● Heap sort

23 78 45 8 32 56 12 64

23

78 45

8 32 56 12

64
Selection

● Heap sort

78 64 56 23 32 45 12 8

78

64 56

23 32 45 12

8
Selection

● Heap sort

8 64 56 23 32 45 12 78

64 56

23 32 45 12
Selection

● Heap sort

64 32 56 23 8 45 12 78

64

32 56

23 8 45 12
Selection

● Heap sort

12 32 56 23 8 45 64 78

12

32 56

23 8 45
Selection

● Heap sort

56 32 45 23 8 12 64 78

56

32 45

23 8 12
Selection

● Heap sort

12 32 45 23 8 56 64 78

12

32 45

23 8
Selection

● Heap sort

45 32 12 23 8 56 64 78

45

32 12

23 8
Selection

● Heap sort

8 32 12 23 45 56 64 78

32 12

23
Selection

● Heap sort

32 23 12 8 45 56 64 78

32

23 12

8
Selection

● Heap sort

8 23 12 32 45 56 64 78

23 12
Selection

● Heap sort

23 8 12 32 45 56 64 78

23

8 12
Selection

● Heap sort

12 8 23 32 45 56 64 78

8
Selection

● Heap sort

8 12 23 32 45 56 64 78

8
Selection

● Heap sort

8 12 23 32 45 56 64 78
Selection

● Complexity:
○ Selection sort:
■ f (n) = n(n+1)/2 = O(n2)
○ Heap sort:
■ f (n) = O(nlog2n)
Exchange

● Bubble sort idea:

○ Split the array into 2 parts: sorted and unsorted
○ Sort the sized-2 array from the end to the beginning of the unsorted

Sorted Unsorted
Exchange

● Bubble sort
23 78 45 8 32 56 12 64

8 23 78 45 12 32 56 64

8 12 23 78 45 32 56 64

8 12 23 32 78 45 56 64

8 12 23 32 45 78 56 64
Exchange

● Bubble sort
8 12 23 32 45 78 56 64

8 12 23 32 45 56 78 64

8 12 23 32 45 56 64 78

8 12 23 32 45 56 64 78
Exchange

● Complexity:
○ Bubble sort:
■ f (n) = n(n+1)/2 = O(n2)
Divide-and-conquer

● Quick sort idea:

○ Choosing pivot
○ Divide array into 2 sub-arrays:
■ 1 less than & 1 greater than the pivot
○ Sort the sub-arrays by recursion
Divide-and-conquer

● Quick sort
23 78 45 8 32 56 12 64

8 12 23 78 45 32 56 64

8 12 23 78 45 32 56 64

8 12 23 45 32 56 64 78
Divide-and-conquer

● Quick sort
8 12 23 45 32 56 64 78

8 12 23 32 45 56 64 78

8 12 23 32 45 56 64 78

8 12 23 32 45 56 64 78
Divide-and-conquer

● Merge sort idea:

○ Split array into 2 half sub-arrays
○ Recursively sort the sub-arrays
Divide-and-conquer

● Merge sort
23 78 45 8 32 56 12 64

23 78 45 8 32 56 12 64

23 78 45 8 32 56 12 64

23 78 8 45 32 56 12 64
Divide-and-conquer

● Merge sort
23 78 8 45 32 56 12 64

8 23 45 78 12 32 56 64

8 12 23 32 45 56 64 78
Divide-and-conquer

● Complexity:
○ Quick sort:
■ Average: f (n) = O(nlog2n)
■ Worst: f (n) = O(n2)
○ Merge sort:
■ f (n) = O(nlog2n)