Lecture 7-8.

Heapsort

Introduction to Algorithms
Da Nang University of Science and Technology

Dang Thien Binh

[email protected]

Intelligent Networking LaboratoryCopyright 2000-2022 Intelligent

Dang
Networking
Thien Binh
Laboratory
34/82
 Introduction for Heapsort

 Heapsort
 Running time: O(n lg n)
 Like merge sort
 Sorts in place: only a constant number of array elements
are stored outside the input array at any time
 Like insertion sort

 Heap
 A data structure used by Heapsort to manage information
during the execution of the algorithm
 Can be used as an efficient priority queue

Intelligent Networking Laboratory Dang Thien Binh 35/82

 Perfect Binary Tree

 For binary tree with height h

 All nodes at levels h–1 or less have 2 children (full)

h=0

h=1 h=2 h=3

Intelligent Networking Laboratory Dang Thien Binh 36/82

 Complete Binary Trees

 For binary tree with height h

 All nodes at levels h–2 or less have 2 children (full)
 All leaves on level h are as far left as possible

h=0

h=2

h=1

Intelligent Networking Laboratory Dang Thien Binh 37/82

 Complete Binary Trees

h=3
Intelligent Networking Laboratory Dang Thien Binh 38/82
 Heaps
 Two key properties
 Complete binary tree
 Value at node
 Smaller than or equal to values in subtrees
 Greater than or equal to values in subtrees

 Example max-heap
YX
X
ZX

Y Z

Intelligent Networking Laboratory Dang Thien Binh 39/82

 Heap and Non-heap Examples

5
5

8
5 45

6 45
5 2
2
6 22 6 22
6 22
25 8 45 25
8 45 25
Min-heaps Non-heaps
Intelligent Networking Laboratory Dang Thien Binh 40/82
 Binary Heap
 An array object that can be viewed as a nearly
complete binary tree
 Each tree node corresponds to an array element
that stores the value in the tree node
 The tree is completely filled on all levels except possibly the
lowest, which is filled from the left up to a point
 A has two attributes
 length[A]: number of elements in the array
 heap-size[A]: number of elements in the heap stored within A
 heap-size[A]  length[A]
 max-heap and min-heap

Intelligent Networking Laboratory Dang Thien Binh 41/82

 Max-heap

Intelligent Networking Laboratory Dang Thien Binh 42/82

 Length and Heap-Size

11 7

11 7

Length = 10
Heap-Size = 7

Intelligent Networking Laboratory Dang Thien Binh 43/82

 Heap Computation
 Given the index i of a node, the indices of its parent,
left child, and right child can be computed simply:

 i / 2
PARENT (i ) : return　　
 LEFT (i ) : return　　
2i
2i + 1
 RIGHT (i ) : return　　

Intelligent Networking Laboratory Dang Thien Binh 44/82

 Heap Computation
1
parent(i) = floor(i/2)
0 16 left-child(i) = 2i
2 3 right-child(i)= 2i +1
1 14 10
4 5 6 7
2 8 7 9 3

8 9 10
1 2 3 4 5 6 7 8 9 10
3 2 4 1 16 14 10 8 7 9 3 2 4 1

Intelligent Networking Laboratory Dang Thien Binh 45/82

 Heap Property

 Heap property
 The property that the values in the node must satisfy
 Max-heap property, for every node i other than the
root
 A[PARENT(i)]  A[i]
 The value of a node is at most the value of its parent
 The largest element in a max-heap is stored at the root
 The subtree rooted at a node contains values
no larger than value of the node itself

Intelligent Networking Laboratory Dang Thien Binh 46/82

 Heap Height

 The height of a node in a heap

 The number of edges on the longest simple downward path
from the node to a leaf
 The height of a heap is the height of its root
 The height of a heap of n elements is (lg n)
 Exercise 6.1-2 on page 153

Intelligent Networking Laboratory Dang Thien Binh 47/82

 Heap Procedures

 MAX-HEAPIFY
 Maintains the max-heap property
 O(lg n)
 BUILD-MAX-HEAP
 Produces a max-heap from an unordered input array
 O(n)
 HEAPSORT
 Sorts an array in place
 O(n lg n)

Intelligent Networking Laboratory Dang Thien Binh 48/82

 Maintaining the Heap Property

 MAX-HEAPIFY
 Inputs: an array A and an index i into the array
 Assume the binary trees rooted at LEFT(i) and RIGHT(i) are
max-heaps, but A[i] may be smaller than its children
 violate the max-heap property
 MAX-HEAPIFY lets the value at A[i] “float down” in the
max-heap

Intelligent Networking Laboratory Dang Thien Binh 49/82

 MAX-HEAPIFY

Extract the indices of LEFT and RIGHT

children of i

Choose the largest of A[i], A[l], A[r]

Float down A[i] recursively

Intelligent Networking Laboratory Dang Thien Binh 50/82

 Example of MAX-HEAPIFY (1/3)

16

4 10
4 < 14
14 7 9 3

2 8 1

Intelligent Networking Laboratory Dang Thien Binh 51/82

 Example of MAX-HEAPIFY (2/3)

16

14 10

4 7 9 3
4<8

2 8 1

Intelligent Networking Laboratory Dang Thien Binh 52/82

 Example of MAX-HEAPIFY (3/3)

16

14 10

8 7 9 3

2 4 1

Intelligent Networking Laboratory Dang Thien Binh 53/82

 MAX-HEAPIFY

Iterative version Recursive version

Intelligent Networking Laboratory Dang Thien Binh 54/82

 Running Time of MAX-HEAPIFY

 (1) to find out the largest among

A[i], A[LEFT(i)], and A[RIGHT(i)]
 Plus the time to run MAX-HEAPIFY on a
subtree rooted at one of the children of node i
 The children’s subtrees each have size
at most 2n/3 (why?)
 the worst case occurs when the last row of the tree is
exactly half full

 T(n)  T(2n/3) + (1)

 By case 2 of the master theorem
 T(n) = O(lg n)
7/11 = 0.63

Intelligent Networking Laboratory Dang Thien Binh 55/82

 Heapify Example

Intelligent Networking Laboratory Dang Thien Binh 56/82

 Building a Max-Heap

 Observation: A[(n/2+1)..n] are all leaves of the tree

 Exercise 6.1-7 on page 154
 Each is a 1-element heap to begin with
 Upper bound on the running time
 O(lg n) for each call to MAX-HEAPIFY, and call n times → O(n lg n)
 Not tight

Intelligent Networking Laboratory Dang Thien Binh 57/82

 Building a
Max-Heap

Intelligent Networking Laboratory Dang Thien Binh 58/82

 Loop Invariant
 At the start of each iteration of the for loop of lines 2-3, each node i+1,
i+2, .., n is the root of a max-heap
 Initialization: Prior to the first iteration of the loop, i = n/2. Each
node n/2+1, n/2+2,.., n is a leaf and the root of a trivial max-heap.
 Maintenance: Observe that the children of node i are numbered
higher than i. By the loop invariant, therefore, they are both roots of
max-heaps. This is precisely the condition required for the call
MAX-HEAPIFY(A, i) to make node i a max-heap root. Moreover, the
MAX-HEAPIFY call preserves the property that nodes i+1, i+2, …, n
are all roots of max-heaps. Decrementing i in the for loop update
reestablishes the loop invariant for the next iteration.
 Termination: At termination, i=0. By the loop invariant, each node 1, 2, …,
n is the root of a max-heap. In particular, node 1 is.

Intelligent Networking Laboratory Dang Thien Binh 59/82

 Cost for Build-MAX-HEAP
 Heap-properties of an n-element heap
 Height = lg n
 At most n/2h+1 nodes of any height h
 Exercise 6.3-3 on page 159

lg n  lg n  h 
 n  h

h =0
 2 h +1  O(h) = O(n  2 h ) = O(n 2 h ) = O(n)
h =0 h =0

Ignore the constant ½  h 1

 = 2 =2
h =0 2
h
(1 − 1 ) 2
2


x
 kxk =
k =0 (1 − x) 2 (for |x| < 1)

Intelligent Networking Laboratory Dang Thien Binh 60/82

 Heapsort

 Using BUILD-MAX-HEAP to build a max-heap on the input array A[1..n],

where n=length[A]
 Put the maximum element, A[1], to A[n]
 Then discard node n from the heap by decrementing heap-size(A)
 A[2..n-1] remain max-heaps, but A[1] may violate
 call MAX-HEAPIFY(A, 1) to restore the max-heap property
for A[1..n-1]
 Repeat the above process from n down to 2
 Cost: O(n lg n)
 BUILD-MAX-HEAP: O(n)
 Each of the n-1 calls to MAX-HEAPIFY takes time O(lg n)

Intelligent Networking Laboratory Dang Thien Binh 61/82

 Heapsort Algorithm

Intelligent Networking Laboratory Dang Thien Binh 62/82

 Example: Heapsort (1/8)
1
16
2 3
14 10
4 5 6 7
8 7 9 3

8 9 10
2 4 1 16 14 10 8 7 9 3 2 4 1

Intelligent Networking Laboratory Dang Thien Binh 63/82

 Example: Heapsort (2/8)
1
14
2 3
8 10
4 5 6 7
4 7 9 3

8 9 10
2 1 16
Intelligent Networking Laboratory Dang Thien Binh 64/82
 Example: Heapsort (3/8)
1
10
2 3
8 9
4 5 6 7
4 7 1 3

8 9 10
2 14 16
Intelligent Networking Laboratory Dang Thien Binh 65/82
 Example: Heapsort (4/8)
1
9
2 3
8 3
4 5 6 7
4 7 1 2

8 9 10
10 14 16
Intelligent Networking Laboratory Dang Thien Binh 66/82
 Example: Heapsort (5/8)
1
7
2 3
4 3
4 5 6 7
1 2 8 9

8 9 10
10 14 16
Intelligent Networking Laboratory Dang Thien Binh 67/82
 Example: Heapsort (6/8)
1
4
2 3
2 3
4 5 6 7
1 7 8 9

8 9 10
10 14 16
Intelligent Networking Laboratory Dang Thien Binh 68/82
 Example: Heapsort (7/8)
1
1
2 3
2 3
4 5 6 7
4 7 8 9

8 9 10
10 14 16
Intelligent Networking Laboratory Dang Thien Binh 69/82
 Example: Heapsort (8/8)
1
1
2 3
2 3
4 5 6 7
4 7 8 9
8 9 10
10 14 16
1 2 3 4 7 8 9 10 14 16

Intelligent Networking Laboratory Dang Thien Binh 70/82

 Heapsort Algorithm

Intelligent Networking Laboratory Dang Thien Binh 71/82

 Heap & Heap Sort Algorithm
 Video Content
 An illustration of Heap and Heap Sort.

Intelligent Networking Laboratory Dang Thien Binh 72/82

 Heap & Heap Sort Algorithm

Intelligent Networking Laboratory Dang Thien Binh 73/82

 Priority Queues

 We can implement the priority queue ADT with a heap.

The operations are:
 Maximum(A) returns the maximum element
 Extract-Max(A) removes and returns the maximum element
 Increase-Key(A,i,key) increases value of ith node to key
 Insert(A,key) inserts element key into A

Intelligent Networking Laboratory Dang Thien Binh 74/82

 Extract-Max
(1)

O(lg n)

Intelligent Networking Laboratory Dang Thien Binh 75/82

 Increase-Key

O(lg n)

Intelligent Networking Laboratory Dang Thien Binh 76/82

 Example: increase key (1/4)
1
16
2 3
14 10
4 5 6 7
8 7 9 3

8 9 10
2 4 1 increase 4 to 15
Intelligent Networking Laboratory Dang Thien Binh 77/82
 Example: increase key (2/4)
1
16
2 3
14 10
4 5 6 7
8 7 9 3

8 9 10
2 15 1
Intelligent Networking Laboratory Dang Thien Binh 78/82
 Example: increase key (3/4)
1
16
2 3
14 10
4 5 6 7
15 7 9 3

8 9 10
2 8 1
Intelligent Networking Laboratory Dang Thien Binh 79/82
 Example: increase key (4/4)
1
16
2 3
15 10
4 5 6 7
14 7 9 3

8 9 10
2 8 1
Intelligent Networking Laboratory Dang Thien Binh 80/82
 Insert-Max

O(lg n)

Intelligent Networking Laboratory Dang Thien Binh 81/82

 Practice Problems
 Show the resulting heap after insert 20 into the following
heap

Intelligent Networking Laboratory Dang Thien Binh 82/82

 Thanks to contributors
Mr. Pham Van Nguyen (2022)
Dr. Thien-Binh Dang (2017 – 2022)
Prof. Hyunseung Choo (2001 – 2022)

Intelligent Networking LaboratoryCopyright 2000-2022 Intelligent

Dang
Networking
Thien Binh
Laboratory
83/82