Design and Analysis of Algorithm

(Heap Sort)

Lecture -14 - 17
 Overview

• O(n lg n) worst case time complexity.

• Sorts in place-like insertion sort.

• O(n) is the tight bound analysis of Build Heap.

Heap data structure
Example
 Example

• Given an array of size N. The task is to sort the

array elements by using Heap Sort.
– Input:
– N=10
– Arr[]:{16, 4, 10, 14, 7, 9, 3, 2, 8, 1}
– Output: 1 2 3 4 7 8 9 10 14 16
 Example

• Given an array of size N. The task is to sort the

array elements by using Heap Sort.
– Input:
– N = 10
– arr[] = {10,9,8,7,6,5,4,3,2,1}
– Output:1 2 3 4 5 6 7 8 9 10
 Heap property

• For max-heaps (largest element at root),

max-heap property: for all nodes i ,
excluding the root, A[PARENT(i )] ≥ A[i ].
• For min-heaps (smallest element at root),
min-heap property: for all nodes i ,
excluding the root, A[PARENT(i )] ≤ A[i ].
Maintaining the heap property
The Time Complexity
of MAX-HEAPIFY (A,i,n)
is Ο(log 𝑛).
[Because any element
insert in a tree
required maximum
log n time]
Building a heap
 Example
Building a max-heap from the following unsorted array results in the
first heap example.
 Correctness
Initialization:

Initialization: we know that each node n / 2 + 1, n / 2 + 2, . . . , n is a

leaf, which is the root of a trivial max-heap. Since i = n / 2 before the
first iteration of the for loop, the invariant is initially true.

Maintenance: Children of node i are indexed higher than i , so by the

loop invariant, they are both roots of max-heaps. Correctly assuming
that i+1, i+2, . . . , n are all roots of max-heaps, MAX-HEAPIFY makes
node i a max-heap root. Decrementing i re-establishes the loop
invariant at each iteration.

Termination: When i = 0, the loop terminates. By the loop invariant,

each node, notably node 1, is the root of a max-heap.
 Analysis
• Simple bound: O(n) calls to MAX-HEAPIFY,
each of which takes O(lg n) time ⇒ O(n lg n).

•Tighter analysis observation:

An n element heap has height log 𝑛 and at
𝑛
most nodes of any height h.
2ℎ +1
 Tighter analysis Proof

BUILD-MAX-HEAP(A,n) 9
𝒇𝒐𝒓 𝒊 ← 𝒏/𝟐 𝒅𝒐𝒘𝒏𝒕𝒐 𝟏
do MAX-HEAPIFY (A,i,n) 6 5

0 8 2 1
1 2 3 4 5 6 7 8
9 6 5 0 8 2 1 3
3
 Tighter analysis Proof
• For easy understanding, Let us take a complete binary
Tree,

• The height of a node is the number of edges from the

node to the deepest leaf.
• The depth of a node is the no of edges from the root of
the node.
 Tighter analysis Proof
• All the leaves are of height 0, therefore
there are 8 nodes at height 0.
• 4 numbers of nodes at height 1.
• 2 numbers of nodes at height 2.
• and one node at height 3.
 Tighter analysis Proof
• Hence the question is how many nodes are
there at height ‘h’ in a complete binary tree?
• The answer is :
• If there are n nodes in tree, then at most
𝑛
nodes are available at height h.
2ℎ+1
 Tighter analysis Proof
• Now if we apply MAX-HEAPIFY() on any node of any
level, then the time taken by MAX-HEAPIFY() is the
𝑛
height of the node.(i.e. ℎ+1 Ο(ℎ))
2
• Hence in case of root the time taken is log 𝒏.
• Hence
log 𝑛
𝑛
𝑇 𝑛 = ෍ Ο(ℎ)
2ℎ+1
ℎ=0
 Tighter analysis Proof
log 𝑛
𝑛
𝑇 𝑛 = ෍ Ο(ℎ)
2ℎ+1
ℎ=0
ℎ
≤ Ο 𝑛 σ∞
ℎ=0 2ℎ 2
𝑛 ∞ ℎ
≤Ο σℎ=0 ℎ
2 2
𝑛 ∞ 1 ℎ
≤Ο σ ℎ 2
2 ℎ=0

𝑛 1Τ 𝑛
≤Ο 2
⟹ Ο(2 2 )
2 1−1Τ2 2
T(n) ⟹ Ο 𝑛
Hence the running time of BUILD-MAX-HEAP(A,n) is Ο 𝑛 in tight bound .
 Tighter analysis Proof
∞
1
෍ 𝑥𝑘 = value of Infanite G P Series
1−𝑥
𝑘=0
∞

෍ 𝑥 𝑘 = (1 − 𝑥)−1
𝑘=0
Differentiate both side:
∞
1
෍ 𝑘. 𝑥 𝑘−1 = −1 1 − 𝑥 −2
−1 =
(1 − 𝑥)2
𝑘=0
Multiply x both side
∞
𝑥
෍ 𝑘. 𝑥 𝑘 ==
(1 − 𝑥)2
𝑘=0
 The heapsort algorithm
Given an input array, the heapsort algorithm acts as
follows:
• Builds a max-heap from the array.
• Starting with the root (the maximum element), the
algorithm places the maximum element into the
correct place in the array by swapping it with the
element in the last position in the array.
• “Discard” this last node (knowing that it is in its
correct place) by decreasing the heap size, and calling
MAX-HEAPIFY on the new (possibly incorrectly-placed)
root.
• Repeat this “discarding” process until only one node
(the smallest element) remains, and therefore is in the
correct place in the array.
Example
 Analysis
• BUILD-MAX-HEAP: O(n)
• for loop: n − 1 times
• exchange elements: O(1)
• MAX-HEAPIFY: O(lg n)

Total time: O(n lg n).

 Heap implementation of
priority queue
• Heaps efficiently implement priority
queues. These notes will deal with max
priority queues implemented with max-
heaps. Min-priority queues are
implemented with min-heaps similarly.
• A heap gives a good compromise between
fast insertion but slow extraction and vice
versa. Both operations take O(lg n) time.
 Priority queue
• Maintains a dynamic set S of elements.
• Each set element has a key-an associated value.
• Max-priority queue supports dynamic-set operations:
• INSERT(S, x): inserts element x into set S.
• MAXIMUM(S): returns element of S with largest key.
• EXTRACT-MAX(S): removes and returns element of S
with largest key.
• INCREASE-KEY(S, x, k): increases value of element x’s
key to k. Assume k ≥ x’s current key value.
• Example max-priority queue application: schedule jobs on
shared computer.
• Min-priority queue supports similar operations:
• INSERT(S, x): inserts element x into set S.
• MINIMUM(S): returns element of S with
smallest key.
• EXTRACT-MIN(S): removes and returns
element of S with smallest key.
• DECREASE-KEY(S, x, k): decreases value of
element x’s key to k. Assume k ≤ x’s current
key value.
• Example min-priority queue application:
event - driven simulator.
Finding the maximum element

Getting the maximum element is easy: it’s

the root.
HEAP-MAXIMUM(A)
return A[1]
Finding the maximum element

Getting the maximum element is easy: it’s

the root.
HEAP-MAXIMUM(A)
return A[1]
Time: 𝚶(1).
Extracting max element
Given the array A:
• Make sure heap is not empty.
• Make a copy of the maximum element (the root).
• Make the last node in the tree the new root.
• Re-heapify the heap, with one fewer node.
• Return the copy of the maximum element.
Extracting max element
Given the array A:
• Make sure heap is not empty.
• Make a copy of the maximum element (the root).
• Make the last node in the tree the new root.
• Re-heapify the heap, with one fewer node.
• Return the copy of the maximum element.
HEAP-EXTRACT-MAX(A, n)
if n < 1
then error .heap underflow.
max ← A[1]
A[1] ← A[n]
MAX-HEAPIFY(A, 1, n − 1) remakes heap
return max
Extracting max element
Given the array A:
• Make sure heap is not empty.
• Make a copy of the maximum element (the root).
• Make the last node in the tree the new root.
• Re-heapify the heap, with one fewer node.
• Return the copy of the maximum element.
HEAP-EXTRACT-MAX(A, n) Time Complexity
if n < 1
then error .heap underflow.
is Ο(log 𝑛)
max ← A[1]
A[1] ← A[n]
MAX-HEAPIFY(A, 1, n − 1) remakes heap
return max
• HEAP-INCREASE-KEY(A,i,key)
1. If key<A[i]
2. error” new key is smaller than the
current key”.
3. A[i]= key
4. While i>1 and A[parent(i)]<A[i]
5. swap(A[parent(i)], A[i])
6. i=parent(i)
• HEAP-INCREASE-KEY(A,i,key)
1. If key<A[i]
2. error” new key is smaller than the
current key”.
3. A[i]= key
4. While i>1 and A[parent(i)]<A[i]
5. swap(A[parent(i)], A[i])
6. i=parent(i)
The running time of HEAP-INCREASE-
KEY(A,i,key) is Ο(log 𝑛)
• Example(from own)
• MAX-HEAP-INSERT(A,key)
1. A.heap-size= A.heap-size+1
2. A[heap-size]=-∞
3. HEAP-INCREASE-KEY(A,heap-size,key)
• MAX-HEAP-INSERT(A,key)
1. A.heap-size= A.heap-size+1
2. A[heap-size]=-∞
3. HEAP-INCREASE-KEY(A,heap-size,key)
The running time of MAX-HEAP-INSERT(A,key) is
Ο(log 𝑛).