Heap

◼ The height of a node in a heap is the

number of edges on the longest simple
downward path from the node to a leaf.
◼ The height of the heap is the height of
its root.
◼ Since a heap of n elements is based on
a complete binary tree, its height is
O(lg n)
Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Min Heap

◼ For every node i other than the root,

A[PARENT(i)] ≤ A[i]
1 13

2 21 14
3

4 24 5 31 19
6 7 68

8 65 9 26 32 10

array 13 21 14 24 31 19 68 65 26 32
index 1 2 3 4 5 6 7 8 9 10 11
 MAX-HEAP
◼ The binary trees rooted at LEFT(i) and RIGHT(i) are max heaps and A[i] might violate the
max-heap property.
◼ MAX-HEAPIFY lets the value at A[i] float down so that the subtree rooted at index i
becomes a max-heap.
◼ MAX-HEAPIFY(A, i)
1. L = Left(i)
2. R = Right(i)
3. if L ≤ A.heap-size and A[L] > A[i]
4. largest = L
5. else largest = i
6. if R ≤ A.heap-size and A[R] > A[largest]
7. then largest = R
8. if largest ≠ i
9. exchange A[i] with A[largest]
10. MAX-HEAPIFY(A, largest)
Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Action of MAX-HEAPIFY(A,2)

68

13 65

32 31 19 14

26 24 21

array 68 13 65 32 31 19 14 26 24 21
index 0 1 2 3 4 5 6 7 8 9 10 11
 Action of MAX-HEAPIFY(A,2)

68

13 65

32 31 19 14

26 24 21

array 68 13 65 32 31 19 14 26 24 21
index 0 1 2 3 4 5 6 7 8 9 10 11
 Action of MAX-HEAPIFY(A,2)

68

32 65
MAX-HEAPIFY(A,4)

13 31 19 14

26 24 21

array 68 32 65 13 31 19 14 26 24 21
index 0 1 2 3 4 5 6 7 8 9 10 11
 Action of MAX-HEAPIFY(A,2)

68

32 65

26 31 19 14

13 24 21

MAX-HEAPIFY(A,8)

array 68 32 65 26 31 19 14 13 24 21
index 0 1 2 3 4 5 6 7 8 9 10 11
 Action of MAX-HEAPIFY(A,2)

68

32 65

26 31 19 14

13 24 21

array 68 32 65 13 31 19 14 26 24 21
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP(A)
◼ We can use MAX-HEAPIFY in a bottom-up manner to convert array A[1..n],
where n = A.length, into a max-heap

BUILD-MAX-HEAP(A)
1. A.heap-size = A.length
2. for i = 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ/2 downto 1
3. MAX-HEAPIFY(A, i)
 BUILD-MAX-HEAP

21

65 68

31 13 24 19

14 32 26

array 21 65 68 31 13 24 19 14 32 26
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

21

65 68

31 MAX-HEAPIFY(A,5)
13 24 19

14 32 26

array 21 65 68 31 13 24 19 14 32 26
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

21

65 68

31 MAX-HEAPIFY(A,5)
26 24 19

14 32 13

array 21 65 68 31 26 24 19 14 32 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

21

65 68

MAX-HEAPIFY(A,4) 31 26 24 19

14 32 13

array 21 65 68 31 26 24 19 14 32 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

21

65 68

32 26 24 19

14 31 13

array 21 65 68 32 26 24 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

21

65 MAX-HEAPIFY(A,3) 68

32 26 24 19

14 31 13

array 21 65 68 32 26 24 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

21

MAX-HEAPIFY(A,2) 65 68

32 26 24 19

14 31 13

array 21 65 68 32 26 24 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

MAX-HEAPIFY(A,1)
21

65 68

32 26 24 19

14 31 13

array 21 65 68 32 26 24 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

68

65 21

32 26 24 19

14 31 13

array 68 65 21 32 26 24 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

68

65 21

32 26 24 19

14 31 13

array 68 65 21 32 26 24 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

68

65 24

32 26 21 19

14 31 13

array 68 65 24 32 26 21 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

68

65 24

32 26 21 19

14 31 13

array 68 65 24 32 26 21 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP

68

65 24

32 26 21 19

14 31 13

array 68 65 24 32 26 21 19 14 31 13
index 0 1 2 3 4 5 6 7 8 9 10 11
 BUILD-MAX-HEAP(A)
◼ We can use MAX-HEAPIFY in a bottom-up manner to
convert array A[1..n], where n = A.length, into a max-
heap

BUILD-MAX-HEAP(A)
1. A.heap-size = A.length
2. for i = 𝐴. 𝑙𝑒𝑛𝑔𝑡ℎ/2 downto 1
3. MAX-HEAPIFY(A, i)
 A Simple Upper Bound
◼ BUILD-MAX-HEAP takes O(n lg n) time.
◼ MAX-HAEP makes O(n) calls to MAX-HEAPIFY which costs O(lg n)
time.

Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Another Analysis of
Building a Heap
◼ Let’s assume the tree is complete : 𝑛 = 2ℎ − 1
◼ There is one key at the height=h ,which might shift down h levels
◼ There is two keys at the height=h-1 ,which might shift down h-1 levels
◼ There is four key at the height=h-2 ,which might shift down h-2 levels
◼ Overall costs S = ℎ + 2 ℎ − 1 + 22 ℎ − 2 + ⋯ + 2ℎ−1 1
2𝑆 = 2ℎ + 22 ℎ − 1 + 23 ℎ − 2 + ⋯ + 2ℎ 1
−𝑆 = ℎ − 21 + 22 + ⋯ + 2ℎ−1 − 2ℎ
𝑆 = −ℎ + (21 + 22 + ⋯ + 2ℎ )
= −ℎ − 1 + 20 + 21 + ⋯ + 2ℎ−1 + 2ℎ
= 2ℎ+1 − ℎ − 2 = 2 ∙ 2lg 𝑛 − lg 𝑛 − 2
≤ 2𝑛
𝛼 lg 𝛽 = 𝛽 lg 𝛼
 Heapsort
◼ Use BUILD-MAX-HEAP to build a max-heap.
◼ Since the maximum element of the array
is stored at the root A[1], put it into its
correct position by exchanging it with A[n].
◼ Restore the max-heap property by calling
MAX-HEAPIFY(A, 1) to generate a max-
heap in A[1..n-1].
◼ Repeat the steps for the max-heap of size
n-1 down to a heap of size 2
Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Heapsort(A)
HAEPSORT(A)
1. BUILD-MAX-HEAP(A)
2. for i = A.length downto 2
3. exchange A[1] with A[i]
4. A.heap-size = A.heap-size-1
5. MAX-HEAPIFY(A,1)

Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Operation of Heap Sort

97

53 59

26 41 58 31

array 97 53 59 26 41 58 31
index 0 1 2 3 4 5 6 7 8 9 10 11
 Operation of Heap Sort

59

53 58

26 41 31 97

array 59 53 58 26 41 31 97
index 0 1 2 3 4 5 6 7 8 9 10 11
 Operation of Heap Sort

58

53 31

26 41 59 97

array 58 53 31 26 41 59 97
index 0 1 2 3 4 5 6 7 8 9 10 11
 Operation of Heap Sort

53

41 31

26 58 59 97

array 53 41 31 26 58 59 97
index 0 1 2 3 4 5 6 7 8 9 10 11
 Operation of Heap Sort

41

26 31

53 58 59 97

array 41 26 31 53 58 59 97
index 0 1 2 3 4 5 6 7 8 9 10 11
 Operation of Heap Sort

31

26 41

53 58 59 97

array 31 26 41 53 58 59 97
index 0 1 2 3 4 5 6 7 8 9 10 11
 Operation of Heap Sort

26

31 41

53 58 59 97

array 26 31 41 53 58 59 97
index 0 1 2 3 4 5 6 7 8 9 10 11
 Heapsort(A)
HAEPSORT(A)
1. BUILD-MAX-HEAP(A)
2. for i = A.length downto 2
3. exchange A[1] with A[i]
4. A.heap-size = A.heap-size-1
5. MAX-HEAPIFY(A,1)

Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Heapsort
◼ It takes O(n lg n) time:
◼ Calls to BUILD-MAX-HEAP takes O(n) time.
◼ Each of the n-1 calls to MAX-HEAPIFY takes O(lg n) time.

Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Priority Queues
◼ HEAP-MAXIMUM(A)
1. Return A[1]

Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 Priority Queues
◼ HEAP-EXTRACT-MAX(A)
1. if A.heap-size < 1
2. error “heap underflow”
3. max = A[1]
4. A[1] = A[A.heap-size]
5. A.heap-size = A.heap-size – 1
6. MAX-HEAPIFY(A, 1)
7. return max

Thomas T. Cormen , Charles E. Leiserson , Ronald L. Rivest, Clifford Stein, Introduction to Algorithms, MIT Press, Cambridge, MA, 2009
 INSERT

68

65 24

32 26 21 19

14 31

array 68 65 24 32 26 21 19 14 31
index 0 1 2 3 4 5 6 7 8 9 10 11
 INSERT

68

65 24

32 26 21 19

14 31 66

array 68 65 24 32 26 21 19 14 31 66
index 0 1 2 3 4 5 6 7 8 9 10 11
 INSERT

68

65 24

32 66 21 19

14 31 26

array 68 65 24 32 26 21 19 14 31 66
index 0 1 2 3 4 5 6 7 8 9 10 11
 INSERT

68

66 24

32 65 21 19

14 31 26

array 68 66 24 32 26 21 19 14 31 65
index 0 1 2 3 4 5 6 7 8 9 10 11
 Heap: Priority Queues
◼ Max(min) priority queue
◼ An element with highest(lowest) priority is
deleted
◼ An element with arbitrary priority can be
inserted
◼ Frequently implemented using max(min)
heap

95
 Priority Queues
◼ Abstract class in C++
template <class T>
class MaxPQ {
public:
virtual ~MaxPQ(){}
// virtual destructor
virtual bool IsEmpty() const = 0;
// return true if the priority queue is empty
virtual const T& Top() const = 0;
// return reference to max element
virtual void Push(const T&) = 0;
// add an element to the priority queue
virtual void Pop() = 0;
// delete element with max priority
};

96
 Definition of a Heap
◼ Max(min) tree
◼ A tree in which the key value in each node is
no smaller (larger) than the
key values in its children (if any)
◼ The key in the root is the largest
(smallest)
◼ Max(min) heap
◼ A complete binary tree that is also
a max(min) tree

97
Definition of a Heap
14 9 30

12 7 6 3 25

10 8 6 5

Figure 5.24 : Max heaps

2 10 11

7 4 20 83 21

10 8 6 50

Figure 5.25 : Min heaps

98
 Definition of a Heap
◼ Basic operations of a max heap
◼ Creation of an empty heap
◼ Insertion of a new element into the heap
◼ Deletion of the largest element from the heap
◼ Private data members of class MaxHeap
private:
T *heap; // element array
int heapSize; // number of elements in heap
int capacity; // size of the array heap

99
Definition of a Heap
template <class T>
MaxHeap<T>::MaxHeap(int theCapacity = 10)
{
if (theCapacity < 1) throw “Capacity must be >= 1”;
capacity = theCapacity;
heapSize = 0;
heap = new T[capacity+1]; // heap[0] is not used
}
---------------------------------------------
Program 5.15 : Max heap constructor

100
 Insertion into Max Heap
◼ Examples
20
after insert

15 2

14 10

(a) (b)
Insert 21
Insert 5
20 5 21

15 15 20

14 10 2 14 10 2

(c) (d)

Figure 5.26 : Insertion into a max heap 101

 Insertion into Max Heap
◼ Implementation
◼ Need to move from child to parent
◼ Heap is complete binary tree

◼ Use formula-based representation

◼ Lemma 5.4 : parent(i) is at ⌊i/2⌋ if i≠1

◼ Complexity is O(log n)

102
 Insertion into Max Heap
template <class T>
void MaxHeap<T>::Push(const T& e)
{ // Insert e into the max heap
if (heapSize == capacity) { // double the capacity
ChangeSize1D(heap, capacity, 2*capacity);
capacity *= 2;
}
int currentNode = ++heapSize;
while (currentNode != 1 && heap[currentNode / 2] < e)
{ // bubble up
heap[currentNode] = heap[currentNode / 2];
// move parent down
currentNode /= 2;
}
heap[currentNode] = e;
}
--------------------------------------------------
Program 5.16 : Insertion into a max heap 103
 Deletion from Max Heap
◼ Example
delete 21 2 delete 20
reinsert 2 15

15 20 14 2

14 10 6 10

(a) (b) (c)

Figure 5.27 : Deletion from a heap

104
Deletion from Max Heap
template <class T>
void MaxHeap<T>::Pop()
{ // Delete max element
if (IsEmpty()) throw “Heap is empty. Cannot delete.”;
heap[1].~T(); // delete max element

// remove last element from heap

T lastE = heap[heapSize--];

// trickle down
int currentNode = 1; // root
int child = 2; // a child of currentNode
while (child <= heapSize)
{
// set child to larger child of currentNode
if (child<heapSize && heap[child]<heap[child+1]) child++;
// can we put lastE in current Node?
if (lastE>=heap[child]) break; // yes

// no
heap[currentNode] = heap[child];
currentNode = child; child *= 2;
}
heap[currentNode] = lastE;
}
----------------------------------------------------------
Program 5.17 : Deletion from a max heap
105
 Binary Search Trees: Definition
◼ Binary tree which may be empty
◼ If not empty
◼ (1) every element has a distinct key
◼ (2) keys in left subtree < root key
◼ (3) keys in right subtree > root key
◼ (4) left and right subtrees are also binary search trees

20 30 60

15 25 5 40 70

12 16
10 22 2 65 80

(a) (b) (c)

Figure 5.28 : Binary trees
106
 Searching Binary Search Tree
◼ Recursive search by key value
◼ definition of binary search tree is recursive
◼ key(element) = x
◼ x=root key : element=root
◼ x<root key : search left subtree
◼ x>root key : search right subtree

107
 Searching Binary Search Tree
template <class K, class E> // Driver
pair<K, E>* BST<K, E>::Get(const K& k)
{ // Search the binary search tree (*this) for a pair with key k
// If such a pair is found, return a pointer to this pair; otherwise, return 0
return Get(root, k);
}

template <class K, class E> // Workhorse

pair<K, E>* BST<K, E>::Get(TreeNode<pair<K, E> >* p, const K& k)
{
if(!p) return 0;
if(k<p->data.first) return Get(p->leftChild, k);
if(k>p->data.first) return Get(p->rightChild, k);
return &p->data;
}
--------------------------------------------------------
Program 5.18 : Recursive search of a binary search tree
108
 Searching Binary Search Tree
template <class K, class E> // Driver
pair<K, E>* BST<K, E>::Get(const K& k)
{
TreeNode<pair<K,E> > *currentNode = root;
while(currentNode)
{
if (k<currentNode->data.first)
currentNode = currentNode->leftChild;
else if ( k>currentNode->data.first)
currentNode = currentNode->rightChild;
else return &currentNode->data;
}
//no matching pair
return 0;
}
--------------------------------------------------------
Program 5.19 : Iterative search of a binary search tree
109
 Searching Binary Search Tree
◼ Search by rank
◼ node needs LeftSize field
◼ LeftSize=1 + #elements in left subtree

3 30

2 5 40 1

1 2

110
 Searching Binary Search Tree
template <class K, class E> // search by rank
pair<K,E>* BST<K,E>::RankGet(int r)
{ // Search the binary search tree for the rth smallest pair
TreeNode<pair<K,E> > *currentNode = root;
while (currentNode)
if(r<currentNode->leftSize)
currentNode = currentNode->leftChild;
else if (r>currentNode->leftSize)
{
r -= currentNode->leftSize;
currentNode = currentNode->rightChild;
}
else return &currentNode->data;
return 0;
}
-------------------------------------------------
Program 5.20 : Searching a binary search tree by rank
111
 Insertion into Binary Search Tree

◼ New element x
◼ search x in the tree
◼ success : x is in the tree
◼ fail : insert x at the point the search terminated

30 30

5 40 5 40

2 80 2 35 80

(a) Insert 80 (b) Insert 35

Figure 5.29 : Inserting into a binary search tree

112
Insertion into Binary Search Tree
template <class K, class E>
void BST<K, E>::Insert(const pair<K, E>& thePair)
{ // Insert thePair into the binary search tree.
// search for thePair.first, pp is parent of p
TreeNode<pair<K, E> > *p = root, *pp = 0;
while(p) {
pp = p;
if (thePair.first < p->data.first) p = p->leftChild;
else if(thePair.first > p->data.first) p = p->rightChild;
else // duplicate, update associated element
{ p->data.second = thePair.second; return; }
}
//perform insertion
p = new TreeNode<pair<K, E> >(thePair);
if(root) // tree not empty
if (thePair.first<pp->data.first) pp->leftChild=p;
else pp->rightChild = p;
else root = p;
}
-----------------------------------------------------
Program 5.21 : Insertion into a binary search tree 113