Heap sort

Algorithm and Data structure

Lecture slides by:

Dr. Inayat-ur-Rehman
 Priority Queues
 In many situations, simple queues are inadequate, since first in/first
out scheduling has to be overruled using some priority criteria.
 E.g. in a sequence of processes, process P2 may need to be
executed before process P1 for the proper functioning of a system,
even though P1 was put on the queue of waiting processes before
P2.
 Banks managing customer information
 Often will remove or get the information about the customer
with the minimum bank account balance
 Shared Printer queue
 list of print jobs, must get next job with highest priority, and
higher-priority jobs always print before lower-priority jobs
 In situations like these, a modified queue or priority queue, is
needed.
 In priority queues, elements are dequeued according to their
priority and their current queue positions.
Priority Queues
 The problem with a priority queue is in finding an
efficient implementation which allows relatively fast
enqueuing and dequeuing
 Since elements may arrive randomly to the queue,
there is no guarantee that the front elements will be
the most likely to be dequeued and that the elements
put at the end will be the last candidates for
dequeuing.
 The situation is complicated because a wide spectrum
of possible priority criteria can be used in different
cases.
Heaps as Priority Queues
 A heap is an excellent way to implement a
priority queue.
 3 jobs have been submitted to a printer,
the jobs have sizes 100, 10, 1 page.
 Average waiting time for FIFO service,
(100+110+111)/3 = 107 time units
 Average waiting time for shortest job first
service, (1+10+111)/3 = 36.67 time units
 Need to have a queue which does insert
and deletemin
 Priority Queue
Heaps
 A particular kind of binary tree, called a
heap, has the following properties.
1. The value of each node is not less than the
values stored in each of its children
2. The tree is an almost complete binary tree.
 These two properties define a max heap.
 If “less” in the first property is replaced
with “greater”; then the definition specifies
a min heap.
Heaps
 Almost complete binary tree
 All levels are complete except the lowest one
 (book says complete binary tree, I will define complete
and almost complete whenever I use them)
 Note down example from the board
 If such a tree has height h, then it has between
2h-1 + 2 and 2h + 1 nodes. Thus h is O(log n)
 Mistake in the expression in most of the books, so
plz concentrate only this
Almost Complete Binary Trees
 If the height of the tree is d, then all levels
except possibly level d are completely full.
The bottom level has all nodes to the left
side.
Almost Complete Binary Tree
0
A

1 2
B k C

3 5 6
4
D 2k E 2k+1 F G
7 8 9 10 11
H I J K L

Numbering of a tree’s nodes for storage in an array

A B C D E F G H I J K L
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Finding parent/child or siblings
Numbering of a tree’s nodes for storage in an array

A B C D E F G H I J K L
0 1 2 3 4 5 6 7 8 9 10 11 12 13

Root always at [0]

For [I]. Parent is at [(I-1)/2], using integer divide
For element at [I], left child is at 2i+1, right child
at 2i+2
Can also find siblings
Min-Heap

9 12

37 16 47 22

41 44 32 19 50

3 9 12 37 16 47 22 41 44 32 19 50

0 1 2 3 4 5 6 7 8 9 10 11 12
Almost Complete Binary Tree

32 47

41 16 50 22

37 44 3 19 12

9 32 47 41 16 50 22 37 44 3 19 12
0 1 2 3 4 5 6 7 8 9 10 11 12
 9

32 47

41 16 50 22

37 44 3 19 12

Start with the last node that is not a leaf

If the node is smaller than its child(ren), do nothing
If the node is larger than its child(ren), swap the node with the
smaller of the children (percolatedown)
keep moving backward through the tree.
Insert a new element
3

9 12

37 16 47 22

41 44 32 19 50 5

3 9 12 37 16 47 22 41 44 32 19 50 5
0 1 2 3 4 5 6 7 8 9 10 11 12
Inserting a new element
 Insert the new element into the next
available slot in the array (preserving
the structure)
 To preserve the order, check against
its parent. If larger than parent, fine.
If smaller than parent, swap it with
the parent (percolating up).
 Keep checking until the the new
element is in correct order.
Insert a new element

9 12

37 16 5 22

41 44 32 19 50 47

3 9 12 37 16 5 22 41 44 32 19 50 47
0 1 2 3 4 5 6 7 8 9 10 11 12
Insert a new element
3

9 5

37 16 12 22

41 44 32 19 50 47

3 9 5 37 16 12 22 41 44 32 19 50 47
0 1 2 3 4 5 6 7 8 9 10 11 12
Delete the minimum node
 Minimum is at the root
 Remove it and pass it to the calling
routine
 Now there is a hole at the root
 Put last element into root. Decrement
current size by 1.
 Percolate the new root down.
DeleteMin
3

9 5

37 16 12 22

41 44 32 19 50 47

3X 9 5 37 16 12 22 41 44 32 19 50 47
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Percolate down
47

Compare left and

9 right (9 & 5) and then 5
smaller with root(47).

37 16 12 22

41 44 32 19 50

47 9 5 37 16 12 22 41 44 32 19 50
0 1 2 3 4 5 6 7 8 9 10 11 12
Percolate down
5

Swap result
9 47

37 16 12 22

41 44 32 19 50

5 9 47 37 16 12 22 41 44 32 19 50
0 1 2 3 4 5 6 7 8 9 10 11 12
Percolate down
5

9 47

37 16 12 22
Compare left and
right (12 & 22) and
41 44 32 19 50 then smaller one with
root(47).

5 9 47 37 16 12 22 41 44 32 19 50
0 1 2 3 4 5 6 7 8 9 10 11 12
Heap after DeleteMin

9 12

37 16 47 22
Swap result

41 44 32 19 50

5 9 12 37 16 47 22 41 44 32 19 50
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Heap after DeleteMin

9 12

37 16 47 22
Compare 47 with 50. Now
no need to swap, just stop
41 44 32 19 50 there.
Else we have to check till
Last index.
5 9 12 37 16 47 22 41 44 32 19 50
0 1 2 3 4 5 6 7 8 9 10 11 12 13
Are these Heaps?
10
15

8 7
6

2 3 6

20

10 12
What about these?
10 12
6

15 2 7 10 21

8 3 6
10

6 8 7

15 2 3 6
 Which are min-heaps?
10 10 10

20 80 20 80
20 80
30 15 40 60 99
40 60 85 99
10 20
50 700
20 80 10 10 80
40 60 85 99 40 60 85 99
20 80
50 700 50 700
40 60
Which are max-heaps?
wrong!
30 48 80
10 20 21 10 25 30
14 24

50
33 30

10 17 10 30 35
40
22 28 18 9
7 3
wrong! 11
Enqueuing an element to a heap
 To enqueue an element, the element is added at the
end of the heap as the last leaf.
 Restoring the heap property in the case of enqueuing
is achievied by moving from the last leaf toward the
root.
 The algorithm for enqueuing is as follows
heapEnqueue(el)
put el at the end of heap;
while el is not in the root and el > parent(el)
swap el with its parent
Enqueuing an element to a heap
20

10 15

8 7 13 14
20
2 5 6

10 15

7 13 14
8
2 5 6 15
Enqueuing an element to a heap
20

10 15

15 13 14 20
8
2 5 6 7
15 15

10 13 14
8
2 5 6 7
Dequeuing an element from heap
 Dequeuing an element from the heap
consists of removing the root element from
the heap, since
 By the heap property it is the element with
the greatest priority.
 Then the last leaf is put in its place,
 and the heap property almost certainly has
to be restored,
 This time by moving down the root down
the tree.
Dequeuing an element from heap
 The algorithm for dequeuing is as follows:
heapDequeue()
extract the element from the root;
put the element from the last leaf in its
place;
remove the last leaf;
//both subtrees of the root are heaps;
p = root;
while p is not a leaf and p < any of its
children
swap p with the larger child;
 Dequeuing an element from heap
dequeue
20

10 15

8 7 13 14
2 5 6 6

10 15

8 7 13 14

2 5
 Dequeuing an element from heap

15

10 6

8 7 13 14
15
2 5
10 14

8 7 13 6
2 5
Dequeuing an element from heap
heapDequeue()
extract the element from the root;
put the element from the last leaf in its place;
remove the last leaf;
//both subtrees of the root are heaps;
p = root;
while p is not a leaf and p < any of its children
swap p with the larger child;
 The last three lines of this algorithm can be treated as the
separate algorithm that
 Restore the heap property only if it has been violated by
the root of the tree.
Array implementation
 We start from position 1 in the array.
 The first element contains the root
 Left child of element at position j is at
position 2j, right child is at position 2j
+1
 Parent of element at position j is at
j/2
 Need to know the size of the heap.
Insertion
 Find an empty position in the heap
 If some nodes have one child, then the empty
position is the other child
 If all nodes have 0 or 2 children, then the empty
position is the left child of a leaf.
Insert the element there.
 Let the element be inserted at position j. If
the parent of the element is more than the
element, then interchange the parent and
child
 Interchange elements at j/2 and j.
 If necessary, interchange elements at j/2
with  j/2 /2
 And so on till we reach the root.
We are maintaining the heap property at
every stage.
 For (k = empty_pos; k >1; k = k/2 )
If (Heap[k] < Heap[k/2 ])
interchange(Heap[k] , Heap[k/2 ]) ;
DeleteMin
 Delete the root.
 Compare the two children of the root
 Make the lesser of the two root.
 An empty spot is created.
 Bring the lesser of the two children of
the empty spot to the empty spot.
 A new empty spot is created.
 Continue
pseudocode
 Delete root; k =1;
 Do
 {
 If A[k] is empty, break;
 If A[k]  min(A[2k], A[2k + 1]), break;
 If A[2k] < A[2k + 1] j = 2k, else j = 2k+1;
 A[k] = A[j];
 k = j;
 }Insert A[currsize] at A[k]; }

Implementation of Heap
Main()
{
/* This variable is used to indicate
size of the array acting as binary heap */
unsigned int HeapSize;
/* This pointer points to the array
which behaves as binary heap */
int *heap;
/*This variable indicates the last insertion
point I.e. points to last inserted leaf */
int last;

Continue on next slide…

Implementation of Heap
//This constructor creates a array for binary
heap
Heap(unsigned int Max_Size)
{
last = -1;
heapSize = Max_Size;
heap = new int[Max_Size];
for(int I = 0; I < Max_Size;
I++)//initializ array
heapI] = 0;
}

Continue on next slide…

 Implementation of Heap
Void insert(int el) //inserts an element in heap
{ if(last == HeapSize-1) cout<<“no space in heap”;
else { heap[++last] = el; //insert an element in heap
int tLast = last; Bool adjusted = false;
while((!adjusted)&&(tlast != 0))//Restore heap property
{ if(tlast%2!=0)//find parent index when odd child index
int p = ceil((tlast – 1)/2)
else //find parent index when child index is even
p = (tlast-2)/2
if(data[tlast] > data[p])//child value > parent value
{ swap(data[talst], data[p])//swap the parent & child
tlast = p; //parent becomes the
child
}
else adjusted = true;
}
}
}
Dequeuing an element from heap
TdeleteFromHeap(void) //removes an element from heap
{ if(last == -1) cout<<“empty heap”;
else { int value = heap[0]; //remove root value;
heap[0] = heap[last]; //move last inserted leaf to root
heap[last- -] = 0; //remove last inserted leaf
int first = 0 largest = 1 ;
while(largest <= last) //Restore heap property
{ if(largest < last && //first has two children (at
2*first + 1 and
heap[largest] < heap[largest+1]) //2*first+2) and
the second
largest++; //is larger than the first
If(heap[first] < heap[largest]) //if necessary,
{ swap(heap[first], heap[largest]);//swap child
and parent,
first = largest; largest = 2 * first +1; //and
move down
} else largest = last + 1; //to exit the
loop, the heap property not violated at heat[fist]
} return value}
Reference
 Mohammad Ikramulhaq
 FAST
 Data structures and algorithm in C++ , 2nd Edition,
By Adam Drozdek (second edition), Brooks / Cole,
Thomson Learning Inc.
 Introduction to Data Structures and Algorithms with
C++ By Glenn W. Rowe, Prentice-Hall, India
 Data structures and algorithm analysis in C++, By
Mark Allen Weiss, Benjamin/Cummings Publishing
Company, Inc.
 Data structures and Program Design in C, By Robert
L. Kruse …..
 \\juweb\jinnah\Dr.A Qadir\CS2114 Summer2004