CSE408
Heap & Heap sort,Hashing
Lecture #18
Special Types of Trees
• Def: Full binary tree = a 4
binary tree in which each node 1 3
is either a leaf or has degree
2 16 9 10
exactly 2. 14 8 7
12
Full binary tree
4
• Def: Complete binary tree = a
1 3
binary tree in which all leaves
2 16 9 10
are on the same level and all
internal nodes have degree 2. Complete binary tree
Definitions
• Height of a node = the number of edges on the longest
simple path from the node down to a leaf
• Level of a node = the length of a path from the root to
the node
• Height of tree = height of root node
4 Height of root = 3
1 3
Height of (2)= 1 2 16 9 10 Level of (10)= 2
14 8
Useful Properties
height
height
(see Ex 6.1-2, page 129)
d 1
d
2 1
n 2l 2d 1 1 4 Height of root = 3
l 0 2 1
1 3
Height of (2)= 1 2 16 9 10 Level of (10)= 2
14 8
The Heap Data Structure
• Def: A heap is a nearly complete binary tree with
the following two properties:
– Structural property: all levels are full, except
possibly the last one, which is filled from left to right
– Order (heap) property: for any node x
Parent(x) ≥ x
8 From the heap property, it
follows that:
7 4 “The root is the maximum
5 2 element of the heap!”
Heap
A heap is a binary tree that is filled in order
Array Representation of Heaps
• A heap can be stored as an
array A.
– Root of tree is A[1]
– Left child of A[i] = A[2i]
– Right child of A[i] = A[2i + 1]
– Parent of A[i] = A[ i/2 ]
– Heapsize[A] ≤ length[A]
• The elements in the subarray
A[(n/2+1) .. n] are leaves
Heap Types
• Max-heaps (largest element at root), have the
max-heap property:
– for all nodes i, excluding the root:
A[PARENT(i)] ≥ A[i]
• Min-heaps (smallest element at root), have the
min-heap property:
– for all nodes i, excluding the root:
A[PARENT(i)] ≤ A[i]
Adding/Deleting Nodes
• New nodes are always inserted at the bottom
level (left to right)
• Nodes are removed from the bottom level (right
to left)
Operations on Heaps
• Maintain/Restore the max-heap property
– MAX-HEAPIFY
• Create a max-heap from an unordered array
– BUILD-MAX-HEAP
• Sort an array in place
– HEAPSORT
• Priority queues
Maintaining the Heap Property
• Suppose a node is smaller than a
child
– Left and Right subtrees of i are max-heaps
• To eliminate the violation:
– Exchange with larger child
– Move down the tree
– Continue until node is not smaller than
children
Example
MAX-HEAPIFY(A, 2, 10)
A[2] A[4]
A[2] violates the heap property A[4] violates the heap property
A[4] A[9]
Heap property restored
Maintaining the Heap Property
• Assumptions: Alg: MAX-HEAPIFY(A, i, n)
– Left and Right 1. l ← LEFT(i)
subtrees of i are 2. r ← RIGHT(i)
max-heaps
3. if l ≤ n and A[l] > A[i]
– A[i] may be
4. then largest ←l
smaller than its
children 5. else largest ←i
6. if r ≤ n and A[r] > A[largest]
7. then largest ←r
8. if largest i
9. then exchange A[i] ↔ A[largest]
10. MAX-HEAPIFY(A, largest, n)
MAX-HEAPIFY Running Time
• Intuitively:
- h
-
- 2h
- O(h)
• Running time of MAX-HEAPIFY is O(lgn)
• Can be written in terms of the height of the heap,
as being O(h)
– Since the height of the heap is lgn
Building a Heap
• Convert an array A[1 … n] into a max-heap (n = length[A])
• The elements in the subarray A[(n/2+1) .. n] are leaves
• Apply MAX-HEAPIFY on elements between 1 and n/2
Alg: BUILD-MAX-HEAP(A) 1
4
1. n = length[A]
2 3
2. for i ← n/2 downto 1 4
1
5 6
3
7
3. do MAX-HEAPIFY(A, i, n) 8
2 9 10
16 9 10
14 8 7
A: 4 1 3 2 16 9 10 14 8 7
Example: A 4 1 3 2 16 9 10 14 8 7
i=5 i=4 i=3
1 1 1
4 4 4
2 3 2 3 2 3
1 3 1 3 1 3
4 5 6 7 4 5 6 7 4 5 6 7
8
2 9 10
16 9 10 8 2 9 10
16 9 10 8 14 9 10
16 9 10
14 8 7 14 8 7 2 8 7
i=2 i=1
1 1 1
4 4 16
2 3 2 3 2 3
1 10 16 10 14 10
4 5 6 7 4 5 6 7 4 5 6 7
8
14 9 10
16 9 3 8
14 9 10
7 9 3 8
8 9 10
7 9 3
2 8 7 2 8 1 2 4 1
Running Time of BUILD MAX HEAP
Alg: BUILD-MAX-HEAP(A)
1. n = length[A]
2. for i ← n/2 downto 1
O(n)
3. do MAX-HEAPIFY(A, i, n) O(lgn)
Running time: O(nlgn)
• This is not an asymptotically tight upper bound
Running Time of BUILD MAX HEAP
• HEAPIFY takes O(h) the cost of HEAPIFY on a node i is
proportional to the height ofh the node i in the tree
h
T (n) ni hi 2i h i O(n)
Height i 0 i 0 Level No. of nodes
h0 = 3 (lgn) i=0 20
h1 = 2 i=1 21
h2 = 1 i=2 22
h3 = 0 i = 3 (lgn) 23
hi = h – i height of the heap rooted at level i
ni = 2i number of nodes at level i
Running Time of BUILD MAX HEAP
h
T (n) ni hi Cost of HEAPIFY at level i number of nodes at that level
i 0
h
2i h i Replace the values of ni and hi computed before
i 0
h
hi h
h i
2 Multiply by 2h both at the nominator and denominator and
1
i 0 2 write 2i as i
2
h
k
2 k
h
Change variables: k = h - i
k 0 2
k
n k
The sum above is smaller than the sum of all elements to
k 0 2 and h = lgn
O(n) The sum above is smaller than 2
Running time of BUILD-MAX-HEAP: T(n) = O(n)
Heapsort
• Goal:
– Sort an array using heap representations
• Idea:
– Build a max-heap from the array
– Swap the root (the maximum element) with the last
element in the array
– “Discard” this last node by decreasing the heap size
– Call MAX-HEAPIFY on the new root
– Repeat this process until only one node remains
Example: A=[7, 4, 3, 1, 2]
MAX-HEAPIFY(A, 1, 4) MAX-HEAPIFY(A, 1, 3) MAX-HEAPIFY(A, 1, 2)
MAX-HEAPIFY(A, 1, 1)
Alg: HEAPSORT(A)
1. BUILD-MAX-HEAP(A) O(n)
2. for i ← length[A] downto 2
n-1 times
3. do exchange A[1] ↔ A[i]
4. MAX-HEAPIFY(A, 1, i - 1) O(lgn)
• Running time: O(nlgn) --- Can be
shown to be Θ(nlgn)
Priority Queues
12 4
Operations on Priority Queues
• Max-priority queues support the following
operations:
– INSERT(S, x): inserts element x into set S
– EXTRACT-MAX(S): removes and returns element of
S with largest key
– MAXIMUM(S): returns element of S with largest key
– INCREASE-KEY(S, x, k): increases value of element
x’s key to k (Assume k ≥ x’s current key value)
HEAP-MAXIMUM
Goal:
– Return the largest element of the heap
Running time: O(1)
Alg: HEAP-MAXIMUM(A)
1. return A[1]
Heap A:
Heap-Maximum(A) returns 7
HEAP-EXTRACT-MAX
Goal:
– Extract the largest element of the heap (i.e., return the max
value and also remove that element from the heap
Idea:
– Exchange the root element with the last
– Decrease the size of the heap by 1 element
– Call MAX-HEAPIFY on the new root, on a heap of size n-1
Heap A: Root is the largest element
Example: HEAP-EXTRACT-MAX
16 1
14 10 max = 16 14 10
8 7 9 3 8 7 9 3
2 4 1 2 4
Heap size decreased with 1
14
Call MAX-HEAPIFY(A, 1, n-1)
8 10
4 7 9 3
2 1
HEAP-EXTRACT-MAX
Alg: HEAP-EXTRACT-MAX(A, n)
1. if n < 1
2. then error “heap underflow”
3. max ← A[1]
4. A[1] ← A[n]
5. MAX-HEAPIFY(A, 1, n-1) remakes heap
6. return max
Running time: O(lgn)
HEAP-INCREASE-KEY
• Goal:
– Increases the key of an element i in the heap
• Idea:
– Increment the key of A[i] to its new value
– If the max-heap property does not hold anymore:
traverse a path toward the root to find the proper
place for the newly increased key
16
14 10
8 7 9 3
i
Key [i] ← 15 2 4 1
Example: HEAP-INCREASE-KEY
16 16
14 10 14 10
8 7 9 3 8 7 9 3
i i
2 4 1 2 15 1
Key [i ] ← 15
16 16
i
14 10 15 10
i
15 7 9 3 14 7 9 3
2 8 1 2 8 1
HEAP-INCREASE-KEY
Alg: HEAP-INCREASE-KEY(A, i, key)
1. if key < A[i]
2. then error “new key is smaller than current key”
3. A[i] ← key
4. while i > 1 and A[PARENT(i)] < A[i] 16
5. do exchange A[i] ↔ A[PARENT(i)]
14 10
6. i ← PARENT(i)
8 7 9 3
i
2 4 1
• Running time: O(lgn)
Key [i] ← 15
MAX-HEAP-INSERT
• Goal:
16
– Inserts a new element into a max-
heap 14 10
8 7 9 3
• Idea:
2 4 1 -
– Expand the max-heap with a new
16
element whose key is -
– Calls HEAP-INCREASE-KEY to 14 10
set the key of the new node to its 8 7 9 3
correct value and maintain the 2 4 1 15
max-heap property
Example: MAX-HEAP-INSERT
Insert value 15: Increase the key to 15
- Start by inserting - Call HEAP-INCREASE-KEY on A[11] = 15
16 16
14 10 14 10
8 7 9 3 8 7 9 3
2 4 1 - 2 4 1 15
The restored heap containing
the newly added element
16 16
14 10 15 10
8 15 9 3 8 14 9 3
2 4 1 7 2 4 1 7
MAX-HEAP-INSERT
16
Alg: MAX-HEAP-INSERT(A, key, n)
14 10
1. heap-size[A] ← n + 1 8 7 9 3
2 4 1 -
2. A[n + 1] ← -
3. HEAP-INCREASE-KEY(A, n + 1, key)
Running time: O(lgn)
Summary
• We can perform the following operations on
heaps:
– MAX-HEAPIFY O(lgn)
– BUILD-MAX-HEAP O(n)
– HEAP-SORT O(nlgn)
– MAX-HEAP-INSERT O(lgn)
– HEAP-EXTRACT-MAX O(lgn)
Average
– HEAP-INCREASE-KEY O(lgn) O(lgn)
– HEAP-MAXIMUM O(1)
Hash Functions
• If the input keys are integers then simply Key
mod TableSize is a general strategy.
– Unless key happens to have some undesirable
properties. (e.g. all keys end in 0 and we use mod 10)
• If the keys are strings, hash function needs more
care.
– First convert it into a numeric value.
Some methods
• Truncation:
– e.g. 123456789 map to a table of 1000 addresses
by picking 3 digits of the key.
• Folding:
– e.g. 123|456|789: add them and take mod.
• Key mod N:
– N is the size of the table, better if it is prime.
• Squaring:
– Square the key and then truncate
• Radix conversion:
– e.g. 1 2 3 4 treat it to be base 11, truncate if
necessary.
Hash Function 1
• Add up the ASCII values of all characters of the key.
int hash(const string &key, int tableSize)
{
int hasVal = 0;
for (int i = 0; i < key.length(); i++)
hashVal += key[i];
return hashVal % tableSize;
}
• Simple to implement and fast.
• However, if the table size is large, the function does not distribute the
keys well.
• e.g. Table size =10000, key length <= 8, the hash function
can assume values only between 0 and 1016
Hash Function 2
• Examine only the first 3 characters of the key.
int hash (const string &key, int tableSize)
{
return (key[0]+27 * key[1] + 729*key[2]) % tableSize;
• In theory, 26 * 26 * 26 = 17576 different words can be generated.
However, English is not random, only 2851 different combinations are
possible.
• Thus, this function although easily computable, is also not appropriate if
the hash table is reasonably large.
Hash Function 3
KeySize1
hash(key ) Key
i 0
[ KeySize i 1] 37 i
int hash (const string &key, int tableSize)
{
int hashVal = 0;
for (int i = 0; i < key.length(); i++)
hashVal = 37 * hashVal + key[i];
hashVal %=tableSize;
if (hashVal < 0) /* in case overflows occurs */
hashVal += tableSize;
return hashVal;
};
Hash function for strings:
98 108 105 key[i]
key a l i
0 1 2 i
KeySize = 3;
hash(“ali”) = (105 * 1 + 108*37 + 98*372) % 10,007 = 8172
0
1
2
“ali” hash ……
function 8172
ali
……
10,006 (TableSize)
Double Hashing
• A second hash function is used to drive the collision
resolution.
– f(i) = i * hash2(x)
• We apply a second hash function to x and probe at
a distance hash2(x), 2*hash2(x), … and so on.
• The function hash2(x) must never evaluate to zero.
– e.g. Let hash2(x) = x mod 9 and try to insert 99 in the
previous example.
• A function such as hash2(x) = R – ( x mod R) with
R a prime smaller than TableSize will work well.
– e.g. try R = 7 for the previous example.(7 - x mode 7)
Hashing Applications
• Compilers use hash tables to implement the
symbol table (a data structure to keep track of
declared variables).
• Game programs use hash tables to keep track of
positions it has encountered (transposition table)
• Online spelling checkers.
Summary
• Hash tables can be used to implement the insert
and find operations in constant average time.
– it depends on the load factor not on the number of
items in the table.
• It is important to have a prime TableSize and a
correct choice of load factor and hash function.
• For separate chaining the load factor should be
close to 1.
• For open addressing load factor should not
exceed 0.5 unless this is completely
unavoidable.
– Rehashing can be implemented to grow (or shrink)
the table.
