Data Structures and

Algorithms
"
Priority Queues!
 Outline"
• Priority Queues!
• Heaps!
• Adaptable Priority Queues!

Phạm Bảo Sơn - DSA

Priority Queues"
 Priority Queue ADT"
• A priority queue stores a • Additional methods!
collection of entries! – min()
• Each entry is a pair returns, but does not
(key, value)! remove, an entry with
smallest key!
• Main methods of the Priority – size(), isEmpty()!
Queue ADT!
– insert(k, x)
inserts an entry with key k
and value x! • Applications:!
– removeMin() – Standby flyers!
removes and returns the – Auctions!
entry with smallest key! – Stock market!

Phạm Bảo Sơn - DSA

 Total Order Relations"
• Keys in a priority • Mathematical concept
queue can be of total order relation ≤!
arbitrary objects – Reflexive property:
on which an order x≤x
is defined! – Antisymmetric property:
x ≤ y ∧ y ≤ x ⇒ x = y!
• Two distinct
entries in a priority – Transitive property:
x≤y∧y≤z⇒x≤z
queue can have
the same key

Phạm Bảo Sơn - DSA

 Entry ADT"
• An entry in a priority • As a Java interface:!
queue is simply a key- /** !
value pair! * Interface for a key-value!
• Priority queues store * pair entry !
entries to allow for **/!
efficient insertion and public interface Entry {!
removal based on keys! public Object key();!
• Methods:! public Object value();!
– key(): returns the key for }!
this entry!
– value(): returns the value
associated with this
entry!
Phạm Bảo Sơn - DSA
 Comparator ADT"
• A comparator encapsulates
the action of comparing two
objects according to a given
total order relation!
• A generic priority queue • The primary method of the
uses an auxiliary Comparator ADT:!
comparator! – compare(x, y): Returns an
• The comparator is external integer i such that i < 0 if a <
to the keys being compared! b, i = 0 if a = b, and i > 0 if a
• When the priority queue > b; an error occurs if a and b
needs to compare two keys, cannot be compared.!
it uses its comparator!

Phạm Bảo Sơn - DSA

•
Example Comparator
Lexicographic comparison of 2-D
"
points:! • Point objects:!
!
!
/** Comparator for 2D points under the
standard lexicographic order. */! /** Class representing a point in the
plane with integer coordinates */!
public class Lexicographic implements
Comparator {! public class Point2D !{!
int xa, ya, xb, yb;! protected int xc, yc; // coordinates!
public int compare(Object a, Object b) public Point2D(int x, int y) {!
throws ClassCastException {! xc = x;!
xa = ((Point2D) a).getX();! yc = y;!
ya = ((Point2D) a).getY();! }!
xb = ((Point2D) b).getX();! public int getX() { !
yb = ((Point2D) b).getY();! ! !return xc; !!
if (xa != xb)! }!
" "return (xb - xa);! public int getY() { !
else" ! !return yc; !!
" "return (yb - ya);! }!
}! }!
}!
Phạm Bảo Sơn - DSA
 Priority Queue Sorting"
• We can use a priority Algorithm PQ-Sort(S, C)
queue to sort a set of Input sequence S, comparator C
comparable elements! for the elements of S
1. Insert the elements one Output sequence S sorted in
by one with a series of increasing order according to C
insert operations!
2. Remove the elements in P ← priority queue with
sorted order with a series comparator C
of removeMin operations! while ¬S.isEmpty ()
• The running time of this e ← S.removeFirst ()
sorting method depends on P.insert (e, 0)
the priority queue
while ¬P.isEmpty()
implementation!
e ← P.removeMin().key()
S.insertLast(e)

Phạm Bảo Sơn - DSA

 Sequence-based Priority
Queue"
• Implementation with an • Implementation with a
unsorted list! sorted list!
4 5 2 3 1 1 2 3 4 5

• Performance:!
– insert takes O(1) time
• Performance:!
since we can insert the – insert takes O(n) time
item at the beginning or since we have to find the
end of the sequence! place where to insert the
– removeMin and min take item!
O(n) time since we have – removeMin and min take
to traverse the entire O(1) time, since the
sequence to find the
smallest key ! smallest key is at the
beginning!
Phạm Bảo Sơn - DSA
 Selection-Sort"
• Selection-sort is the variation of PQ-sort where the
priority queue is implemented with an unsorted
sequence!
• Running time of Selection-sort:!
1. Inserting the elements into the priority queue with n insert
operations takes O(n) time!
2. Removing the elements in sorted order from the priority
queue with n removeMin operations takes time
proportional to
! ! !1 + 2 + …+ n!
• Selection-sort runs in O(n2) time !
Phạm Bảo Sơn - DSA
 Selection-Sort Example"
!Sequence S ! !Priority Queue P!!
Input: ! !(7,4,8,2,5,3,9) ! !() !!
!
Phase 1!!
!(a) ! !(4,8,2,5,3,9) ! !(7) !!
!(b) ! !(8,2,5,3,9) ! !(7,4) !!
!.. ! !.. !.. !!
!. ! !. !. !!
!(g) ! !() ! ! !(7,4,8,2,5,3,9) !!
Phase 2!!
!(a) ! !(2) ! ! !(7,4,8,5,3,9) !!
!(b) ! !(2,3) ! ! !(7,4,8,5,9) !!
!(c) ! !(2,3,4) ! ! !(7,8,5,9)!!
!(d) ! !(2,3,4,5)! ! !(7,8,9) !!
!(e) ! !(2,3,4,5,7) ! !(8,9) !!
!(f) ! !(2,3,4,5,7,8) ! !(9) !!
!(g) ! !(2,3,4,5,7,8,9) ! !()!
Phạm Bảo Sơn - DSA
 Insertion-Sort"
• Insertion-sort is the variation of PQ-sort where the
priority queue is implemented with a sorted
sequence!
• Running time of Insertion-sort:!
1. Inserting the elements into the priority queue with n insert
operations takes time proportional to
! ! !1 + 2 + …+ n!
2. Removing the elements in sorted order from the priority
queue with a series of n removeMin operations takes
O(n) time!
• Insertion-sort runs in O(n2) time !

Phạm Bảo Sơn - DSA

 Insertion-Sort Example"
! ! !Sequence S !Priority queue P!!
Input: ! !(7,4,8,2,5,3,9) ! !() !!
!
Phase 1!!
(a) ! !(4,8,2,5,3,9) ! !(7) !!
!(b) ! !(8,2,5,3,9) ! !(4,7) !!
!(c) ! !(2,5,3,9)! ! !(4,7,8) !!
!(d) ! !(5,3,9) ! ! !(2,4,7,8)!!
!(e) ! !(3,9) ! ! !(2,4,5,7,8) !!
!(f) ! !(9) ! ! !(2,3,4,5,7,8) !!
!(g) ! !() ! ! !(2,3,4,5,7,8,9) !!
!
Phase 2!!
!(a) ! !(2) ! ! !(3,4,5,7,8,9) !!
!(b) ! !(2,3) ! ! !(4,5,7,8,9) !!
!.. ! !.. ! ! !.. !!
!. ! !. ! ! !. !!
!(g) ! !(2,3,4,5,7,8,9) ! !()!
Phạm Bảo Sơn - DSA
 In-place Insertion-sort
5 4 2
"
3 1
• Instead of using an
external data structure,
we can implement 5 4 2 3 1
selection-sort and
insertion-sort in-place! 4 5 2 3 1
• A portion of the input
sequence itself serves as
2 4 5 3 1
the priority queue!
• For in-place insertion-sort!
2 3 4 5 1
– We keep sorted the initial
portion of the sequence!
– We can use swaps 1 2 3 4 5
instead of modifying the
sequence! 1 2 3 4 5
Phạm Bảo Sơn - DSA
Heaps"

5 6

9 7
 Recall Priority Queue ADT"
• A priority queue stores a • Additional methods!
collection of entries! – min()
• Each entry is a pair returns, but does not
(key, value)! remove, an entry with
smallest key!
• Main methods of the Priority – size(), isEmpty()!
Queue ADT!
– insert(k, x)
inserts an entry with key k
and value x! • Applications:!
– removeMin() – Standby flyers!
removes and returns the – Auctions!
entry with smallest key! – Stock market!

Phạm Bảo Sơn - DSA

 Recall Priority Queue
Sorting"
• We can use a priority
queue to sort a set of
comparable elements! Algorithm PQ-Sort(S, C)
– Insert the elements with a Input sequence S, comparator C
series of insert operations! for the elements of S
– Remove the elements in Output sequence S sorted in
sorted order with a series increasing order according to C
of removeMin operations!
• The running time depends P ← priority queue with
on the priority queue comparator C
implementation:! while ¬S.isEmpty ()
– Unsorted sequence gives
selection-sort: O(n2) time! e ← S.remove (S. first ())
– Sorted sequence gives P.insertItem(e, e)
insertion-sort: O(n2) time! while ¬P.isEmpty()
• Can we do better?! e ← P.removeMin()
Phạm Bảo Sơn - DSA
S.insertLast(e)
 Heaps"
• A heap is a binary tree • The last node of a heap
storing keys at its nodes is the rightmost node of
and satisfying the following depth h
properties:!
– Heap-Order: for every
internal node v other than the 2
root,
key(v) ≥ key(parent(v)) 5 6
– Complete Binary Tree: let h
be the height of the heap! 9 7
• for i = 0, … , h - 1, there are
2i nodes of depth i!
• at depth h, the internal
nodes are to the left of the
external nodes! last node
Phạm Bảo Sơn - DSA
 Height of a Heap"
• Theorem: A heap storing n keys has height O(log n)
!Proof: (we apply the complete binary tree property)!
– Let h be the height of a heap storing n keys!
– Since there are 2i keys at depth i = 0, … , h - 1 and at least one key
at depth h, we have n ≥ 1 + 2 + 4 + … + 2h-1 + 1
– Thus, n ≥ 2h , i.e., h ≤ log n

depth keys
0 1

1 2
h-1 2h-1

h 1

Phạm Bảo Sơn - DSA

Heaps and Priority Queues"
• We can use a heap to implement a priority queue!
• We store a (key, element) item at each internal node!
• We keep track of the position of the last node!
• For simplicity, we show only the keys in the pictures!
(2, Sue)

(5, Pat) (6, Mark)

(9, Jeff) (7, Anna)

Phạm Bảo Sơn - DSA

 Insertion into a Heap"
• Method insertItem of the 2
priority queue ADT 5 6
corresponds to the z
9 7
insertion of a key k to
the heap!
insertion node
• The insertion algorithm
consists of three steps! 2

– Find the insertion node z 5 6

(the new last node)! z
9 7 1
– Store k at z!
– Restore the heap-order
property (discussed next)!
Phạm Bảo Sơn - DSA
 Upheap"
• After the insertion of a new key k, the heap-order property may be
violated!
• Algorithm upheap restores the heap-order property by swapping k
along an upward path from the insertion node!
• Upheap terminates when the key k reaches the root or a node
whose parent has a key smaller than or equal to k !
• Since a heap has height O(log n), upheap runs in O(log n) time!

2 1

5 1 5 2

9 7
z 6 9 7
z 6

Phạm Bảo Sơn - DSA

 Removal from a Heap"
• Method removeMin of 2
the priority queue ADT
corresponds to the 5 6
removal of the root key w
9 7
from the heap!
• The removal algorithm last node
consists of three steps!
– Replace the root key with 7
the key of the last node w!
5 6
– Remove w ! w
– Restore the heap-order 9
property (discussed next)!
new last node
Phạm Bảo Sơn - DSA
 Downheap"
• After replacing the root key with the key k of the last node, the
heap-order property may be violated!
• Algorithm downheap restores the heap-order property by
swapping key k along a downward path from the root!
• Downheap terminates when key k reaches a leaf or a node whose
children have keys greater than or equal to k !
• Since a heap has height O(log n), downheap runs in O(log n) time!

7 5

5 6 7 6
w w
9 9

Phạm Bảo Sơn - DSA

 Updating the Last Node"
• The insertion node can be found by traversing a path of O(log n)
nodes!
– Go up until a left child or the root is reached!
– If a left child is reached, go to the right child!
– Go down left until a leaf is reached!
• Similar algorithm for updating the last node after a removal!

Phạm Bảo Sơn - DSA

 Heap-Sort"
• Consider a priority
• Using a heap-based
queue with n items
priority queue, we can
implemented by means sort a sequence of n
of a heap! elements in O(n log n)
– the space used is O(n)! time
– methods insert and • The resulting algorithm
removeMin take O(log n) is called heap-sort!
time!
• Heap-sort is much
– methods size, isEmpty,
and min take time O(1)
faster than quadratic
time! sorting algorithms, such
as insertion-sort and
Phạm Bảo Sơn - DSA selection-sort!
 Vector-based Heap
Implementation"
• We can represent a heap with n
keys by means of a vector of 2
length n + 1!
• For the node at rank i! 5 6
– the left child is at rank 2i
– the right child is at rank 2i + 1 9 7
• Links between nodes are not
explicitly stored!
• The cell of at rank 0 is not used!
• Operation insert corresponds to
inserting at rank n + 1 2 5 6 9 7
• Operation removeMin 0 1 2 3 4 5
corresponds to removing at rank 1
• Yields in-place heap-sort!

Phạm Bảo Sơn - DSA

 Merging Two Heaps
3
" 2

• We are given two 8 5 4 6

heaps and a key k
• We create a new heap 7
with the root node 3 2
storing k and with the
8 5 4 6
two heaps as subtrees!
• We perform downheap
to restore the heap- 2
order property ! 3 4
8 5 7 6

Phạm Bảo Sơn - DSA

 Bottom-up Heap
Construction"
• We can construct a
heap storing n given
keys in using a bottom-
2i -1 2i -1
up construction with log
n phases!
• In phase i, pairs of
heaps with 2i -1 keys are
merged into heaps with
2i+1-1 keys!
2i+1-1

Phạm Bảo Sơn - DSA

 Example"

16 15 4 12 6 7 23 20

25 5 11 27

16 15 4 12 6 7 23 20

Phạm Bảo Sơn - DSA

 Example (contd.)"

25 5 11 27

16 15 4 12 6 9 23 20

15 4 6 20

16 25 5 12 11 9 23 27

Phạm Bảo Sơn - DSA

 Example (contd.)"
7 8

15 4 6 20

16 25 5 12 11 9 23 27

4 6

15 5 8 20

16 25 7 12 11 9 23 27

Phạm Bảo Sơn - DSA

 Example (end)"
10

4 6

15 5 8 23

16 25 7 12 11 9 27 20

5 6

15 7 8 23

16 25 10 12 11 9 27 20

Phạm Bảo Sơn - DSA

 Analysis"
• We visualize the worst-case time of a downheap with a proxy path
that goes first right and then repeatedly goes left until the bottom
of the heap (this path may differ from the actual downheap path)!
• Since each node is traversed by at most two proxy paths, the total
number of nodes of the proxy paths is O(n) !
• Thus, bottom-up heap construction runs in O(n) time !
• Bottom-up heap construction is faster than n successive insertions
and speeds up the first phase of heap-sort!

Phạm Bảo Sơn - DSA

 3 a
Adaptable
Priority Queues"

5 g 4 e
Recall the Entry and Priority
Queue ADTs"
• An entry stores a (key, • Priority Queue ADT:!
value) pair within a data – insert(k, x)
structure! inserts an entry with
key k and value x!
• Methods of the entry
ADT:! – removeMin()
removes and returns
– key(): returns the key the entry with
associated with this smallest key!
entry!
– min()
– value(): returns the value returns, but does not
paired with the key remove, an entry
associated with this with smallest key!
entry!
– size(), isEmpty()!
Phạm Bảo Sơn - DSA
 Motivating Example"
• Suppose we have an online trading system where orders to
purchase and sell a given stock are stored in two priority queues
(one for sell orders and one for buy orders) as (p,s) entries:!
– The key, p, of an order is the price!
– The value, s, for an entry is the number of shares!
– A buy order (p,s) is executed when a sell order (p’,s’) with price
p’<p is added (the execution is complete if s’>s)!
– A sell order (p,s) is executed when a buy order (p’,s’) with price
p’>p is added (the execution is complete if s’>s)!
• What if someone wishes to cancel their order before it
executes?!
• What if someone wishes to update the price or number of
shares for their order?!

Phạm Bảo Sơn - DSA

 Methods of the Adaptable
Priority Queue ADT"
• remove(e): Remove from P and return
entry e.!!
• replaceKey(e,k): Replace with k and
return the key of entry e of P; an !error
condition occurs if k is invalid (that is, k
cannot be compared with other keys). !!
• replaceValue(e,x): Replace with x and
return the value of entry e of P. !!
Phạm Bảo Sơn - DSA
Operation

Example


Output
"

P


insert(5,A) e1

(5,A)
insert(3,B) e2

(3,B),(5,A)
insert(7,C) e3

(3,B),(5,A),(7,C)
min() e2

(3,B),(5,A),(7,C)
key(e2) 3

(3,B),(5,A),(7,C)
remove(e1) e1

(3,B),(7,C)
replaceKey(e2,9) 3

(7,C),(9,B)
replaceValue(e3,D) C

(7,D),(9,B)
remove(e2) e2

(7,D) !
Phạm Bảo Sơn - DSA
 Locating Entries"
• In order to implement the operations
remove(k), replaceKey(e), and
replaceValue(k), we need fast ways of
locating an entry e in a priority queue.!
• We can always just search the entire
data structure to find an entry e, but
there are better ways for locating
entries.!
Phạm Bảo Sơn - DSA
 Location-Aware Entries"
• A locator-aware entry identifies and tracks the
location of its (key, value) object within a data
structure!
• Intuitive notion:!
– Coat claim check!
– Valet claim ticket!
– Reservation number!
• Main idea:!
– Since entries are created and returned from the
data structure itself, it can return location-aware
entries, thereby making future updates easier!
Phạm Bảo Sơn - DSA
 List Implementation"
• A location-aware list entry is an object storing!
– key!
– value!
– position (or rank) of the item in the list!
• In turn, the position (or array cell) stores the entry!
• Back pointers (or ranks) are updated during swaps!

header nodes/positions trailer

2 c 4 c 5 c 8 c

entries
Phạm Bảo Sơn - DSA
 Heap Implementation"
• A location-aware 2 d
heap entry is an
object storing! 4 a 6 b
– key!
– value!
– position of the entry
in the underlying
heap!
• In turn, each heap
position stores an
entry!
• Back pointers are
updated during 8 g 5 e 9 c
entry swaps!
Phạm Bảo Sơn - DSA
 Performance"
• Using location-aware entries we can achieve
the following running times (times better than
those achievable without location-aware
entries are highlighted in red):!
Method Unsorted List Sorted List Heap
size, isEmpty O(1) O(1) O(1)
insert O(1) O(n) O(log n)
min O(n) O(1) O(1)
removeMin O(n) O(1) O(log n)
remove O(1) O(1) O(log n)
replaceKey O(1) O(n) O(log n)
replaceValue O(1) O(1) O(1)"
Phạm Bảo Sơn - DSA