Minimum Spanning Trees (Ch. 23)! Minimum Spanning Trees!

Definition: Given an undirected graph G = (V, E) with weights on

Definition: Given an undirected graph G = (V, E), a the edges, a minimum spanning tree of G is a subgraph T ! E
such that T:
spanning tree of G is any subgraph of G that is a tree
o has no cycles,
o! connects all nodes in V, and
o! has a sum of edge weights that is minimum over
a b c all possible spanning trees.

d e
1! 6!
f g h a 3! b c
5! 3!
6! 2!
d e 8!
5! 2!
f g h
4! 7!

Minimum Spanning Trees! Minimum Spanning Trees!

We will look at two “greedy algorithms” to find an MST of a Question: (True or False) Adding an edge to a spanning tree
weighted graph: Kruskal’s and Prim’s algorithms of a graph G always creates a cycle.

A greedy algorithm makes choices in sequence such that each

Justify your answer.
individual choice is best according to some limited “short-term”
criterion that is not too expensive to evaluate.

1! 6!
a 3! b c
5! 3!
1! 6! 6! 2!
a d e
3! b 5! 3!
c
5! 2! 8!
6! 2! f g h
d e 8! 4! 7!
5! 2!
f g h
4! 7!

Properties of MSTs! Kruskal’s MST Algorithm!

Idea:
MST property: Let G = (V, E) and let T be any spanning tree of - use a greedy strategy
G. Suppose that for every edge (u,v) of G that is not in T, if (u,v)
- consider edges in increasing order of weight (sort edges)
is added to T it creates a cycle such that (u,v) is a maximum
- add edge to spanning forest F if it doesn’t create a cycle.
weight edge on that cycle. Then T has the MST property.

If there are 2 spanning trees T1 and T2 on G that both have the Algorithm MST-Kruskal (G)
MST property, then T1 and T2 have same total weight. R = E // R is initially set of all edges
F = " // F is set of edges in a spanning tree of a subgraph of G
0. sort all edges of R in increasing order
1. while (R is not empty)
2. do remove the lightest-weight edge, (v,w), from R
3. if (v,w) does not make a cycle in F
4. then add (v,w) to F
5. return F
 Disjoint Sets (Ch. 21)! Data Structures for Disjoint Sets!
A disjoint-set data structure Comment 1: The Make-Set operation is only used during the
o! maintains a collection of disjoint subsets C = s1,s2,
initialization of a particular algorithm.
…,sm, where each si is identified by a representative
element (set id).
o! set of underlying elements make up the sets Comment 2: We assume there is an array of pointers to each
U = {1,2,…,n} x # U (so we never have to search for a particular
Operations on C: element, just which id the set has). Thus the problems
•! Make-Set(x): x # U, creates singleton set {x} we’re trying to solve are how to join the sets (Union)
•! Union(x,y): x, y # U and are id’s of their resp. sets,
sx and sy; replaces sets sx and sy with a set that is and how to find the id of the set containing a particular
sx $ sy and returns the id of the new set. element (Find-Set) efficiently.
•! Find-Set(x): x # U, returns the id of the set
containing x.

Rooted Tree Representation of Sets! Weighted Union Implementation"

Idea: Organize elements of each set as a tree with id = element for Trees!
at the root, and a pointer from every child to its parent (assume
Idea: Add weight field to each node holding the number of
we have an array of pointers to each element in the tree).
nodes in subtree rooted at this node (only care about weight
y!
Make-Set(x): O(1) time field of roots, even though other nodes maintain value too).
When doing a Union, make smaller tree subtree of larger tree.
Find-Set(x):
- start at x (using pointer provided) Make-Set(x): O(1)
x! and follow pointers up to the root.
-! return id of root Find-Set(x):
- ??? See next slide y!
x!
w-c running time is O(n), n = |U| x! $! =!
y!
Union(x, y):
x! $!
y! y! Union(x, y): - x and y are ids (roots of trees).
=!
- x and y are ids (roots of trees). - make node (x or y) with smaller
x!
-! make x a child of y and return y weight the child of the other
- O(1) time
running time is O(1)

Weighted Union! Path Compression Implementation!

Theorem: Any k-node tree created by k-1 weighted Unions has height
O(lg k) (assume we start with Make-Set on k singleton sets). Idea: extend the idea of weighted union (i.e., unions still
Want to show that trees stay “short”. weighted), but on a Find-Set(x) operation, make every node on
the path from x to the root (the node with the set id) a child of
Proof: By induction on k, the number of nodes. the root.
Basis: k = 1, height = 0 = lg 1 <true>
Inductive Hypothesis: Assume true for all i < k.
Inductive Step: Show true for k. Suppose the last operation performed
was union(x,y) and that m = wt(x) ! wt(y) = k – m, so that m ! k/2.
x!
a!
a!
y! b!
x! y! x!
x! b! c! d!
=! h! c!
m! hx! $! k - m! hy! k - m!

nodes! nodes!
m!
nodes!
nodes! d!
So Find-Set(x) still has worst-case time of O(lg n), but subsequent
Show h = max(hx + 1, hy) % lg k Find-Sets for nodes that used to be ancestors of x (or subsequent
•! hx + 1 % lg m + 1 % lg(k/2) + 1 = lg k – 1 + 1 = lg k finds for x itself) will now be very fast -- O(1).
•! hy % lg(k – m) % lg k (for positive m)
 Path Compression Analysis! Kruskal’s MST Algorithm!
Idea:
- Sorts edges, then add the minimum-weight edge (u,v) to the
Analysis in book (section 21.4). They prove that the running time
MST if u and v aren’t in same subgraph. Uses union-
for m disjoint-set operations on n elements is
find operations to determine when nodes are in same subgraph

O(mlg*n) MST-Kruskal (G)

1. T = "
2. for each v # V
You knew that lg* would turn up somewhere, right? do make-set(v)
3. sort edges in E by increasing (non-decreasing) weight
4. for each (u,v) # E
do if find-set(u) " find-set(v)
/** if (u,v) doesn’t create a cycle **/
T = T $ {(u,v)} /** add edge to MST **/
union(find-set(u), find-set(v))
5. return t

Kruskal’s MST Algorithm! MST-Kruskal (G)

1.! T = "
2.! for each v # V
Running Time makeset(v)
* initialization (lines 1-3) – 3. sort edges in E by increasing weight
O(1) + O(V) + O(E lg E) = O(V + E lg E) 4. for each (u,v) # E
if find (u) " find(v) /** doesn’t create a cycle **/
* E iterations of for-loop
T = T $ {(u,v)}/** add edge to MST **/
- 2E finds – O(E lg*E) time
union(find(u), find(v))
- O(V) unions – O(V) time (at most V – 1 unions)
5. return t
* total: O(V + E lg E) = O(E lg V) time
- note lg E = O(lgV) since E = O (V2), so lgE = O(2 lg V) 2! 6!
a 3! b c
5! 3!
6! 2!
d e 8!
1!
f 5! g h
4! 7!
Number the edges in the above graph in a possible order
they are added to the MST by Kruskal’s algorithm.

Correctness of Kruskal’s Algorithm! Correctness of Kruskal’s Algorithm

Theorem: Kruskal’s algorithm produces an MST.
(cont.)!
•! let e* be edge with minimum weight on c that is not in e1, e2, …, ei-1
Proof: Clearly, the algorithm produces a spanning tree. We need
Then wt(ei) < wt(e*), otherwise algorithm would have picked e*
to argue that it is an MST.
next in sorted order when it picked ei.
Suppose, in contradiction, the algorithm does not produce an MST.
Suppose that the algorithm adds edges to the tree in order e1, e2, …, en-1. Claim: T’ = T – {e*} $ {ei} is an MST
Let i be the value such that e1, e2, …, ei-1 is a subset of some MST T, •! T’ is a spanning tree since it contains all nodes and has no cycles.
but e1, e2, …, ei-1, ei is not a subset of any MST. •! wt(T’) < wt(T), so T is not an MST (contradiction)
Consider T $ {ei}
•!T $ {ei} must have a cycle c involving ei This contradiction means our original assumption must be wrong
•!In the cycle c there is at least one edge that is not in e1, e2, …, ei-1 (since and therefore the algorithm always finds an MST.
the algorithm doesn’t pick an edge that creates a cycle and it picked ei).
 Prim’s MST Algorithm! Prim’s MST Algorithm!
Algorithm starts by selecting an arbitrary starting vertex, and Idea: Use a priority queue Q
then “branching out” from the part of the tree constructed so – associate with each node v MST-Prim (G, r)!
two fields: (edges are included 1.! insert each v #V into Q with key[v] & ',"
far by choosing a new vertex and edge at each iteration. #[v] & "!
in the MST T when their end- 2.! key[r] & 0 // root of MST!
points are extracted from Q) 3.! while Q # "!
Idea:
- key[v]: if v isn’t in T, then 4.! u & extract-min(Q)!
- always maintain one connected subgraph (different from Kruskal’s)
holds the min wt of all the 5.! for each neighbor v of u!
- at each iteration, choose the lowest weight edge that goes out from
edges from v to a node in T. 6.! if v # Q and wt(u,v) < key[v] then!
the current tree (a greedy strategy). 7.! # [v] & u!
- ![v]: if v isn’t in T, holds
8.! key[v] & wt(u,v)!
the name of the node u in T
9.! end while!
such that wt(u,v) is
v’s best edge to node in T.

iteration 2: Q = Q – {b}!
Prim’s Improved MST Algorithm! Q! xa xb c d e f g h! MST-Prim (G, r)!
MST-Prim (G, r)! 1.! insert each v #V into Q with key[v] & ',"
1.! insert each v #V into Q with key[v] & ',"
$! " a "! b a a " " "! #[v] & "!
Start at node a:! #[v] & "! key! 0 1 %! 6 6 3 ' ' '! 2.! key[r] & 0 // root of MST!
2.! key[r] & 0 // root of MST! 3.! while Q # "!
Q! a b c d e f g h! 3.! while Q # "! T = {(b,a)}! 4.! u & extract-min(Q)!
$! " " " " " " " "! 4.! u & extract-min(Q)! iteration 3: Q = Q – {e}! 5.! for each neighbor v of u!
5.! for each neighbor v of u!
key! 0 ' ' ' ' ' ' '! 6.! if v # Q and wt(u,v) < key[v] then!
Q! xa xb c d xe f g h! 6.! if v # Q and wt(u,v) < key[v] then!
7.! # [v] & u!
7.! # [v] & u! $! " a e e a " e e! 8.! key[v] & wt(u,v)!
iteration 1: Q = Q – {a}! 8.! key[v] & wt(u,v)! 9.! end while!
9.! end while!
key! 0 1 3 2 3 ' 2 8!
Q! ax b c d e f g h!
$! " a " a a " " "! T = {(b,a), (e,a)}!
key! 0 1 ' 6 3 ' ' '! 1! 6! iteration 4: Q = Q – {d}! 1! 6!
a 3! b c a 3! b 5! c
5! 3! Q! ax xb c dx xe f g h! 3!
6! 6!
T = {"}! d 2! e $! " a e e a " e e! d 2! e 8!
2! 8! 2!
f 5! g h
key! 0 1 3 2 3 ' 2 8! f 5! g h
4! 4! 7!
7! T = {(b,a), (e,a), (d,e)}!

iteration 5: Q = Q – {g}!
Q! xa xb c dx xe f gx h! MST-Prim (G, r)! MST-Prim (G, r)!
1.! insert each v #V into Q with key[v] & '," 1.! insert each v #V into Q with key[v] & ',"
$! " a e e a g e g! #[v] & "! #[v] & "!
key! 0 1 3 2 3 4 2 7! 2.! key[r] & 0 // root of MST! 2.! key[r] & 0 // root of MST!
3.! while Q # "! 3.! while Q # "!
T = {(b,a), (e,a), (d,e), (g,e)}! 4.! u & extract-min(Q)! iteration 7: Q = Q – {h} = "! 4.! u & extract-min(Q)!
iteration 6: Q = Q – {c}! 5.! for each neighbor v of u! 5.! for each neighbor v of u!
6.! if v # Q and wt(u,v) < key[v] then!
Q! xa xb xc dx xe xf gx h! 6.! if v # Q and wt(u,v) < key[v] then!
Q! xa xb xc dx xe f gx h! 7.! # [v] & u! $! " a e e a g e g! 7.! # [v] & u!
8.! key[v] & wt(u,v)! 8.! key[v] & wt(u,v)!
$! " a e e a g e g! 9.! end while!
key! 0 1 3 2 3 4 2 7! 9.! end while!
key! 0 1 3 2 3 4 2 7!
T = {(b,a), (e,a), (d,e), (g,e), (c,e), (f,g), (h,g)}!
T = {(b,a), (e,a), (d,e), (g,e), (c,e)}!
iteration 6: Q = Q – {f}! 1! 6!
a 3! b 5! c DONE!
a
1! 6!
Q! ax xb xc dx xe xf gx h! 3! 3! b 5! 3!
c
6!
$! " a e e a g e g! d 2! e 8! 6!
d 2! e
2! 2! 8!
key! 0 1 3 2 3 4 2 7! f 5! g h f 5! g h
4! 7! 4! 7!
T = {(b,a), (e,a), (d,e), (g,e), (c,e), (f,g)}!
 Running Time of Prim’s MST Algorithm! Correctness of Prim’s Algorithm!
•! Assume Q is implemented with a binary min-heap (heapsort)
•! How can we tell if v # Q without searching heap? (line 6) Let Ti be the tree after the ith iteration of the while loop
- keep an array for nodes with boolean flag indicating if in heap
Lemma: For all i, Ti is a subtree of some MST of G.
Running time: Proof: by induction on i
•! initialize Q: O(V) time
•! decrease v0’s key = O(lg V) time Basis: when i = 0, T0 = ", ok because trivial MST subtree
•! while loop... IHOP: Assume Ti is a subtree of some MST M
in each of V iterations of while loop:
Induction Step: Now show that Ti +1 is a subtree of some MST (possibly
extract min = O(lg V) time
different from M)
update T = O(1) time ==> O(V lg V) total
over all iterations (combined):
Let (u,v) be the edge added in iteration i + 1
check neighbors of u (line 9): O(E) executions
when condition test and update ( = O(1) time
when decreasing v’s key = O(lg V) time = O(E lg V)
So, the grand total is:
O(V lg V + E lg V) = O(E lg V) ( asymptotically the
same as Kruskal’s)

Correctness of Prim’s Algorithm!

case 1: (u, v) is an edge of M.
Then clearly Ti+1 is a subtree of M (ok)

case 2: (u, v) is not an edge of M

We know there is a path p in M from u to v (because M is a ST)

Let (x, y) be the first edge in p with x in Ti and y not in Ti. We know
this edge exists because the algorithm will not add edge (u,v) to a cycle.
How do we know this?

M’ = M – {(x, y)} $ {(u, v)} is another spanning tree.

Now we note that

wt(M’) = wt(M) – wt(x, y) + wt(u, v) % wt(M)
since (u, v) is the minimum weight outgoing edge from Ti
Therefore, M’ is also a MST of G and Ti+1 is a subtree of M'.