Algorithms R OBERT S EDGEWICK | K EVIN W AYNE

4.3 M INIMUM S PANNING T REES 4.3 M INIMUM S PANNING T REES

‣ introduction ‣ introduction
‣ greedy algorithm ‣ greedy algorithm
‣ edge-weighted graph API ‣ edge-weighted graph API
Algorithms Algorithms
‣ Kruskal's algorithm ‣ Kruskal's algorithm
F O U R T H E D I T I O N

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Prim's algorithm R OBERT S EDGEWICK | K EVIN W AYNE
‣ Prim's algorithm
http://algs4.cs.princeton.edu ‣ context http://algs4.cs.princeton.edu ‣ context

Minimum spanning tree Minimum spanning tree

Def. A spanning tree of G is a subgraph T that is: Def. A spanning tree of G is a subgraph T that is:
Connected. Connected.
Acyclic. Acyclic.
Includes all of the vertices. Includes all of the vertices.

graph G not connected

3 4
Minimum spanning tree Minimum spanning tree

Def. A spanning tree of G is a subgraph T that is: Def. A spanning tree of G is a subgraph T that is:
Connected. Connected.
Acyclic. Acyclic.
Includes all of the vertices. Includes all of the vertices.

not acyclic not spanning

5 6

Minimum spanning tree Minimum spanning tree

Given. Undirected graph G with positive edge weights (connected). Given. Undirected graph G with positive edge weights (connected).
Goal. Find a min weight spanning tree. Goal. Find a min weight spanning tree.

24 24
4 4

6 23 9 6 23 9
18 18
5 5
11 11
16 16
8 8
7 7
10 14 10 14

21 21

edge-weighted graph G minimum spanning tree T

(cost = 50 = 4 + 6 + 8 + 5 + 11 + 9 + 7)

Brute force. Try all spanning trees?

7 8
Network design Models of nature
MST of bicycle routes in North Seattle MST of random graph

http://www.flickr.com/photos/ewedistrict/21980840 http://algo.inria.fr/broutin/gallery.html
9 10

Medical image processing Medical image processing

MST dithering

MST describes arrangement of nuclei in the epithelium for cancer research

http://www.bccrc.ca/ci/ta01_archlevel.html

http://www.flickr.com/photos/quasimondo/2695389651
11 12
Applications

MST is fundamental problem with diverse applications.

Dithering.
Cluster analysis.
Max bottleneck paths.
Real-time face verification. 4.3 M INIMUM S PANNING T REES
LDPC codes for error correction.
Image registration with Renyi entropy. ‣ introduction
Find road networks in satellite and aerial imagery.
‣ greedy algorithm
Reducing data storage in sequencing amino acids in a protein.
‣ edge-weighted graph API
Model locality of particle interactions in turbulent fluid flows. Algorithms
Autoconfig protocol for Ethernet bridging to avoid cycles in a network. ‣ Kruskal's algorithm
Approximation algorithms for NP-hard problems (e.g., TSP, Steiner tree).
R OBERT S EDGEWICK | K EVIN W AYNE
‣ Prim's algorithm
Network design (communication, electrical, hydraulic, computer, road). http://algs4.cs.princeton.edu ‣ context
http://www.ics.uci.edu/~eppstein/gina/mst.html

13

Simplifying assumptions Cut property

Graph is connected. Def. A cut in a graph is a partition of its vertices into two (nonempty) sets.
Edge weights are distinct. Def. A crossing edge connects a vertex in one set with a vertex in the other.

Cut property. Given any cut, the crossing edge of min weight is in the MST.
Consequence. MST exists and is unique.

crossing edge separating

gray and white vertices

5
7
2 10 no two edge
6 4
weights are equal
13

9 1
3
12 8

14 16

20

minimum-weight crossing edge

must be in the MST
15 16
Cut property: correctness proof Greedy MST algorithm demo

Def. A cut in a graph is a partition of its vertices into two (nonempty) sets. Start with all edges colored gray.
Def. A crossing edge connects a vertex in one set with a vertex in the other. Find cut with no black crossing edges; color its min-weight edge black.
Repeat until V - 1 edges are colored black.
Cut property. Given any cut, the crossing edge of min weight is in the MST.
0-7 0.16
Pf. Suppose min-weight crossing edge e is not in the MST.
2-3 0.17
Adding e to the MST creates a cycle. 1 1-7 0.19
3
Some other edge f in cycle must be a crossing edge. 0-2 0.26
5
Removing f and adding e is also a spanning tree. 5-7 0.28
7 1-3 0.29
Since weight of e is less than the weight of f, 2
1-5 0.32
that spanning tree is lower weight. f 0 2-7 0.34
Contradiction. ▪ 4-5 0.35

4 6 1-2 0.36
e 4-7 0.37
0-4 0.38
an edge-weighted graph 6-2 0.40
the MST does
not contain e 3-6 0.52
adding e to MST 6-0 0.58
creates a cycle 6-4 0.93
17 18

Greedy MST algorithm demo Greedy MST algorithm: correctness proof

Start with all edges colored gray. Proposition. The greedy algorithm computes the MST.
Find cut with no black crossing edges; color its min-weight edge black.
Repeat until V - 1 edges are colored black. Pf.
Any edge colored black is in the MST (via cut property).
Fewer than V - 1 black edges ⇒ cut with no black crossing edges.
1 (consider cut whose vertices are any one connected component)
3
5

7
2

4 6
fewer than V-1 edges colored black aacut
cutwith
withnonoblack
blackcrossing
crossingedges
edges fewer
fewerthan
than V-1
V-1 edges
edgescolored
coloredblack
black a cut with no black cro

MST edges
0-2 5-7 6-2 0-7 2-3 1-7 4-5

19 20
 2 6 0.17 1 5 0.02
0 4 -0.99
weights can be 0 or negative 1 6 0
4 6 0.62 0 2 0.22
Greedy MST algorithm: efficient implementations Removing two simplifying assumptions5 6 0.88
1 5 0.02
1
1
2 0.50
3 0.97
0 4 -0.99 2 6 0.17
1 6 0
MST may not be unique
Proposition. The greedy algorithm computes the MST. Q. What if edge weights0are not all distinct?
2 0.22
when weights have equal values
1 2 0.50
A. Greedy MST algorithm 1 3still correct if equal weights are present!
0.97 1 2 1.00
2 6 0.17 1 3 0.50
Efficient implementations. Choose cut? Find min-weight edge? (our correctness proof fails, but that can be fixed) 2 4 1.00
MST may not be unique
3 4 0.50
Ex 1. Kruskal's algorithm. [stay tuned] when weights have equal values

1 2 1.00 1 2 1.00
Ex 2. Prim's algorithm. [stay tuned] 1 3 0.50 1 3 0.50
2 4 1.00 2 4 1.00
Ex 3. Borüvka's algorithm. 3 4 0.50 3 4 0.50

1 2 1.00
1 3 0.50 Various MST anomalies
2 4 1.00
Q. What if graph is not 3 4 0.50
connected?
A. Compute minimum spanning forest = MST of each component.
Various MST anomalies
no MST if graph is not connected
4 5 0.61
4 6 0.62
5 6 0.88
1 5 0.11
2 3 0.35
0 3 0.6
1 6 0.10
0 2 0.22
can independently compute
MSTs of components

21 22
weights need not be
proportional to distance

4 6 0.62
5 6 0.88
Greed is good 1 5 0.02
0 4 0.64
1 6 0.90
0 2 0.22
1 2 0.50
1 3 0.97
2 6 0.17

weights can be 0 or negative

4 6 0.62
5 6 0.88

4.3 M INIMUM S PANNING T REES

1
0
5 0.02
4 -0.99
1 6 0
0 2 0.22
‣ introduction 1
1
2 0.50
3 0.97

‣ greedy algorithm 2 6 0.17

MST may not be unique

‣ edge-weighted graph API
when weights have equal values
Algorithms 1 2 1.00
‣ Kruskal's1 algorithm
3 0.50
2 4 1.00

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Prim's algorithm
3 4 0.50

1 2 1.00
http://algs4.cs.princeton.edu ‣ context 1 3 0.50
2 4 1.00
3 4 0.50
Gordon Gecko (Michael Douglas) address to Teldar Paper Stockholders in Wall Street (1986)

Various MST anomalies

23
Weighted edge API Weighted edge: Java implementation

Edge abstraction needed for weighted edges.

public class Edge implements Comparable<Edge>
{
private final int v, w;
public class Edge implements Comparable<Edge> private final double weight;

Edge(int v, int w, double weight) create a weighted edge v-w public Edge(int v, int w, double weight)
{ constructor
this.v = v;
int either() either endpoint
this.w = w;
this.weight = weight;
int other(int v) the endpoint that's not v }

int compareTo(Edge that) compare this edge to that edge public int either()
{ return v; } either endpoint

double weight() the weight

public int other(int vertex)
{
String toString() string representation if (vertex == v) return w; other endpoint
else return v;
}

weight public int compareTo(Edge that)

v w {
if (this.weight < that.weight) return -1;
compare edges by weight
else if (this.weight > that.weight) return +1;
else return 0;
}
Idiom for processing an edge e: int v = e.either(), w = e.other(v); }
25 26

Edge-weighted graph API Edge-weighted graph: adjacency-lists representation

Maintain vertex-indexed array of Edge lists.

public class EdgeWeightedGraph

EdgeWeightedGraph(int V) create an empty graph with V vertices

EdgeWeightedGraph(In in) create a graph from input stream

tinyEWG.txt 6 0 .58 0 2 .26 0 4 .38 0 7 .16 Bag

void addEdge(Edge e) add weighted edge e to this graph V objects
8 E
16 1 3 .29 1 2 .36 1 7 .19 1 5 .32
Iterable<Edge> adj(int v) edges incident to v adj[]
4 5 0.35
4 7 0.37 0
6 2 .40 2 7 .34 1 2 .36 0 2 .26 2 3 .17
5 7 0.28 1
Iterable<Edge> edges() all edges in this graph
0 7 0.16 2
1 5 0.32 3 6 .52 1 3 .29 2 3 .17
0 4 0.38 3
int V() number of vertices
2 3 0.17 4
1 7 0.19 6 4 .93 0 4 .38 4 7 .37 4 5 .35
5
int E() number of edges 0 2 0.26
1 2 0.36 6 references to the
1 5 .32 5 7 .28 4 5 .35 same Edge object
1 3 0.29 7
String toString() string representation 2 7 0.34
6 2 0.40 6 4 .93 6 0 .58 3 6 .52 6 2 .40
3 6 0.52
6 0 0.58 2 7 .34 1 7 .19 0 7 .16 5 7 .28 5 7 .28
6 4 0.93

Edge-weighted graph representation

Conventions. Allow self-loops and parallel edges.

27 28
Edge-weighted graph: adjacency-lists implementation Minimum spanning tree API

Q. How to represent the MST?

public class EdgeWeightedGraph
{ public class MST
private final int V; same as Graph, but adjacency
private final Bag<Edge>[] adj; lists of Edges instead of integers MST(EdgeWeightedGraph G) constructor

public EdgeWeightedGraph(int V) Iterable<Edge> edges() edges in MST

{
this.V = V; constructor
double weight() weight of MST
adj = (Bag<Edge>[]) new Bag[V];
for (int v = 0; v < V; v++)
adj[v] = new Bag<Edge>(); tinyEWG.txt
} V 8
E
16
% java MST tinyEWG.txt
4 5 0.35 MST edge
public void addEdge(Edge e) 4 7 0.37 (black) 0-7 0.16
{ 5 7 0.28 1-7 0.19
0 7 0.16
int v = e.either(), w = e.other(v); add edge to both 1 5 0.32
0-2 0.26
adj[v].add(e); adjacency lists 0 4 0.38 2-3 0.17
adj[w].add(e); 2 3 0.17 5-7 0.28
1 7 0.19
} 0 2 0.26 4-5 0.35
1 2 0.36 6-2 0.40
1 3 0.29
public Iterable<Edge> adj(int v) 1.81
2 7 0.34
{ return adj[v]; } 6 2 0.40 non-MST edge
(gray)
} 3 6 0.52
6 0 0.58
6 4 0.93
29 30
An edge-weighted graph and its MST

Minimum spanning tree API

Q. How to represent the MST?

public class MST

MST(EdgeWeightedGraph G) constructor

Iterable<Edge> edges() edges in MST 4.3 M INIMUM S PANNING T REES

double weight() weight of MST
‣ introduction
‣ greedy algorithm
public static void main(String[] args) % java MST tinyEWG.txt ‣ edge-weighted graph API
{ 0-7 0.16 Algorithms
In in = new In(args[0]); 1-7 0.19 ‣ Kruskal's algorithm
EdgeWeightedGraph G = new EdgeWeightedGraph(in); 0-2 0.26
MST mst = new MST(G); 2-3 0.17
R OBERT S EDGEWICK | K EVIN W AYNE
‣ Prim's algorithm
for (Edge e : mst.edges()) 5-7 0.28
StdOut.println(e); 4-5 0.35 http://algs4.cs.princeton.edu ‣ context
StdOut.printf("%.2f\n", mst.weight()); 6-2 0.40
} 1.81

31
Kruskal's algorithm demo Kruskal's algorithm demo

Consider edges in ascending order of weight. Consider edges in ascending order of weight.
Add next edge to tree T unless doing so would create a cycle. graph edges
Add next edge to tree T unless doing so would create a cycle.
sorted by weight

0-7 0.16 0-7 0.16

2-3 0.17 2-3 0.17
1 1-7 0.19 1 1-7 0.19
3 3
0-2 0.26 0-2 0.26
5 5
5-7 0.28 5-7 0.28
7 1-3 0.29 7 1-3 0.29
2 2
1-5 0.32 1-5 0.32
0 2-7 0.34 0 2-7 0.34
4-5 0.35 4-5 0.35

4 6 1-2 0.36 4 6 1-2 0.36

4-7 0.37 4-7 0.37
0-4 0.38 0-4 0.38
an edge-weighted graph 6-2 0.40 a minimum spanning tree 6-2 0.40
3-6 0.52 3-6 0.52
6-0 0.58 6-0 0.58
6-4 0.93 6-4 0.93
33 34

Kruskal's algorithm: visualization Kruskal's algorithm: correctness proof

Proposition. [Kruskal 1956] Kruskal's algorithm computes the MST.

Pf. Kruskal's algorithm is a special case of the greedy MST algorithm.

Suppose Kruskal's algorithm colors the edge e = v–w black.
Cut = set of vertices connected to v in tree T.
No crossing edge is black.
No crossing edge has lower weight. Why?

add edge to tree adding edge to tree

would create a cycle

35 36
Kruskal's algorithm: implementation challenge Kruskal's algorithm: implementation challenge

Challenge. Would adding edge v–w to tree T create a cycle? If not, add it. Challenge. Would adding edge v–w to tree T create a cycle? If not, add it.

How difficult? Efficient solution. Use the union-find data structure.

E+V Maintain a set for each connected component in T.
run DFS from v, check if w is reachable
V (T has at most V – 1 edges) If v and w are in same set, then adding v–w would create a cycle.
log V To add v–w to T, merge sets containing v and w.
log* V use the union-find data structure !

add edge to tree adding edge to tree w

would create a cycle

v w v

Case 1: adding v–w creates a cycle Case 2: add v–w to T and merge sets containing v and w

37 38

Kruskal's algorithm: Java implementation Kruskal's algorithm: running time

Proposition. Kruskal's algorithm computes MST in time proportional to

public class KruskalMST
E log E (in the worst case).
{
private Queue<Edge> mst = new Queue<Edge>();

public KruskalMST(EdgeWeightedGraph G) build priority queue Pf. operation frequency time per op
{ (or sort)
MinPQ<Edge> pq = new MinPQ<Edge>(G.edges()); build pq 1 E

UF uf = new UF(G.V()); delete-min E log E

while (!pq.isEmpty() && mst.size() < G.V()-1)
{ union V log* V †
Edge e = pq.delMin(); greedily add edges to MST
int v = e.either(), w = e.other(v);
connected E log* V †
if (!uf.connected(v, w)) edge v–w does not create cycle
{
uf.union(v, w); merge sets † amortized bound using weighted quick union with path compression
mst.enqueue(e); add edge to MST
}
}
}
recall: log* V ≤ 5 in this universe
public Iterable<Edge> edges()
{ return mst; }
}
Remark. If edges are already sorted, order of growth is E log* V.
39 40
 Prim's algorithm demo

Start with vertex 0 and greedily grow tree T.

Add to T the min weight edge with exactly one endpoint in T.
Repeat until V - 1 edges.

0-7 0.16
4.3 M INIMUM S PANNING T REES 2-3 0.17
1 1-7 0.19
3
‣ introduction 5
0-2
5-7
0.26
0.28
‣ greedy algorithm 7
1-3 0.29
2 1-5 0.32
‣ edge-weighted graph API
Algorithms 0
2-7
4-5
0.34
0.35
‣ Kruskal's algorithm 1-2 0.36
4-7 0.37
R OBERT S EDGEWICK | K EVIN W AYNE
‣ Prim's algorithm 4 6
0-4 0.38

http://algs4.cs.princeton.edu ‣ context 6-2

3-6
0.40
0.52
an edge-weighted graph
6-0 0.58
6-4 0.93

42

Prim's algorithm demo Prim’s algorithm: visualization

Start with vertex 0 and greedily grow tree T.

Add to T the min weight edge with exactly one endpoint in T.
Repeat until V - 1 edges.

1
3
5

7
2

4 6

MST edges
0-7 1-7 0-2 2-3 5-7 4-5 6-2

43 44
Prim's algorithm: proof of correctness Prim's algorithm: implementation challenge

Proposition. [Jarník 1930, Dijkstra 1957, Prim 1959] Challenge. Find the min weight edge with exactly one endpoint in T.
Prim's algorithm computes the MST.
How difficult?
Pf. Prim's algorithm is a special case of the greedy MST algorithm. E try all edges

Suppose edge e = min weight edge connecting a vertex on the tree V

to a vertex not on the tree. log E use a priority queue!

Cut = set of vertices connected on tree. log* E

No crossing edge is black. l
No crossing edge has lower weight.
1-7is min weight edge with
edge e = 7-5 added to tree exactly one endpoint in T
priority queue
of crossing edges
1-7 0.19
0-2 0.26
5-7 0.28
2-7 0.34
4-7 0.37
0-4 0.38
6-0 0.58

45 46

Prim's algorithm: lazy implementation Prim's algorithm (lazy) demo

Challenge. Find the min weight edge with exactly one endpoint in T. Start with vertex 0 and greedily grow tree T.
Add to T the min weight edge with exactly one endpoint in T.
Lazy solution. Maintain a PQ of edges with (at least) one endpoint in T. Repeat until V - 1 edges.
Key = edge; priority = weight of edge.
0-7 0.16
Delete-min to determine next edge e = v–w to add to T.
2-3 0.17
Disregard if both endpoints v and w are marked (both in T). 1 1-7 0.19
3
Otherwise, let w be the unmarked vertex (not in T ): 0-2 0.26
5
– add to PQ any edge incident to w (assuming other endpoint not in T) 5-7 0.28
7 1-3 0.29
– add e to T and mark w 2
1-5 0.32
1-7 is min weight edge with 0 2-7 0.34
exactly one endpoint in T
priority queue 4-5 0.35
of crossing edges
4 6 1-2 0.36
1-7 0.19 4-7 0.37
0-2 0.26
0-4 0.38
5-7 0.28
2-7 0.34 an edge-weighted graph 6-2 0.40
4-7 0.37 3-6 0.52
0-4 0.38
6-0 0.58
6-0 0.58
6-4 0.93
47 48
Prim's algorithm (lazy) demo Prim's algorithm: lazy implementation

Start with vertex 0 and greedily grow tree T.

public class LazyPrimMST
Add to T the min weight edge with exactly one endpoint in T. {
private boolean[] marked; // MST vertices
Repeat until V - 1 edges.
private Queue<Edge> mst; // MST edges
private MinPQ<Edge> pq; // PQ of edges

public LazyPrimMST(WeightedGraph G)
1 {
3 pq = new MinPQ<Edge>();
5 mst = new Queue<Edge>();
marked = new boolean[G.V()];
7 visit(G, 0); assume G is connected
2

while (!pq.isEmpty() && mst.size() < G.V() - 1)

0
{ repeatedly delete the
Edge e = pq.delMin(); min weight edge e = v–w from PQ

4 6 int v = e.either(), w = e.other(v); ignore if both endpoints in T

if (marked[v] && marked[w]) continue;
add edge e to tree
mst.enqueue(e);
if (!marked[v]) visit(G, v); add v or w to tree
if (!marked[w]) visit(G, w);
MST edges }
0-7 1-7 0-2 2-3 5-7 4-5 6-2 }
}
49 50

Prim's algorithm: lazy implementation Lazy Prim's algorithm: running time

Proposition. Lazy Prim's algorithm computes the MST in time proportional

private void visit(WeightedGraph G, int v)

to E log E and extra space proportional to E (in the worst case).
{
marked[v] = true; add v to T
for (Edge e : G.adj(v)) Pf. operation frequency binary heap
if (!marked[e.other(v)]) for each edge e = v–w, add to
pq.insert(e); PQ if w not already in T
delete min E log E
}

public Iterable<Edge> mst() insert E log E

{ return mst; }

51 52
Prim's algorithm: eager implementation Prim's algorithm (eager) demo

Challenge. Find min weight edge with exactly one endpoint in T. Start with vertex 0 and greedily grow tree T.
Add to T the min weight edge with exactly one endpoint in T.
pq has at most one entry per vertex
Repeat until V - 1 edges.
Eager solution. Maintain a PQ of vertices connected by an edge to T,
0-7 0.16
where priority of vertex v = weight of shortest edge connecting v to T.
2-3 0.17
Delete min vertex v and add its associated edge e = v–w to T. 1 1-7 0.19
3
Update PQ by considering all edges e = v–x incident to v 0-2 0.26
5
– ignore if x is already in T 5-7 0.28
7 1-3 0.29
– add x to PQ if not already on it 2
1-5 0.32
– decrease priority of x if v–x becomes shortest edge connecting x to T 0 2-7 0.34
4-5 0.35
0 4 6 1-2 0.36
1 1-7 0.19
4-7 0.37
2 0-2 0.26 red: on PQ
3 1-3 0.29 0-4 0.38
4 0-4 0.38
5 5-7 0.28 an edge-weighted graph 6-2 0.40
6 6-0 0.58 3-6 0.52
7 0-7 0.16
6-0 0.58
6-4 0.93
black: on MST
53 54

Prim's algorithm (eager) demo Indexed priority queue

Start with vertex 0 and greedily grow tree T. Associate an index between 0 and N - 1 with each key in a priority queue.
Add to T the min weight edge with exactly one endpoint in T. Supports insert and delete-the-minimum.
Repeat until V - 1 edges. Supports decrease-key given the index of the key.

v edgeTo[] distTo[]
0 - -
public class IndexMinPQ<Key extends Comparable<Key>>
1
3 7 0–7 0.16 create indexed priority queue
IndexMinPQ(int N)
5 1 1–7 0.19 with indices 0, 1, …, N – 1
2 0–2 0.26
7 void insert(int i, Key key) associate key with index i
2 3 2–3 0.17
5 5–7 0.28 void decreaseKey(int i, Key key) decrease the key associated with index i
0 4 4–5 0.35
boolean contains(int i) is i an index on the priority queue?
6 6–2 0.40
4 6 remove a minimal key and return its
int delMin()
associated index

boolean isEmpty() is the priority queue empty?

MST edges int size() number of keys in the priority queue

0-7 1-7 0-2 2-3 5-7 4-5 6-2

55 56
Indexed priority queue implementation Prim's algorithm: which priority queue?

Binary heap implementation. [see Section 2.4 of textbook] Depends on PQ implementation: V insert, V delete-min, E decrease-key.
Start with same code as MinPQ.
Maintain parallel arrays keys[], pq[], and qp[] so that: PQ implementation insert delete-min decrease-key total
– keys[i] is the priority of i
– pq[i] is the index of the key in heap position i unordered array 1 V 1 V2

– qp[i] is the heap position of the key with index i

binary heap log V log V log V E log V
Use swim(qp[i]) to implement decreaseKey(i, key).
d-way heap logd V d logd V logd V E logE/V V
i 0 1 2 3 4 5 6 7 8
keys[i] A S O R T I N G - Fibonacci heap 1† log V † 1† E + V log V
pq[i] - 0 6 7 2 1 5 4 3
qp[i] 1 5 4 8 7 6 2 3 -
† amortized
1
A Bottom line.
2 N 3 G
Array implementation optimal for dense graphs.
Binary heap much faster for sparse graphs.
4 O 5 S 6 I 7 T
4-way heap worth the trouble in performance-critical situations.
8 R Fibonacci heap best in theory, but not worth implementing.
57 58

Does a linear-time MST algorithm exist?

deterministic compare-based MST algorithms

year worst case discovered by

1975 E log log V Yao

4.3 M INIMUM S PANNING T REES 1976 E log log V Cheriton-Tarjan

1984 E log* V, E + V log V Fredman-Tarjan

‣ introduction
1986 E log (log* V) Gabow-Galil-Spencer-Tarjan
‣ greedy algorithm
1997 E α(V) log α(V) Chazelle
‣ edge-weighted graph API
Algorithms 2000 E α(V) Chazelle
‣ Kruskal's algorithm
2002 optimal Pettie-Ramachandran

R OBERT S EDGEWICK | K EVIN W AYNE

‣ Prim's algorithm 20xx ???
E
http://algs4.cs.princeton.edu ‣ context

Remark. Linear-time randomized MST algorithm (Karger-Klein-Tarjan 1995).

60
Euclidean MST Scientific application: clustering

Given N points in the plane, find MST connecting them, where the distances k-clustering. Divide a set of objects classify into k coherent groups.
between point pairs are their Euclidean distances. Distance function. Numeric value specifying "closeness" of two objects.

Goal. Divide into clusters so that objects in different clusters are far apart.

outbreak of cholera deaths in London in 1850s (Nina Mishra)

Applications.
Routing in mobile ad hoc networks.
Document categorization for web search.
Brute force. Compute ~ N 2/ 2 distances and run Prim's algorithm. Similarity searching in medical image databases.
Ingenuity. Exploit geometry and do it in ~ c N log N. Skycat: cluster 109 sky objects into stars, quasars, galaxies.
61 62

Single-link clustering Single-link clustering algorithm

k-clustering. Divide a set of objects classify into k coherent groups. “Well-known” algorithm in science literature for single-link clustering:
Distance function. Numeric value specifying "closeness" of two objects. Form V clusters of one object each.
Find the closest pair of objects such that each object is in a different
Single link. Distance between two clusters equals the distance cluster, and merge the two clusters.
between the two closest objects (one in each cluster). Repeat until there are exactly k clusters.

Single-link clustering. Given an integer k, find a k-clustering that Observation. This is Kruskal's algorithm.
maximizes the distance between two closest clusters. (stopping when k connected components)

distance between two clusters distance between

two closest clusters

4-clustering
Alternate solution. Run Prim; then delete k – 1 max weight edges.
63 64
Dendrogram of cancers in human

Tumors in similar tissues cluster together.

gene 1

gene n

gene expressed
Reference: Botstein & Brown group
gene not expressed
65