Minimum Spanning Tree (MST)

MST Kruskal’s Algorithm

MST Prim’s Algorithm
Week-11, Lecture-01

Course Code: CSE221 Course Teacher: Tanzina Afroz Rimi

Course Title: Algorithms Designation: Lecturer
Program: B.Sc. in CSE Email: [email protected]
Problem: Laying Telephone Wire

Central office

2
Wiring: Naïve Approach

Central office

Expensive!

3
 Wiring: Better Approach

Central office

Minimize the total length of wire connecting the customers

4
 A Networking Problem

Problem: The vertices represent 8 regional data centers which

need to be connected with high-speed data lines. Feasibility
studies show that the links illustrated above are possible, and the
cost in millions of dollars is shown next to the link. Which links
should be constructed to enable full communication (with relays
allowed) and keep the total cost minimal.

5
 Minimum Spanning Trees
• Undirected, connected graph G =
(V,E)
• Weight function W: E → R

■ Spanning tree: tree that connects all the vertices

(above?)
■ Minimum spanning tree: tree that connects all
the vertices and minimizes

6
Minimum Spanning Tree (MST)
A minimum spanning tree is a subgraph of an
undirected weighted graph G, such that

• it is a tree (i.e., it is acyclic)

• it covers all the vertices V
– contains |V| - 1 edges
• the total cost associated with tree edges is the
minimum among all possible spanning trees
• not necessarily unique

7
How Can We Generate a MST?

9 9
b b
a 2 6 a 2 6
d d
4 5 4 5
5 4 5 4

5 e 5 e
c c

8
 Greedy Choice
We will show two ways to build a minimum spanning tree.
• A MST can be grown from the current spanning tree by adding the
nearest vertex and the edge connecting the nearest vertex to the
MST. (Prim's algorithm)

• A MST can be grown from a forest of spanning trees by adding the

smallest edge connecting two spanning trees. (Kruskal's algorithm)

9
 Notation
• Tree-vertices: in the tree constructed so far
• Non-tree vertices: rest of vertices

Prim’s Selection rule

• Select the minimum weight edge between a
tree-node and a non-tree node and add to the
tree

10
The Prim algorithm Main Idea
Select a vertex to be a tree-node

while (there are non-tree vertices) { a 4

6
if there is no edge connecting a tree node with a
non-tree node 5
b u
return “no spanning tree”
14 2
10
select an edge of minimum weight between a tree c v
node and a non-tree node
3
8 15
d
add the selected edge and its new vertex to the
tree f
}
return tree

11
 Prim’s Algorithm
• Vertex based algorithm
• Grows one tree T, one vertex at a time

12
 Prim Algorithm (2)
MST-Prim(G,w,r)
01 Q ← V[G] // Q – vertices out of T
02 for each u ∈ Q
03 key[u] ← ∞
04 key[r] ← 0 // r is the first tree node, let r=1
05 π[r] ← NIL
06 while Q ≠ ∅ do
07 u ← ExtractMin(Q) // making u part of T
08 for each v ∈ Adj[u] do
09 if v ∈ Q and w(u,v) < key[v] then
10 π[v] ← u
11 key[v] ← w(u,v)

13
 Prim Algorithm:Variables
• r:
• Grow the minimum spanning tree from the root vertex “r”.
• Q:
• is a priority queue, holding all vertices that are not in the tree now.
• key[v]:
• is the minimum weight of any edge connecting v to a vertex in the
tree.
• π [v]:
• names the parent of v in the tree.
• T[v] –
• Vertex v is already included in MST if T[v]==1, otherwise, it is not
included yet.

14
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 1 14 e
i 4
1 7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 0 0 0 0 0 0 0 0
Key 0 - - - - - - - -
π -1 - - - - - - - -

15
The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 1 14 e
i 4
1 7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 0 0 0 0 0 0 0 0
Key 0 4 - - - - - 8 -
π -1 a - - - - - a -

16
The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2
Important: Update Key[v] only if T[v]==0

V a b c d e f g h i
T 1 1 0 0 0 0 0 0 0
Key 0 4 8 - - - - 8 -
π -1 a b - - - - a -

17
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 0 0 0 0
Key 0 4 8 7 - 4 - 8 2
π -1 a b c - c - a c

18
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 0 0 0 1
Key 0 4 8 7 - 4 6 7 2
π -1 a b c - c i i c

19
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 1 0 0 1
Key 0 4 8 7 10 4 2 7 2
π -1 a b c f c f i c

20
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 1 1 0 1
Key 0 4 8 7 10 4 2 1 2
π -1 a b c f c f g c

21
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 0 0 1 1 1 1
Key 0 4 8 7 10 4 2 1 2
π -1 a b c f c f g c

22
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 1 0 1 1 1 1
Key 0 4 8 7 9 4 2 1 2
π -1 a b c d c f g c

23
 The execution of Prim's algorithm(moderate part)
8 7
the root b c d 9
vertex 4
2
a 11 14 e
i 4
7 6
10
8
h g f
1 2

V a b c d e f g h i
T 1 1 1 1 1 1 1 1 1
Key 0 4 8 7 9 4 2 1 2
π -1 a b c d c f g c

24
 Complexity: Prim Algorithm
MST-Prim(G,w,r)
01 Q ← V[G] // Q – vertices out of T
02 for each u ∈ Q
03 key[u] ← ∞ O(V)
04 key[r] ← 0
05 π[r] ← NIL
06 while Q ≠ ∅ do
O(V)
07 u ← ExtractMin(Q) // making u part of T
08 for each v ∈ Adj[u] do Heap: O(lgV)
09 if v ∈ Q and w(u,v) < key[v] then Overall: O(E)
10 π[v] ← u
11 key[v] ← w(u,v)
Decrease Key: O(lgV)

Overall complexity: O(V)+O(V lg V+E lg V) = O(E lg V)

25
 Overall Complexity Analysis
• O(V2)
• When we don’t use heap
• To find the minimum element, we traverse the “KEY” array from beginning to
end
• We use adjacency matrix to update KEY.
• O(ElogV)
• When min-heap is used to find the minimum element from “KEY”.
• O(E+VlogV)
• When fibonacci heap is used to find the minimum element from “KEY”.

26
 Kruskal’s Algorithm

• A MST can be grown from a forest of spanning trees

by adding the smallest edge connecting two spanning
trees.

27
 Some Definition
• Cut:
• Partition of V. Ex: (S, V-S)
• Cross:
• Edge (u,v) crosses the cut (S, V-S) if one of its endpoints is in S and the other is
in V-S.
• Light edge:
• An edge crossing a cut if its weight is the minimum of any edge crossing the
cut.

28
 Kruskal's Algorithm
• Edge based algorithm
• Add the edges one at a time, in increasing weight order
• The algorithm maintains A – a forest of trees. An edge is accepted it if
connects vertices of distinct trees
• We need a data structure that maintains a partition, i.e.,a collection
of disjoint sets
• MakeSet(S,x)
• Union(Si,Sj)
• FindSet(S, x)

29
Kruskal – Step 1

30
Kruskal – Step 2

31
Kruskal – Step 3

32
Kruskal – Step 4

33
Kruskal – Step 5

34
Kruskal – Step 6

35
Why Avoiding Cycles Matters

Up to this point, we have simply taken the

edges in order of their weight. But now we
will have to reject an edge since it forms a
cycle when added to those already chosen.

36
Forms a Cycle

So we cannot take the blue edge having weight 55.

37
 Kruskal – Step 7 DONE!!

Weight (T) = 23 + 29 + 31 + 32 + 47 + 54 + 66 = 282

38
Kruskal's Algorithm
• The algorithm adds the cheapest edge that connects
two trees of the forest

MST-Kruskal(G,w)
01 A ← ∅
02 for each vertex v ∈ V[G] do
O(V)
03 Make-Set(v)
04 sort the edges of E by non-decreasing weight w O(ElogE)
05 for each edge (u,v) ∈ E, in order by
O(E)
non-decreasing weight do
06 if Find-Set(u) ≠ Find-Set(v) then
07 A ← A ∪ {(u,v)} O(V)
08 Union(u,v)
09 return A

Overall Complexity: O(VE)

39
 Textbooks & Web References
• Text Book (Chapter 23)
• www.geeksforgeeks.org
 Thank you
&
Any question?