Graph

Terminology
Paths and cycles
• A path is a sequence of nodes
v1, v2, …, vN such that (vi,vi+1)E for 0<i<N

• The length of the path is N-1.

• Simple path: all vi are distinct, 0<i<N

• A cycle is a path such that v1=vN

• An acyclic graph has no cycles

2
Cycles
BOS

SFO DTW

PIT
JFK

LAX

3
More useful definitions
• In a directed graph:

• The indegree of a node v is the number of distinct edges (w,v)E.

• The outdegree of a node v is the number of distinct edges (v,w)E.

• A node with indegree 0 is a root.

4
Trees are graphs
• A dag is a directed acyclic graph.

• A tree is a connected acyclic undirected graph.

• A forest is an acyclic undirected graph (not necessarily connected),

i.e., each connected component is a tree.

5
Example DAG
Undershorts
Socks
Watch
Pants Shoes
Shirt
a DAG implies an
Belt Tie ordering on events

Jacket

6
Example DAG
Undershorts
Socks
Watch
Pants Shoes
Shirt
In a complex DAG, it
Belt Tie can be hard to find a
schedule that obeys
Jacket all the constraints.

7
Topological Sort
Topological Sort
• For a directed acyclic graph G = (V,E)
• A topological sort is an ordering of all of G’s vertices v1, v2, …, vn such
that...

Formally: for every edge (vi,vk) in E, i<k.

Visually: all arrows are pointing to the right

9
Topological sort
• There are often many possible topological sorts of a given DAG
• Topological orders for this DAG :
• 1,2,5,4,3,6,7 1 2

• 2,1,5,4,7,3,6
3 4 5
• 2,5,1,4,7,3,6
• Etc.
6 7

• Each topological order is a feasible schedule.

10
Topological Sorts for Cyclic
Graphs?
1 2
Impossible!
3

• If v and w are two vertices on a cycle, there

exist paths from v to w and from w to v.
• Any ordering will contradict one of these paths

11
Topological sort algorithm
• Algorithm
• Assume indegree is stored with each node.
• Repeat until no nodes remain:
• Choose a root and output it.
• Remove the root and all its edges.

• Performance
• O(V2 + E), if linear search is used to find a root.

12
Better topological sort
• Algorithm:
• Scan all nodes, pushing roots onto a stack.
• Repeat until stack is empty:
• Pop a root r from the stack and output it.
• For all nodes n such that (r,n) is an edge, decrement n’s indegree. If
0 then push onto the stack.
• O( V + E ), so still O(V2) in worst case, but better for
sparse graphs.
• Q: Why is this algorithm correct?

13
Correctness
• Clearly any ordering produced by this algorithm is a topological order

But...

• Does every DAG have a topological order, and if so, is this algorithm
guaranteed to find one?

14
Graph Traversals
Graph Traversals

•Both take time: O(V+E)

16
Use of a stack
• It is very common to use a stack to keep track of:
• nodes to be visited next, or
• nodes that we have already visited.
• Typically, use of a stack leads to a depth-first visit order.
• Depth-first visit order is “aggressive” in the sense that it examines
complete paths.

17
Topological Sort as DFS
• do a DFS of graph G
• as each vertex v is “finished” (all of it’s children processed), insert it
onto the front of a linked list
• return the linked list of vertices

• why is this correct?

18
Use of a queue
• It is very common to use a queue to keep track of:
• nodes to be visited next, or
• nodes that we have already visited.
• Typically, use of a queue leads to a breadth-first visit order.
• Breadth-first visit order is “cautious” in the sense that it examines
every path of length i before going on to paths of length i+1.

19
Graph Searching ???
• Graph as state space (node = state, edge = action)
• For example, game trees, mazes, ...
• BFS and DFS each search the state space for a best move. If the search is
exhaustive they will find the same solution, but if there is a time limit and the
search space is large...
• DFS explores a few possible moves, looking at the effects far in the future
• BFS explores many solutions but only sees effects in the near future (often finds
shorter solutions)

20
Minimum Spanning Trees

21
Problem: Laying Telephone Wire

Central office

22
Wiring: Naïve Approach

Central office

Expensive!

23
Wiring: Better Approach

Central office

Minimize the total length of wire connecting the customers

24
Minimum Spanning Tree (MST)
(see Weiss, Section 24.2.2)

A minimum spanning tree is a subgraph of an

undirected weighted graph G, such that

• it is a tree (i.e., it is acyclic)

• it covers all the vertices V
– contains |V| - 1 edges
• the total cost associated with tree edges is the
minimum among all possible spanning trees
• not necessarily unique

25
How Can We Generate a MST?

9 9
b b
a 2 6 a 2 6
d d
4 5 4 5
5 4 5 4

5 e 5 e
c c

27
Prim’s Algorithm
Initialization
a. Pick a vertex r to be the root
b. Set D(r) = 0, parent(r) = null
c. For all vertices v  V, v  r, set D(v) = 
d. Insert all vertices into priority queue P,
using distances as the keys
9 Vertex Parent
b e -
a 2 6 e a b c d
d
4 5
5 4 0    
5 e
c

28
Prim’s Algorithm
While P is not empty:

1. Select the next vertex u to add to the tree

u = P.deleteMin()

2. Update the weight of each vertex w adjacent to

u which is not in the tree (i.e., w  P)
If weight(u,w) < D(w),
a. parent(w) = u
b. D(w) = weight(u,w)
c. Update the priority queue to reflect
new distance for w

29
Prim’s algorithm
Vertex Parent
e -
e d b c a b -
0     c -
9 d -
b
a 2 6
d Vertex Parent
4 5
5 4 e -
5 e d b c a b e
c 4 5 5  c e
d e

The MST initially consists of the vertex e, and we update

the distances and parent for its adjacent vertices

30
Prim’s algorithm
Vertex Parent
e -
d b c a b e
4 5 5  c e
9 d e
b
a 2 6
d
4 5
5 4
Vertex Parent
5 e e -
c a c b b e
2 4 5 c d
d e
a d

31
Prim’s algorithm
Vertex Parent
e -
a c b b e
2 4 5 c d
9 d e
b
a d
a 2 6
d
4 5
5 4

5 e
c Vertex Parent
e -
c b b e
4 5 c d
d e
a d

32
Prim’s algorithm
Vertex Parent
e -
c b b e
9 4 5 c d
b d e
a 2 6
a d
d
4 5
5 4

5 e Vertex Parent
c e -
b b e
5 c d
d e
a d

33
Prim’s algorithm
Vertex Parent
e -
b b e
5 c d
9
b d e
a 2 6 a d
d
4 5
5 4

5 e
c Vertex Parent
e -
b e
The final minimum spanning tree c d
d e
a d

34
Running time of Prim’s algorithm
(without heaps)

Initialization of priority queue (array): O(|V|)

Update loop: |V| calls

• Choosing vertex with minimum cost edge: O(|V|)
• Updating distance values of unconnected
vertices: each edge is considered only once
during entire execution, for a total of O(|E|)
updates
Overall cost without heaps: O(|E| + |V| 2)

When heaps are used, apply same analysis as for

Dijkstra’s algorithm (p.469) (good exercise)

35
Prim’s Algorithm Invariant
• At each step, we add the edge (u,v) s.t. the weight of
(u,v) is minimum among all edges where u is in the
tree and v is not in the tree

• Each step maintains a minimum spanning tree of the

vertices that have been included thus far

• When all vertices have been included, we have a MST

for the graph!

36
Correctness of Prim’s
• This algorithm adds n-1 edges without creating a cycle, so
clearly it creates a spanning tree of any connected graph
(you should be able to prove this).

But is this a minimum spanning tree?

Suppose it wasn't.

• There must be point at which it fails, and in particular there

must a single edge whose insertion first prevented the
spanning tree from being a minimum spanning tree.

37
Correctness of Prim’s

• Let G be a connected,
undirected graph
• Let S be the set of
edges chosen by Prim’s x
algorithm before y
choosing an errorful
edge (x,y)
• Let V' be the vertices incident with edges in S
• Let T be a MST of G containing all edges in S, but not (x,y).

38
Correctness of Prim’s

w
v
• Edge (x,y) is not in T, so
there must be a path in
T from x to y since T is x
connected. y
• Inserting edge (x,y) into
T will create a cycle
• There is exactly one edge on this cycle with exactly one
vertex in V’, call this edge (v,w)

39
Correctness of Prim’s
• Since Prim’s chose (x,y) over (v,w), w(v,w) >= w(x,y).
• We could form a new spanning tree T’ by swapping (x,y) for
(v,w) in T (prove this is a spanning tree).
• w(T’) is clearly no greater than w(T)
• But that means T’ is a MST
• And yet it contains all the edges in S, and also (x,y)

...Contradiction

40
Another Approach
• Create a forest of trees from the vertices
• Repeatedly merge trees by adding “safe edges”
until only one tree remains
• A “safe edge” is an edge of minimum weight which
does not create a cycle

9
b
a 2 6
d forest: {a}, {b}, {c}, {d}, {e}
4 5
5 4

5 e
c

41
Kruskal’s algorithm
Initialization
a. Create a set for each vertex v  V
b. Initialize the set of “safe edges” A
comprising the MST to the empty set
c. Sort edges by increasing weight

9
b
a 2 6 F = {a}, {b}, {c}, {d}, {e}
d A= 
4 5
5 4
E = {(a,d), (c,d), (d,e), (a,c),
5 e (b,e), (c,e), (b,d), (a,b)}
c

42
Kruskal’s algorithm
For each edge (u,v)  E in increasing order
while more than one set remains:
If u and v, belong to different sets U and V
a. add edge (u,v) to the safe edge set
A = A  {(u,v)}
b. merge the sets U and V
F = F - U - V + (U  V)

Return A

• Running time bounded by sorting (or findMin)

• O(|E|log|E|), or equivalently, O(|E|log|V|) (why???)

43
Kruskal’s algorithm
9
b
a 2 6
E = {(a,d), (c,d), (d,e), (a,c),
d
5 4
4 5 (b,e), (c,e), (b,d), (a,b)}
5 e
c
Forest A
{a}, {b}, {c}, {d}, {e} 
{a,d}, {b}, {c}, {e} {(a,d)}
{a,d,c}, {b}, {e} {(a,d), (c,d)}
{a,d,c,e}, {b} {(a,d), (c,d), (d,e)}
{a,d,c,e,b} {(a,d), (c,d), (d,e), (b,e)}

44
 Kruskal’s Algorithm Invariant

• After each iteration, every tree in the forest is a MST of the vertices it connects

• Algorithm terminates when all vertices are connected into one tree

45
Correctness of Kruskal’s
• This algorithm adds n-1 edges without creating a cycle, so
clearly it creates a spanning tree of any connected graph
(you should be able to prove this).

But is this a minimum spanning tree?

Suppose it wasn't.

• There must be point at which it fails, and in particular there

must a single edge whose insertion first prevented the
spanning tree from being a minimum spanning tree.

46
Correctness of Kruskal’s
K T
S
e
• Let e be this first errorful edge.
• Let K be the Kruskal spanning tree
• Let S be the set of edges chosen by Kruskal’s algorithm before
choosing e
• Let T be a MST containing all edges in S, but not e.

47
Correctness of Kruskal’s
Lemma: w(e’) >= w(e) for all edges e’ in T - S
Proof (by contradiction):
• Assume there exists some K T
edge e’ in T - S, w(e’) < w(e)
S
• Kruskal’s must have
considered e’ before e e
• However, since e’ is not in K (why??), it must have
been discarded because it caused a cycle with some of
the other edges in S.
• But e’ + S is a subgraph of T, which means it cannot
form a cycle ...Contradiction
48
Correctness of Kruskal’s
• Inserting edge e into T will create a cycle
• There must be an edge on this cycle which is not in K (why??). Call this edge e’
• e’ must be in T - S, so (by our lemma) w(e’) >= w(e)
• We could form a new spanning tree T’ by swapping e for e’ in T (prove this is a
spanning tree).
• w(T’) is clearly no greater than w(T)
• But that means T’ is a MST
• And yet it contains all the edges in S, and also e
...Contradiction

49
Greedy Approach
• Like Dijkstra’s algorithm, both Prim’s and Kruskal’s
algorithms are greedy algorithms

• The greedy approach works for the MST problem;

however, it does not work for many other problems!

50