Intro to Graphs

Week 9: Graphs: Non-linear DS

Learning outcomes:
• What are graphs, Graph representation: Vertices (nodes) and
Edges?
• Implementation of graphs?
– Twitter (directed)
– Facebook (undirected - Reciprocal)
• Graph examples
– Word game / PageRank algorithm
– https://csacademy.com/lesson/introduction_to_graphs/
– https://csacademy.com/lesson/graph_representation/
• Depth first search
• Breadth first search
• Dijkstra's Algorithm: Shortest Path Tree/Self-balancing Briary
Tree
– Uses, limitations, Runtime 2
– Demonstrate on a graph / Route finding algorithm:
 Graphs
• A graph is composed of edges E and vertices V
that link the nodes together. A graph G is often
denoted G=(V,E) where V is the set of vertices and
E the set of edges.
• Two types of graphs:
– Directed graphs: G=(V,E) where E is composed of
ordered pairs of vertices; i.e. the edges have direction
and point from one vertex to another.
– Undirected graphs: G=(V,E) where E is composed of
unordered pairs of vertices; i.e. the edges are
bidirectional.
 Directed Graph

1 2

3 4

If we allow multiple edges between the same pair of

vertices it is called a multigraph
 Undirected Graph

1 2

3 4
 Graph Terminology
• The degree of a vertex in an undirected graph is
the number of edges that leave/enter the vertex.
• The degree of a vertex in a directed graph is the
same, but we distinguish between in-degree and
out-degree. Degree = in-degree + out-degree.
• A path from u to z is <u,v,w,x,y,z> and denoted
uvwxyz
• The running time of a graph algorithm expressed
in terms of the cardinality of E and V, where E = |
E| and V=|V|; e.g. G=O(EV) is |E| * |V|
 Implementing a Graph
• Implement a graph in three ways:
– Adjacency List
– Adjacency-Matrix
– Pointers/memory for each node (actually a form
of adjacency list)
 Adjacency List
• List of pointers for each vertex

1 2
1  2  3
2 
5 3  4
4  1
3 4 5  2  4
 Undirected Adjacency List

1 2
1  2  3  4
2  1  5
5
3  4  1
4  1  3
3 4 5  2  4
 Adjacency List
• The sum of the lengths of the adjacency
lists is 2|E| in an undirected graph, and |E| in
a directed graph.
• The amount of memory to store the array
for the adjacency list is
O(max(V,E))=O(V+E).
 Adjacency Matrix

1 2 3 4 5
1 2
1 0 1 1 0 0
2 0 0 0 0 0
5
3 0 0 0 1 0
3 4 1 0 0 0 0
4
5 0 1 0 1 0
 Undirected Adjacency Matrix

1 2 3 4 5
1 2
1 0 1 1 1 0
5
2 1 0 0 0 1
3 1 0 0 1 0
3 4 4 1 0 1 0 1
5 0 1 0 1 0
 Adjacency Matrix vs. List?
• The matrix always uses Θ(v2) memory. Usually
easier to implement and perform lookup than an
adjacency list.
• Sparse graph: very few edges.
• Dense graph: lots of edges. Up to O(v2) edges if
fully connected.
• The adjacency matrix is a good way to represent a
weighted graph. In a weighted graph, the edges
have weights associated with them. Update matrix
entry to contain the weight. Weights could
indicate distance, cost, etc.
 Introductory Remarks
• Given a path v1..vn, if v1 = vn, and the edges
don’t repeat, it is a circuit; if the vertices in
a circuit are different, it is a cycle
• A complete graph of n vertices, denoted Kn,
has exactly one edge between each pair of
vertices
• The edge count = = = = O
 Searching a Graph
• Search: The goal is to methodically explore
every vertex and every edge; perhaps to do
some processing on each.
• For the most part in our algorithms we will
assume an adjacency-list representation of
the input graph.
 Breadth First Search
• Example 1: Tree. This is a special case of a graph.

– The order of search is across levels.

– The root is examined first; then both children of the
root; then the children of those nodes, etc.

1
2 3 4
5 6 7 8
 Breadth First Search
• Example 2: Directed Graph

• Pick a source vertex S to start.

• Find (or discover) the vertices that are adjacent to S.
• Pick each child of S in turn and discover their vertices
adjacent to that child.
• Done when all children have been discovered and
examined.

• This results in a tree that is rooted at the source vertex S.

• The idea is to find the distance from some Source vertex
by expanding the “frontier” of what we have visited.
 BFS Properties
• Memory required: Need to maintain Q, which contains a
list of all fringe vertices we need to explore, O(V)
• Runtime: O(V+E) ; O(E) to scan through adjacency list
and O(V) to visit each vertex. This is considered linear
time in the size of G.
• Claim: BFS always computes the shortest path distance in
d[i] between S and vertex I. We will skip the proof.
• What if some nodes are unreachable from the source?
(reverse c-e,f-h edges). What values do these nodes get?
 Depth First Search
• Example 1: DFS on tree. Specialized case of more general
graph. The order of the search is down paths and from left
to right.
– The root is examined first; then the left child of the root; then the
left child of this node, etc. until a leaf is found. At a leaf,
backtrack to the lowest right child and repeat.

1
2 5 6
3 4 7 8
 Depth First Search
• Example 2: DFS on directed graph.

• Start at some source vertex S.

• Find (or explore) the first vertex that is adjacent to S.
• Repeat with this vertex and explore the first vertex that is
adjacent to it.
• When a vertex is found that has no unexplored vertices
adjacent to it then backtrack up one level
• Done when all children have been discovered and examined.

• Results in a forest of trees.

 DFS Runtime
• O(V2) - DFS loop goes O(V) times once for each vertex
(can’t be more than once, because a vertex does not stay
white), and the loop over Adj runs up to V times.
• But…
– The for loop in DFS-Visit looks at every element in Adj once. It is
charged once per edge for a directed graph, or twice if undirected. A
small part of Adj is looked at during each recursive call but over the
entire time the for loop is executed only the same number of times as
the size of the adjacency list which is (E).
– Since the initial loop takes (V) time, the total runtime is (V+E).
• Note: Don’t have to track the backtracking/fringe as in BFS
since this is done for us in the recursive calls and the stack.
The amount of storage needed is linear in terms of the depth
of the tree.
 More Graph Algorithms

Minimum Spanning Trees, Shortest

Path Algorithms
 Spanning Tree
• Definition
– A spanning tree of a graph G is a tree (acyclic) that
connects all the vertices of G once
• i.e. the tree “spans” every vertex in G
– A Minimum Spanning Tree (MST) is a spanning tree on
a weighted graph that has the minimum total weight

w(T )   w( u, v ) such that w(T) is minimum

u , v T

Where might this be useful? Minimum material to connect nodes

and also approximation for some hard problems.
 Sample MST
• Which links to make this a MST?
6 4
5 9

14
2
10

15
3 8

Optimal substructure: A subtree of the MST must in turn be a MST of the

nodes that it spans. This idea is common in recursion and
dynamic programming.
 MST Claim
• Claim: Say that M is a MST
– If we remove any edge (u,v) from M then this
results in two trees, T1 and T2.
– T1 is a MST of its subgraph while T2 is a MST
of its subgraph.
– Then the MST of the entire graph is T1 + T2 +
the smallest edge between T1 and T2
– If some other edge was used, we wouldn’t have
the minimum spanning tree overall
 Greedy Algorithm
• We can use a greedy algorithm to find the
MST
– A greedy algorithm is one that makes what
looks like the best decision at a point in time
and never backtracks to undo that decision
– Two common algorithms
• Kruskal
• Prim
 Kruskal’s MST Algorithm
• Idea: Greedily construct the MST
– Go through the list of edges and make a forest that is a
MST
– At each vertex, sort the edges
– Edges with smallest weights examined and possibly
added to MST before edges with higher weights
– Edges added must be “safe edges” that do not ruin the tree
property.
 Kruskal’s MST Algorithm
• https://www.programiz.com/dsa/kruskal-alg
orithm
• Kruskal's algorithm is a minimum spanning
tree algorithm that takes a graph as input
and finds the subset of the edges of that
graph which
– form a tree that includes every vertex
– has the minimum sum of weights among all the
trees that can be formed from the graph
 Steps for implementing Kruskal's
• We start from the edges with the lowest weight
and keep adding edges until we reach our goal.
• The steps for implementing Kruskal's algorithm
are as follows:
• Sort all the edges from low weight to high
• Take the edge with the lowest weight and add it
to the spanning tree. If adding the edge created a
cycle, then reject this edge.
• Keep adding edges until we reach all vertices.
 Prim's algorithm
• https://www.programiz.com/dsa/prim-algorith
m
• Prim's algorithm is a minimum spanning tree
algorithm that takes a graph as input and
finds the subset of the edges of that graph
which
– form a tree that includes every vertex
– has the minimum sum of weights among all the
trees that can be formed from the graph
 Kruskal’s vs Prim's algorithm
• Instead of starting from a vertex, Kruskal's algorithm
sorts all the edges from low weight to high and keeps
adding the lowest edges, ignoring those edges that create a
cycle.
• Prim’s Algorithm does not pre-sort all edges. It uses a
Priority Queue to maintain a running sorted next possible
vertices. Each time it greedily select top of the priority
queue and add the edge into the final MST if the enqueued
neighbor hasn’t been already visited.
• The time complexity of Prim's algorithm is O(E log V).
• The time complexity Of Kruskal's Algorithm: O(E log E).
 Dijkstra vs. Prim’s MST
(Greedy Algorithms)
• Very similar to Prim’s MST algorithm, but the
two compute different things. Not the shortest
path in MST, but picks shortest edge overall to
connect all edges
– (what case would the MST pick a different edge than
Dijkstra?)
– single-source shortest paths problem (SSSP)
– Given a graph and a source vertex in the graph, find
the shortest paths from the source to all vertices in the
given graph.
https://opendsa-server.cs.vt.edu/ODSA/Books/Everything/html/GraphShortes
t.html
 Shortest Path Algorithms
Goal: Find the shortest path between vertices in a weighted graph. We denote the shortest path between
vertex u and v as  (u,v). This is a very practical problem - the weights may represent the shortest
distance, shortest time, shortest cost, etc. There are several forms of this problem:

1. Single-source shortest path. Find the shortest distance from a source vertex s to every other vertex in
the graph.
2. Single-destination shortest path. Find a shortest path to a given destination vertex t from every vertex
v. This is just the reverse of single-source.
3. Single-pair shortest path. Find a shortest path between a pair of vertices. No algorithm is known for
this problem that runs asymptotically faster than the best single-source algorithms.
4. All-pairs shortest path. Find the shortest path between every vertex in the graph.

Note that BFS computes the shortest path on an unweighted graph.

 Properties of Shortest Paths
1. The shortest path problem has optimal substructure. That is, the subpath of a shortest path is a shortest
path. If pik is the shortest path from I to K, then for any node j along this path such that I J K, pij is
the shortest path from I to J. (If there was a shorter path from I to J then the IK path wasn’t the
shortest!)

2.  (s,v)   (s,u) + w(u,v)

Relaxation: The process of relaxing an edge (u,v) consists of testing whether or not we can improve the
shortet path to v by going through some other path through u.
 Dynamic Programming
• Dynamic Programming (DP) is a similar
approach to Divide & Conquer. It also breaks
the problem into similar subproblems, but they
are actually overlapping and codependent —
they’re not solved independently.
• DP applications include Fibonacci number
series, Tower of Hanoi, Dijkstra etc.
 Single-Source Shortest Path: Dijkstra's
Algorithm
Dijkstra’s algorithm presents an algorithm to solve the single-source shortest-paths
problem.
Given Vertex S in Graph G, find a shortest path from S to every other vertex in G.

The algorithm processes a set of labelled vertices v 0 to vn-1, with S = v0.

Assume that we have processed in order of distance from S to the first i - 1 vertices
that are closest to S; call this set of vertices S. We are now about to process the ith
closest vertex; call it X. A shortest path from S to X must have its next-to-last
vertex in S. Thus,

In other words, the shortest path from S to X is the minimum over all paths that go
from S to U, then have an edge from U to X, where U is some vertex in S.

How it Works: It works by maintaining a distance estimate D(X) for all vertices X in
V. The elements of D are initialized to the value INFINITE. Vertices are processed in
order of distance from S. Whenever a vertex V is processed, D(X) is updated for every
neighbor X of V.
https://opendsa-server.cs.vt.edu/ODSA/Books/Everything/html/
GraphShortest.html
https://www.programiz.com/dsa/dijkstra-algorithm
https://visualgo.net/en/sssp
 Summary - Take-home Messages
• Given a weighted directed acyclic graph, we can find
the shortest distance between two vertices by:
– starting with a trivial path containing the initial
vertex
– growing this path by always going to the next
vertex which has the shortest current path
 Dijkstra’s Algorithm time complexity in the worst-

case input depends on the ADT used to store the

graph’s vertices (Adjacency Matrix/List, MinHeap)
 Breadth First Search Algorithm
https://opendsa-server.cs.vt.edu/ODSA/Books/Everything/html/Graph
Traversal.html

• Pseudocode: Uses FIFO Queue Q

BFS(s) ; s is our source vertex

for each u ÎV - {s} ; Initialize unvisited vertices to ¥
do d[u] ¬ ¥
d[s] ¬ 0 ; distance to source vertex is 0
Q.add(s) ; Queue of vertices to visit
while Q not empty do
u = Q.remove()
for each v Î Adj[u] do ; Get adjacent vertices
if d[v]= ¥
then d[v] ¬ d[u]+1 ; Increment depth
Q.add(v) ; Add to nodes to explore

https://visualgo.net/en/sssp
BFS(s) ; s is our source vertex
for each u ÎV - {s} ; Initialize unvisited vertices to ¥
do d[u] ¬ ¥
d[s] ¬ 0 ; distance to source vertex is 0
Q.add(s) ; Queue of vertices to visit
while Q not empty do
u = Q.remove()
for each v Î Adj[u] do ; Get adjacent vertices
if d[v]= ¥
then d[v] ¬ d[u]+1 ; Increment depth
Q.add(v) ; Add to nodes to explore

a d

b e

f i
g h
 BFS Example
• Final State shown Can create tree out of
0 3
order we visit nodes
a d
1 a 0
c
1 2
b e
b c 1
2 3
f i
g h e f 2
3 3

d h i g 3
 DFS Algorithm
https://opendsa-server.cs.vt.edu/ODSA/Books/Everything/html/Graph
Traversal.html
• Pseudocode
DFS(s)
for each vertex u V
do color[u]  White ; not visited
time  1 ; time stamp
for each vertex u V
do if color[u]=White
then DFS-Visit(u,time)

DFS-Visit(u,time)
color[u]  Gray ; in progress nodes
d[u]  time ; d=discover time
time  time+1
for each v Adj[u] do
if color[u]=White
then DFS-Visit(v,time)
color[u]  Black
f[u]  time  time+1 ; f=finish time
DFS(s)
for each vertex u V
do color[u]  White ; not visited
time  1 ; time stamp
for each vertex u V
do if color[u]=White
then DFS-Visit(u,time)

DFS-Visit(u,time)
color[u]  Gray ; in progress nodes
d[u]  time ; d=discover time
time  time+1
for each v Adj[u] do
if color[u]=White
then DFS-Visit(v,time)
color[u]  Black
f[u]  time  time+1 ; f=finish time

a d

b e

f i
g h
 DFS Example
• Result (start/finish time): Tree:
a
10/11
1/18
d
a c
2/17
c
5/6 f
9/14
b e 12/13
3/16
g h
i
f

g h b e
4/7 8/15

d i
 DFS Example
• What if some nodes are unreachable? We still visit those
nodes in DFS. Consider if c-e, f-h links were reversed.
Then we end up with two separate trees
– Still visit all vertices and get a forest: a set of unconnected graphs
without cycles (a tree is a connected graph without cycles).
11/12
1/10
d
a
2/9
c
5/6
13/18
b
e 15/16
3/8
i
f

g h
4/7 14/17
 What is next

Chapter 9: Sorting Algorithms

46