Shortest path algorithms

Introduction
• Given a directed graph G with edge weights,
find
– The shortest path from a given vertex s to all other
vertices (Single Source Shortest Paths)
– The shortest paths between all pairs of vertices (All
Pairs Shortest Paths)
• where the length of a path is the sum of its
edge weights.
 Shortest Paths: Applications
• Flying times between cities
• Distance between street corners
• Cost of doing an activity
– Vertices are states
– Edge weights are costs of moving between
states
 Shortest Paths: Algorithms
• Single-Source Shortest Paths (SSSP)
– Dijkstra’s
– Bellman-Ford
– DAG Shortest Paths
• All-Pairs Shortest Paths (APSP)
– Floyd-Warshall
– Johnson’s
 A Fact About Shortest Paths

• Theorem: If p is a shortest path from u to v,

then any subpath of p is also a shortest path.
• Proof: Consider a subpath of p from x to y. If
there were a shorter path from x to y, then
there would be a shorter path from u to v.
shorter?

u x y v
 Example
• Imagine you are
9
D 4 planning a road
E C
trip from one
2 6 another city.
14 F 15 • Your task is to find
9 8 the shortest path
7
A B distance to reach
your destination.

Figure: Cities in the map are circled. The distances

between cities are the labelled lines.
 Analysis of previous approaches
• Breadth First search (BFS)
– We could perform a BFS on the map by using a
queue.
– We have seen how BFS could be used to construct
the shortest path between two cities.
– We could accumulate the distances as we visit each
city in BFS order.
– Once we reach the destination we know the total
distance and we can reconstruct the path.
 Example
D 4
9

E C
2 6

14 F 15
9 8

7
A B

Example: Path(A,E,D) , Total distance: 14+9=23

• Clearly, calculating the shortest path does not always result in the
shortest distance.
• Unless the distance between all cities is the same, BFS will not
always compute the shortest path.
 Analysis of previous approaches
• Depth First Search (DFS):
– We could perform a DFS on the map by using a
stack.
– Similar to BFS we accumulate the distances as we
visit each city in depth first order.
– Once we reach the distances we know the total
distance and we can reconstruct the path.
 Example
D 4
9

E C
2 6

15
14 F
9 8

7
A B

Example: Path(A, B, C, D) , Total distance: 7+15+4=26

• Calculating the shortest path does not always result in the shortest
distance.
• This approach only works if we stumble upon the optimal distance by
virtue of always choosing the correct next neighbor.
 Analysis of previous approaches
• Exhaustive:
– Exhaustive search is always an option.
– Continue generating different paths even when a valid
one is produced.
– Once generated we would then choose the path with
the lowest cumulative distance.
– But this approach will work at the cost of enumerating
all possible paths between two cities.
• As the number of cities and roads are increase, the cost to
compute the shortest distance can increase exponentially.
• This is because each new city added causes all existing
cities and paths to consider the new city.
 Analysis of previous approaches
• Natural greedy approach:
– The idea is that one can make the most progress to
the goal by always choosing the best choice
available at each point in time.
– We could construct an algorithm that selects roads
based on the shortest path road length.
– Once we have a path between the start and finish
cities and then compute the total distance between
the start and finish cities.
 Example
D 4
9

E C
2 6

14 F 15
9 8

7
A B

Initially no roads in the map are selected.

 Example
D 4
9

E C
2 6

14 F 15
9 8

7
A B

The road between E and F is selected first because it

is the shortest. No path between A and D exists –
continue selecting
 Example

D 4
9

E C
2 6

14 F 15
9 8

7
A B

The road between C and D is selected next shortest path.

No path between A and D exists – continue selecting
 Example
D 4
9

E C
2 6

14 F 15
9 8

7
A B

The road between C and F is selected as next shortest path.

No path between A and D exists – continue selecting
 Example
D 4
9

E C
2 6

14 F 15
9 8

7
A B

The road between A and B is selected as next shortest

path. No path between A and D exists – continue selecting
 Example
D 4
9

E C
2 6

14 F 15
9 8

7
A B

The road between B and F is selected as next shortest path.

A path between A and D exists; (A, B, F, C, D): Total
distance = 25
 Analysis of previous approaches
• Greedy approach: Conclusion
– This approach also produces a non-optimal result
• because it does not take into account the total sum of the
roads needed to connect the two cities.
– There are other greedy approaches, such as
choosing the shortest road out of the current city
and following it.
• But this is also not optimal and can be easily verified.
• Note: However, one greedy approach fails, it
doesn’t mean that all of them will.
 Dijkstra’s Algorithm
• Main idea of this algorithm is thinking optimally.
• The logic behind Dijkstar’s algorthm is based on
the principle of Optimality.

“In an optimal sequence based on choices, each

contiguous subsequence must also be optimal.”
 Dijkstra’s Algorithm
Dijkstra(G)
for each v  V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  Adj[u]
if (d[v] > d[u]+w(u,v))
Note: this
is really a d[v] = d[u]+w(u,v);
call to Q->DecreaseKey()
David Luebke

21

12/21/2021
 Example
D 4
9

E C
2 6

14 F 15
9 8

7
A B
 Example
D 4
(14, A) 9

E C
2 6

14 F 15
9 8

7
A B

• Current vertex is A.
• A’s neighbours are updated.
• A is finalized.
 Example
D 4
(14, A) 9

E C (22, B)
2 6

14 F 15
9 8

7
A B

• Current vertex is B.
• B’s neighbours are updated.
• B is finalized.
 Example
D 4
(11, F) 9

E C (15, F)
2 6

14 F 15
9 8

7
A B

• Current vertex is F.
• F’s neighbours are updated.
• F is finalized.
 Example
(20, F)
D 4
(11, F) 9

E C (15, F)
2 6

14 F 15
9 8

7
A B

• Current vertex is E.
• E’s neighbours are updated.
• E is finalized.
 Example
(19, C)
D 4
(11, F) 9

E C (15, F)
2 6

14 F 15
9 8

7
A B

• Current vertex is C.
• C’s neighbours are updated.
• C is finalized.
 Example
(19, C)
D 4
(11, F) 9

E C (15, F)
2 6

14 F 15
9 8

7
A B

• Current vertex is D.
• D’s neighbours are updated.
• D is finalized. Algorithm done.
 Example
(19, C)
D 4
(11, F) 9

E C (15, F)
2 6

14 F 15
9 8

7
A B

• Minimum cost from A to D is 19

• The path is found by tracing backwards from D to A
using the predecessor’s: (D, C, F, A).
• Reverse this to get (A, F, C, D).
 Dijkstra’s Algorithm
Dijkstra(G)
for each v  V
d[v] = ;
d[s] = 0; S = ; Q = V;
while (Q  )
u = ExtractMin(Q);
S = S U {u};
for each v  u->Adj[]
if (d[v] > d[u]+w(u,v))
Note: this
is really a d[v] = d[u]+w(u,v);
call to Q->DecreaseKey()
David Luebke

30

12/21/2021
 Time Complexity: Using List
The simplest implementation of the Dijkstra's
algorithm stores vertices in an ordinary linked list
or array
– Good for dense graphs (many edges)
• |V| vertices and |E| edges
• Initialization O(|V|)
• While loop O(|V|)
– Find and remove min distance vertices O(|V|)
– Potentially |E| updates
• Update costs O(1)
• Reconstruct path O(|E|)
Total time O(|V2| + |E|) = O(|V2| )
31
Time Complexity: Priority Queue
For sparse graphs, (i.e. graphs with much less than |V2|
edges) Dijkstra's implemented more efficiently by
priority queue
• Initialization O(|V|) using O(|V|) build Heap
• While loop O(|V|)
• Find and remove min distance vertices O(log |V|)
using O(log |V|) deleteMin
• Potentially |E| updates
• Update costs O(log |V|) using decreaseKey
Total time O(|V|log|V| + |E|log|V|) = O(|E|log|V|)
|V| = O(|E|) assuming a connected graph
32
 Correctness of Dijkstra's Alg.

• Let Ti be the tree constructed after i-th iteration of

the while loop:
– The nodes in Ti are not in Q
– The edges in Ti are indicated by parent variables
• Show by induction on i that the path in Ti from s to
u is a shortest path and has distance d[u], for all u
in Ti.
• Basis: i = 1.
s is the only node in T1 and d[s] = 0.
33
 Correctness of Dijkstra's Alg.

• Induction: Assume Ti is a correct shortest path tree.

We need to show that Ti+1 is a correct shortest path
tree as well.
• Let u be the node added in iteration i.
• Let x = parent(u).
Need to show
path in Ti+1
Ti Ti+1 from s to u is a
s x u shortest path,
and has
distance d[u]
34
Correctness of Dijkstra's Alg

P, path in Ti+1
Ti Ti+1
s from s to u
x u
a P', another
path from s to u
b
(a,b) is first edge in P' that
leaves Ti

35
 Correctness of Dijkstra's Alg

Let P1 be part of P' before (a,b).

Ti
Let P2 be part of P' after (a,b). s P Ti+1
x u
w(P') = w(P1) + w(a,b) + w(P2) a

≥ w(P1) + w(a,b) (nonneg wts) b

P'
≥ wt of path in Ti from s to a + w(a,b) (inductive hypothesis)
≥ w(s->x path in Ti) + w(x,u) (alg chose u in iteration i and
d-values are accurate, by inductive hypothesis
= w(P).
So P is a shortest path, and d[u] is accurate after iteration i+1.
36
 Graph with –ve edges : How to get
the shortest paths?
• Some edges may have negative weights

• If there is a negative cycle reachable from s:

– Shortest path is no longer well-defined
– Example

• Otherwise, it is fine
 Cycles
• Shortest path cannot have cycles inside
– Negative cycles : already eliminated
– Positive cycles: can be removed
– 0-weight cycles: can be removed

• So each shortest path does not have cycles

What About Negative Edge Weights?

• Dijkstra's SSSP algorithm requires all edge

weights to be nonnegative. This is too
restrictive, since it suffices to outlaw negative
weight cycles.
• Bellman-Ford SSSP algorithm can handle
negative edge weights. [It even can
detect negative weight cycles if they exist.]

40
The Bellman-Ford Algorithm

41
 Bellman-Ford: Algorithm
BELLMAN-FORD(G, w, s)
1 For each vertex v V[G] do //INIT_SINGLE_SOURCE
2 d[v] 
3 p[v] NIL
4 d[s] 0
5 for i  1 to |V[G]|-1 do > each iteration is a “pass”
6 for each edge (u,v) in E[G] do
7 if d[u] + w(u,v) < d[v] then
8 d[v] := d[u] + w(u,v)
9 parent[v] := u
10 for each edge (u,v) in E[G] do > check for negative cycles
11 if d[v] > d[u] + w(u,v) then
11 return FALSE
12 return TRUE
 Running Time of Bellman-Ford
• O(V) iterations of outer for loop
• O(E) iterations of inner for loop
• O(VE) time total

43
Example
 Correctness of Bellman-Ford
• First prove that if G = (V, E) contains no negative-
weight cycles, then after the Bellman-Ford
algorithm executes, d[v] = d(s, v) for all v  V.
• Let v be reachable from s, and let p = < v0 , v1 , . . .
, vk> be a shortest path from s to v, where v0 = s
and vk = v.
• Since p is acyclic, so k≤ |V| − 1 edges.
• Each iteration of the for loop relaxes all edges:
• First iteration relaxes (v0, v1).
• Second iteration relaxes (v1, v2).
• kth iteration relaxes (vk-1, vk).
By the path-relaxation property (if d[u] + w(u,v) <
d[v] then d[v] := d[u] + w(u,v)),
d[v] = d[vk ] = δ(s, vk ) = δ(s, v)
• Suppose there is no negative-weight cycle
reachable from s.
• At termination, for all (u, v) ∈ E,
d[v] = δ(s, v)
≤ δ(s, u) + w(u, v) (triangle inequality)
= d[u] + w(u, v) .
• So BELLMAN-FORD returns TRUE.
• Now suppose there exists negative-weight cycle c =
v0, v1, . . . , vk, where v0= vk , reachable from s.
• Then

• Suppose the contradiction that Bellman-Ford

returns TRUE.
• Then d[vi ] ≤ d[vi-1] + w(vi-1, vi ) for i = 1, 2, . . . , k.
• Summing around c
• Since v0= vk, each vertex in c appears exactly
once in each of the summations

• So
=
• Hence

• This contradicts c being a negative-weight cycle.

and concludes that the Bellman-Ford algorithm returns
TRUE if graph G contains no negative-weight cycles
reachable from the source, and FALSE otherwise.
Single source shortest path for DAG
• There is no cycle in a DAG. Hence, no
negative-weight cycle can exists in a DAG,
and shortest paths are well defined.
• Single source shortest paths problem for DAG
can be solved more efficiently by using
topological sort.
• Reason: the only paths to a vertex come from
vertices before it in the topological sort.
 SSSP-DAG Algorithm
DAG-SHORTEST-PATHS(G,w,s)
1 topologically sort the vertices of G
2 initialize d and p as in previous algorithms
3 for each vertex u in topological sort order do
4 for each vertex v in Adj[u] do
5 RELAX(u, v, w)

• Running time: q(V+E), same as topological

sort.
 Example

DAG and its corresponding topological sort

 Example

6 iterations corresponding to RELAX operations

Floyd’s Algorithm

All pairs shortest path

 All Pairs Shortest Paths (APSP)
• Given : directed graph G = ( V, E ),
weight function ω : E → R, |V| = n

• Goal : create an n × n matrix D = ( dij ) of shortest

path distances
i.e., dij = δ ( vi , vj )

The All-Pairs Shortest Path Problem asks to find the

length of the shortest path between any pair of
vertices in G.

• trivial solution : run a SSSP algorithm n times, one for

each vertex as the source. 54
 All Pairs Shortest Paths (APSP)
► all edge weights are nonnegative : use Dijkstra’s
algorithm
– PQ = linear array : O ( V3 + VE ) = O ( V3 )
– PQ = binary heap : O ( V2lgV + EVlgV ) = O ( V2lgV )

► negative edge weights : use Bellman-Ford algorithm

– O ( V2E )

55
 Dynamic Programming
(1) Characterize the structure of an optimal solution.
(2) Recursively define the value of an optimal
solution.
(3) Compute the value of an optimal solution in a
bottom-up manner.
(4) Construct an optimal solution from information
constructed in (3).

56
 Floyd-Warshall
•We will now investigate a dynamic
programming solution that solved the problem in
O(n3) time for a graph with n vertices.
This algorithm is known as the Floyd-Warshall
algorithm.

57
The structure of a shortest path:
• Given a path p=(v1, v2,…, vm) in the graph, we will
call the vertices vk with index k in {2,…,m-1} the
intermediate vertices of p.
• For any pair of vertices i,j  V, consider all paths
from i to j whose intermediate vertices are all drawn
from {1,2,…,k},and let p be a minimum-weight path
from among them.
• The Floyd-Warshall algorithm exploits a relationship
between path p and shortest paths from i to j with all
intermediate vertices in the set {1,2,…,k-1}.
58
 Cont.
• Consider k is not an intermediate vertex of
path p.
• All intermediate vertices of path p are in the
set {1,2,…,k-1}. Thus dij(k) = dij(k-1) .

59
 All intermediate vertices in {1,2,…,k-1}

p1 p2
k

i j

P:all intermediate vertices in {1,2,…,k}

• If the vertex k is an intermediate vertex on

p, then dij(k) = dik(k-1) + dkj(k-1)
A recursive solution to the all-pairs
shortest paths problem:
• A recursive definition is given by
• dij(k)= wij

min(dij(k-1), dik(k-1)+dkj(k-1))
if k=0,
if k 1.

• The matrix D(n)=(dij(n)) gives the final answer.

61
 Computing the shortest-path
Floyd//Computes shortest distance between all pairs of
//nodes, and saves P to enable finding shortest paths
1. D0  W // initialize D array to W [ ]
2. P  0 // initialize P array to [0]
3. for k  1 to n
4. do for i  1 to n
5. do for j  1 to n
6. if (Dk-1[ i, j ] > Dk-1 [ i, k ] + Dk-1 [ k, j ] )
7. then Dk[ i, j ]  Dk-1 [ i, k ] + Dk-1 [ k, j ]
8. P[ i, j ]  k;
9. else Dk[ i, j ]  Dk-1 [ i, j ]

62
 Example
1 2 3
1 0 4 5
W= D0 =
2 2 0 
1 5
3  -3 0
4 2 3
1 2 3
-3 1 0 0 0
2
P= 2 0 0 0
3 0 0 0
 1 1 2 3
5 D0 =
1 0 4 5 k=1
4 3 Vertex 1 can be
2 2 2 0 
2
-3
3  -3 0
intermediate
1 2 3
node
1 0 4 5 D1[2,3] = min( D0[2,3], D0[2,1]+D0[1,3] )
D1 =
2 2 0 7
= min (, 7)
=7
3  -3 0

1 2 3
D1[3,2] = min( D0[3,2], D0[3,1]+D0[1,2] )
1 0 0 0 = min (-3,)
P= 2 0 0 1 = -3
3 0 0 0
 1 2 3
1 5 D1 = 1
0 4 5
4 3 2 2 0 7
k=2
2
-3 3
Vertices 1, 2 can
2  -3 0
be intermediate
1 2 3
1 0 4 5 D2[1,3] = min( D1[1,3], D1[1,2]+D1[2,3] )
D2 =
2 2 0 7
= min (5, 4+7)
=5
3 -1 -3 0

1 2 3
1 0 0 0 D2[3,1] = min( D1[3,1], D1[3,2]+D1[2,1] )
P= 2 0 0 1 = min (, -3+2)
3 2 0 0 = -1
 1 1 2 3
5 D2 =
1 0 4 5
4 3
2 2 2 0 7 k=3
2
-3 Vertices 1, 2, 3 can
3 -1 -3 0
be intermediate
1 2 3
1 0 2 5
D3 = D3[1,2] = min(D2[1,2], D2[1,3]+D2[3,2] )
2 2 0 7 = min (4, 5+(-3))
3 -1 -3 0 =2

1 2 3
1 0 3 0 D3[2,1] = min(D2[2,1], D2[2,3]+D2[3,1] )
= min (2, 7+ (-1))
P= 2 0 0 1
=2
3 2 0 0
 Johnsons Algorithm
• Let the graph be sparse.
• Makes clever use of Bellman-Ford and Dijkstra
to do All-Pairs-Shortest-Paths efficiently on
sparse graphs.
• It is a better idea to run Dijkstras algorithm V
times
• However, Dijkstras algorithm operates only for
edges with nonnegative weights
• Can we change the edge weights to make
them nonnegative?
• Reweight the edges so that:
1. No edge weight is negative.
2. Shortest paths are preserved. (A shortest path
in the original graph is still one in the new,
reweighted graph.)

• So we need to reweight such that the

shortest paths remain the same with the new
weights
 Reweighting scheme
Let h(u) be a real number associated with vertex
u (we will discuss later how to choose them)

wnew(u,v) = w(u,v) + h(u) – h(v)

Let us compute the weight of any path p under

this new scheme
p = (v0, v1,…..vk-1, vk)

wnew(p) = w(v0 ,v1) + h(v0) – h(v1) + w(v1 ,v2) + h(v1) – h(v2)+…

+ w(vk-2 ,vk-1) + h(vk-2) – h(vk-1) + w(vk_1 ,vk) + h(vk-1) – h(vk)
wnew(p) = w(v0 ,v1) + w(v1 ,v2) +… + w(vk-2 ,vk-1) + w(vk_1 ,vk) + h(v0) – h(vk)

wnew(p) = w(p) + h(v0) – h(vk)

Under the new weighting scheme, weight of every path

between v0 and vk is incremented by constant amount
(decremented if the constant is negeative).
In other words, we have adjusted the lengths of all paths by
the same amount. So this will not affect the relative ordering
of the paths— shortest paths will be preserved.

So shortest paths remain the same under the new weights.

Still need to ensure that weight of every edge is nonnegative
wnew(u,v) = w(u,v) + h(u) – h(v) >= 0
• Will choose h(u)s appropriately
• Will add a node s to the existing network, and add an edge
from s to every node .
u
• Weight of new edges are 0 s w(u, v)

– d(s, u) is the shortest path weight between s andv u under

the old weights. h(u) =d(s,u)
d ( s, u )  w(u , v)  d ( s, v )
Rewriting :
w(u , v )  d ( s, u )  d ( s, v)  0

• wnew(u,v) = w(u,v) + d(s, u) –d(s,v) >= 0

 Johnson’s: Algorithm
1. Compute G’, which consists of G augmented
with s and a zero-weight edge from s to every
vertex in G.
2. Run Bellman-Ford(G’, w, s) to obtain the
d(s,v)’s
3. Reweight by computing ŵ for each edge
d̂
4. Run Dijkstra on each vertex to compute
5. Undo reweighting factors to compute d
Johnson’s: CLRS
 Complexity Analysis
• Adding new vertex and edges takes O(V)
• Bellman-ford takes O((V+1)(V + E)) or O(V(V +
E))
• Computing the new weights takes O(E)
• Dijkstras algorithm takes O(V(VlogV + E))
• O(V + V(V + E) + V(VlogV + E))
– O(V(VlogV + E))
 Johnson: reweighting

ŵ(u, v) = w(u, v) + h (u) - h(v)

=w(u, v) + d(s, u) - d(s, v)