Jim Anderson Comp 122, Fall 2003 Single-source SPs - 1

Chapter 24: Single-Source Shortest Paths

Given: A single source vertex in a weighted,
directed graph.
Want to compute a shortest path for each
possible destination.
Similar to BFS.
We will assume either
no negative-weight edges, or
no reachable negative-weight cycles.
Algorithm will compute a shortest-path tree.
Similar to BFS tree.
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 2
Outline
General Lemmas and Theorems.
CLRS now does this last. Well still do it first.
Bellman-Ford algorithm.
DAG algorithm.
Dijkstras algorithm.
We will skip Section 24.4.
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 3
Corollary: Let p = SP from s to v, where p = s u v. Then,
(s, v) = (s, u) + w(u, v).
General Results (Relaxation)
Lemma 24.1: Let p = v
1
, v
2
, , v
k
be a SP from v
1
to v
k
. Then,
p
ij
= v
i
, v
i+1
, , v
j
is a SP from v
i
to v
j
, where 1 s i s j s k.
So, we have the optimal-substructure property.

Bellman-Fords algorithm uses dynamic programming.

Dijkstras algorithm uses the greedy approach.

Let (u, v) = weight of SP from u to v.

p'
Lemma 24.10: Let s e V. For all edges (u,v) e E, we have
(s, v) s (s, u) + w(u,v).
Jim Anderson
Lemma 24.1 holds because one edge gives the
shortest path, so the other edges must give sums
that are at least as large.
Comp 122, Fall 2003 Single-source SPs - 4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 5
Relaxation
Initialize(G, s)
for each v e V[G] do
d[v] := ;
t[v] := NIL
od;
d[s] := 0
Relax(u, v, w)
if d[v] > d[u] + w(u, v) then
d[v] := d[u] + w(u, v);
t[v] := u
fi
Algorithms keep track of d[v], t[v]. Initialized as follows:
These values are changed when an edge (u, v) is relaxed:
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 6
Properties of Relaxation
d[v], if not , is the length of some path from s
to v.
d[v] either stays the same or decreases with time
Therefore, if d[v] = o(s, v) at any time, this
holds thereafter
Note that d[v] > o(s, v) always
After i iterations of relaxing on all (u,v), if the
shortest path to v has i edges, then d[v] = o(s, v).
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 7
Properties of Relaxation
Consider any algorithm in which d[v], and t[v] are first initialized
by calling Initialize(G, s) [s is the source], and are only changed by
calling Relax. We have:
Lemma 24.11: ( v:: d[v] > o(s, v)) is an invariant.
Implies d[v] doesnt change once d[v] = o(s, v).
Proof:
Initialize(G, s) establishes invariant. If call to Relax(u, v, w)
changes d[v], then it establishes:
d[v] = d[u] + w(u, v)
> o(s, u) + w(u, v) , invariant holds before call.
> o(s, v) , by Lemma 24.10.
Corollary 24.12: If there is no path from s to v, then
d[v] = (s, v) = is an invariant.
Jim Anderson
For lemma 24.11, note that initialization makes
the invariant true at the beginning.
Comp 122, Fall 2003 Single-source SPs - 8
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 9
More Properties
Lemma 24.14: Let p = SP from s to v, where p = s u v.
If d[u] = (s, u) holds at any time prior to calling Relax(u, v, w),
then d[v] = (s, v) holds at all times after the call.

p'
Proof:

After the call we have:
d[v] s d[u] + w(u, v) , by Lemma 24.13.
= o(s, u) + w(u, v) , d[u] = o(s, u) holds.
= o(s, v) , by corollary to Lemma 24.1.

By Lemma 24.11, d[v] > (s, v), so d[v] = (s, v).

Lemma 24.13: Immediately after relaxing edge (u, v) by calling
Relax(u, v, w), we have d[v] s d[u] + w(u, v).
Jim Anderson
Lemma 24.13 follows simply from the structure
of Relax.
Lemma 24.14 shows that the shortest path will
be found one vertex at a time, if not faster.
Thus after a number of iterations of Relax equal
to V(G) - 1, all shortest paths will be found.
Comp 122, Fall 2003 Single-source SPs - 10
Jim Anderson


Bellman-Ford returns a compact representation
of the set of shortest paths from s to all other
vertices in the graph reachable from s. This is
contained in the predecessor subgraph.
Comp 122, Fall 2003 Single-source SPs - 11
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 12
Predecessor Subgraph
Lemma 24.16: Assume given graph G has no negative-weight cycles
reachable from s. Let G
t
= predecessor subgraph. G
t
is always a
tree with root s (i.e., this property is an invariant).
Proof:
Two proof obligations:
(1) G
t
is acyclic.
(2) There exists a unique path from source s to each vertex in V
t
.

Proof of (1):

Suppose there exists a cycle c = v
0
, v
1
, , v
k
, where v
0
= v
k
.
We have t[v
i
] = v
i-1
for i = 1, 2, , k.

Assume relaxation of (v
k-1
, v
k
) created the cycle.
We show cycle has a negative weight.
Note: Cycle must be reachable from s. (Why?)
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 13
Proof of (1) (Continued)
Before call to Relax(v
k-1
, v
k
, w):

t[v
i
] = v
i-1
for i = 1, , k1.
Implies d[v
i
] was last updated by d[v
i
] := d[v
i-1
] + w(v
i-1
, v
i
)
for i = 1, , k1. [Because Relax updates t.]
Implies d[v
i
] > d[v
i-1
] + w(v
i-1
,

v
i
) for i = 1, , k1. [Lemma 24.13]

Because t[v
k
] is changed by call, d[v
k
] > d[v
k-1
] + w(v
k-1
,

v
k
). Thus,

cycle! weight - neg. i.e., 0, ) v , w(v , ] d[v ] d[v Because
) v , w(v ] d[v
)) v , w(v ] (d[v ] d[v
k
1 i
i 1 i
k
1 i
1 i
k
1 i
i
k
1 i
k
1 i
i 1 i 1 i
k
1 i
k
1 i
i 1 i 1 i i
< =
+ =
+ >



=

=

=
= =

= =

Jim Anderson
Comment on Proof
d[v
i
] > d[v
i-1
] + w(v
i-1
,

v
i
) for i = 1, , k1
because when Relax(v
i-1
,

v
i
, w) was called,
there was an equality, and d[v
i-1
] may have
gotten smaller by further calls to Relax.
d[v
k
] > d[v
k-1
] + w(v
k-1
,

v
k
) before the last call to
Relax because that last call changed d[v
k
].
Comp 122, Fall 2003 Single-source SPs - 14
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 15
Proof of (2)
Proof of (2):

( v: v e V
t
:: (- path from s to v)) is an invariant.

So, for any v in V
t
, - at least 1 path from s to v.

Show s 1 path.

Assume 2 paths.




s u
y
x
z v
impossible!
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 16
Lemma 24.17
Lemma 24.17: Same conditions as before. Call Initialize & repeatedly
call Relax until d[v] = (s, v) for all v in V. Then, G
t
is a shortest-path
tree rooted at s.
Proof:

Key Proof Obligation: For all v in V
t
, the unique simple path p from
s to v in G
t
(path exists by Lemma 24.16) is a shortest path from s to v
in G.

Let p = v
0
, v
1
, , v
k
, where v
0
= s and v
k
= v.

We have d[v
i
] = (s, v
i
)
d[v
i
] > d[v
i-1
] + w(v
i-1
,

v
i
) (reasoning as before)

Implies w(v
i-1
,

v
i
) s (s, v
i
) (s, v
i-1
).
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 17
Proof (Continued)
) v , (s
) v , (s ) v , (s
)) v , (s ) v , (s (
) v , w(v
w(p)
k
0 k
1 - i i
k
1 i
k
1 i
i 1 i
=
=
s
=

=
=

So, equality holds and p is a shortest path.
Jim Anderson
And note that this shortest path tree will be
found after V(G) - 1 iterations of Relax.
Comp 122, Fall 2003 Single-source SPs - 18
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 19
Bellman-Ford Algorithm
Can have negative-weight edges. Will detect reachable negative-weight
cycles.
Initialize(G, s);
for i := 1 to |V[G]| 1 do
for each (u, v) in E[G] do
Relax(u, v, w)
od
od;
for each (u, v) in E[G] do
if d[v] > d[u] + w(u, v) then
return false
fi
od;
return true
Time
Complexity
is O(VE).
Jim Anderson
So if Bellman-Ford has not converged after
V(G) - 1 iterations, then there cannot be a
shortest path tree, so there must be a negative
weight cycle.
Comp 122, Fall 2003 Single-source SPs - 20
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 21
Example
0


z
u v
x y
6
5
3
9
7
7
8
2
4
2
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 22
Example
0
7
6
z
u v
x y
6
5
3
9
7
7
8
2
4
2
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 23
Example
0
2 7
4 6
z
u v
x y
6
5
3
9
7
7
8
2
4
2
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 24
Example
0
2 7
4 2
z
u v
x y
6
5
3
9
7
7
8
2
4
2
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 25
Example
0
-2 7
4 2
z
u v
x y
6
5
3
9
7
7
8
2
4
2
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 26
Another Look
Note: This is essentially dynamic programming.

Let d(i, j) = cost of the shortest path from s to i that is at most j hops.


d(i, j) =
0 if i = s . j = 0
if i = s . j = 0
min({d(k, j1) + w(k, i): i e Adj(k)}
{d(i, j1)}) if j > 0
z u v x y
1 2 3 4 5
0 0
1 0 6 7
2 0 6 4 7 2
3 0 2 4 7 2
4 0 2 4 7 2
j
i
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 27
Lemma 24.2
Lemma 24.2: Assuming no negative-weight cycles reachable from
s, d[v] = o(s, v) holds upon termination for all vertices v reachable
from s.
Proof:

Consider a SP p, where p = v
0
, v
1
, , v
k
, where v
0
= s and v
k
= v.

Assume k s |V| 1, otherwise p has a cycle.

Claim: d[v
i
] = o(s, v
i
) holds after the i
th
pass over edges.
Proof follows by induction on i.

By Lemma 24.11, once d[v
i
] = o(s, v
i
) holds, it continues to hold.
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 28
Correctness
Claim: Algorithm returns the correct value.

(Part of Theorem 24.4. Other parts of the theorem follow easily from earlier results.)

Case 1: There is no reachable negative-weight cycle.

Upon termination, we have for all (u, v):
d[v] = o(s, v) , by lemma 24.2 (last slide) if v is reachable;
d[v] = o(s, v) = otherwise.
s o(s, u) + w(u, v) , by Lemma 24.10.
= d[u] + w(u, v)

So, algorithm returns true.
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 29
Case 2
Case 2: There exists a reachable negative-weight cycle
c = v
0
, v
1
, , v
k
, where v
0
= v
k
.

We have E
i = 1, , k
w(v
i-1
, v
i
) < 0. (*)

Suppose algorithm returns true. Then, d[v
i
] s d[v
i-1
] + w(v
i-1
, v
i
) for
i = 1, , k. (because Relax didnt change any d[v
i
] ). Thus,

E
i = 1, , k
d[v
i
] s E
i = 1, , k
d[v
i-1
] + E
i = 1, , k
w(v
i-1
, v
i
)

But, E
i = 1, , k
d[v
i
] = E
i = 1, , k
d[v
i-1
].

Can show no d[v
i
] is infinite. Hence, 0 s E
i = 1, , k
w(v
i-1
, v
i
).

Contradicts (*). Thus, algorithm returns false.

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 30
Shortest Paths in DAGs
Topologically sort vertices in G;
Initialize(G, s);
for each u in V[G] (in order) do
for each v in Adj[u] do
Relax(u, v, w)
od
od
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 31
Example
0
r s t
u
v w
5 2 7 1 2
6 1
3
2
4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 32
Example
0
r s t
u
v w
5 2 7 1 2
6 1
3
2
4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 33
Example
0 2 6
r s t
u
v w
5 2 7 1 2
6 1
3
2
4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 34
Example
0 2 6 6 4
r s t
u
v w
5 2 7 1 2
6 1
3
2
4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 35
Example
0 2 6 5 4
r s t
u
v w
5 2 7 1 2
6 1
3
2
4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 36
Example
0 2 6 5 3
r s t
u
v w
5 2 7 1 2
6 1
3
2
4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 37
Example
0 2 6 5 3
r s t
u
v w
5 2 7 1 2
6 1
3
2
4
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 38
Dijkstras Algorithm
Assumes no negative-weight edges.
Maintains a set S of vertices whose SP from s has been determined.
Repeatedly selects u in VS with minimum SP estimate (greedy choice).
Store VS in priority queue Q.
Initialize(G, s);
S := C;
Q := V[G];
while Q = C do
u := Extract-Min(Q);
S := S {u};
for each v e Adj[u] do
Relax(u, v, w)
od
od
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 39
Example
0


s
u v
x y
10
1
9
2
4 6
5
2 3
7
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 40
Example
0
5
10
s
u v
x y
10
1
9
2
4 6
5
2 3
7
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 41
Example
0
7 5
14 8
s
u v
x y
10
1
9
2
4 6
5
2 3
7
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 42
Example
0
7 5
13 8
s
u v
x y
10
1
9
2
4 6
5
2 3
7
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 43
Example
0
7 5
9 8
s
u v
x y
10
1
9
2
4 6
5
2 3
7
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 44
Example
0
7 5
9 8
s
u v
x y
10
1
9
2
4 6
5
2 3
7
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 45
Correctness
Theorem 24.6: Upon termination, d[u] = (s, u) for all u in V
(assuming non-negative weights).
Proof:

By Lemma 24.11, once d[u] = (s, u) holds, it continues to hold.

We prove: For each u in V, d[u] = o(s, u) when u is inserted in S.

Suppose not. Let u be the first vertex such that d[u] = o(s, u) when
inserted in S.

Note that d[s] = o(s, s) = 0 when s is inserted, so u = s.

S = C just before u is inserted (in fact, s e S).
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 46
Proof (Continued)
Note that there exists a path from s to u, for otherwise
d[u] = o(s, u) = by Corollary 24.12.

there exists a SP from s to u. SP looks like this:
x
s
y
u
S
p
1

p
2

Jim Anderson Comp 122, Fall 2003 Single-source SPs - 47
Proof (Continued)
Claim: d[y] = o(s, y) when u is inserted into S.

We had d[x] = o(s, x) when x was inserted into S.

Edge (x, y) was relaxed at that time.

By Lemma 24.14, this implies the claim.

Now, we have: d[y] = o(s, y) , by Claim.
s o(s, u) , nonnegative edge weights.
s d[u] , by Lemma 24.11.

Because u was added to S before y, d[u] s d[y].

Thus, d[y] = o(s, y) = o(s, u) = d[u].

Contradiction.
Jim Anderson Comp 122, Fall 2003 Single-source SPs - 48
Complexity
Running time is

O(V
2
) using linear array for priority queue.

O((V + E) lg V) using binary heap.

O(V lg V + E) using Fibonacci heap.

(See book.)