Lecture 7: Synchronous Network Algorithms

Basic technology of distributed algorithms

(1) Correctness of algorithms
(2) Efficiency of algorithms
(3) Fault-tolerance
Complexity of distributed algorithms
• Communication complexity: (1) the number of
messages transmitted in whole system, (2) the number
of message bits transmitted in whole system
• Memory complexity: (1) the amount of memory used
in whole system, (2) the amount of memory used in
one process.
• Computation complexity: (1) processing time in
processes, (2) upper bound of transmission delay in
communication links.
Problems 1 Leader Election in A synchronous Ring
Assumption of the network
• Network graph G=(V,E) is a ring, where node set V={1,2,…,n} and edge set
E={(i,i+1), where i=1,2,…,n and i is counted mod n}.
• Each process has a unique distinct identifier (UID).
• Each node uses only unidirectional communication and its UID (it does not
know the size of the ring, its own index and anything of its neighbors).
• Only the leader performs an output.

LCR algorithm (informal)

(1) Each process sends its identifier around the ring.
(2) When a process receives an incoming identifier, it compares that identifier to
its own. If the incoming identifier is greater than its own, it keeps passing the
identifier; if it is less than its own, it discards the incoming identifier; if it is equal
to its own, the process declares itself the leader.
 LCR algorithm (formal)
Suppose that each node i has
(1) A UID u, initially i’s UID,
(2) send, a UID or null, initially i’s UID,
(3) A status with value in {unknown, leader}, initially unknown.
Algorithm
Send the current value of send to clockwise neighbor
send := null
if the incoming message is v, a UID, than
case
Theorem LCR solves the leader-election
v>u: send :=v
problem in a synchronous ring.
v=u: status := leader
Proof: exact one process outputs the value
v<u: do nothing
leader.
endcase
Analysis of the Complexity of LCR Algorithm
• Time complexity: n rounds until a leader is announced.
• Communication complexity: O ( n 2 ) in worst case.

Remark 1: If it is required that all the nodes halt, the time

complexity is 2n and the communication complexity is
still O ( n 2 ).
Remark 2: If it is required that the leader and nonleaders
all provide output and all processes halt, the extra cost of
obtaining the extra outputs and the halting is only n
rounds and n messages.
Improved Leader-Election Algorithm
In the following algorithm, the communication is supposed to be
bidirectional.
HS algorithm (informal)
Each process i operates in phases 0,1,2,….
In each phase l, process i send out “tokens” consists of UID ui in both
directions. These are intended to travel distance 2l then return to their
original i. If both tokens make it back safely, process i continues with the
following phase. However, the tokens might not make it back safely.
While a ui token is proceeding in the outbound direction, each other process
j onui path comparesui with its own UIDu j . Ifui  u j then j simply
ui  u jif
discards the token, whereas ui u. jIf
, thenuji relays , then it
means that process j has received its own UID before the token has turned
around, so process j elects itself as the leader.
All processes always relay all tokens in the inbound direction.
 i

distance 20
find leader
distance 21
relay
distance 22

distance 2log n

Analysis of the Complexity of HS Algorithm

• Time complexity: O(n) rounds (the time for each phase is 2 2l ).
 n 
• Communication complexity: O(n log n) (at most  2l  1  1 processes
altogether initiate tokens at phase l. Therefore, the total messages
 l n 
sent out at phase l is bounded by  2 2l  1  1 8n ).
4
 
Problem2 Leader Election in a General Network
Assumption of the network
• An strongly connected network digraph G=(V,E) having n nodes.
• Processes do not know their indices, nor those of their neighbors, but
refer to their neighbors by local names.
• If a process i has the same process j for both incoming and outgoing
neighbor, then i knows that the two processes are the same.

FloodMax algorithm (informal)

Suppose that each process has a unique distinct UID and it knows
diam, the diameter of network.
• Each process maintains a record of the maximum UID it has seen so
far (initially its own). At each round, each process propagates its
maximum on all of its outgoing edges.
• After diam rounds, if the maximum value seen is the process’s own
UID, the process elects itself the leader; otherwise, it is a non-leader.
Theorem FloodMax algorithm solves the leader-election problem
in a synchronous general network.
Proof: exact one process outputs the value leader.

Analysis of the Complexity of FloodMax Algorithm

• Time complexity: diam (The time until the leader is elected and
all other processes know that they are not the leader is diam
rounds.) diam| E |
• Communication
diam| E | complexity: (The number of messages
is , where |E| is the number of directed edges in the
digraph, because a message is sent on every directed edge for each
of the first diam rounds.)
 Problem 3 Breadth-First Search
Bread-First Search Given a graph G=(V,E) and a distinguished
source vertex i, breadth-first search explores the edges of G to
discover every vertex that is reachable from s. It provides the
distance from i to all such reachable vertices. It also provide a
breadth-first tree with root i that contains all such reachable
vertices.
s s

Bread-First Search in an undirected graphBread-First Search in a directed graph

Spanning Tree A spanning tree of graph G=(V,E) is a
rooted tree which contains all vertices of V and the path
from the root to any vertex.
• A breadth-firs tree is a spanning tree.
Assumption of the network
• An strongly connected network digraph G=(V,E) having n nodes and a
distinguished source note s.
• Output is the structure of a breadth-first search tree of the network graph with
root s in a distributed fashion: each process other than s should have a parent
component that gets set to indicate the node that is its parent in the tree.
• Processes know their neighbors’ indices. They have no knowledge of the size or
diameter of the network. UIDs are not needed.
SynchBFS algorithms
• At any point during execution, there is some set of processes that is “marked”,
initially just s.
• Process s sends out a search message at round 1, to all of its outgoing
neighbors.
• At any round, if an unmarked process receives a search message, it marks itself
and chooses one of the processes from which the search has arrived as its parent,
then it sends a search message to all of its outgoing neighbors.
Theorem SynchBFS algorithm solves the Breadth-First
Search problem.

Analysis of the complexity of SynchBFS algorithm

• Time complexity: At most diam rounds
• Communication complexity: |E|

Application to Message Broadcast Problem

SynchBFS algorithm can be used to Message Broadcast
problem: piggyback the message m in the Breadth-First
Search.
Problem 4 Shortest Path Problem

Shortest Path Problem Consider a strong directed graph

G=(V,E), where ach edge e=(i,j) in E has a nonnegative real-
value weight w(i,j). The problem is to find a shortest path
form a distinguished node s to each other node in G.

2 1 s
2 1 s
1 4 3
1 4 3
1 1
1 1 1
1 3 1
3 1

Shortest paths from s

 Assumption of network
• Each process initially knows its neighbors’ indices and the weight of
all its incident edges.
• Each process knows the number n of nodes in the network.
• Output: each process knows its parent in the shortest path tree, and its
distance from s.
BellmanFord algorithm
• Each process i keeps track of parent and dist, the shortest distance from s to i it
knows so far. Initially, dist(s)=0, dist(i)= for i s and the parent components
are not defined.
• At each round process i sends its dist(i) to all its outgoing neighbors. Then
each process i updates its dist(i) jto min {dist (i ), dist ( j )  weight( j, i )}.
be
{neighbors of i}

If dist(i) is updated, the parent is also updated accordingly.

• After n-1 rounds, dist contains the shortest distance, and parent contains the
parent in the shortest tree.
 
s  2 
1 s
 2 1 s
2 1 0
0 1 4 3
1 4 3 1 4 3
1 1  1 1 3 1
1 1
1 1 1 3 1
3 1 3
  1


7 5
 2 1 s 3 2
6
1 s 3 2 1 s
0 0 0
1 4 3 4 3 4 3
1 1
1 2 1 1 2 1 1 2 1
1 1 1
3 1 1 3 1
3
6 1 6 6 1
1
Analysis of the complexity of BellFord algorithm
• Time complexity: n-1
• Communication complexity: (n-1)|E|
Assignment
(1) Improve algorithm FloodMax to reduce the communication
complexity.
(2) Describe an algorithm that extends SynchBFS to allow the
source process s to broadcast a message to all other
processes and obtain an acknowledge that all processes have
received it. Your algorithm should use O(|E|) messages and
O(diam) time. You may assume that the network graph is
undireted.