Distributed Mutual Exclusion

Dr. Rajiv Misra

Associate Professor
Dept. of Computer Science & Engg.
Indian Institute of Technology Patna
[email protected]
Cloud Computing and DistributedVuSystems
Pham Distributed Mutual Exclusion
Preface
Content of this Lecture:

In this lecture, we will discuss about the ‘Concepts of

Mutual Exclusion’, Classical algorithms for distributed
computing systems and Industry systems for Mutual
Exclusion.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Need of Mutual Exclusion in Cloud
Bank’s Servers in the Cloud: Two customers
make simultaneous deposits of 10,000 Rs. into
your bank account, each from a separate ATM.
Both ATMs read initial amount of 1000 Rs.
concurrently from the bank’s cloud server
Both ATMs add 10,000 Rs. to this amount (locally at
the ATM)
Both write the final amount to the server
What’s wrong? 11000Rs. (or 21000Rs.)

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Need of Mutual Exclusion in Cloud
Bank’s Servers in the Cloud: Two customers make
simultaneous deposits of 10,000 Rs. into your bank
account, each from a separate ATM.
Both ATMs read initial amount of 1000 Rs. concurrently
from the bank’s cloud server
Both ATMs add 10,000 Rs. to this amount (locally at the
ATM)
Both write the final amount to the server
You lost 10,000 Rs.!
The ATMs need mutually exclusive access to your
account entry at the server
or, mutually exclusive access to executing the code that
modifies the account entry

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Some other Mutual Exclusion use
Distributed File systems
o Locking of files and directories
Accessing objects in a safe and consistent way
o Ensure at most one server has access to object at any
point of time
Server coordination
o Work partitioned across servers
o Servers coordinate using locks
In industry
o Chubby is Google’s locking service
o Many cloud stacks use Apache Zookeeper for
coordination among servers

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Problem Statement for Mutual Exclusion
• Critical Section Problem: Piece of code (at all
processes) for which we need to ensure there
is at most one process executing it at any point
of time.
• Each process can call three functions
o enter() to enter the critical section (CS)
o AccessResource() to run the critical section code
o exit() to exit the critical section

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Bank Example

ATM1: ATM2:
enter(S); enter(S);
// AccessResource() // AccessResource()
obtain bank amount; obtain bank amount;
add in deposit; add in deposit;
update bank amount; update bank amount;
// AccessResource() end // AccessResource() end
exit(S); // exit exit(S); // exit

7
Cloud Computing and Distributed Systems
Vu Pham Distributed Mutual Exclusion
Approaches to Solve Mutual Exclusion
• Single OS:

• If all processes are running in one OS

on a machine (or VM), then
• Semaphores, mutexes, condition
variables, monitors, etc.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Approaches to Solve Mutual Exclusion (2)
• Distributed system:
• Processes communicating by passing messages

Need to guarantee 3 properties:

o Safety (essential): At most one process executes in

CS (Critical Section) at any time

o Liveness (essential): Every request for a CS is
granted eventually
o Fairness (desirable): Requests are granted in the

order they were made

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Processes Sharing an OS: Semaphores
Semaphore == an integer that can only be accessed via two special
functions
Semaphore S=1; // Max number of allowed accessors
1. wait(S) (or P(S) or down(S)):
while(1) { // each execution of the while loop is atomic
if (S > 0) {
enter() S--;
break;
}
}

Each while loop execution and S++ are each atomic operations – supported
via hardware instructions such as compare-and-swap, test-and-set, etc.
exit() 2. signal(S) (or V(S) or up(s)):
S++; // atomic

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Bank Example Using Semaphores
Semaphore S=1; // shared
ATM1: Semaphore S=1; // shared
wait(S); ATM2:
// AccessResource() wait(S);
obtain bank amount; // AccessResource()
add in deposit; obtain bank amount;
update bank amount; add in deposit;
// AccessResource() end update bank amount;
signal(S); // exit // AccessResource() end
signal(S); // exit

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Next
• In a distributed system, cannot share variables like
semaphores

• So how do we support mutual exclusion in a

distributed system?

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
System Model
Before solving any problem, specify its System Model:

Each pair of processes is connected by reliable channels

(such as TCP).
Messages are eventually delivered to recipient, and in
FIFO (First In First Out) order.
Processes do not fail.
• Fault-tolerant variants exist in literature.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Central Solution
o Elect a central master (or leader)
o Use one of our election algorithms!
o Master keeps
o A queue of waiting requests from processes who wish to
access the CS
o A special token which allows its holder to access CS
o Actions of any process in group:
o enter()
o Send a request to master
o Wait for token from master
o exit()
o Send back token to master

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Central Solution
o Master Actions:
o On receiving a request from process Pi
if (master has token)
Send token to Pi
else
Add Pi to queue
o On receiving a token from process Pi
if (queue is not empty)
Dequeue head of queue (say Pj), send that process the token
else
Retain token

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Analysis of Central Algorithm
o Safety – at most one process in CS
o Exactly one token
o Liveness – every request for CS granted eventually
o With N processes in system, queue has at most N
processes
o If each process exits CS eventually and no failures,
liveness guaranteed
o FIFO Ordering is guaranteed, in order of requests received
at master

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Performance Analysis
Efficient mutual exclusion algorithms use fewer messages, and
make processes wait for shorter durations to access resources.
Three metrics:
Bandwidth: the total number of messages sent in each enter
and exit operation.
Client delay: the delay incurred by a process at each enter
and exit operation (when no other process is in, or waiting)
(We will prefer mostly the enter operation.)
Synchronization delay: the time interval between one
process exiting the critical section and the next process
entering it (when there is only one process waiting)

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Analysis of Central Algorithm
o Bandwidth: the total number of messages sent in each enter
and exit operation.
o 2 messages for enter
o 1 message for exit
o Client delay: the delay incurred by a process at each enter and
exit operation (when no other process is in, or waiting)
o 2 message latencies (request + grant)
o Synchronization delay: the time interval between one process
exiting the critical section and the next process entering it
(when there is only one process waiting)
o 2 message latencies (release + grant)

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
But…
The master is the performance bottleneck and SPoF
(single point of failure)

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
 Ring-based Mutual Exclusion

Currently holds token,

N12 N3 can access CS

N6
N32

Token: N80 N5

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
 Ring-based Mutual Exclusion

Cannot access CS anymore

N12 N3
Here’s the token!

N6
N32

Token: N80 N5

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
 Ring-based Mutual Exclusion

N12 N3

N6 Currently holds token,

N32 can access CS

Token: N80 N5

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Ring-based Mutual Exclusion
N Processes organized in a virtual ring
Each process can send message to its successor in ring
Exactly 1 token
enter()
Wait until you get token
exit() // already have token
Pass on token to ring successor
If receive token, and not currently in enter(), just pass
on token to ring successor

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Analysis of Ring-based Mutual Exclusion
• Safety
• Exactly one token
• Liveness
• Token eventually loops around ring and reaches
requesting process (no failures)
• Bandwidth
• Per enter(), 1 message by requesting process but up to
N messages throughout system
• 1 message sent per exit()

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Analysis of Ring-Based Mutual Exclusion (2)

• Client delay: 0 to N message transmissions after entering

enter()
• Best case: already have token
• Worst case: just sent token to neighbor
• Synchronization delay between one process’ exit() from the CS
and the next process’ enter():
• Between 1 and (N-1) message transmissions.
• Best case: process in enter() is successor of process in exit()
• Worst case: process in enter() is predecessor of process in
exit()

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Next
Client/Synchronization delay to access CS still
O(N) in Ring-Based approach.
Can we make this faster?

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
System Model
Before solving any problem, specify its System Model:
Each pair of processes is connected by reliable
channels (such as TCP).
Messages are eventually delivered to recipient, and
in FIFO (First In First Out) order.
Processes do not fail.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Lamport’s Algorithm

Requests for CS are executed in the increasing order of

timestamps and time is determined by logical clocks.
Every site Si keeps a queue, request_queuei which contains
mutual exclusion requests ordered by their timestamps.
This algorithm requires communication channels to deliver
messages the FIFO order. Three types of messages are used-
Request, Reply and Release. These messages with timestamps
also updates logical clock.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
 The Algorithm
Requesting the critical section:
When a site Si wants to enter the CS, it broadcasts a REQUEST(tsi , i )
message to all other sites and places the request on request_queuei .
((tsi , i ) denotes the timestamp of the request.)
When Sj receives the REQUEST(tsi , i ) message from site Si , Sj places site Si ’s
request on request_queuej and it returns a timestamped REPLY message to Si .

Executing the critical section: Site Si enters the CS when the following two
conditions hold:
L1: Si has received a message with timestamp larger than (tsi , i ) from all other
sites.
L2: Si ’s request is at the top of request _queuei .

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
The Algorithm

Releasing the critical section:

Site Si , upon exiting the CS, removes its request from the top of its
request queue and broadcasts a timestamped RELEASE message to all
other sites.
When a site Sj receives a RELEASE message from site Si , it removes Si ’s
request from its request queue.

When a site removes a request from its request queue, its own request
may come at the top of the queue, enabling it to enter the CS.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Correctness
Theorem: Lamport’s algorithm achieves mutual exclusion.

Proof:
Proof is by contradiction. Suppose two sites Si and Sj are executing the CS
concurrently. For this to happen conditions L1 and L2 must hold at both the sites
concurrently.
This implies that at some instant in time, say t, both Si and Sj have their own
requests at the top of their request_queues and condition L1 holds at them.
Without loss of generality, assume that Si ’s request has smaller timestamp than
the request of Sj .
From condition L1 and FIFO property of the communication channels, it is clear
that at instant t the request of Si must be present in request_queuej when Sj was
executing its CS. This implies that Sj ’s own request is at the top of its own
request_queue when a smaller timestamp request, Si ’s request, is present in the
request queuej – a contradiction!

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Correctness
Theorem: Lamport’s algorithm is fair.

Proof:
The proof is by contradiction. Suppose a site Si ’s request has a smaller
timestamp than the request of another site Sj and Sj is able to execute the
CS before Si .
For Sj to execute the CS, it has to satisfy the conditions L1 and L2. This
implies that at some instant in time say t, Sj has its own request at the top
of its queue and it has also received a message with timestamp larger than
the timestamp of its request from all other sites.
But request_queue at a site is ordered by timestamp, and according to
our assumption Si has lower timestamp. So Si ’s request must be placed
ahead of the Sj ’s request in the request _queuej . This is a contradiction!

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Lamport’s Algorithm Example:
Sites S1 and S2 are Making Requests for the CS
Sites S1 enter the CS
(1,1)
S1

(1,2)

S2

S3

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Lamport’s Algorithm Example:
Site S1 exits the CS
and sends RELEASE
Site S1 enters the CS
messages

(1,1) (1,1), (1,2)

S1

(1,2) (1,2)
S2
(1,1), (1,2)

S3

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Lamport’s Algorithm Example:
Site S1 exits the CS
and sends RELEASE
Site S1 enters the CS
messages

(1,1) (1,1), (1,2)

S1

(1,2)

S2
(1,1), (1,2)
Site S2 enters the CS

S3

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Performance

For each CS execution, Lamport’s algorithm requires (N − 1)

REQUEST messages, (N − 1) REPLY messages, and (N − 1)
RELEASE messages.

Thus, Lamport’s algorithm requires 3(N − 1) messages per CS

invocation.

Synchronization delay in the algorithm is T .

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
An Optimization
In Lamport’s algorithm, REPLY messages can be omitted in certain
situations. For example, if site Sj receives a REQUEST message from site
Si after it has sent its own REQUEST message with timestamp higher
than the timestamp of site Si ’s request, then site Sj need not send a
REPLY message to site Si .

This is because when site Si receives site Sj ’s request with timestamp

higher than its own, it can conclude that site Sj does not have any
smaller timestamp request which is still pending.

With this optimization, Lamport’s algorithm requires between

3(N − 1) and 2(N − 1) messages per CS execution.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Ricart-Agrawala’s Algorithm
Classical algorithm from 1981
Invented by Glenn Ricart (NIH) and Ashok
Agrawala (U. Maryland)

No token
Uses the notion of causality and multicast
Has lower waiting time to enter CS than Ring-
Based approach

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Key Idea: Ricart-Agrawala Algorithm
• enter() at process Pi
• multicast a request to all processes
• Request: <T, Pi>, where T = current Lamport
timestamp at Pi
• Wait until all other processes have responded
positively to request
• Requests are granted in order of causality
• <T, Pi> is used lexicographically: Pi in request <T, Pi> is
used to break ties (since Lamport timestamps are not
unique for concurrent events)

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Ricart-Agrawala Algorithm
The Ricart-Agrawala algorithm assumes the communication channels are
FIFO. The algorithm uses two types of messages: REQUEST and REPLY.
A process sends a REQUEST message to all other processes to request their
permission to enter the critical section. A process sends a REPLY message to a
process to give its permission to that process.
Processes use Lamport-style logical clocks to assign a timestamp to critical
section requests and timestamps are used to decide the priority of requests.
Each process pi maintains the Request-Deferred array, RDi , the size of which
is the same as the number of processes in the system.
Initially, ∀i ∀j: RDi [j]=0. Whenever pi defer the request sent by pj , it sets
RDi [j]=1 and after it has sent a REPLY message to pj , it sets RDi [j]=0.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Description of the Algorithm
Requesting the critical section:
(a) When a site Si wants to enter the CS, it broadcasts a timestamped
REQUEST message to all other sites.
(b) When site Sj receives a REQUEST message from site Si , it sends a
REPLY message to site Si if site Sj is neither requesting nor
executing the CS, or if the site Sj is requesting and Si ’s request’s
timestamp is smaller than site Sj ’s own request’s timestamp.
Otherwise, the reply is deferred and Sj sets RDj [i]=1
Executing the critical section:
(c) Site Si enters the CS after it has received a REPLY message from
every site it sent a REQUEST message to.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Contd…
Releasing the critical section:
(d) When site Si exits the CS, it sends all the deferred REPLY
messages: ∀j if RDi [j]=1, then send a REPLY message to Sj and set
RDi [j]=0.

Notes:
When a site receives a message, it updates its clock using the
timestamp in the message.
When a site takes up a request for the CS for processing, it updates its
local clock and assigns a timestamp to the request.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Correctness

Theorem: Ricart-Agrawala algorithm achieves mutual exclusion.

Proof:
Proof is by contradiction. Suppose two sites Si and Sj ‘ are executing the
CS concurrently and Si ’s request has higher priority than the request of
Sj . Clearly, Si received Sj ’s request after it has made its own request.
Thus, Sj can concurrently execute the CS with Si only if Si returns a REPLY
to Sj (in response to Sj ’s request) before Si exits the CS.
However, this is impossible because Sj ’s request has lower priority.
Therefore, Ricart-Agrawala algorithm achieves mutual exclusion.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Ricart–Agrawala algorithm Example:
Sites S1 and S2 are Making Requests for the CS

(1,1)
S1

(1,2)

S2

S3

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Ricart–Agrawala algorithm Example:

Request is deferred Site S1 enters the CS

(1,1)
S1

(1,2)

S2

S3

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Ricart–Agrawala algorithm Example:

Request is deferred Site S1 enters the CS

(1,1)
S1
Site S1
exits the
(1,2) CS
S2
And send a REPLY
message to S2’s
deferred request

S3

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Performance

For each CS execution, Ricart-Agrawala algorithm requires

(N − 1) REQUEST messages and (N − 1) REPLY messages.

Thus, it requires 2(N − 1) messages per CS execution.

Synchronization delay in the algorithm is T .

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Comparison
Compared to Ring-Based approach, in
Ricart-Agrawala approach
Client/synchronization delay has now gone
down to O(1)
But bandwidth has gone up to O(N)
Can we get both down?

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Quorum-based approach

In the ‘quorum-based approach’, each site requests

permission to execute the CS from a subset of sites
(called a quorum).

The intersection property of quorums make sure that

only one request executes the CS at any time.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Quorum-Based Mutual Exclusion Algorithms
Quorum-based mutual exclusion algorithms are different in two
ways:

1. A site does not request permission from all other sites, but
only from a subset of the sites.
The request set of sites are chosen such that
∀ i ∀ j : 1 ≤ i , j ≤ N : : Ri ∩ Rj ≠ Φ.
Consequently, every pair of sites has a site which
mediates conflicts between that pair.

2. A site can send out only one REPLY message at any time.
A site can send a REPLY message only after it has received
a RELEASE message for the previous REPLY message.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Contd…

Notion of ‘Coteries’ and ‘Quorums’:

A coterie C is defined as a set of sets, where each set g ∈ C is called
a quorum.

The following properties hold for quorums in a coterie:

Intersection property: For every quorum g, h ∈ C, g ∩ h ≠ ∅.
For example, sets {1,2,3}, {2,5,7} and {5,7,9} cannot be quorums in a
coterie, because first and third sets do not have a common element.

Minimality property: There should be no quorums g, h in coterie C

such that g ⊇ h i.e g is superset of h.
For example, sets {1,2,3} and {1,3} cannot be quorums in a coterie
because the first set is a superset of the second.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Maekawa’s Algorithm

Maekawa’s algorithm was first quorum-based mutual exclusion algorithm.

The request sets for sites (i.e., quorums) in Maekawa’s algorithm are
constructed to satisfy the following conditions:
M1: (∀i ∀j : i ≠ j, 1 ≤ i , j ≤ N : : Ri ∩ Rj ≠ φ)
M2: (∀i : 1 ≤ i ≤ N : : Si ∈ Ri )
M3: (∀i : 1 ≤ i ≤ N : : |Ri | = K )
M4: Any site Sj is contained in K number of Ri s, 1 ≤ i , j ≤ N .
Maekawa used the theory of projective planes and showed that
N = K (K − 1) + 1. This relation gives |Ri | =

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Maekawa’s Algorithm

Conditions M1 and M2 are necessary for correctness; whereas

conditions M3 and M4 provide other desirable features to the
algorithm.
Condition M3 states that the size of the requests sets of all sites
must be equal implying that all sites should have to do an equal
amount of work to invoke mutual exclusion.
Condition M4 enforces that exactly the same number of sites
should request permission from any site, which implies that all sites
have “equal responsibility” in granting permission to other sites.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
The Algorithm
A site Si executes the following steps to execute the CS.

Requesting the critical section

(a) A site Si requests access to the CS by sending REQUEST( i )
messages to all sites in its request set Ri .
(b) When a site Sj receives the REQUEST( i ) message, it sends a
REPLY( j ) message to Si provided it hasn’t sent a REPLY message to
a site since its receipt of the last RELEASE message. Otherwise, it
queues up the REQUEST( i ) for later consideration.

Executing the critical section

(c) Site Si executes the CS only after it has received a REPLY message
from every site in Ri .

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
The Algorithm

Releasing the critical section

(d) After the execution of the CS is over, site Si sends a RELEASE( i )
message to every site in Ri .
(e) When a site Sj receives a RELEASE( i ) message from site Si , it
sends a REPLY message to the next site waiting in the queue and deletes
that entry from the queue.
If the queue is empty, then the site updates its state to reflect that it
has not sent out any REPLY message since the receipt of the last
RELEASE message.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Correctness

Theorem: Maekawa’s algorithm achieves mutual exclusion.

Proof:

Proof is by contradiction. Suppose two sites Si and Sj are concurrently

executing the CS.

This means site Si received a REPLY message from all sites in Ri and
concurrently site Sj was able to receive a REPLY message from all sites in Rj .

If Ri ∩ Rj = {Sk }, then site Sk must have sent REPLY messages to both

Si and Sj concurrently, which is a contradiction.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Performance

Since the size of a request set is , an execution of the CS requires

REQUEST, REPLY, and RELEASE messages, resulting in 3
messages per CS execution.

Synchronization delay in this algorithm is 2T . This is because after a

site Si exits the CS, it first releases all the sites in Ri and then one of those
sites sends a REPLY message to the next site that executes the CS.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Problem of Deadlocks

Maekawa’s algorithm can deadlock because a site is exclusively locked

by other sites and requests are not prioritized by their timestamps.
Assume three sites Si , Sj , and Sk simultaneously invoke mutual exclusion.
Suppose Ri ∩ Rj = {Sij }, Rj ∩ Rk = {Sjk }, and Rk ∩ Ri = {Ski }.

Consider the following scenario:

1. Sij has been locked by Si (forcing Sj to wait at Si j).
2. Sjk has been locked by Sj (forcing Sk to wait at Sjk ).
3. Ski has been locked by Sk (forcing Si to wait at Ski ).

This state represents a deadlock involving sites Si , Sj , and Sk .

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Handling Deadlocks

Maekawa’s algorithm handles deadlocks by requiring a

site to yield a lock if the timestamp of its request is
larger than the timestamp of some other request
waiting for the same lock.

A site suspects a deadlock (and initiates message

exchanges to resolve it) whenever a higher priority
request arrives and waits at a site because the site has
sent a REPLY message to a lower priority request.

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Message types for Handling Deadlocks
Deadlock handling requires three types of messages:

FAILED: A FAILED message from site Si to site Sj indicates that Si can

not grant Sj’s request because it has currently granted permission
to a site with a higher priority request.

INQUIRE: An INQUIRE message from Si to Sj indicates that Si would like

to find out from Sj if it has succeeded in locking all the sites in its
request set.

YIELD: A YIELD message from site Si to Sj indicates that Si is returning

the permission to Sj (to yield to a higher priority request at Sj ).

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Handling Deadlocks
Maekawa’s algorithm handles deadlocks as follows:
When a REQUEST(ts , i ) from site Si blocks at site Sj because Sj has currently
granted permission to site Sk , then Sj sends a FAILED( j) message to Si if Si ’s
request has lower priority. Otherwise, Sj sends an INQUIRE( j) message to site Sk .
In response to an INQUIRE( j) message from site Sj , site Sk sends a YIELD(k )
message to Sj provided Sk has received a FAILED message from a site in its request
set and if it sent a YIELD to any of these sites, but has not received a new REPLY
from it.
In response to a YIELD(k ) message from site Sk , site Sj assumes as if it has been
released by Sk , places the request of Sk at appropriate location in the request
queue, and sends a REPLY( j) to the top request’s site in the queue.

Maximum number of messages required per CS execution in this case is 5

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Handling Deadlocks: Case-I
When a REQUEST(ts , i ) from site Si blocks at site Sj because Sj has currently
granted permission to site Sk , then Sj sends a FAILED( j ) message to Si if Si ’s
request has lower priority. Otherwise, Sj sends an INQUIRE( j ) message to site Sk .

Si
REQUEST(ts,i) FAILED (j)
Block at Sj IF (ts,i) < (ts,k)
Sj

REQUEST(ts,k) REPLY (j) INQUIRE (j)

Else(ts,i) > (ts,k)

Sk

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Handling Deadlocks: Case-II
In response to an INQUIRE(j) message from site Sj , site Sk sends a YIELD(k ) message
to Sj provided Sk has received a FAILED message from a site in its request set and if
it sent a YIELD to any of these sites, but has not received a new REPLY from it.

Si

Sj

INQUIRE (j)
YIELD (k)

Sk

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Handling Deadlocks: Case-III
In response to a YIELD(k ) message from site Sk , site Sj assumes as if it has
been released by Sk , places the request of Sk at appropriate location in the
request queue, and sends a REPLY( j) to the top request’s site in the queue.

Si
Sj assumes as if it has been
released by Sk REPLY ( j) to top
request site i.e. Si
Sj

YIELD (k)

Sk

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Failures?

other ways to handle failures: Use Paxos like!

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Industry Mutual Exclusion : Chubby
Google’s system for locking
Used underneath Google’s systems like
BigTable, Megastore, etc.
Chubby provides Advisory locks only
Doesn’t guarantee mutual exclusion unless every
client checks lock before accessing resource

Reference: http://research.google.com/archive/chubby.html

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion
Chubby
Can use not only for locking but also
writing small configuration files
Relies on Paxos-like (consensus) protocol Server A
Group of servers with one elected as
Master Server B
All servers replicate same information
Clients send read requests to Master,
Server C
which serves it locally
Clients send write requests to Master,
which sends it to all servers, gets majority Server D Master
(quorum) among servers, and then
responds to client
Server E
On master failure, run election protocol
On replica failure, just replace it and have
it catch up
Cloud Computing and DistributedVuSystems
Pham Distributed Mutual Exclusion
Conclusion
Mutual exclusion important problem in cloud
computing systems
Classical algorithms
Central
Ring-based
Lamport’s Algorithm
Ricart-Agrawala
Maekawa
Industry systems
Chubby: a coordination service
Similarly, Apache Zookeeper for coordination

Cloud Computing and DistributedVuSystems

Pham Distributed Mutual Exclusion